summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
authorNicolas Dechesne <nicolas.dechesne@linaro.org>2020-06-26 19:10:51 +0200
committerRichard Purdie <richard.purdie@linuxfoundation.org>2020-09-17 10:09:33 +0100
commit9bd69b1f1d71a9692189beeac75af9dfbad816cc (patch)
tree305347fca899074aed5610e0e82eaec180bf630c /documentation
parentc40a8d5904c29046f1cbbeb998e6cd7c24f9b206 (diff)
downloadpoky-9bd69b1f1d71a9692189beeac75af9dfbad816cc.tar.gz
sphinx: initial sphinx support
This commit is autogenerated pandoc to generate an inital set of reST files based on DocBook XML files. A .rst file is generated for each .xml files in all manuals with this command: cd <manual> for i in *.xml; do \ pandoc -f docbook -t rst --shift-heading-level-by=-1 \ $i -o $(basename $i .xml).rst \ done The conversion was done with: pandoc 2.9.2.1-91 (Arch Linux). Also created an initial top level index file for each document, and added all 'books' to the top leve index.rst file. The YP manuals layout is organized as: Book Chapter Section Section Section Sphinx uses section headers to create the document structure. ReStructuredText defines sections headers like that: To break longer text up into sections, you use section headers. These are a single line of text (one or more words) with adornment: an underline alone, or an underline and an overline together, in dashes "-----", equals "======", tildes "~~~~~~" or any of the non-alphanumeric characters = - ` : ' " ~ ^ _ * + # < > that you feel comfortable with. An underline-only adornment is distinct from an overline-and-underline adornment using the same character. The underline/overline must be at least as long as the title text. Be consistent, since all sections marked with the same adornment style are deemed to be at the same level: Let's define the following convention when converting from Docbook: Book => overline === (Title) Chapter => overline *** (1.) Section => ==== (1.1) Section => ---- (1.1.1) Section => ~~~~ (1.1.1.1) Section => ^^^^ (1.1.1.1.1) During the conversion with pandoc, we used --shift-heading-level=-1 to convert most of DocBook headings automatically. However with this setting, the Chapter header was removed, so I added it back manually. Without this setting all headings were off by one, which was more difficult to manually fix. At least with this change, we now have the same TOC with Sphinx and DocBook. (From yocto-docs rev: 3c73d64a476d4423ee4c6808c685fa94d88d7df8) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r--documentation/adt-manual/adt-command.rst178
-rw-r--r--documentation/adt-manual/adt-intro.rst136
-rw-r--r--documentation/adt-manual/adt-manual-intro.rst22
-rw-r--r--documentation/adt-manual/adt-manual.rst13
-rw-r--r--documentation/adt-manual/adt-package.rst68
-rw-r--r--documentation/adt-manual/adt-prepare.rst753
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst360
-rw-r--r--documentation/bsp-guide/bsp-guide.rst9
-rw-r--r--documentation/bsp-guide/bsp.rst1450
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.rst10227
-rw-r--r--documentation/dev-manual/dev-manual-intro.rst62
-rw-r--r--documentation/dev-manual/dev-manual-qemu.rst429
-rw-r--r--documentation/dev-manual/dev-manual-start.rst873
-rw-r--r--documentation/dev-manual/dev-manual.rst12
-rw-r--r--documentation/index.rst11
-rw-r--r--documentation/kernel-dev/kernel-dev-advanced.rst762
-rw-r--r--documentation/kernel-dev/kernel-dev-common.rst1728
-rw-r--r--documentation/kernel-dev/kernel-dev-concepts-appx.rst409
-rw-r--r--documentation/kernel-dev/kernel-dev-faq.rst43
-rw-r--r--documentation/kernel-dev/kernel-dev-intro.rst178
-rw-r--r--documentation/kernel-dev/kernel-dev-maint-appx.rst226
-rw-r--r--documentation/kernel-dev/kernel-dev.rst14
-rw-r--r--documentation/overview-manual/overview-manual-concepts.rst2133
-rw-r--r--documentation/overview-manual/overview-manual-development-environment.rst656
-rw-r--r--documentation/overview-manual/overview-manual-intro.rst74
-rw-r--r--documentation/overview-manual/overview-manual-yp-intro.rst941
-rw-r--r--documentation/overview-manual/overview-manual.rst12
-rw-r--r--documentation/profile-manual/profile-manual-arch.rst28
-rw-r--r--documentation/profile-manual/profile-manual-examples.rst20
-rw-r--r--documentation/profile-manual/profile-manual-intro.rst67
-rw-r--r--documentation/profile-manual/profile-manual-usage.rst2018
-rw-r--r--documentation/profile-manual/profile-manual.rst12
-rw-r--r--documentation/ref-manual/faq.rst418
-rw-r--r--documentation/ref-manual/migration.rst5081
-rw-r--r--documentation/ref-manual/ref-classes.rst2881
-rw-r--r--documentation/ref-manual/ref-devtool-reference.rst533
-rw-r--r--documentation/ref-manual/ref-features.rst353
-rw-r--r--documentation/ref-manual/ref-images.rst137
-rw-r--r--documentation/ref-manual/ref-kickstart.rst205
-rw-r--r--documentation/ref-manual/ref-manual.rst24
-rw-r--r--documentation/ref-manual/ref-qa-checks.rst524
-rw-r--r--documentation/ref-manual/ref-release-process.rst182
-rw-r--r--documentation/ref-manual/ref-structure.rst871
-rw-r--r--documentation/ref-manual/ref-system-requirements.rst378
-rw-r--r--documentation/ref-manual/ref-tasks.rst834
-rw-r--r--documentation/ref-manual/ref-terms.rst369
-rw-r--r--documentation/ref-manual/ref-variables.rst7924
-rw-r--r--documentation/ref-manual/ref-varlocality.rst164
-rw-r--r--documentation/ref-manual/resources.rst207
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing-standard.rst32
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing.rst347
-rw-r--r--documentation/sdk-manual/sdk-appendix-obtain.rst270
-rw-r--r--documentation/sdk-manual/sdk-extensible.rst1230
-rw-r--r--documentation/sdk-manual/sdk-intro.rst223
-rw-r--r--documentation/sdk-manual/sdk-manual.rst15
-rw-r--r--documentation/sdk-manual/sdk-using.rst136
-rw-r--r--documentation/sdk-manual/sdk-working-projects.rst284
-rw-r--r--documentation/test-manual/test-manual-intro.rst486
-rw-r--r--documentation/test-manual/test-manual-test-process.rst103
-rw-r--r--documentation/test-manual/test-manual-understand-autobuilder.rst287
-rw-r--r--documentation/test-manual/test-manual.rst12
-rw-r--r--documentation/toaster-manual/toaster-manual-intro.rst97
-rw-r--r--documentation/toaster-manual/toaster-manual-reference.rst515
-rw-r--r--documentation/toaster-manual/toaster-manual-setup-and-use.rst495
-rw-r--r--documentation/toaster-manual/toaster-manual-start.rst46
-rw-r--r--documentation/toaster-manual/toaster-manual.rst12
66 files changed, 49599 insertions, 0 deletions
diff --git a/documentation/adt-manual/adt-command.rst b/documentation/adt-manual/adt-command.rst
new file mode 100644
index 0000000000..41bc41e149
--- /dev/null
+++ b/documentation/adt-manual/adt-command.rst
@@ -0,0 +1,178 @@
1**********************
2Using the Command Line
3**********************
4
5Recall that earlier the manual discussed how to use an existing
6toolchain tarball that had been installed into the default installation
7directory, ``/opt/poky/DISTRO``, which is outside of the `Build
8Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__ (see the section
9"`Using a Cross-Toolchain
10Tarball) <#using-an-existing-toolchain-tarball>`__". And, that sourcing
11your architecture-specific environment setup script initializes a
12suitable cross-toolchain development environment.
13
14During this setup, locations for the compiler, QEMU scripts, QEMU
15binary, a special version of ``pkgconfig`` and other useful utilities
16are added to the ``PATH`` variable. Also, variables to assist
17``pkgconfig`` and ``autotools`` are also defined so that, for example,
18``configure.sh`` can find pre-generated test results for tests that need
19target hardware on which to run. You can see the "`Setting Up the
20Cross-Development
21Environment <#setting-up-the-cross-development-environment>`__" section
22for the list of cross-toolchain environment variables established by the
23script.
24
25Collectively, these conditions allow you to easily use the toolchain
26outside of the OpenEmbedded build environment on both Autotools-based
27projects and Makefile-based projects. This chapter provides information
28for both these types of projects.
29
30Autotools-Based Projects
31========================
32
33Once you have a suitable cross-toolchain installed, it is very easy to
34develop a project outside of the OpenEmbedded build system. This section
35presents a simple "Helloworld" example that shows how to set up,
36compile, and run the project.
37
38Creating and Running a Project Based on GNU Autotools
39-----------------------------------------------------
40
41Follow these steps to create a simple Autotools-based project:
42
431. *Create your directory:* Create a clean directory for your project
44 and then make that directory your working location: $ mkdir
45 $HOME/helloworld $ cd $HOME/helloworld
46
472. *Populate the directory:* Create ``hello.c``, ``Makefile.am``, and
48 ``configure.in`` files as follows:
49
50 - For ``hello.c``, include these lines: #include <stdio.h> main() {
51 printf("Hello World!\n"); }
52
53 - For ``Makefile.am``, include these lines: bin_PROGRAMS = hello
54 hello_SOURCES = hello.c
55
56 - For ``configure.in``, include these lines: AC_INIT(hello.c)
57 AM_INIT_AUTOMAKE(hello,0.1) AC_PROG_CC AC_PROG_INSTALL
58 AC_OUTPUT(Makefile)
59
603. *Source the cross-toolchain environment setup file:* Installation of
61 the cross-toolchain creates a cross-toolchain environment setup
62 script in the directory that the ADT was installed. Before you can
63 use the tools to develop your project, you must source this setup
64 script. The script begins with the string "environment-setup" and
65 contains the machine architecture, which is followed by the string
66 "poky-linux". Here is an example that sources a script from the
67 default ADT installation directory that uses the 32-bit Intel x86
68 Architecture and the DISTRO_NAME Yocto Project release: $ source
69 /opt/poky/DISTRO/environment-setup-i586-poky-linux
70
714. *Generate the local aclocal.m4 files and create the configure
72 script:* The following GNU Autotools generate the local
73 ``aclocal.m4`` files and create the configure script: $ aclocal $
74 autoconf
75
765. *Generate files needed by GNU coding standards:* GNU coding
77 standards require certain files in order for the project to be
78 compliant. This command creates those files: $ touch NEWS README
79 AUTHORS ChangeLog
80
816. *Generate the configure file:* This command generates the
82 ``configure``: $ automake -a
83
847. *Cross-compile the project:* This command compiles the project using
85 the cross-compiler. The
86 ```CONFIGURE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-CONFIGURE_FLAGS>`__
87 environment variable provides the minimal arguments for GNU
88 configure: $ ./configure ${CONFIGURE_FLAGS}
89
908. *Make and install the project:* These two commands generate and
91 install the project into the destination directory: $ make $ make
92 install DESTDIR=./tmp
93
949. *Verify the installation:* This command is a simple way to verify
95 the installation of your project. Running the command prints the
96 architecture on which the binary file can run. This architecture
97 should be the same architecture that the installed cross-toolchain
98 supports. $ file ./tmp/usr/local/bin/hello
99
10010. *Execute your project:* To execute the project in the shell, simply
101 enter the name. You could also copy the binary to the actual target
102 hardware and run the project there as well: $ ./hello As expected,
103 the project displays the "Hello World!" message.
104
105Passing Host Options
106--------------------
107
108For an Autotools-based project, you can use the cross-toolchain by just
109passing the appropriate host option to ``configure.sh``. The host option
110you use is derived from the name of the environment setup script found
111in the directory in which you installed the cross-toolchain. For
112example, the host option for an ARM-based target that uses the GNU EABI
113is ``armv5te-poky-linux-gnueabi``. You will notice that the name of the
114script is ``environment-setup-armv5te-poky-linux-gnueabi``. Thus, the
115following command works to update your project and rebuild it using the
116appropriate cross-toolchain tools: $ ./configure
117--host=armv5te-poky-linux-gnueabi \\ --with-libtool-sysroot=sysroot_dir
118
119.. note::
120
121 If the
122 configure
123 script results in problems recognizing the
124 --with-libtool-sysroot=
125 sysroot-dir
126 option, regenerate the script to enable the support by doing the
127 following and then run the script again:
128 ::
129
130 $ libtoolize --automake
131 $ aclocal -I ${OECORE_NATIVE_SYSROOT}/usr/share/aclocal \
132 [-I dir_containing_your_project-specific_m4_macros]
133 $ autoconf
134 $ autoheader
135 $ automake -a
136
137
138Makefile-Based Projects
139=======================
140
141For Makefile-based projects, the cross-toolchain environment variables
142established by running the cross-toolchain environment setup script are
143subject to general ``make`` rules.
144
145To illustrate this, consider the following four cross-toolchain
146environment variables:
147`CC <&YOCTO_DOCS_REF_URL;#var-CC>`__\ =i586-poky-linux-gcc -m32
148-march=i586 --sysroot=/opt/poky/1.8/sysroots/i586-poky-linux
149`LD <&YOCTO_DOCS_REF_URL;#var-LD>`__\ =i586-poky-linux-ld
150--sysroot=/opt/poky/1.8/sysroots/i586-poky-linux
151`CFLAGS <&YOCTO_DOCS_REF_URL;#var-CFLAGS>`__\ =-O2 -pipe -g
152-feliminate-unused-debug-types
153`CXXFLAGS <&YOCTO_DOCS_REF_URL;#var-CXXFLAGS>`__\ =-O2 -pipe -g
154-feliminate-unused-debug-types Now, consider the following three cases:
155
156- *Case 1 - No Variables Set in the ``Makefile``:* Because these
157 variables are not specifically set in the ``Makefile``, the variables
158 retain their values based on the environment.
159
160- *Case 2 - Variables Set in the ``Makefile``:* Specifically setting
161 variables in the ``Makefile`` during the build results in the
162 environment settings of the variables being overwritten.
163
164- *Case 3 - Variables Set when the ``Makefile`` is Executed from the
165 Command Line:* Executing the ``Makefile`` from the command line
166 results in the variables being overwritten with command-line content
167 regardless of what is being set in the ``Makefile``. In this case,
168 environment variables are not considered unless you use the "-e" flag
169 during the build: $ make -e file If you use this flag, then the
170 environment values of the variables override any variables
171 specifically set in the ``Makefile``.
172
173.. note::
174
175 For the list of variables set up by the cross-toolchain environment
176 setup script, see the "
177 Setting Up the Cross-Development Environment
178 " section.
diff --git a/documentation/adt-manual/adt-intro.rst b/documentation/adt-manual/adt-intro.rst
new file mode 100644
index 0000000000..4e44afacbe
--- /dev/null
+++ b/documentation/adt-manual/adt-intro.rst
@@ -0,0 +1,136 @@
1*****************************************
2The Application Development Toolkit (ADT)
3*****************************************
4
5Part of the Yocto Project development solution is an Application
6Development Toolkit (ADT). The ADT provides you with a custom-built,
7cross-development platform suited for developing a user-targeted product
8application.
9
10Fundamentally, the ADT consists of the following:
11
12- An architecture-specific cross-toolchain and matching sysroot both
13 built by the `OpenEmbedded build
14 system <&YOCTO_DOCS_DEV_URL;#build-system-term>`__. The toolchain and
15 sysroot are based on a `Metadata <&YOCTO_DOCS_DEV_URL;#metadata>`__
16 configuration and extensions, which allows you to cross-develop on
17 the host machine for the target hardware.
18
19- The Eclipse IDE Yocto Plug-in.
20
21- The Quick EMUlator (QEMU), which lets you simulate target hardware.
22
23- Various user-space tools that greatly enhance your application
24 development experience.
25
26The Cross-Development Toolchain
27===============================
28
29The `Cross-Development
30Toolchain <&YOCTO_DOCS_DEV_URL;#cross-development-toolchain>`__ consists
31of a cross-compiler, cross-linker, and cross-debugger that are used to
32develop user-space applications for targeted hardware. This toolchain is
33created either by running the ADT Installer script, a toolchain
34installer script, or through a `Build
35Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__ that is based on
36your Metadata configuration or extension for your targeted device. The
37cross-toolchain works with a matching target sysroot.
38
39Sysroot
40=======
41
42The matching target sysroot contains needed headers and libraries for
43generating binaries that run on the target architecture. The sysroot is
44based on the target root filesystem image that is built by the
45OpenEmbedded build system and uses the same Metadata configuration used
46to build the cross-toolchain.
47
48.. _eclipse-overview:
49
50Eclipse Yocto Plug-in
51=====================
52
53The Eclipse IDE is a popular development environment and it fully
54supports development using the Yocto Project. When you install and
55configure the Eclipse Yocto Project Plug-in into the Eclipse IDE, you
56maximize your Yocto Project experience. Installing and configuring the
57Plug-in results in an environment that has extensions specifically
58designed to let you more easily develop software. These extensions allow
59for cross-compilation, deployment, and execution of your output into a
60QEMU emulation session. You can also perform cross-debugging and
61profiling. The environment also supports a suite of tools that allows
62you to perform remote profiling, tracing, collection of power data,
63collection of latency data, and collection of performance data.
64
65For information about the application development workflow that uses the
66Eclipse IDE and for a detailed example of how to install and configure
67the Eclipse Yocto Project Plug-in, see the "`Working Within
68Eclipse <&YOCTO_DOCS_DEV_URL;#adt-eclipse>`__" section of the Yocto
69Project Development Manual.
70
71The QEMU Emulator
72=================
73
74The QEMU emulator allows you to simulate your hardware while running
75your application or image. QEMU is made available a number of ways:
76
77- If you use the ADT Installer script to install ADT, you can specify
78 whether or not to install QEMU.
79
80- If you have cloned the ``poky`` Git repository to create a `Source
81 Directory <&YOCTO_DOCS_DEV_URL;#source-directory>`__ and you have
82 sourced the environment setup script, QEMU is installed and
83 automatically available.
84
85- If you have downloaded a Yocto Project release and unpacked it to
86 create a `Source Directory <&YOCTO_DOCS_DEV_URL;#source-directory>`__
87 and you have sourced the environment setup script, QEMU is installed
88 and automatically available.
89
90- If you have installed the cross-toolchain tarball and you have
91 sourced the toolchain's setup environment script, QEMU is also
92 installed and automatically available.
93
94User-Space Tools
95================
96
97User-space tools are included as part of the Yocto Project. You will
98find these tools helpful during development. The tools include
99LatencyTOP, PowerTOP, OProfile, Perf, SystemTap, and Lttng-ust. These
100tools are common development tools for the Linux platform.
101
102- *LatencyTOP:* LatencyTOP focuses on latency that causes skips in
103 audio, stutters in your desktop experience, or situations that
104 overload your server even when you have plenty of CPU power left.
105
106- *PowerTOP:* Helps you determine what software is using the most
107 power. You can find out more about PowerTOP at
108 ` <https://01.org/powertop/>`__.
109
110- *OProfile:* A system-wide profiler for Linux systems that is capable
111 of profiling all running code at low overhead. You can find out more
112 about OProfile at ` <http://oprofile.sourceforge.net/about/>`__. For
113 examples on how to setup and use this tool, see the
114 "`OProfile <&YOCTO_DOCS_PROF_URL;#profile-manual-oprofile>`__"
115 section in the Yocto Project Profiling and Tracing Manual.
116
117- *Perf:* Performance counters for Linux used to keep track of certain
118 types of hardware and software events. For more information on these
119 types of counters see ` <https://perf.wiki.kernel.org/>`__. For
120 examples on how to setup and use this tool, see the
121 "`perf <&YOCTO_DOCS_PROF_URL;#profile-manual-perf>`__" section in the
122 Yocto Project Profiling and Tracing Manual.
123
124- *SystemTap:* A free software infrastructure that simplifies
125 information gathering about a running Linux system. This information
126 helps you diagnose performance or functional problems. SystemTap is
127 not available as a user-space tool through the Eclipse IDE Yocto
128 Plug-in. See ` <http://sourceware.org/systemtap>`__ for more
129 information on SystemTap. For examples on how to setup and use this
130 tool, see the
131 "`SystemTap <&YOCTO_DOCS_PROF_URL;#profile-manual-systemtap>`__"
132 section in the Yocto Project Profiling and Tracing Manual.
133
134- *Lttng-ust:* A User-space Tracer designed to provide detailed
135 information on user-space activity. See ` <http://lttng.org/ust>`__
136 for more information on Lttng-ust.
diff --git a/documentation/adt-manual/adt-manual-intro.rst b/documentation/adt-manual/adt-manual-intro.rst
new file mode 100644
index 0000000000..6e914dc8ef
--- /dev/null
+++ b/documentation/adt-manual/adt-manual-intro.rst
@@ -0,0 +1,22 @@
1************
2Introduction
3************
4
5Welcome to the Yocto Project Application Developer's Guide. This manual
6provides information that lets you begin developing applications using
7the Yocto Project.
8
9The Yocto Project provides an application development environment based
10on an Application Development Toolkit (ADT) and the availability of
11stand-alone cross-development toolchains and other tools. This manual
12describes the ADT and how you can configure and install it, how to
13access and use the cross-development toolchains, how to customize the
14development packages installation, how to use command-line development
15for both Autotools-based and Makefile-based projects, and an
16introduction to the Eclipse IDE Yocto Plug-in.
17
18.. note::
19
20 The ADT is distribution-neutral and does not require the Yocto
21 Project reference distribution, which is called Poky. This manual,
22 however, uses examples that use the Poky distribution.
diff --git a/documentation/adt-manual/adt-manual.rst b/documentation/adt-manual/adt-manual.rst
new file mode 100644
index 0000000000..d53c38b550
--- /dev/null
+++ b/documentation/adt-manual/adt-manual.rst
@@ -0,0 +1,13 @@
1===========================================
2Yocto Project Application Developer's Guide
3===========================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 adt-manual-intro
10 adt-intro
11 adt-prepare
12 adt-package
13 adt-command
diff --git a/documentation/adt-manual/adt-package.rst b/documentation/adt-manual/adt-package.rst
new file mode 100644
index 0000000000..6a6da1dd8f
--- /dev/null
+++ b/documentation/adt-manual/adt-package.rst
@@ -0,0 +1,68 @@
1************************************************************
2Optionally Customizing the Development Packages Installation
3************************************************************
4
5Because the Yocto Project is suited for embedded Linux development, it
6is likely that you will need to customize your development packages
7installation. For example, if you are developing a minimal image, then
8you might not need certain packages (e.g. graphics support packages).
9Thus, you would like to be able to remove those packages from your
10target sysroot.
11
12Package Management Systems
13==========================
14
15The OpenEmbedded build system supports the generation of sysroot files
16using three different Package Management Systems (PMS):
17
18- *OPKG:* A less well known PMS whose use originated in the
19 OpenEmbedded and OpenWrt embedded Linux projects. This PMS works with
20 files packaged in an ``.ipk`` format. See
21 ` <http://en.wikipedia.org/wiki/Opkg>`__ for more information about
22 OPKG.
23
24- *RPM:* A more widely known PMS intended for GNU/Linux distributions.
25 This PMS works with files packaged in an ``.rpm`` format. The build
26 system currently installs through this PMS by default. See
27 ` <http://en.wikipedia.org/wiki/RPM_Package_Manager>`__ for more
28 information about RPM.
29
30- *Debian:* The PMS for Debian-based systems is built on many PMS
31 tools. The lower-level PMS tool ``dpkg`` forms the base of the Debian
32 PMS. For information on dpkg see
33 ` <http://en.wikipedia.org/wiki/Dpkg>`__.
34
35Configuring the PMS
36===================
37
38Whichever PMS you are using, you need to be sure that the
39```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
40variable in the ``conf/local.conf`` file is set to reflect that system.
41The first value you choose for the variable specifies the package file
42format for the root filesystem at sysroot. Additional values specify
43additional formats for convenience or testing. See the
44``conf/local.conf`` configuration file for details.
45
46.. note::
47
48 For build performance information related to the PMS, see the "
49 package.bbclass
50 " section in the Yocto Project Reference Manual.
51
52As an example, consider a scenario where you are using OPKG and you want
53to add the ``libglade`` package to the target sysroot.
54
55First, you should generate the IPK file for the ``libglade`` package and
56add it into a working ``opkg`` repository. Use these commands: $ bitbake
57libglade $ bitbake package-index
58
59Next, source the cross-toolchain environment setup script found in the
60`Source Directory <&YOCTO_DOCS_DEV_URL;#source-directory>`__. Follow
61that by setting up the installation destination to point to your sysroot
62as sysroot_dir. Finally, have an OPKG configuration file conf_file that
63corresponds to the ``opkg`` repository you have just created. The
64following command forms should now work: $ opkg-cl –f conf_file -o
65sysroot_dir update $ opkg-cl –f cconf_file -o sysroot_dir \\
66--force-overwrite install libglade $ opkg-cl –f cconf_file -o
67sysroot_dir \\ --force-overwrite install libglade-dbg $ opkg-cl –f
68conf_file> -osysroot_dir> \\ --force-overwrite install libglade-dev
diff --git a/documentation/adt-manual/adt-prepare.rst b/documentation/adt-manual/adt-prepare.rst
new file mode 100644
index 0000000000..fa7657c616
--- /dev/null
+++ b/documentation/adt-manual/adt-prepare.rst
@@ -0,0 +1,753 @@
1*************************************
2Preparing for Application Development
3*************************************
4
5In order to develop applications, you need set up your host development
6system. Several ways exist that allow you to install cross-development
7tools, QEMU, the Eclipse Yocto Plug-in, and other tools. This chapter
8describes how to prepare for application development.
9
10.. _installing-the-adt:
11
12Installing the ADT and Toolchains
13=================================
14
15The following list describes installation methods that set up varying
16degrees of tool availability on your system. Regardless of the
17installation method you choose, you must ``source`` the cross-toolchain
18environment setup script, which establishes several key environment
19variables, before you use a toolchain. See the "`Setting Up the
20Cross-Development
21Environment <#setting-up-the-cross-development-environment>`__" section
22for more information.
23
24.. note::
25
26 Avoid mixing installation methods when installing toolchains for
27 different architectures. For example, avoid using the ADT Installer
28 to install some toolchains and then hand-installing cross-development
29 toolchains by running the toolchain installer for different
30 architectures. Mixing installation methods can result in situations
31 where the ADT Installer becomes unreliable and might not install the
32 toolchain.
33
34 If you must mix installation methods, you might avoid problems by
35 deleting ``/var/lib/opkg``, thus purging the ``opkg`` package
36 metadata.
37
38- *Use the ADT installer script:* This method is the recommended way to
39 install the ADT because it automates much of the process for you. For
40 example, you can configure the installation to install the QEMU
41 emulator and the user-space NFS, specify which root filesystem
42 profiles to download, and define the target sysroot location.
43
44- *Use an existing toolchain:* Using this method, you select and
45 download an architecture-specific toolchain installer and then run
46 the script to hand-install the toolchain. If you use this method, you
47 just get the cross-toolchain and QEMU - you do not get any of the
48 other mentioned benefits had you run the ADT Installer script.
49
50- *Use the toolchain from within the Build Directory:* If you already
51 have a `Build Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__,
52 you can build the cross-toolchain within the directory. However, like
53 the previous method mentioned, you only get the cross-toolchain and
54 QEMU - you do not get any of the other benefits without taking
55 separate steps.
56
57Using the ADT Installer
58-----------------------
59
60To run the ADT Installer, you need to get the ADT Installer tarball, be
61sure you have the necessary host development packages that support the
62ADT Installer, and then run the ADT Installer Script.
63
64For a list of the host packages needed to support ADT installation and
65use, see the "ADT Installer Extras" lists in the "`Required Packages for
66the Host Development
67System <&YOCTO_DOCS_REF_URL;#required-packages-for-the-host-development-system>`__"
68section of the Yocto Project Reference Manual.
69
70Getting the ADT Installer Tarball
71~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
72
73The ADT Installer is contained in the ADT Installer tarball. You can get
74the tarball using either of these methods:
75
76- *Download the Tarball:* You can download the tarball from
77 ` <&YOCTO_ADTINSTALLER_DL_URL;>`__ into any directory.
78
79- *Build the Tarball:* You can use
80 `BitBake <&YOCTO_DOCS_DEV_URL;#bitbake-term>`__ to generate the
81 tarball inside an existing `Build
82 Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__.
83
84 If you use BitBake to generate the ADT Installer tarball, you must
85 ``source`` the environment setup script
86 (````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
87 ```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
88 located in the Source Directory before running the ``bitbake``
89 command that creates the tarball.
90
91 The following example commands establish the `Source
92 Directory <&YOCTO_DOCS_DEV_URL;#source-directory>`__, check out the
93 current release branch, set up the build environment while also
94 creating the default Build Directory, and run the ``bitbake`` command
95 that results in the tarball
96 ``poky/build/tmp/deploy/sdk/adt_installer.tar.bz2``:
97
98 .. note::
99
100 Before using BitBake to build the ADT tarball, be sure to make
101 sure your
102 local.conf
103 file is properly configured. See the "
104 User Configuration
105 " section in the Yocto Project Reference Manual for general
106 configuration information.
107
108 $ cd ~ $ git clone git://git.yoctoproject.org/poky $ cd poky $ git
109 checkout -b DISTRO_NAME origin/DISTRO_NAME $ source OE_INIT_FILE $
110 bitbake adt-installer
111
112Configuring and Running the ADT Installer Script
113~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
114
115Before running the ADT Installer script, you need to unpack the tarball.
116You can unpack the tarball in any directory you wish. For example, this
117command copies the ADT Installer tarball from where it was built into
118the home directory and then unpacks the tarball into a top-level
119directory named ``adt-installer``: $ cd ~ $ cp
120poky/build/tmp/deploy/sdk/adt_installer.tar.bz2 $HOME $ tar -xjf
121adt_installer.tar.bz2 Unpacking it creates the directory
122``adt-installer``, which contains the ADT Installer script
123(``adt_installer``) and its configuration file (``adt_installer.conf``).
124
125Before you run the script, however, you should examine the ADT Installer
126configuration file and be sure you are going to get what you want. Your
127configurations determine which kernel and filesystem image are
128downloaded.
129
130The following list describes the configurations you can define for the
131ADT Installer. For configuration values and restrictions, see the
132comments in the ``adt-installer.conf`` file:
133
134- ``YOCTOADT_REPO``: This area includes the IPKG-based packages and the
135 root filesystem upon which the installation is based. If you want to
136 set up your own IPKG repository pointed to by ``YOCTOADT_REPO``, you
137 need to be sure that the directory structure follows the same layout
138 as the reference directory set up at
139 ` <http://adtrepo.yoctoproject.org>`__. Also, your repository needs
140 to be accessible through HTTP.
141
142- ``YOCTOADT_TARGETS``: The machine target architectures for which you
143 want to set up cross-development environments.
144
145- ``YOCTOADT_QEMU``: Indicates whether or not to install the emulator
146 QEMU.
147
148- ``YOCTOADT_NFS_UTIL``: Indicates whether or not to install user-mode
149 NFS. If you plan to use the Eclipse IDE Yocto plug-in against QEMU,
150 you should install NFS.
151
152 .. note::
153
154 To boot QEMU images using our userspace NFS server, you need to be
155 running
156 portmap
157 or
158 rpcbind
159 . If you are running
160 rpcbind
161 , you will also need to add the
162 -i
163 option when
164 rpcbind
165 starts up. Please make sure you understand the security
166 implications of doing this. You might also have to modify your
167 firewall settings to allow NFS booting to work.
168
169- ``YOCTOADT_ROOTFS_``\ arch: The root filesystem images you want to
170 download from the ``YOCTOADT_IPKG_REPO`` repository.
171
172- ``YOCTOADT_TARGET_SYSROOT_IMAGE_``\ arch: The particular root
173 filesystem used to extract and create the target sysroot. The value
174 of this variable must have been specified with
175 ``YOCTOADT_ROOTFS_``\ arch. For example, if you downloaded both
176 ``minimal`` and ``sato-sdk`` images by setting
177 ``YOCTOADT_ROOTFS_``\ arch to "minimal sato-sdk", then
178 ``YOCTOADT_ROOTFS_``\ arch must be set to either "minimal" or
179 "sato-sdk".
180
181- ``YOCTOADT_TARGET_SYSROOT_LOC_``\ arch: The location on the
182 development host where the target sysroot is created.
183
184After you have configured the ``adt_installer.conf`` file, run the
185installer using the following command: $ cd adt-installer $
186./adt_installer Once the installer begins to run, you are asked to enter
187the location for cross-toolchain installation. The default location is
188``/opt/poky/``\ release. After either accepting the default location or
189selecting your own location, you are prompted to run the installation
190script interactively or in silent mode. If you want to closely monitor
191the installation, choose “I” for interactive mode rather than “S” for
192silent mode. Follow the prompts from the script to complete the
193installation.
194
195Once the installation completes, the ADT, which includes the
196cross-toolchain, is installed in the selected installation directory.
197You will notice environment setup files for the cross-toolchain in the
198installation directory, and image tarballs in the ``adt-installer``
199directory according to your installer configurations, and the target
200sysroot located according to the ``YOCTOADT_TARGET_SYSROOT_LOC_``\ arch
201variable also in your configuration file.
202
203.. _using-an-existing-toolchain-tarball:
204
205Using a Cross-Toolchain Tarball
206-------------------------------
207
208If you want to simply install a cross-toolchain by hand, you can do so
209by running the toolchain installer. The installer includes the pre-built
210cross-toolchain, the ``runqemu`` script, and support files. If you use
211this method to install the cross-toolchain, you might still need to
212install the target sysroot by installing and extracting it separately.
213For information on how to install the sysroot, see the "`Extracting the
214Root Filesystem <#extracting-the-root-filesystem>`__" section.
215
216Follow these steps:
217
2181. *Get your toolchain installer using one of the following methods:*
219
220 - Go to ` <&YOCTO_TOOLCHAIN_DL_URL;>`__ and find the folder that
221 matches your host development system (i.e. ``i686`` for 32-bit
222 machines or ``x86_64`` for 64-bit machines).
223
224 Go into that folder and download the toolchain installer whose
225 name includes the appropriate target architecture. The toolchains
226 provided by the Yocto Project are based off of the
227 ``core-image-sato`` image and contain libraries appropriate for
228 developing against that image. For example, if your host
229 development system is a 64-bit x86 system and you are going to use
230 your cross-toolchain for a 32-bit x86 target, go into the
231 ``x86_64`` folder and download the following installer:
232 poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
233
234 - Build your own toolchain installer. For cases where you cannot use
235 an installer from the download area, you can build your own as
236 described in the "`Optionally Building a Toolchain
237 Installer <#optionally-building-a-toolchain-installer>`__"
238 section.
239
2402. *Once you have the installer, run it to install the toolchain:*
241
242 .. note::
243
244 You must change the permissions on the toolchain installer script
245 so that it is executable.
246
247 The following command shows how to run the installer given a
248 toolchain tarball for a 64-bit x86 development host system and a
249 32-bit x86 target architecture. The example assumes the toolchain
250 installer is located in ``~/Downloads/``. $
251 ~/Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
252 The first thing the installer prompts you for is the directory into
253 which you want to install the toolchain. The default directory used
254 is ``/opt/poky/DISTRO``. If you do not have write permissions for the
255 directory into which you are installing the toolchain, the toolchain
256 installer notifies you and exits. Be sure you have write permissions
257 in the directory and run the installer again.
258
259 When the script finishes, the cross-toolchain is installed. You will
260 notice environment setup files for the cross-toolchain in the
261 installation directory.
262
263.. _using-the-toolchain-from-within-the-build-tree:
264
265Using BitBake and the Build Directory
266-------------------------------------
267
268A final way of making the cross-toolchain available is to use BitBake to
269generate the toolchain within an existing `Build
270Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__. This method does
271not install the toolchain into the default ``/opt`` directory. As with
272the previous method, if you need to install the target sysroot, you must
273do that separately as well.
274
275Follow these steps to generate the toolchain into the Build Directory:
276
2771. *Set up the Build Environment:* Source the OpenEmbedded build
278 environment setup script (i.e.
279 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
280 ```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
281 located in the `Source
282 Directory <&YOCTO_DOCS_DEV_URL;#source-directory>`__.
283
2842. *Check your Local Configuration File:* At this point, you should be
285 sure that the ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__
286 variable in the ``local.conf`` file found in the ``conf`` directory
287 of the Build Directory is set for the target architecture. Comments
288 within the ``local.conf`` file list the values you can use for the
289 ``MACHINE`` variable. If you do not change the ``MACHINE`` variable,
290 the OpenEmbedded build system uses ``qemux86`` as the default target
291 machine when building the cross-toolchain.
292
293 .. note::
294
295 You can populate the Build Directory with the cross-toolchains for
296 more than a single architecture. You just need to edit the
297 MACHINE
298 variable in the
299 local.conf
300 file and re-run the
301 bitbake
302 command.
303
3043. *Make Sure Your Layers are Enabled:* Examine the
305 ``conf/bblayers.conf`` file and make sure that you have enabled all
306 the compatible layers for your target machine. The OpenEmbedded build
307 system needs to be aware of each layer you want included when
308 building images and cross-toolchains. For information on how to
309 enable a layer, see the "`Enabling Your
310 Layer <&YOCTO_DOCS_DEV_URL;#enabling-your-layer>`__" section in the
311 Yocto Project Development Manual.
312
3134. *Generate the Cross-Toolchain:* Run ``bitbake meta-ide-support`` to
314 complete the cross-toolchain generation. Once the ``bitbake`` command
315 finishes, the cross-toolchain is generated and populated within the
316 Build Directory. You will notice environment setup files for the
317 cross-toolchain that contain the string "``environment-setup``" in
318 the Build Directory's ``tmp`` folder.
319
320 Be aware that when you use this method to install the toolchain, you
321 still need to separately extract and install the sysroot filesystem.
322 For information on how to do this, see the "`Extracting the Root
323 Filesystem <#extracting-the-root-filesystem>`__" section.
324
325Setting Up the Cross-Development Environment
326============================================
327
328Before you can develop using the cross-toolchain, you need to set up the
329cross-development environment by sourcing the toolchain's environment
330setup script. If you used the ADT Installer or hand-installed
331cross-toolchain, then you can find this script in the directory you
332chose for installation. For this release, the default installation
333directory is ````. If you installed the toolchain in the `Build
334Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__, you can find the
335environment setup script for the toolchain in the Build Directory's
336``tmp`` directory.
337
338Be sure to run the environment setup script that matches the
339architecture for which you are developing. Environment setup scripts
340begin with the string "``environment-setup``" and include as part of
341their name the architecture. For example, the toolchain environment
342setup script for a 64-bit IA-based architecture installed in the default
343installation directory would be the following:
344YOCTO_ADTPATH_DIR/environment-setup-x86_64-poky-linux When you run the
345setup script, many environment variables are defined:
346```SDKTARGETSYSROOT`` <&YOCTO_DOCS_REF_URL;#var-SDKTARGETSYSROOT>`__ -
347The path to the sysroot used for cross-compilation
348```PKG_CONFIG_PATH`` <&YOCTO_DOCS_REF_URL;#var-PKG_CONFIG_PATH>`__ - The
349path to the target pkg-config files
350```CONFIG_SITE`` <&YOCTO_DOCS_REF_URL;#var-CONFIG_SITE>`__ - A GNU
351autoconf site file preconfigured for the target
352```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__ - The minimal command and
353arguments to run the C compiler
354```CXX`` <&YOCTO_DOCS_REF_URL;#var-CXX>`__ - The minimal command and
355arguments to run the C++ compiler
356```CPP`` <&YOCTO_DOCS_REF_URL;#var-CPP>`__ - The minimal command and
357arguments to run the C preprocessor
358```AS`` <&YOCTO_DOCS_REF_URL;#var-AS>`__ - The minimal command and
359arguments to run the assembler ```LD`` <&YOCTO_DOCS_REF_URL;#var-LD>`__
360- The minimal command and arguments to run the linker
361```GDB`` <&YOCTO_DOCS_REF_URL;#var-GDB>`__ - The minimal command and
362arguments to run the GNU Debugger
363```STRIP`` <&YOCTO_DOCS_REF_URL;#var-STRIP>`__ - The minimal command and
364arguments to run 'strip', which strips symbols
365```RANLIB`` <&YOCTO_DOCS_REF_URL;#var-RANLIB>`__ - The minimal command
366and arguments to run 'ranlib'
367```OBJCOPY`` <&YOCTO_DOCS_REF_URL;#var-OBJCOPY>`__ - The minimal command
368and arguments to run 'objcopy'
369```OBJDUMP`` <&YOCTO_DOCS_REF_URL;#var-OBJDUMP>`__ - The minimal command
370and arguments to run 'objdump' ```AR`` <&YOCTO_DOCS_REF_URL;#var-AR>`__
371- The minimal command and arguments to run 'ar'
372```NM`` <&YOCTO_DOCS_REF_URL;#var-NM>`__ - The minimal command and
373arguments to run 'nm'
374```TARGET_PREFIX`` <&YOCTO_DOCS_REF_URL;#var-TARGET_PREFIX>`__ - The
375toolchain binary prefix for the target tools
376```CROSS_COMPILE`` <&YOCTO_DOCS_REF_URL;#var-CROSS_COMPILE>`__ - The
377toolchain binary prefix for the target tools
378```CONFIGURE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-CONFIGURE_FLAGS>`__ - The
379minimal arguments for GNU configure
380```CFLAGS`` <&YOCTO_DOCS_REF_URL;#var-CFLAGS>`__ - Suggested C flags
381```CXXFLAGS`` <&YOCTO_DOCS_REF_URL;#var-CXXFLAGS>`__ - Suggested C++
382flags ```LDFLAGS`` <&YOCTO_DOCS_REF_URL;#var-LDFLAGS>`__ - Suggested
383linker flags when you use CC to link
384```CPPFLAGS`` <&YOCTO_DOCS_REF_URL;#var-CPPFLAGS>`__ - Suggested
385preprocessor flags
386
387Securing Kernel and Filesystem Images
388=====================================
389
390You will need to have a kernel and filesystem image to boot using your
391hardware or the QEMU emulator. Furthermore, if you plan on booting your
392image using NFS or you want to use the root filesystem as the target
393sysroot, you need to extract the root filesystem.
394
395Getting the Images
396------------------
397
398To get the kernel and filesystem images, you either have to build them
399or download pre-built versions. For an example of how to build these
400images, see the "`Buiding
401Images <&YOCTO_DOCS_QS_URL;#qs-buiding-images>`__" section of the Yocto
402Project Quick Start. For an example of downloading pre-build versions,
403see the "`Example Using Pre-Built Binaries and
404QEMU <#using-pre-built>`__" section.
405
406The Yocto Project ships basic kernel and filesystem images for several
407architectures (``x86``, ``x86-64``, ``mips``, ``powerpc``, and ``arm``)
408that you can use unaltered in the QEMU emulator. These kernel images
409reside in the release area - ` <&YOCTO_MACHINES_DL_URL;>`__ and are
410ideal for experimentation using Yocto Project. For information on the
411image types you can build using the OpenEmbedded build system, see the
412"`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
413Project Reference Manual.
414
415If you are planning on developing against your image and you are not
416building or using one of the Yocto Project development images (e.g.
417``core-image-*-dev``), you must be sure to include the development
418packages as part of your image recipe.
419
420If you plan on remotely deploying and debugging your application from
421within the Eclipse IDE, you must have an image that contains the Yocto
422Target Communication Framework (TCF) agent (``tcf-agent``). You can do
423this by including the ``eclipse-debug`` image feature.
424
425.. note::
426
427 See the "
428 Image Features
429 " section in the Yocto Project Reference Manual for information on
430 image features.
431
432To include the ``eclipse-debug`` image feature, modify your
433``local.conf`` file in the `Build
434Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__ so that the
435```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
436variable includes the "eclipse-debug" feature. After modifying the
437configuration file, you can rebuild the image. Once the image is
438rebuilt, the ``tcf-agent`` will be included in the image and is launched
439automatically after the boot.
440
441Extracting the Root Filesystem
442------------------------------
443
444If you install your toolchain by hand or build it using BitBake and you
445need a root filesystem, you need to extract it separately. If you use
446the ADT Installer to install the ADT, the root filesystem is
447automatically extracted and installed.
448
449Here are some cases where you need to extract the root filesystem:
450
451- You want to boot the image using NFS.
452
453- You want to use the root filesystem as the target sysroot. For
454 example, the Eclipse IDE environment with the Eclipse Yocto Plug-in
455 installed allows you to use QEMU to boot under NFS.
456
457- You want to develop your target application using the root filesystem
458 as the target sysroot.
459
460To extract the root filesystem, first ``source`` the cross-development
461environment setup script to establish necessary environment variables.
462If you built the toolchain in the Build Directory, you will find the
463toolchain environment script in the ``tmp`` directory. If you installed
464the toolchain by hand, the environment setup script is located in
465``/opt/poky/DISTRO``.
466
467After sourcing the environment script, use the ``runqemu-extract-sdk``
468command and provide the filesystem image.
469
470Following is an example. The second command sets up the environment. In
471this case, the setup script is located in the ``/opt/poky/DISTRO``
472directory. The third command extracts the root filesystem from a
473previously built filesystem that is located in the ``~/Downloads``
474directory. Furthermore, this command extracts the root filesystem into
475the ``qemux86-sato`` directory: $ cd ~ $ source
476/opt/poky/DISTRO/environment-setup-i586-poky-linux $ runqemu-extract-sdk
477\\ ~/Downloads/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2
478\\ $HOME/qemux86-sato You could now point to the target sysroot at
479``qemux86-sato``.
480
481Optionally Building a Toolchain Installer
482=========================================
483
484As an alternative to locating and downloading a toolchain installer, you
485can build the toolchain installer if you have a `Build
486Directory <&YOCTO_DOCS_DEV_URL;#build-directory>`__.
487
488.. note::
489
490 Although not the preferred method, it is also possible to use
491 bitbake meta-toolchain
492 to build the toolchain installer. If you do use this method, you must
493 separately install and extract the target sysroot. For information on
494 how to install the sysroot, see the "
495 Extracting the Root Filesystem
496 " section.
497
498To build the toolchain installer and populate the SDK image, use the
499following command: $ bitbake image -c populate_sdk The command results
500in a toolchain installer that contains the sysroot that matches your
501target root filesystem.
502
503Another powerful feature is that the toolchain is completely
504self-contained. The binaries are linked against their own copy of
505``libc``, which results in no dependencies on the target system. To
506achieve this, the pointer to the dynamic loader is configured at install
507time since that path cannot be dynamically altered. This is the reason
508for a wrapper around the ``populate_sdk`` archive.
509
510Another feature is that only one set of cross-canadian toolchain
511binaries are produced per architecture. This feature takes advantage of
512the fact that the target hardware can be passed to ``gcc`` as a set of
513compiler options. Those options are set up by the environment script and
514contained in variables such as ```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__
515and ```LD`` <&YOCTO_DOCS_REF_URL;#var-LD>`__. This reduces the space
516needed for the tools. Understand, however, that a sysroot is still
517needed for every target since those binaries are target-specific.
518
519Remember, before using any BitBake command, you must source the build
520environment setup script (i.e.
521````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
522```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
523located in the Source Directory and you must make sure your
524``conf/local.conf`` variables are correct. In particular, you need to be
525sure the ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable
526matches the architecture for which you are building and that the
527```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__ variable is
528correctly set if you are building a toolchain designed to run on an
529architecture that differs from your current development host machine
530(i.e. the build machine).
531
532When the ``bitbake`` command completes, the toolchain installer will be
533in ``tmp/deploy/sdk`` in the Build Directory.
534
535.. note::
536
537 By default, this toolchain does not build static binaries. If you
538 want to use the toolchain to build these types of libraries, you need
539 to be sure your image has the appropriate static development
540 libraries. Use the
541 IMAGE_INSTALL
542 variable inside your
543 local.conf
544 file to install the appropriate library packages. Following is an
545 example using
546 glibc
547 static development libraries:
548 ::
549
550 IMAGE_INSTALL_append = " glibc-staticdev"
551
552
553Optionally Using an External Toolchain
554======================================
555
556You might want to use an external toolchain as part of your development.
557If this is the case, the fundamental steps you need to accomplish are as
558follows:
559
560- Understand where the installed toolchain resides. For cases where you
561 need to build the external toolchain, you would need to take separate
562 steps to build and install the toolchain.
563
564- Make sure you add the layer that contains the toolchain to your
565 ``bblayers.conf`` file through the
566 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable.
567
568- Set the
569 ```EXTERNAL_TOOLCHAIN`` <&YOCTO_DOCS_REF_URL;#var-EXTERNAL_TOOLCHAIN>`__
570 variable in your ``local.conf`` file to the location in which you
571 installed the toolchain.
572
573A good example of an external toolchain used with the Yocto Project is
574Mentor Graphics Sourcery G++ Toolchain. You can see information on how
575to use that particular layer in the ``README`` file at
576` <http://github.com/MentorEmbedded/meta-sourcery/>`__. You can find
577further information by reading about the
578```TCMODE`` <&YOCTO_DOCS_REF_URL;#var-TCMODE>`__ variable in the Yocto
579Project Reference Manual's variable glossary.
580
581.. _using-pre-built:
582
583Example Using Pre-Built Binaries and QEMU
584=========================================
585
586If hardware, libraries and services are stable, you can get started by
587using a pre-built binary of the filesystem image, kernel, and toolchain
588and run it using the QEMU emulator. This scenario is useful for
589developing application software.
590
591|Using a Pre-Built Image|
592
593For this scenario, you need to do several things:
594
595- Install the appropriate stand-alone toolchain tarball.
596
597- Download the pre-built image that will boot with QEMU. You need to be
598 sure to get the QEMU image that matches your target machine’s
599 architecture (e.g. x86, ARM, etc.).
600
601- Download the filesystem image for your target machine's architecture.
602
603- Set up the environment to emulate the hardware and then start the
604 QEMU emulator.
605
606Installing the Toolchain
607------------------------
608
609You can download a tarball installer, which includes the pre-built
610toolchain, the ``runqemu`` script, and support files from the
611appropriate directory under ` <&YOCTO_TOOLCHAIN_DL_URL;>`__. Toolchains
612are available for 32-bit and 64-bit x86 development systems from the
613``i686`` and ``x86_64`` directories, respectively. The toolchains the
614Yocto Project provides are based off the ``core-image-sato`` image and
615contain libraries appropriate for developing against that image. Each
616type of development system supports five or more target architectures.
617
618The names of the tarball installer scripts are such that a string
619representing the host system appears first in the filename and then is
620immediately followed by a string representing the target architecture.
621
622::
623
624 poky-glibc-host_system-image_type-arch-toolchain-release_version.sh
625
626 Where:
627 host_system is a string representing your development system:
628
629 i686 or x86_64.
630
631 image_type is a string representing the image you wish to
632 develop a Software Development Toolkit (SDK) for use against.
633 The Yocto Project builds toolchain installers using the
634 following BitBake command:
635
636 bitbake core-image-sato -c populate_sdk
637
638 arch is a string representing the tuned target architecture:
639
640 i586, x86_64, powerpc, mips, armv7a or armv5te
641
642 release_version is a string representing the release number of the
643 Yocto Project:
644
645 DISTRO, DISTRO+snapshot
646
647
648For example, the following toolchain installer is for a 64-bit
649development host system and a i586-tuned target architecture based off
650the SDK for ``core-image-sato``:
651poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
652
653Toolchains are self-contained and by default are installed into
654``/opt/poky``. However, when you run the toolchain installer, you can
655choose an installation directory.
656
657The following command shows how to run the installer given a toolchain
658tarball for a 64-bit x86 development host system and a 32-bit x86 target
659architecture. You must change the permissions on the toolchain installer
660script so that it is executable.
661
662The example assumes the toolchain installer is located in
663``~/Downloads/``.
664
665.. note::
666
667 If you do not have write permissions for the directory into which you
668 are installing the toolchain, the toolchain installer notifies you
669 and exits. Be sure you have write permissions in the directory and
670 run the installer again.
671
672$ ~/Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
673
674For more information on how to install tarballs, see the "`Using a
675Cross-Toolchain
676Tarball <&YOCTO_DOCS_ADT_URL;#using-an-existing-toolchain-tarball>`__"
677and "`Using BitBake and the Build
678Directory <&YOCTO_DOCS_ADT_URL;#using-the-toolchain-from-within-the-build-tree>`__"
679sections in the Yocto Project Application Developer's Guide.
680
681Downloading the Pre-Built Linux Kernel
682--------------------------------------
683
684You can download the pre-built Linux kernel suitable for running in the
685QEMU emulator from ` <&YOCTO_QEMU_DL_URL;>`__. Be sure to use the kernel
686that matches the architecture you want to simulate. Download areas exist
687for the five supported machine architectures: ``qemuarm``, ``qemumips``,
688``qemuppc``, ``qemux86``, and ``qemux86-64``.
689
690Most kernel files have one of the following forms: \*zImage-qemuarch.bin
691vmlinux-qemuarch.bin Where: arch is a string representing the target
692architecture: x86, x86-64, ppc, mips, or arm.
693
694You can learn more about downloading a Yocto Project kernel in the
695"`Yocto Project Kernel <&YOCTO_DOCS_DEV_URL;#local-kernel-files>`__"
696bulleted item in the Yocto Project Development Manual.
697
698Downloading the Filesystem
699--------------------------
700
701You can also download the filesystem image suitable for your target
702architecture from ` <&YOCTO_QEMU_DL_URL;>`__. Again, be sure to use the
703filesystem that matches the architecture you want to simulate.
704
705The filesystem image has two tarball forms: ``ext3`` and ``tar``. You
706must use the ``ext3`` form when booting an image using the QEMU
707emulator. The ``tar`` form can be flattened out in your host development
708system and used for build purposes with the Yocto Project.
709core-image-profile-qemuarch.ext3 core-image-profile-qemuarch.tar.bz2
710Where: profile is the filesystem image's profile: lsb, lsb-dev, lsb-sdk,
711lsb-qt3, minimal, minimal-dev, sato, sato-dev, or sato-sdk. For
712information on these types of image profiles, see the
713"`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
714Project Reference Manual. arch is a string representing the target
715architecture: x86, x86-64, ppc, mips, or arm.
716
717Setting Up the Environment and Starting the QEMU Emulator
718---------------------------------------------------------
719
720Before you start the QEMU emulator, you need to set up the emulation
721environment. The following command form sets up the emulation
722environment. $ source
723YOCTO_ADTPATH_DIR/environment-setup-arch-poky-linux-if Where: arch is a
724string representing the target architecture: i586, x86_64, ppc603e,
725mips, or armv5te. if is a string representing an embedded application
726binary interface. Not all setup scripts include this string.
727
728Finally, this command form invokes the QEMU emulator $ runqemu qemuarch
729kernel-image filesystem-image Where: qemuarch is a string representing
730the target architecture: qemux86, qemux86-64, qemuppc, qemumips, or
731qemuarm. kernel-image is the architecture-specific kernel image.
732filesystem-image is the .ext3 filesystem image.
733
734Continuing with the example, the following two commands setup the
735emulation environment and launch QEMU. This example assumes the root
736filesystem (``.ext3`` file) and the pre-built kernel image file both
737reside in your home directory. The kernel and filesystem are for a
73832-bit target architecture. $ cd $HOME $ source
739YOCTO_ADTPATH_DIR/environment-setup-i586-poky-linux $ runqemu qemux86
740bzImage-qemux86.bin \\ core-image-sato-qemux86.ext3
741
742The environment in which QEMU launches varies depending on the
743filesystem image and on the target architecture. For example, if you
744source the environment for the ARM target architecture and then boot the
745minimal QEMU image, the emulator comes up in a new shell in command-line
746mode. However, if you boot the SDK image, QEMU comes up with a GUI.
747
748.. note::
749
750 Booting the PPC image results in QEMU launching in the same shell in
751 command-line mode.
752
753.. |Using a Pre-Built Image| image:: figures/using-a-pre-built-image.png
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst
new file mode 100644
index 0000000000..eaf19d3f74
--- /dev/null
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst
@@ -0,0 +1,360 @@
1=========================
2Yocto Project Quick Build
3=========================
4
5Welcome!
6========
7
8Welcome! This short document steps you through the process for a typical
9image build using the Yocto Project. The document also introduces how to
10configure a build for specific hardware. You will use Yocto Project to
11build a reference embedded OS called Poky.
12
13.. note::
14
15 - The examples in this paper assume you are using a native Linux
16 system running a recent Ubuntu Linux distribution. If the machine
17 you want to use Yocto Project on to build an image (`build
18 host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__) is not
19 a native Linux system, you can still perform these steps by using
20 CROss PlatformS (CROPS) and setting up a Poky container. See the
21 `Setting Up to Use CROss PlatformS
22 (CROPS) <&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops>`__" section
23 in the Yocto Project Development Tasks Manual for more
24 information.
25
26 - You may use Windows Subsystem For Linux v2 to set up a build host
27 using Windows 10.
28
29 .. note::
30
31 The Yocto Project is not compatible with WSLv1, it is
32 compatible but not officially supported nor validated with
33 WSLv2, if you still decide to use WSL please upgrade to WSLv2.
34
35 See the `Setting Up to Use Windows Subsystem For
36 Linux <&YOCTO_DOCS_DEV_URL;#setting-up-to-use-wsl>`__" section in
37 the Yocto Project Development Tasks Manual for more information.
38
39If you want more conceptual or background information on the Yocto
40Project, see the `Yocto Project Overview and Concepts
41Manual <&YOCTO_DOCS_OM_URL;>`__.
42
43Compatible Linux Distribution
44=============================
45
46Make sure your `build
47host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ meets the
48following requirements:
49
50- 50 Gbytes of free disk space
51
52- Runs a supported Linux distribution (i.e. recent releases of Fedora,
53 openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux
54 distributions that support the Yocto Project, see the "`Supported
55 Linux
56 Distributions <&YOCTO_DOCS_REF_URL;#detailed-supported-distros>`__"
57 section in the Yocto Project Reference Manual. For detailed
58 information on preparing your build host, see the "`Preparing the
59 Build Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__"
60 section in the Yocto Project Development Tasks Manual.
61
62-
63
64 - Git 1.8.3.1 or greater
65
66 - tar 1.28 or greater
67
68 - Python 3.5.0 or greater.
69
70 - gcc 5.0 or greater.
71
72 If your build host does not meet any of these three listed version
73 requirements, you can take steps to prepare the system so that you
74 can still use the Yocto Project. See the "`Required Git, tar, Python
75 and gcc
76 Versions <&YOCTO_DOCS_REF_URL;#required-git-tar-python-and-gcc-versions>`__"
77 section in the Yocto Project Reference Manual for information.
78
79Build Host Packages
80===================
81
82You must install essential host packages on your build host. The
83following command installs the host packages based on an Ubuntu
84distribution:
85
86.. note::
87
88 For host package requirements on all supported Linux distributions,
89 see the "
90 Required Packages for the Build Host
91 " section in the Yocto Project Reference Manual.
92
93$ sudo apt-get install UBUNTU_HOST_PACKAGES_ESSENTIAL
94
95Use Git to Clone Poky
96=====================
97
98Once you complete the setup instructions for your machine, you need to
99get a copy of the Poky repository on your build host. Use the following
100commands to clone the Poky repository. $ git clone
101git://git.yoctoproject.org/poky Cloning into 'poky'... remote: Counting
102objects: 432160, done. remote: Compressing objects: 100%
103(102056/102056), done. remote: Total 432160 (delta 323116), reused
104432037 (delta 323000) Receiving objects: 100% (432160/432160), 153.81
105MiB \| 8.54 MiB/s, done. Resolving deltas: 100% (323116/323116), done.
106Checking connectivity... done. Move to the ``poky`` directory and take a
107look at the tags: $ cd poky $ git fetch --tags $ git tag 1.1_M1.final
1081.1_M1.rc1 1.1_M1.rc2 1.1_M2.final 1.1_M2.rc1 . . . yocto-2.5
109yocto-2.5.1 yocto-2.5.2 yocto-2.6 yocto-2.6.1 yocto-2.6.2 yocto-2.7
110yocto_1.5_M5.rc8 For this example, check out the branch based on the
111DISTRO_REL_TAG release: $ git checkout tags/DISTRO_REL_TAG -b
112my-DISTRO_REL_TAG Switched to a new branch 'my-DISTRO_REL_TAG' The
113previous Git checkout command creates a local branch named
114my-DISTRO_REL_TAG. The files available to you in that branch exactly
115match the repository's files in the "DISTRO_NAME_NO_CAP" development
116branch at the time of the Yocto Project DISTRO_REL_TAG release.
117
118For more options and information about accessing Yocto Project related
119repositories, see the "`Locating Yocto Project Source
120Files <&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files>`__"
121section in the Yocto Project Development Tasks Manual.
122
123Building Your Image
124===================
125
126Use the following steps to build your image. The build process creates
127an entire Linux distribution, including the toolchain, from source.
128
129.. note::
130
131 - If you are working behind a firewall and your build host is not
132 set up for proxies, you could encounter problems with the build
133 process when fetching source code (e.g. fetcher failures or Git
134 failures).
135
136 - If you do not know your proxy settings, consult your local network
137 infrastructure resources and get that information. A good starting
138 point could also be to check your web browser settings. Finally,
139 you can find more information on the "`Working Behind a Network
140 Proxy <https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy>`__"
141 page of the Yocto Project Wiki.
142
1431. *Initialize the Build Environment:* From within the ``poky``
144 directory, run the
145 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ environment
146 setup script to define Yocto Project's build environment on your
147 build host. $ cd ~/poky $ source OE_INIT_FILE You had no
148 conf/local.conf file. This configuration file has therefore been
149 created for you with some default values. You may wish to edit it to,
150 for example, select a different MACHINE (target hardware). See
151 conf/local.conf for more information as common configuration options
152 are commented. You had no conf/bblayers.conf file. This configuration
153 file has therefore been created for you with some default values. To
154 add additional metadata layers into your configuration please add
155 entries to conf/bblayers.conf. The Yocto Project has extensive
156 documentation about OE including a reference manual which can be
157 found at: http://yoctoproject.org/documentation For more information
158 about OpenEmbedded see their website: http://www.openembedded.org/
159 ### Shell environment set up for builds. ### You can now run 'bitbake
160 <target>' Common targets are: core-image-minimal core-image-sato
161 meta-toolchain meta-ide-support You can also run generated qemu
162 images with a command like 'runqemu qemux86-64' Among other things,
163 the script creates the `Build
164 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, which is
165 ``build`` in this case and is located in the `Source
166 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. After the
167 script runs, your current working directory is set to the Build
168 Directory. Later, when the build completes, the Build Directory
169 contains all the files created during the build.
170
1712. *Examine Your Local Configuration File:* When you set up the build
172 environment, a local configuration file named ``local.conf`` becomes
173 available in a ``conf`` subdirectory of the Build Directory. For this
174 example, the defaults are set to build for a ``qemux86`` target,
175 which is suitable for emulation. The package manager used is set to
176 the RPM package manager.
177
178 .. tip::
179
180 You can significantly speed up your build and guard against
181 fetcher failures by using mirrors. To use mirrors, add these lines
182 to your
183 local.conf
184 file in the Build directory:
185 ::
186
187 SSTATE_MIRRORS = "\
188 file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH \n \
189 file://.* http://sstate.yoctoproject.org/YOCTO_DOC_VERSION_MINUS_ONE/PATH;downloadfilename=PATH \n \
190 file://.* http://sstate.yoctoproject.org/YOCTO_DOC_VERSION/PATH;downloadfilename=PATH \n \
191 "
192
193
194 The previous examples showed how to add sstate paths for Yocto
195 Project YOCTO_DOC_VERSION_MINUS_ONE, YOCTO_DOC_VERSION, and a
196 development area. For a complete index of sstate locations, see
197 .
198
1993. *Start the Build:* Continue with the following command to build an OS
200 image for the target, which is ``core-image-sato`` in this example: $
201 bitbake core-image-sato For information on using the ``bitbake``
202 command, see the
203 "`BitBake <&YOCTO_DOCS_OM_URL;#usingpoky-components-bitbake>`__"
204 section in the Yocto Project Overview and Concepts Manual, or see the
205 "`BitBake
206 Command <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-command>`__" section
207 in the BitBake User Manual.
208
2094. *Simulate Your Image Using QEMU:* Once this particular image is
210 built, you can start QEMU, which is a Quick EMUlator that ships with
211 the Yocto Project: $ runqemu qemux86-64 If you want to learn more
212 about running QEMU, see the "`Using the Quick EMUlator
213 (QEMU) <&YOCTO_DOCS_DEV_URL;#dev-manual-qemu>`__" chapter in the
214 Yocto Project Development Tasks Manual.
215
2165. *Exit QEMU:* Exit QEMU by either clicking on the shutdown icon or by
217 typing ``Ctrl-C`` in the QEMU transcript window from which you evoked
218 QEMU.
219
220Customizing Your Build for Specific Hardware
221============================================
222
223So far, all you have done is quickly built an image suitable for
224emulation only. This section shows you how to customize your build for
225specific hardware by adding a hardware layer into the Yocto Project
226development environment.
227
228In general, layers are repositories that contain related sets of
229instructions and configurations that tell the Yocto Project what to do.
230Isolating related metadata into functionally specific layers facilitates
231modular development and makes it easier to reuse the layer metadata.
232
233.. note::
234
235 By convention, layer names start with the string "meta-".
236
237Follow these steps to add a hardware layer:
238
2391. *Find a Layer:* Lots of hardware layers exist. The Yocto Project
240 `Source Repositories <&YOCTO_GIT_URL;>`__ has many hardware layers.
241 This example adds the
242 `meta-altera <https://github.com/kraj/meta-altera>`__ hardware layer.
243
2442. *Clone the Layer* Use Git to make a local copy of the layer on your
245 machine. You can put the copy in the top level of the copy of the
246 Poky repository created earlier: $ cd ~/poky $ git clone
247 https://github.com/kraj/meta-altera.git Cloning into 'meta-altera'...
248 remote: Counting objects: 25170, done. remote: Compressing objects:
249 100% (350/350), done. remote: Total 25170 (delta 645), reused 719
250 (delta 538), pack-reused 24219 Receiving objects: 100% (25170/25170),
251 41.02 MiB \| 1.64 MiB/s, done. Resolving deltas: 100% (13385/13385),
252 done. Checking connectivity... done. The hardware layer now exists
253 with other layers inside the Poky reference repository on your build
254 host as ``meta-altera`` and contains all the metadata needed to
255 support hardware from Altera, which is owned by Intel.
256
2573. *Change the Configuration to Build for a Specific Machine:* The
258 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable in the
259 ``local.conf`` file specifies the machine for the build. For this
260 example, set the ``MACHINE`` variable to "cyclone5". These
261 configurations are used:
262 ` <https://github.com/kraj/meta-altera/blob/master/conf/machine/cyclone5.conf>`__.
263
264 .. note::
265
266 See the "
267 Examine Your Local Configuration File
268 " step earlier for more information on configuring the build.
269
2704. *Add Your Layer to the Layer Configuration File:* Before you can use
271 a layer during a build, you must add it to your ``bblayers.conf``
272 file, which is found in the `Build
273 Directory's <&YOCTO_DOCS_REF_URL;#build-directory>`__ ``conf``
274 directory.
275
276 Use the ``bitbake-layers add-layer`` command to add the layer to the
277 configuration file: $ cd ~/poky/build $ bitbake-layers add-layer
278 ../meta-altera NOTE: Starting bitbake server... Parsing recipes: 100%
279 \|##################################################################\|
280 Time: 0:00:32 Parsing of 918 .bb files complete (0 cached, 918
281 parsed). 1401 targets, 123 skipped, 0 masked, 0 errors. You can find
282 more information on adding layers in the "`Adding a Layer Using the
283 ``bitbake-layers``
284 Script <&YOCTO_DOCS_DEV_URL;#adding-a-layer-using-the-bitbake-layers-script>`__"
285 section.
286
287Completing these steps has added the ``meta-altera`` layer to your Yocto
288Project development environment and configured it to build for the
289"cyclone5" machine.
290
291.. note::
292
293 The previous steps are for demonstration purposes only. If you were
294 to attempt to build an image for the "cyclone5" build, you should
295 read the Altera
296 README
297 .
298
299Creating Your Own General Layer
300===============================
301
302Maybe you have an application or specific set of behaviors you need to
303isolate. You can create your own general layer using the
304``bitbake-layers create-layer`` command. The tool automates layer
305creation by setting up a subdirectory with a ``layer.conf``
306configuration file, a ``recipes-example`` subdirectory that contains an
307``example.bb`` recipe, a licensing file, and a ``README``.
308
309The following commands run the tool to create a layer named
310``meta-mylayer`` in the ``poky`` directory: $ cd ~/poky $ bitbake-layers
311create-layer meta-mylayer NOTE: Starting bitbake server... Add your new
312layer with 'bitbake-layers add-layer meta-mylayer' For more information
313on layers and how to create them, see the "`Creating a General Layer
314Using the ``bitbake-layers``
315Script <&YOCTO_DOCS_DEV_URL;#creating-a-general-layer-using-the-bitbake-layers-script>`__"
316section in the Yocto Project Development Tasks Manual.
317
318Where To Go Next
319================
320
321Now that you have experienced using the Yocto Project, you might be
322asking yourself "What now?" The Yocto Project has many sources of
323information including the website, wiki pages, and user manuals:
324
325- *Website:* The `Yocto Project Website <&YOCTO_HOME_URL;>`__ provides
326 background information, the latest builds, breaking news, full
327 development documentation, and access to a rich Yocto Project
328 Development Community into which you can tap.
329
330- *Developer Screencast:* The `Getting Started with the Yocto Project -
331 New Developer Screencast Tutorial <http://vimeo.com/36450321>`__
332 provides a 30-minute video created for users unfamiliar with the
333 Yocto Project but familiar with Linux build hosts. While this
334 screencast is somewhat dated, the introductory and fundamental
335 concepts are useful for the beginner.
336
337- *Yocto Project Overview and Concepts Manual:* The `Yocto Project
338 Overview and Concepts Manual <&YOCTO_DOCS_OM_URL;>`__ is a great
339 place to start to learn about the Yocto Project. This manual
340 introduces you to the Yocto Project and its development environment.
341 The manual also provides conceptual information for various aspects
342 of the Yocto Project.
343
344- *Yocto Project Wiki:* The `Yocto Project Wiki <&YOCTO_WIKI_URL;>`__
345 provides additional information on where to go next when ramping up
346 with the Yocto Project, release information, project planning, and QA
347 information.
348
349- *Yocto Project Mailing Lists:* Related mailing lists provide a forum
350 for discussion, patch submission and announcements. Several mailing
351 lists exist and are grouped according to areas of concern. See the
352 "`Mailing lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__"
353 section in the Yocto Project Reference Manual for a complete list of
354 Yocto Project mailing lists.
355
356- *Comprehensive List of Links and Other Documentation:* The "`Links
357 and Related
358 Documentation <&YOCTO_DOCS_REF_URL;#resources-links-and-related-documentation>`__"
359 section in the Yocto Project Reference Manual provides a
360 comprehensive list of all related links and other user documentation.
diff --git a/documentation/bsp-guide/bsp-guide.rst b/documentation/bsp-guide/bsp-guide.rst
new file mode 100644
index 0000000000..71ac1d0e97
--- /dev/null
+++ b/documentation/bsp-guide/bsp-guide.rst
@@ -0,0 +1,9 @@
1=====================================================
2Yocto Project Board Support Package Developer's Guide
3=====================================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 bsp
diff --git a/documentation/bsp-guide/bsp.rst b/documentation/bsp-guide/bsp.rst
new file mode 100644
index 0000000000..09e25a061c
--- /dev/null
+++ b/documentation/bsp-guide/bsp.rst
@@ -0,0 +1,1450 @@
1************************************************
2Board Support Packages (BSP) - Developer's Guide
3************************************************
4
5A Board Support Package (BSP) is a collection of information that
6defines how to support a particular hardware device, set of devices, or
7hardware platform. The BSP includes information about the hardware
8features present on the device and kernel configuration information
9along with any additional hardware drivers required. The BSP also lists
10any additional software components required in addition to a generic
11Linux software stack for both essential and optional platform features.
12
13This guide presents information about BSP layers, defines a structure
14for components so that BSPs follow a commonly understood layout,
15discusses how to customize a recipe for a BSP, addresses BSP licensing,
16and provides information that shows you how to create a `BSP
17Layer <#bsp-layers>`__ using the
18```bitbake-layers`` <#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__
19tool.
20
21BSP Layers
22==========
23
24A BSP consists of a file structure inside a base directory.
25Collectively, you can think of the base directory, its file structure,
26and the contents as a BSP layer. Although not a strict requirement, BSP
27layers in the Yocto Project use the following well-established naming
28convention: meta-bsp_root_name The string "meta-" is prepended to the
29machine or platform name, which is bsp_root_name in the above form.
30
31.. note::
32
33 Because the BSP layer naming convention is well-established, it is
34 advisable to follow it when creating layers. Technically speaking, a
35 BSP layer name does not need to start with
36 meta-
37 . However, various scripts and tools in the Yocto Project development
38 environment assume this convention.
39
40To help understand the BSP layer concept, consider the BSPs that the
41Yocto Project supports and provides with each release. You can see the
42layers in the `Yocto Project Source
43Repositories <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__ through
44a web interface at ` <&YOCTO_GIT_URL;>`__. If you go to that interface,
45you will find a list of repositories under "Yocto Metadata Layers".
46
47.. note::
48
49 Layers that are no longer actively supported as part of the Yocto
50 Project appear under the heading "Yocto Metadata Layer Archive."
51
52Each repository is a BSP layer supported by the Yocto Project (e.g.
53``meta-raspberrypi`` and ``meta-intel``). Each of these layers is a
54repository unto itself and clicking on the layer name displays two URLs
55from which you can clone the layer's repository to your local system.
56Here is an example that clones the Raspberry Pi BSP layer: $ git clone
57git://git.yoctoproject.org/meta-raspberrypi
58
59In addition to BSP layers, the ``meta-yocto-bsp`` layer is part of the
60shipped ``poky`` repository. The ``meta-yocto-bsp`` layer maintains
61several "reference" BSPs including the ARM-based Beaglebone, MIPS-based
62EdgeRouter, and generic versions of both 32-bit and 64-bit IA machines.
63
64For information on typical BSP development workflow, see the
65"`Developing a Board Support Package
66(BSP) <#developing-a-board-support-package-bsp>`__" section. For more
67information on how to set up a local copy of source files from a Git
68repository, see the "`Locating Yocto Project Source
69Files <&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files>`__"
70section in the Yocto Project Development Tasks Manual.
71
72The BSP layer's base directory (``meta-bsp_root_name``) is the root
73directory of that Layer. This directory is what you add to the
74```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable in the
75``conf/bblayers.conf`` file found in your `Build
76Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, which is
77established after you run the OpenEmbedded build environment setup
78script (i.e. ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__).
79Adding the root directory allows the `OpenEmbedded build
80system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ to recognize the BSP
81layer and from it build an image. Here is an example: BBLAYERS ?= " \\
82/usr/local/src/yocto/meta \\ /usr/local/src/yocto/meta-poky \\
83/usr/local/src/yocto/meta-yocto-bsp \\ /usr/local/src/yocto/meta-mylayer
84\\ "
85
86.. note::
87
88 Ordering and
89 BBFILE_PRIORITY
90 for the layers listed in
91 BBLAYERS
92 matter. For example, if multiple layers define a machine
93 configuration, the OpenEmbedded build system uses the last layer
94 searched given similar layer priorities. The build system works from
95 the top-down through the layers listed in
96 BBLAYERS
97 .
98
99Some BSPs require or depend on additional layers beyond the BSP's root
100layer in order to be functional. In this case, you need to specify these
101layers in the ``README`` "Dependencies" section of the BSP's root layer.
102Additionally, if any build instructions exist for the BSP, you must add
103them to the "Dependencies" section.
104
105Some layers function as a layer to hold other BSP layers. These layers
106are knows as "`container
107layers <&YOCTO_DOCS_REF_URL;#term-container-layer>`__". An example of
108this type of layer is OpenEmbedded's
109```meta-openembedded`` <https://github.com/openembedded/meta-openembedded>`__
110layer. The ``meta-openembedded`` layer contains many ``meta-*`` layers.
111In cases like this, you need to include the names of the actual layers
112you want to work with, such as: BBLAYERS ?= " \\
113/usr/local/src/yocto/meta \\ /usr/local/src/yocto/meta-poky \\
114/usr/local/src/yocto/meta-yocto-bsp \\ /usr/local/src/yocto/meta-mylayer
115\\ .../meta-openembedded/meta-oe \\ .../meta-openembedded/meta-perl \\
116.../meta-openembedded/meta-networking \\ " and so on.
117
118For more information on layers, see the "`Understanding and Creating
119Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
120section of the Yocto Project Development Tasks Manual.
121
122Preparing Your Build Host to Work With BSP Layers
123=================================================
124
125This section describes how to get your build host ready to work with BSP
126layers. Once you have the host set up, you can create the layer as
127described in the "`Creating a new BSP Layer Using the ``bitbake-layers``
128Script <#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
129section.
130
131.. note::
132
133 For structural information on BSPs, see the
134 Example Filesystem Layout
135 section.
136
1371. *Set Up the Build Environment:* Be sure you are set up to use BitBake
138 in a shell. See the "`Preparing the Build
139 Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section
140 in the Yocto Project Development Tasks Manual for information on how
141 to get a build host ready that is either a native Linux machine or a
142 machine that uses CROPS.
143
1442. *Clone the ``poky`` Repository:* You need to have a local copy of the
145 Yocto Project `Source
146 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (i.e. a local
147 ``poky`` repository). See the "`Cloning the ``poky``
148 Repository <&YOCTO_DOCS_DEV_URL;#cloning-the-poky-repository>`__" and
149 possibly the "`Checking Out by Branch in
150 Poky <&YOCTO_DOCS_DEV_URL;#checking-out-by-branch-in-poky>`__" or
151 "`Checking Out by Tag in
152 Poky <&YOCTO_DOCS_DEV_URL;#checkout-out-by-tag-in-poky>`__" sections
153 all in the Yocto Project Development Tasks Manual for information on
154 how to clone the ``poky`` repository and check out the appropriate
155 branch for your work.
156
1573. *Determine the BSP Layer You Want:* The Yocto Project supports many
158 BSPs, which are maintained in their own layers or in layers designed
159 to contain several BSPs. To get an idea of machine support through
160 BSP layers, you can look at the `index of
161 machines <&YOCTO_RELEASE_DL_URL;/machines>`__ for the release.
162
1634. *Optionally Clone the ``meta-intel`` BSP Layer:* If your hardware is
164 based on current Intel CPUs and devices, you can leverage this BSP
165 layer. For details on the ``meta-intel`` BSP layer, see the layer's
166 ```README`` <http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/README>`__
167 file.
168
169 1. *Navigate to Your Source Directory:* Typically, you set up the
170 ``meta-intel`` Git repository inside the `Source
171 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
172 ``poky``). $ cd /home/you/poky
173
174 2. *Clone the Layer:* $ git clone
175 git://git.yoctoproject.org/meta-intel.git Cloning into
176 'meta-intel'... remote: Counting objects: 15585, done. remote:
177 Compressing objects: 100% (5056/5056), done. remote: Total 15585
178 (delta 9123), reused 15329 (delta 8867) Receiving objects: 100%
179 (15585/15585), 4.51 MiB \| 3.19 MiB/s, done. Resolving deltas:
180 100% (9123/9123), done. Checking connectivity... done.
181
182 3. *Check Out the Proper Branch:* The branch you check out for
183 ``meta-intel`` must match the same branch you are using for the
184 Yocto Project release (e.g. DISTRO_NAME_NO_CAP): $ cd meta-intel $
185 git checkout -b DISTRO_NAME_NO_CAP
186 remotes/origin/DISTRO_NAME_NO_CAP Branch DISTRO_NAME_NO_CAP set up
187 to track remote branch DISTRO_NAME_NO_CAP from origin. Switched to
188 a new branch 'DISTRO_NAME_NO_CAP'
189
190 .. note::
191
192 To see the available branch names in a cloned repository, use
193 the
194 git branch -al
195 command. See the "
196 Checking Out By Branch in Poky
197 " section in the Yocto Project Development Tasks Manual for
198 more information.
199
2005. *Optionally Set Up an Alternative BSP Layer:* If your hardware can be
201 more closely leveraged to an existing BSP not within the
202 ``meta-intel`` BSP layer, you can clone that BSP layer.
203
204 The process is identical to the process used for the ``meta-intel``
205 layer except for the layer's name. For example, if you determine that
206 your hardware most closely matches the ``meta-raspberrypi``, clone
207 that layer: $ git clone git://git.yoctoproject.org/meta-raspberrypi
208 Cloning into 'meta-raspberrypi'... remote: Counting objects: 4743,
209 done. remote: Compressing objects: 100% (2185/2185), done. remote:
210 Total 4743 (delta 2447), reused 4496 (delta 2258) Receiving objects:
211 100% (4743/4743), 1.18 MiB \| 0 bytes/s, done. Resolving deltas: 100%
212 (2447/2447), done. Checking connectivity... done.
213
2146. *Initialize the Build Environment:* While in the root directory of
215 the Source Directory (i.e. ``poky``), run the
216 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ environment
217 setup script to define the OpenEmbedded build environment on your
218 build host. $ source OE_INIT_FILE Among other things, the script
219 creates the `Build
220 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, which is
221 ``build`` in this case and is located in the `Source
222 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. After the
223 script runs, your current working directory is set to the ``build``
224 directory.
225
226.. _bsp-filelayout:
227
228Example Filesystem Layout
229=========================
230
231Defining a common BSP directory structure allows end-users to understand
232and become familiar with that standard. A common format also encourages
233standardization of software support for hardware.
234
235The proposed form described in this section does have elements that are
236specific to the OpenEmbedded build system. It is intended that
237developers can use this structure with other build systems besides the
238OpenEmbedded build system. It is also intended that it will be be simple
239to extract information and convert it to other formats if required. The
240OpenEmbedded build system, through its standard `layers
241mechanism <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__, can
242directly accept the format described as a layer. The BSP layer captures
243all the hardware-specific details in one place using a standard format,
244which is useful for any person wishing to use the hardware platform
245regardless of the build system they are using.
246
247The BSP specification does not include a build system or other tools -
248the specification is concerned with the hardware-specific components
249only. At the end-distribution point, you can ship the BSP layer combined
250with a build system and other tools. Realize that it is important to
251maintain the distinction that the BSP layer, a build system, and tools
252are separate components that could be combined in certain end products.
253
254Before looking at the recommended form for the directory structure
255inside a BSP layer, you should be aware that some requirements do exist
256in order for a BSP layer to be considered compliant with the Yocto
257Project. For that list of requirements, see the "`Released BSP
258Requirements <#released-bsp-requirements>`__" section.
259
260Below is the typical directory structure for a BSP layer. While this
261basic form represents the standard, realize that the actual layout for
262individual BSPs could differ. meta-bsp_root_name/
263meta-bsp_root_name/bsp_license_file meta-bsp_root_name/README
264meta-bsp_root_name/README.sources
265meta-bsp_root_name/binary/bootable_images
266meta-bsp_root_name/conf/layer.conf
267meta-bsp_root_name/conf/machine/*.conf meta-bsp_root_name/recipes-bsp/\*
268meta-bsp_root_name/recipes-core/\*
269meta-bsp_root_name/recipes-graphics/\*
270meta-bsp_root_name/recipes-kernel/linux/linux-yocto_kernel_rev.bbappend
271
272Below is an example of the Raspberry Pi BSP layer that is available from
273the `Source Respositories <&YOCTO_GIT_URL;>`__:
274meta-raspberrypi/COPYING.MIT meta-raspberrypi/README.md
275meta-raspberrypi/classes
276meta-raspberrypi/classes/sdcard_image-rpi.bbclass meta-raspberrypi/conf/
277meta-raspberrypi/conf/layer.conf meta-raspberrypi/conf/machine/
278meta-raspberrypi/conf/machine/raspberrypi-cm.conf
279meta-raspberrypi/conf/machine/raspberrypi-cm3.conf
280meta-raspberrypi/conf/machine/raspberrypi.conf
281meta-raspberrypi/conf/machine/raspberrypi0-wifi.conf
282meta-raspberrypi/conf/machine/raspberrypi0.conf
283meta-raspberrypi/conf/machine/raspberrypi2.conf
284meta-raspberrypi/conf/machine/raspberrypi3-64.conf
285meta-raspberrypi/conf/machine/raspberrypi3.conf
286meta-raspberrypi/conf/machine/include
287meta-raspberrypi/conf/machine/include/rpi-base.inc
288meta-raspberrypi/conf/machine/include/rpi-default-providers.inc
289meta-raspberrypi/conf/machine/include/rpi-default-settings.inc
290meta-raspberrypi/conf/machine/include/rpi-default-versions.inc
291meta-raspberrypi/conf/machine/include/tune-arm1176jzf-s.inc
292meta-raspberrypi/docs meta-raspberrypi/docs/Makefile
293meta-raspberrypi/docs/conf.py meta-raspberrypi/docs/contributing.md
294meta-raspberrypi/docs/extra-apps.md
295meta-raspberrypi/docs/extra-build-config.md
296meta-raspberrypi/docs/index.rst meta-raspberrypi/docs/layer-contents.md
297meta-raspberrypi/docs/readme.md meta-raspberrypi/files
298meta-raspberrypi/files/custom-licenses
299meta-raspberrypi/files/custom-licenses/Broadcom
300meta-raspberrypi/recipes-bsp meta-raspberrypi/recipes-bsp/bootfiles
301meta-raspberrypi/recipes-bsp/bootfiles/bcm2835-bootfiles.bb
302meta-raspberrypi/recipes-bsp/bootfiles/rpi-config_git.bb
303meta-raspberrypi/recipes-bsp/common
304meta-raspberrypi/recipes-bsp/common/firmware.inc
305meta-raspberrypi/recipes-bsp/formfactor
306meta-raspberrypi/recipes-bsp/formfactor/formfactor
307meta-raspberrypi/recipes-bsp/formfactor/formfactor/raspberrypi
308meta-raspberrypi/recipes-bsp/formfactor/formfactor/raspberrypi/machconfig
309meta-raspberrypi/recipes-bsp/formfactor/formfactor_0.0.bbappend
310meta-raspberrypi/recipes-bsp/rpi-u-boot-src
311meta-raspberrypi/recipes-bsp/rpi-u-boot-src/files
312meta-raspberrypi/recipes-bsp/rpi-u-boot-src/files/boot.cmd.in
313meta-raspberrypi/recipes-bsp/rpi-u-boot-src/rpi-u-boot-scr.bb
314meta-raspberrypi/recipes-bsp/u-boot
315meta-raspberrypi/recipes-bsp/u-boot/u-boot
316meta-raspberrypi/recipes-bsp/u-boot/u-boot/*.patch
317meta-raspberrypi/recipes-bsp/u-boot/u-boot_%.bbappend
318meta-raspberrypi/recipes-connectivity
319meta-raspberrypi/recipes-connectivity/bluez5
320meta-raspberrypi/recipes-connectivity/bluez5/bluez5
321meta-raspberrypi/recipes-connectivity/bluez5/bluez5/*.patch
322meta-raspberrypi/recipes-connectivity/bluez5/bluez5/BCM43430A1.hcd
323meta-raspberrypi/recipes-connectivity/bluez5/bluez5brcm43438.service
324meta-raspberrypi/recipes-connectivity/bluez5/bluez5_%.bbappend
325meta-raspberrypi/recipes-core meta-raspberrypi/recipes-core/images
326meta-raspberrypi/recipes-core/images/rpi-basic-image.bb
327meta-raspberrypi/recipes-core/images/rpi-hwup-image.bb
328meta-raspberrypi/recipes-core/images/rpi-test-image.bb
329meta-raspberrypi/recipes-core/packagegroups
330meta-raspberrypi/recipes-core/packagegroups/packagegroup-rpi-test.bb
331meta-raspberrypi/recipes-core/psplash
332meta-raspberrypi/recipes-core/psplash/files
333meta-raspberrypi/recipes-core/psplash/files/psplash-raspberrypi-img.h
334meta-raspberrypi/recipes-core/psplash/psplash_git.bbappend
335meta-raspberrypi/recipes-core/udev
336meta-raspberrypi/recipes-core/udev/udev-rules-rpi
337meta-raspberrypi/recipes-core/udev/udev-rules-rpi/99-com.rules
338meta-raspberrypi/recipes-core/udev/udev-rules-rpi.bb
339meta-raspberrypi/recipes-devtools
340meta-raspberrypi/recipes-devtools/bcm2835
341meta-raspberrypi/recipes-devtools/bcm2835/bcm2835_1.52.bb
342meta-raspberrypi/recipes-devtools/pi-blaster
343meta-raspberrypi/recipes-devtools/pi-blaster/files
344meta-raspberrypi/recipes-devtools/pi-blaster/files/*.patch
345meta-raspberrypi/recipes-devtools/pi-blaster/pi-blaster_git.bb
346meta-raspberrypi/recipes-devtools/python
347meta-raspberrypi/recipes-devtools/python/python-rtimu
348meta-raspberrypi/recipes-devtools/python/python-rtimu/*.patch
349meta-raspberrypi/recipes-devtools/python/python-rtimu_git.bb
350meta-raspberrypi/recipes-devtools/python/python-sense-hat_2.2.0.bb
351meta-raspberrypi/recipes-devtools/python/rpi-gpio
352meta-raspberrypi/recipes-devtools/python/rpi-gpio/*.patch
353meta-raspberrypi/recipes-devtools/python/rpi-gpio_0.6.3.bb
354meta-raspberrypi/recipes-devtools/python/rpio
355meta-raspberrypi/recipes-devtools/python/rpio/*.patch
356meta-raspberrypi/recipes-devtools/python/rpio_0.10.0.bb
357meta-raspberrypi/recipes-devtools/wiringPi
358meta-raspberrypi/recipes-devtools/wiringPi/files
359meta-raspberrypi/recipes-devtools/wiringPi/files/*.patch
360meta-raspberrypi/recipes-devtools/wiringPi/wiringpi_git.bb
361meta-raspberrypi/recipes-graphics
362meta-raspberrypi/recipes-graphics/eglinfo
363meta-raspberrypi/recipes-graphics/eglinfo/eglinfo-fb_%.bbappend
364meta-raspberrypi/recipes-graphics/eglinfo/eglinfo-x11_%.bbappend
365meta-raspberrypi/recipes-graphics/mesa
366meta-raspberrypi/recipes-graphics/mesa/mesa-gl_%.bbappend
367meta-raspberrypi/recipes-graphics/mesa/mesa_%.bbappend
368meta-raspberrypi/recipes-graphics/userland
369meta-raspberrypi/recipes-graphics/userland/userland
370meta-raspberrypi/recipes-graphics/userland/userland/*.patch
371meta-raspberrypi/recipes-graphics/userland/userland_git.bb
372meta-raspberrypi/recipes-graphics/vc-graphics
373meta-raspberrypi/recipes-graphics/vc-graphics/files
374meta-raspberrypi/recipes-graphics/vc-graphics/files/egl.pc
375meta-raspberrypi/recipes-graphics/vc-graphics/files/vchiq.sh
376meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics-hardfp.bb
377meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics.bb
378meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics.inc
379meta-raspberrypi/recipes-graphics/wayland
380meta-raspberrypi/recipes-graphics/wayland/weston_%.bbappend
381meta-raspberrypi/recipes-graphics/xorg-xserver
382meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config
383meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi
384meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf
385meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d
386meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/10-evdev.conf
387meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/98-pitft.conf
388meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/99-calibration.conf
389meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
390meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xorg_%.bbappend
391meta-raspberrypi/recipes-kernel
392meta-raspberrypi/recipes-kernel/linux-firmware
393meta-raspberrypi/recipes-kernel/linux-firmware/files
394meta-raspberrypi/recipes-kernel/linux-firmware/files/brcmfmac43430-sdio.bin
395meta-raspberrypi/recipes-kernel/linux-firmware/files/brcfmac43430-sdio.txt
396meta-raspberrypi/recipes-kernel/linux-firmware/linux-firmware_%.bbappend
397meta-raspberrypi/recipes-kernel/linux
398meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi-dev.bb
399meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi.inc
400meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi_4.14.bb
401meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi_4.9.bb
402meta-raspberrypi/recipes-multimedia
403meta-raspberrypi/recipes-multimedia/gstreamer
404meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx
405meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx/*.patch
406meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx_%.bbappend
407meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
408meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx-1.12
409meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx-1.12/*.patch
410meta-raspberrypi/recipes-multimedia/omxplayer
411meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer
412meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer/*.patch
413meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer_git.bb
414meta-raspberrypi/recipes-multimedia/x264
415meta-raspberrypi/recipes-multimedia/x264/x264_git.bbappend
416meta-raspberrypi/wic meta-raspberrypi/wic/sdimage-raspberrypi.wks
417
418The following sections describe each part of the proposed BSP format.
419
420.. _bsp-filelayout-license:
421
422License Files
423-------------
424
425You can find these files in the BSP Layer at:
426meta-bsp_root_name/bsp_license_file
427
428These optional files satisfy licensing requirements for the BSP. The
429type or types of files here can vary depending on the licensing
430requirements. For example, in the Raspberry Pi BSP, all licensing
431requirements are handled with the ``COPYING.MIT`` file.
432
433Licensing files can be MIT, BSD, GPLv*, and so forth. These files are
434recommended for the BSP but are optional and totally up to the BSP
435developer. For information on how to maintain license compliance, see
436the "`Maintaining Open Source License Compliance During Your Product's
437Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
438section in the Yocto Project Development Tasks Manual.
439
440.. _bsp-filelayout-readme:
441
442README File
443-----------
444
445You can find this file in the BSP Layer at: meta-bsp_root_name/README
446
447This file provides information on how to boot the live images that are
448optionally included in the ``binary/`` directory. The ``README`` file
449also provides information needed for building the image.
450
451At a minimum, the ``README`` file must contain a list of dependencies,
452such as the names of any other layers on which the BSP depends and the
453name of the BSP maintainer with his or her contact information.
454
455.. _bsp-filelayout-readme-sources:
456
457README.sources File
458-------------------
459
460You can find this file in the BSP Layer at:
461meta-bsp_root_name/README.sources
462
463This file provides information on where to locate the BSP source files
464used to build the images (if any) that reside in
465``meta-bsp_root_name/binary``. Images in the ``binary`` would be images
466released with the BSP. The information in the ``README.sources`` file
467also helps you find the `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__
468used to generate the images that ship with the BSP.
469
470.. note::
471
472 If the BSP's
473 binary
474 directory is missing or the directory has no images, an existing
475 README.sources
476 file is meaningless and usually does not exist.
477
478.. _bsp-filelayout-binary:
479
480Pre-built User Binaries
481-----------------------
482
483You can find these files in the BSP Layer at:
484meta-bsp_root_name/binary/bootable_images
485
486This optional area contains useful pre-built kernels and user-space
487filesystem images released with the BSP that are appropriate to the
488target system. This directory typically contains graphical (e.g. Sato)
489and minimal live images when the BSP tarball has been created and made
490available in the `Yocto Project <&YOCTO_HOME_URL;>`__ website. You can
491use these kernels and images to get a system running and quickly get
492started on development tasks.
493
494The exact types of binaries present are highly hardware-dependent. The
495```README`` <#bsp-filelayout-readme>`__ file should be present in the
496BSP Layer and it explains how to use the images with the target
497hardware. Additionally, the
498```README.sources`` <#bsp-filelayout-readme-sources>`__ file should be
499present to locate the sources used to build the images and provide
500information on the Metadata.
501
502.. _bsp-filelayout-layer:
503
504Layer Configuration File
505------------------------
506
507You can find this file in the BSP Layer at:
508meta-bsp_root_name/conf/layer.conf
509
510The ``conf/layer.conf`` file identifies the file structure as a layer,
511identifies the contents of the layer, and contains information about how
512the build system should use it. Generally, a standard boilerplate file
513such as the following works. In the following example, you would replace
514bsp with the actual name of the BSP (i.e. bsp_root_name from the example
515template).
516
517# We have a conf and classes directory, add to BBPATH BBPATH .=
518":${LAYERDIR}" # We have a recipes directory, add to BBFILES BBFILES +=
519"${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
520BBFILE_COLLECTIONS += "bsp" BBFILE_PATTERN_bsp = "^${LAYERDIR}/"
521BBFILE_PRIORITY_bsp = "6" LAYERDEPENDS_bsp = "intel"
522
523To illustrate the string substitutions, here are the corresponding
524statements from the Raspberry Pi ``conf/layer.conf`` file: # We have a
525conf and classes directory, append to BBPATH BBPATH .= ":${LAYERDIR}" #
526We have a recipes directory containing .bb and .bbappend files, add to
527BBFILES BBFILES += "${LAYERDIR}/recipes*/*/*.bb \\
528${LAYERDIR}/recipes*/*/*.bbappend" BBFILE_COLLECTIONS += "raspberrypi"
529BBFILE_PATTERN_raspberrypi := "^${LAYERDIR}/"
530BBFILE_PRIORITY_raspberrypi = "9" # Additional license directories.
531LICENSE_PATH += "${LAYERDIR}/files/custom-licenses" . . .
532
533This file simply makes `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__
534aware of the recipes and configuration directories. The file must exist
535so that the OpenEmbedded build system can recognize the BSP.
536
537.. _bsp-filelayout-machine:
538
539Hardware Configuration Options
540------------------------------
541
542You can find these files in the BSP Layer at:
543meta-bsp_root_name/conf/machine/*.conf
544
545The machine files bind together all the information contained elsewhere
546in the BSP into a format that the build system can understand. Each BSP
547Layer requires at least one machine file. If the BSP supports multiple
548machines, multiple machine configuration files can exist. These
549filenames correspond to the values to which users have set the
550```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable.
551
552These files define things such as the kernel package to use
553(```PREFERRED_PROVIDER`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__
554of
555`virtual/kernel <&YOCTO_DOCS_DEV_URL;#metadata-virtual-providers>`__),
556the hardware drivers to include in different types of images, any
557special software components that are needed, any bootloader information,
558and also any special image format requirements.
559
560This configuration file could also include a hardware "tuning" file that
561is commonly used to define the package architecture and specify
562optimization flags, which are carefully chosen to give best performance
563on a given processor.
564
565Tuning files are found in the ``meta/conf/machine/include`` directory
566within the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
567For example, many ``tune-*`` files (e.g. ``tune-arm1136jf-s.inc``,
568``tune-1586-nlp.inc``, and so forth) reside in the
569``poky/meta/conf/machine/include`` directory.
570
571To use an include file, you simply include them in the machine
572configuration file. For example, the Raspberry Pi BSP
573``raspberrypi3.conf`` contains the following statement: include
574conf/machine/include/rpi-base.inc
575
576.. _bsp-filelayout-misc-recipes:
577
578Miscellaneous BSP-Specific Recipe Files
579---------------------------------------
580
581You can find these files in the BSP Layer at:
582meta-bsp_root_name/recipes-bsp/\*
583
584This optional directory contains miscellaneous recipe files for the BSP.
585Most notably would be the formfactor files. For example, in the
586Raspberry Pi BSP, there is the ``formfactor_0.0.bbappend`` file, which
587is an append file used to augment the recipe that starts the build.
588Furthermore, there are machine-specific settings used during the build
589that are defined by the ``machconfig`` file further down in the
590directory. Here is the ``machconfig`` file for the Raspberry Pi BSP:
591HAVE_TOUCHSCREEN=0 HAVE_KEYBOARD=1 DISPLAY_CAN_ROTATE=0
592DISPLAY_ORIENTATION=0 DISPLAY_DPI=133
593
594.. note::
595
596 If a BSP does not have a formfactor entry, defaults are established
597 according to the formfactor configuration file that is installed by
598 the main formfactor recipe
599 ``meta/recipes-bsp/formfactor/formfactor_0.0.bb``, which is found in
600 the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
601
602.. _bsp-filelayout-recipes-graphics:
603
604Display Support Files
605---------------------
606
607You can find these files in the BSP Layer at:
608meta-bsp_root_name/recipes-graphics/\*
609
610This optional directory contains recipes for the BSP if it has special
611requirements for graphics support. All files that are needed for the BSP
612to support a display are kept here.
613
614.. _bsp-filelayout-kernel:
615
616Linux Kernel Configuration
617--------------------------
618
619You can find these files in the BSP Layer at:
620meta-bsp_root_name/recipes-kernel/linux/linux*.bbappend
621meta-bsp_root_name/recipes-kernel/linux/*.bb
622
623Append files (``*.bbappend``) modify the main kernel recipe being used
624to build the image. The ``*.bb`` files would be a developer-supplied
625kernel recipe. This area of the BSP hierarchy can contain both these
626types of files although, in practice, it is likely that you would have
627one or the other.
628
629For your BSP, you typically want to use an existing Yocto Project kernel
630recipe found in the `Source
631Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ at
632``meta/recipes-kernel/linux``. You can append machine-specific changes
633to the kernel recipe by using a similarly named append file, which is
634located in the BSP Layer for your target device (e.g. the
635``meta-bsp_root_name/recipes-kernel/linux`` directory).
636
637Suppose you are using the ``linux-yocto_4.4.bb`` recipe to build the
638kernel. In other words, you have selected the kernel in your
639bsp_root_name\ ``.conf`` file by adding
640```PREFERRED_PROVIDER`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__
641and
642```PREFERRED_VERSION`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_VERSION>`__
643statements as follows: PREFERRED_PROVIDER_virtual/kernel ?=
644"linux-yocto" PREFERRED_VERSION_linux-yocto ?= "4.4%"
645
646.. note::
647
648 When the preferred provider is assumed by default, the
649 PREFERRED_PROVIDER
650 statement does not appear in the
651 bsp_root_name
652 .conf
653 file.
654
655You would use the ``linux-yocto_4.4.bbappend`` file to append specific
656BSP settings to the kernel, thus configuring the kernel for your
657particular BSP.
658
659You can find more information on what your append file should contain in
660the "`Creating the Append
661File <&YOCTO_DOCS_KERNEL_DEV_URL;#creating-the-append-file>`__" section
662in the Yocto Project Linux Kernel Development Manual.
663
664An alternate scenario is when you create your own kernel recipe for the
665BSP. A good example of this is the Raspberry Pi BSP. If you examine the
666``recipes-kernel/linux`` directory you see the following:
667linux-raspberrypi-dev.bb linux-raspberrypi.inc linux-raspberrypi_4.14.bb
668linux-raspberrypi_4.9.bb The directory contains three kernel recipes and
669a common include file.
670
671Developing a Board Support Package (BSP)
672========================================
673
674This section describes the high-level procedure you can follow to create
675a BSP. Although not required for BSP creation, the ``meta-intel``
676repository, which contains many BSPs supported by the Yocto Project, is
677part of the example.
678
679For an example that shows how to create a new layer using the tools, see
680the "`Creating a New BSP Layer Using the ``bitbake-layers``
681Script <#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
682section.
683
684The following illustration and list summarize the BSP creation general
685workflow.
686
6871. *Set up Your Host Development System to Support Development Using the
688 Yocto Project*: See the "`Preparing the Build
689 Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section
690 in the Yocto Project Development Tasks Manual for options on how to
691 get a system ready to use the Yocto Project.
692
6932. *Establish the ``meta-intel`` Repository on Your System:* Having
694 local copies of these supported BSP layers on your system gives you
695 access to layers you might be able to leverage when creating your
696 BSP. For information on how to get these files, see the "`Preparing
697 Your Build Host to Work with BSP
698 Layers <#preparing-your-build-host-to-work-with-bsp-layers>`__"
699 section.
700
7013. *Create Your Own BSP Layer Using the ``bitbake-layers`` Script:*
702 Layers are ideal for isolating and storing work for a given piece of
703 hardware. A layer is really just a location or area in which you
704 place the recipes and configurations for your BSP. In fact, a BSP is,
705 in itself, a special type of layer. The simplest way to create a new
706 BSP layer that is compliant with the Yocto Project is to use the
707 ``bitbake-layers`` script. For information about that script, see the
708 "`Creating a New BSP Layer Using the ``bitbake-layers``
709 Script <#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
710 section.
711
712 Another example that illustrates a layer is an application. Suppose
713 you are creating an application that has library or other
714 dependencies in order for it to compile and run. The layer, in this
715 case, would be where all the recipes that define those dependencies
716 are kept. The key point for a layer is that it is an isolated area
717 that contains all the relevant information for the project that the
718 OpenEmbedded build system knows about. For more information on
719 layers, see the "`The Yocto Project Layer
720 Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__" section
721 in the Yocto Project Overview and Concepts Manual. You can also
722 reference the "`Understanding and Creating
723 Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
724 section in the Yocto Project Development Tasks Manual. For more
725 information on BSP layers, see the "`BSP Layers <#bsp-layers>`__"
726 section.
727
728 .. note::
729
730 - Five hardware reference BSPs exist that are part of the Yocto
731 Project release and are located in the ``poky/meta-yocto-bsp``
732 BSP layer:
733
734 - Texas Instruments Beaglebone (``beaglebone-yocto``)
735
736 - Ubiquiti Networks EdgeRouter Lite (``edgerouter``)
737
738 - Two general IA platforms (``genericx86`` and
739 ``genericx86-64``)
740
741 - Three core Intel BSPs exist as part of the Yocto Project
742 release in the ``meta-intel`` layer:
743
744 - ``intel-core2-32``, which is a BSP optimized for the Core2
745 family of CPUs as well as all CPUs prior to the Silvermont
746 core.
747
748 - ``intel-corei7-64``, which is a BSP optimized for Nehalem
749 and later Core and Xeon CPUs as well as Silvermont and later
750 Atom CPUs, such as the Baytrail SoCs.
751
752 - ``intel-quark``, which is a BSP optimized for the Intel
753 Galileo gen1 & gen2 development boards.
754
755 When you set up a layer for a new BSP, you should follow a standard
756 layout. This layout is described in the "`Example Filesystem
757 Layout <#bsp-filelayout>`__" section. In the standard layout, notice
758 the suggested structure for recipes and configuration information.
759 You can see the standard layout for a BSP by examining any supported
760 BSP found in the ``meta-intel`` layer inside the Source Directory.
761
7624. *Make Configuration Changes to Your New BSP Layer:* The standard BSP
763 layer structure organizes the files you need to edit in ``conf`` and
764 several ``recipes-*`` directories within the BSP layer. Configuration
765 changes identify where your new layer is on the local system and
766 identifies the kernel you are going to use. When you run the
767 ``bitbake-layers`` script, you are able to interactively configure
768 many things for the BSP (e.g. keyboard, touchscreen, and so forth).
769
7705. *Make Recipe Changes to Your New BSP Layer:* Recipe changes include
771 altering recipes (``*.bb`` files), removing recipes you do not use,
772 and adding new recipes or append files (``.bbappend``) that support
773 your hardware.
774
7756. *Prepare for the Build:* Once you have made all the changes to your
776 BSP layer, there remains a few things you need to do for the
777 OpenEmbedded build system in order for it to create your image. You
778 need to get the build environment ready by sourcing an environment
779 setup script (i.e. ``oe-init-build-env``) and you need to be sure two
780 key configuration files are configured appropriately: the
781 ``conf/local.conf`` and the ``conf/bblayers.conf`` file. You must
782 make the OpenEmbedded build system aware of your new layer. See the
783 "`Enabling Your Layer <&YOCTO_DOCS_DEV_URL;#enabling-your-layer>`__"
784 section in the Yocto Project Development Tasks Manual for information
785 on how to let the build system know about your new layer.
786
7877. *Build the Image:* The OpenEmbedded build system uses the BitBake
788 tool to build images based on the type of image you want to create.
789 You can find more information about BitBake in the `BitBake User
790 Manual <&YOCTO_DOCS_BB_URL;>`__.
791
792 The build process supports several types of images to satisfy
793 different needs. See the
794 "`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
795 Project Reference Manual for information on supported images.
796
797Requirements and Recommendations for Released BSPs
798==================================================
799
800Certain requirements exist for a released BSP to be considered compliant
801with the Yocto Project. Additionally, recommendations also exist. This
802section describes the requirements and recommendations for released
803BSPs.
804
805Released BSP Requirements
806-------------------------
807
808Before looking at BSP requirements, you should consider the following:
809
810- The requirements here assume the BSP layer is a well-formed, "legal"
811 layer that can be added to the Yocto Project. For guidelines on
812 creating a layer that meets these base requirements, see the "`BSP
813 Layers <#bsp-layers>`__" section in this manual and the
814 "`Understanding and Creating
815 Layers" <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
816 section in the Yocto Project Development Tasks Manual.
817
818- The requirements in this section apply regardless of how you package
819 a BSP. You should consult the packaging and distribution guidelines
820 for your specific release process. For an example of packaging and
821 distribution requirements, see the "`Third Party BSP Release
822 Process <https://wiki.yoctoproject.org/wiki/Third_Party_BSP_Release_Process>`__"
823 wiki page.
824
825- The requirements for the BSP as it is made available to a developer
826 are completely independent of the released form of the BSP. For
827 example, the BSP Metadata can be contained within a Git repository
828 and could have a directory structure completely different from what
829 appears in the officially released BSP layer.
830
831- It is not required that specific packages or package modifications
832 exist in the BSP layer, beyond the requirements for general
833 compliance with the Yocto Project. For example, no requirement exists
834 dictating that a specific kernel or kernel version be used in a given
835 BSP.
836
837Following are the requirements for a released BSP that conform to the
838Yocto Project:
839
840- *Layer Name:* The BSP must have a layer name that follows the Yocto
841 Project standards. For information on BSP layer names, see the "`BSP
842 Layers <#bsp-layers>`__" section.
843
844- *File System Layout:* When possible, use the same directory names in
845 your BSP layer as listed in the ``recipes.txt`` file, which is found
846 in ``poky/meta`` directory of the `Source
847 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ or in the
848 OpenEmbedded-Core Layer (``openembedded-core``) at
849 ` <http://git.openembedded.org/openembedded-core/tree/meta>`__.
850
851 You should place recipes (``*.bb`` files) and recipe modifications
852 (``*.bbappend`` files) into ``recipes-*`` subdirectories by
853 functional area as outlined in ``recipes.txt``. If you cannot find a
854 category in ``recipes.txt`` to fit a particular recipe, you can make
855 up your own ``recipes-*`` subdirectory.
856
857 Within any particular ``recipes-*`` category, the layout should match
858 what is found in the OpenEmbedded-Core Git repository
859 (``openembedded-core``) or the Source Directory (``poky``). In other
860 words, make sure you place related files in appropriately-related
861 ``recipes-*`` subdirectories specific to the recipe's function, or
862 within a subdirectory containing a set of closely-related recipes.
863 The recipes themselves should follow the general guidelines for
864 recipes used in the Yocto Project found in the "`OpenEmbedded Style
865 Guide <http://openembedded.org/wiki/Styleguide>`__".
866
867- *License File:* You must include a license file in the
868 ``meta-``\ bsp_root_name directory. This license covers the BSP
869 Metadata as a whole. You must specify which license to use since no
870 default license exists when one is not specified. See the
871 ```COPYING.MIT`` <&YOCTO_GIT_URL;/cgit.cgi/meta-raspberrypi/tree/COPYING.MIT>`__
872 file for the Raspberry Pi BSP in the ``meta-raspberrypi`` BSP layer
873 as an example.
874
875- *README File:* You must include a ``README`` file in the
876 ``meta-``\ bsp_root_name directory. See the
877 ```README.md`` <&YOCTO_GIT_URL;/cgit.cgi/meta-raspberrypi/tree/README.md>`__
878 file for the Raspberry Pi BSP in the ``meta-raspberrypi`` BSP layer
879 as an example.
880
881 At a minimum, the ``README`` file should contain the following:
882
883 - A brief description of the target hardware.
884
885 - A list of all the dependencies of the BSP. These dependencies are
886 typically a list of required layers needed to build the BSP.
887 However, the dependencies should also contain information
888 regarding any other dependencies the BSP might have.
889
890 - Any required special licensing information. For example, this
891 information includes information on special variables needed to
892 satisfy a EULA, or instructions on information needed to build or
893 distribute binaries built from the BSP Metadata.
894
895 - The name and contact information for the BSP layer maintainer.
896 This is the person to whom patches and questions should be sent.
897 For information on how to find the right person, see the
898 "`Submitting a Change to the Yocto
899 Project <&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change>`__" section
900 in the Yocto Project Development Tasks Manual.
901
902 - Instructions on how to build the BSP using the BSP layer.
903
904 - Instructions on how to boot the BSP build from the BSP layer.
905
906 - Instructions on how to boot the binary images contained in the
907 ``binary`` directory, if present.
908
909 - Information on any known bugs or issues that users should know
910 about when either building or booting the BSP binaries.
911
912- *README.sources File:* If you BSP contains binary images in the
913 ``binary`` directory, you must include a ``README.sources`` file in
914 the ``meta-``\ bsp_root_name directory. This file specifies exactly
915 where you can find the sources used to generate the binary images.
916
917- *Layer Configuration File:* You must include a ``conf/layer.conf``
918 file in the ``meta-``\ bsp_root_name directory. This file identifies
919 the ``meta-``\ bsp_root_name BSP layer as a layer to the build
920 system.
921
922- *Machine Configuration File:* You must include one or more
923 ``conf/machine/``\ bsp_root_name\ ``.conf`` files in the
924 ``meta-``\ bsp_root_name directory. These configuration files define
925 machine targets that can be built using the BSP layer. Multiple
926 machine configuration files define variations of machine
927 configurations that the BSP supports. If a BSP supports multiple
928 machine variations, you need to adequately describe each variation in
929 the BSP ``README`` file. Do not use multiple machine configuration
930 files to describe disparate hardware. If you do have very different
931 targets, you should create separate BSP layers for each target.
932
933 .. note::
934
935 It is completely possible for a developer to structure the working
936 repository as a conglomeration of unrelated BSP files, and to
937 possibly generate BSPs targeted for release from that directory
938 using scripts or some other mechanism (e.g.
939 meta-yocto-bsp
940 layer). Such considerations are outside the scope of this
941 document.
942
943Released BSP Recommendations
944----------------------------
945
946Following are recommendations for released BSPs that conform to the
947Yocto Project:
948
949- *Bootable Images:* Released BSPs can contain one or more bootable
950 images. Including bootable images allows users to easily try out the
951 BSP using their own hardware.
952
953 In some cases, it might not be convenient to include a bootable
954 image. If so, you might want to make two versions of the BSP
955 available: one that contains binary images, and one that does not.
956 The version that does not contain bootable images avoids unnecessary
957 download times for users not interested in the images.
958
959 If you need to distribute a BSP and include bootable images or build
960 kernel and filesystems meant to allow users to boot the BSP for
961 evaluation purposes, you should put the images and artifacts within a
962 ``binary/`` subdirectory located in the ``meta-``\ bsp_root_name
963 directory.
964
965 .. note::
966
967 If you do include a bootable image as part of the BSP and the
968 image was built by software covered by the GPL or other open
969 source licenses, it is your responsibility to understand and meet
970 all licensing requirements, which could include distribution of
971 source files.
972
973- *Use a Yocto Linux Kernel:* Kernel recipes in the BSP should be based
974 on a Yocto Linux kernel. Basing your recipes on these kernels reduces
975 the costs for maintaining the BSP and increases its scalability. See
976 the ``Yocto Linux Kernel`` category in the `Source
977 Repositories <&YOCTO_GIT_URL;>`__ for these kernels.
978
979Customizing a Recipe for a BSP
980==============================
981
982If you plan on customizing a recipe for a particular BSP, you need to do
983the following:
984
985- Create a ``*.bbappend`` file for the modified recipe. For information
986 on using append files, see the "`Using .bbappend Files in Your
987 Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
988 Yocto Project Development Tasks Manual.
989
990- Ensure your directory structure in the BSP layer that supports your
991 machine is such that the OpenEmbedded build system can find it. See
992 the example later in this section for more information.
993
994- Put the append file in a directory whose name matches the machine's
995 name and is located in an appropriate sub-directory inside the BSP
996 layer (i.e. ``recipes-bsp``, ``recipes-graphics``, ``recipes-core``,
997 and so forth).
998
999- Place the BSP-specific files in the proper directory inside the BSP
1000 layer. How expansive the layer is affects where you must place these
1001 files. For example, if your layer supports several different machine
1002 types, you need to be sure your layer's directory structure includes
1003 hierarchy that separates the files according to machine. If your
1004 layer does not support multiple machines, the layer would not have
1005 that additional hierarchy and the files would obviously not be able
1006 to reside in a machine-specific directory.
1007
1008Following is a specific example to help you better understand the
1009process. This example customizes customizes a recipe by adding a
1010BSP-specific configuration file named ``interfaces`` to the
1011``init-ifupdown_1.0.bb`` recipe for machine "xyz" where the BSP layer
1012also supports several other machines:
1013
10141. Edit the ``init-ifupdown_1.0.bbappend`` file so that it contains the
1015 following: FILESEXTRAPATHS_prepend := "${THISDIR}/files:" The append
1016 file needs to be in the ``meta-xyz/recipes-core/init-ifupdown``
1017 directory.
1018
10192. Create and place the new ``interfaces`` configuration file in the
1020 BSP's layer here:
1021 meta-xyz/recipes-core/init-ifupdown/files/xyz-machine-one/interfaces
1022
1023 .. note::
1024
1025 If the
1026 meta-xyz
1027 layer did not support multiple machines, you would place the
1028 interfaces
1029 configuration file in the layer here:
1030 ::
1031
1032 meta-xyz/recipes-core/init-ifupdown/files/interfaces
1033
1034
1035 The
1036 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
1037 variable in the append files extends the search path the build system
1038 uses to find files during the build. Consequently, for this example
1039 you need to have the ``files`` directory in the same location as your
1040 append file.
1041
1042BSP Licensing Considerations
1043============================
1044
1045In some cases, a BSP contains separately-licensed Intellectual Property
1046(IP) for a component or components. For these cases, you are required to
1047accept the terms of a commercial or other type of license that requires
1048some kind of explicit End User License Agreement (EULA). Once you accept
1049the license, the OpenEmbedded build system can then build and include
1050the corresponding component in the final BSP image. If the BSP is
1051available as a pre-built image, you can download the image after
1052agreeing to the license or EULA.
1053
1054You could find that some separately-licensed components that are
1055essential for normal operation of the system might not have an
1056unencumbered (or free) substitute. Without these essential components,
1057the system would be non-functional. Then again, you might find that
1058other licensed components that are simply 'good-to-have' or purely
1059elective do have an unencumbered, free replacement component that you
1060can use rather than agreeing to the separately-licensed component. Even
1061for components essential to the system, you might find an unencumbered
1062component that is not identical but will work as a less-capable version
1063of the licensed version in the BSP recipe.
1064
1065For cases where you can substitute a free component and still maintain
1066the system's functionality, the "DOWNLOADS" selection from the
1067"SOFTWARE" tab on the `Yocto Project website <&YOCTO_HOME_URL;>`__ makes
1068available de-featured BSPs that are completely free of any IP
1069encumbrances. For these cases, you can use the substitution directly and
1070without any further licensing requirements. If present, these fully
1071de-featured BSPs are named appropriately different as compared to the
1072names of their respective encumbered BSPs. If available, these
1073substitutions are your simplest and most preferred options. Obviously,
1074use of these substitutions assumes the resulting functionality meets
1075system requirements.
1076
1077.. note::
1078
1079 If however, a non-encumbered version is unavailable or it provides
1080 unsuitable functionality or quality, you can use an encumbered
1081 version.
1082
1083A couple different methods exist within the OpenEmbedded build system to
1084satisfy the licensing requirements for an encumbered BSP. The following
1085list describes them in order of preference:
1086
10871. *Use
1088 the*\ ```LICENSE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS>`__\ *Variable
1089 to Define the Recipes that Have Commercial or Other Types of
1090 Specially-Licensed Packages:* For each of those recipes, you can
1091 specify a matching license string in a ``local.conf`` variable named
1092 ```LICENSE_FLAGS_WHITELIST`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST>`__.
1093 Specifying the matching license string signifies that you agree to
1094 the license. Thus, the build system can build the corresponding
1095 recipe and include the component in the image. See the "`Enabling
1096 Commercially Licensed
1097 Recipes <&YOCTO_DOCS_DEV_URL;#enabling-commercially-licensed-recipes>`__"
1098 section in the Yocto Project Development Tasks Manual for details on
1099 how to use these variables.
1100
1101 If you build as you normally would, without specifying any recipes in
1102 the ``LICENSE_FLAGS_WHITELIST``, the build stops and provides you
1103 with the list of recipes that you have tried to include in the image
1104 that need entries in the ``LICENSE_FLAGS_WHITELIST``. Once you enter
1105 the appropriate license flags into the whitelist, restart the build
1106 to continue where it left off. During the build, the prompt will not
1107 appear again since you have satisfied the requirement.
1108
1109 Once the appropriate license flags are on the white list in the
1110 ``LICENSE_FLAGS_WHITELIST`` variable, you can build the encumbered
1111 image with no change at all to the normal build process.
1112
11132. *Get a Pre-Built Version of the BSP:* You can get this type of BSP by
1114 selecting the "DOWNLOADS" item from the "SOFTWARE" tab on the `Yocto
1115 Project website <&YOCTO_HOME_URL;>`__. You can download BSP tarballs
1116 that contain proprietary components after agreeing to the licensing
1117 requirements of each of the individually encumbered packages as part
1118 of the download process. Obtaining the BSP this way allows you to
1119 access an encumbered image immediately after agreeing to the
1120 click-through license agreements presented by the website. If you
1121 want to build the image yourself using the recipes contained within
1122 the BSP tarball, you will still need to create an appropriate
1123 ``LICENSE_FLAGS_WHITELIST`` to match the encumbered recipes in the
1124 BSP.
1125
1126.. note::
1127
1128 Pre-compiled images are bundled with a time-limited kernel that runs
1129 for a predetermined amount of time (10 days) before it forces the
1130 system to reboot. This limitation is meant to discourage direct
1131 redistribution of the image. You must eventually rebuild the image if
1132 you want to remove this restriction.
1133
1134Creating a new BSP Layer Using the ``bitbake-layers`` Script
1135============================================================
1136
1137The ``bitbake-layers create-layer`` script automates creating a BSP
1138layer. What makes a layer a "BSP layer" is the presence of at least one
1139machine configuration file. Additionally, a BSP layer usually has a
1140kernel recipe or an append file that leverages off an existing kernel
1141recipe. The primary requirement, however, is the machine configuration.
1142
1143Use these steps to create a BSP layer:
1144
1145- *Create a General Layer:* Use the ``bitbake-layers`` script with the
1146 ``create-layer`` subcommand to create a new general layer. For
1147 instructions on how to create a general layer using the
1148 ``bitbake-layers`` script, see the "`Creating a General Layer Using
1149 the ``bitbake-layers``
1150 Script <&YOCTO_DOCS_DEV_URL;#creating-a-general-layer-using-the-bitbake-layers-script>`__"
1151 section in the Yocto Project Development Tasks Manual.
1152
1153- *Create a Layer Configuration File:* Every layer needs a layer
1154 configuration file. This configuration file establishes locations for
1155 the layer's recipes, priorities for the layer, and so forth. You can
1156 find examples of ``layer.conf`` files in the Yocto Project `Source
1157 Repositories <&YOCTO_GIT_URL;>`__. To get examples of what you need
1158 in your configuration file, locate a layer (e.g. "meta-ti") and
1159 examine the
1160 ` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-ti/tree/conf/layer.conf>`__
1161 file.
1162
1163- *Create a Machine Configuration File:* Create a
1164 ``conf/machine/``\ bsp_root_name\ ``.conf`` file. See
1165 ```meta-yocto-bsp/conf/machine`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf/machine>`__
1166 for sample bsp_root_name\ ``.conf`` files. Other samples such as
1167 ```meta-ti`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-ti/tree/conf/machine>`__
1168 and
1169 ```meta-freescale`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/tree/conf/machine>`__
1170 exist from other vendors that have more specific machine and tuning
1171 examples.
1172
1173- *Create a Kernel Recipe:* Create a kernel recipe in
1174 ``recipes-kernel/linux`` by either using a kernel append file or a
1175 new custom kernel recipe file (e.g. ``yocto-linux_4.12.bb``). The BSP
1176 layers mentioned in the previous step also contain different kernel
1177 examples. See the "`Modifying an Existing
1178 Recipe <&YOCTO_DOCS_KERNEL_DEV_URL;#modifying-an-existing-recipe>`__"
1179 section in the Yocto Project Linux Kernel Development Manual for
1180 information on how to create a custom kernel.
1181
1182The remainder of this section provides a description of the Yocto
1183Project reference BSP for Beaglebone, which resides in the
1184```meta-yocto-bsp`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp>`__
1185layer.
1186
1187BSP Layer Configuration Example
1188-------------------------------
1189
1190The layer's ``conf`` directory contains the ``layer.conf`` configuration
1191file. In this example, the ``conf/layer.conf`` is the following: # We
1192have a conf and classes directory, add to BBPATH BBPATH .=
1193":${LAYERDIR}" # We have recipes-\* directories, add to BBFILES BBFILES
1194+= "${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
1195BBFILE_COLLECTIONS += "yoctobsp" BBFILE_PATTERN_yoctobsp =
1196"^${LAYERDIR}/" BBFILE_PRIORITY_yoctobsp = "5" LAYERVERSION_yoctobsp =
1197"4" LAYERSERIES_COMPAT_yoctobsp = "DISTRO_NAME_NO_CAP" The variables
1198used in this file configure the layer. A good way to learn about layer
1199configuration files is to examine various files for BSP from the `Source
1200Repositories <&YOCTO_GIT_URL;>`__.
1201
1202For a detailed description of this particular layer configuration file,
1203see "`step 3 <&YOCTO_DOCS_DEV_URL;#dev-layer-config-file-description>`__
1204in the discussion that describes how to create layers in the Yocto
1205Project Development Tasks Manual.
1206
1207BSP Machine Configuration Example
1208---------------------------------
1209
1210As mentioned earlier in this section, the existence of a machine
1211configuration file is what makes a layer a BSP layer as compared to a
1212general or kernel layer.
1213
1214One or more machine configuration files exist in the
1215bsp_layer\ ``/conf/machine/`` directory of the layer:
1216bsp_layer\ ``/conf/machine/``\ machine1\ ``.conf``
1217bsp_layer\ ``/conf/machine/``\ machine2\ ``.conf``
1218bsp_layer\ ``/conf/machine/``\ machine3\ ``.conf`` ... more ... For
1219example, the machine configuration file for the `BeagleBone and
1220BeagleBone Black development boards <http://beagleboard.org/bone>`__ is
1221located in the layer ``poky/meta-yocto-bsp/conf/machine`` and is named
1222``beaglebone-yocto.conf``: #@TYPE: Machine #@NAME: Beaglebone-yocto
1223machine #@DESCRIPTION: Reference machine configuration for
1224http://beagleboard.org/bone and http://beagleboard.org/black boards
1225PREFERRED_PROVIDER_virtual/xserver ?= "xserver-xorg" XSERVER ?=
1226"xserver-xorg \\ xf86-video-modesetting \\ " MACHINE_EXTRA_RRECOMMENDS =
1227"kernel-modules kernel-devicetree" EXTRA_IMAGEDEPENDS += "u-boot"
1228DEFAULTTUNE ?= "cortexa8hf-neon" include
1229conf/machine/include/tune-cortexa8.inc IMAGE_FSTYPES += "tar.bz2 jffs2
1230wic wic.bmap" EXTRA_IMAGECMD_jffs2 = "-lnp " WKS_FILE ?=
1231"beaglebone-yocto.wks" IMAGE_INSTALL_append = " kernel-devicetree
1232kernel-image-zimage" do_image_wic[depends] +=
1233"mtools-native:do_populate_sysroot
1234dosfstools-native:do_populate_sysroot" SERIAL_CONSOLES ?= "115200;ttyS0
1235115200;ttyO0" SERIAL_CONSOLES_CHECK = "${SERIAL_CONSOLES}"
1236PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
1237PREFERRED_VERSION_linux-yocto ?= "5.0%" KERNEL_IMAGETYPE = "zImage"
1238KERNEL_DEVICETREE = "am335x-bone.dtb am335x-boneblack.dtb
1239am335x-bonegreen.dtb" KERNEL_EXTRA_ARGS +=
1240"LOADADDR=${UBOOT_ENTRYPOINT}" SPL_BINARY = "MLO" UBOOT_SUFFIX = "img"
1241UBOOT_MACHINE = "am335x_evm_defconfig" UBOOT_ENTRYPOINT = "0x80008000"
1242UBOOT_LOADADDRESS = "0x80008000" MACHINE_FEATURES = "usbgadget usbhost
1243vfat alsa" IMAGE_BOOT_FILES ?= "u-boot.${UBOOT_SUFFIX} MLO zImage
1244am335x-bone.dtb am335x-boneblack.dtb am335x-bonegreen.dtb" The variables
1245used to configure the machine define machine-specific properties; for
1246example, machine-dependent packages, machine tunings, the type of kernel
1247to build, and U-Boot configurations.
1248
1249The following list provides some explanation for the statements found in
1250the example reference machine configuration file for the BeagleBone
1251development boards. Realize that much more can be defined as part of a
1252machine's configuration file. In general, you can learn about related
1253variables that this example does not have by locating the variables in
1254the "`Yocto Project Variables
1255Glossary <&YOCTO_DOCS_REF_URL;#ref-variables-glos>`__" in the Yocto
1256Project Reference Manual.
1257
1258- ```PREFERRED_PROVIDER_virtual/xserver`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__:
1259 The recipe that provides "virtual/xserver" when more than one
1260 provider is found. In this case, the recipe that provides
1261 "virtual/xserver" is "xserver-xorg", which exists in
1262 ``poky/meta/recipes-graphics/xorg-xserver``.
1263
1264- ```XSERVER`` <&YOCTO_DOCS_REF_URL;#var-XSERVER>`__: The packages that
1265 should be installed to provide an X server and drivers for the
1266 machine. In this example, the "xserver-xorg" and
1267 "xf86-video-modesetting" are installed.
1268
1269- ```MACHINE_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RRECOMMENDS>`__:
1270 A list of machine-dependent packages not essential for booting the
1271 image. Thus, the build does not fail if the packages do not exist.
1272 However, the packages are required for a fully-featured image.
1273
1274 .. note::
1275
1276 Many
1277 MACHINE\*
1278 variables exist that help you configure a particular piece of
1279 hardware.
1280
1281- ```EXTRA_IMAGEDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGEDEPENDS>`__:
1282 Recipes to build that do not provide packages for installing into the
1283 root filesystem but building the image depends on the recipes.
1284 Sometimes a recipe is required to build the final image but is not
1285 needed in the root filesystem. In this case, the U-Boot recipe must
1286 be built for the image.
1287
1288- ```DEFAULTTUNE`` <&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE>`__: Machines
1289 use tunings to optimize machine, CPU, and application performance.
1290 These features, which are collectively known as "tuning features",
1291 exist in the `OpenEmbedded-Core
1292 (OE-Core) <&YOCTO_DOCS_REF_URL;#oe-core>`__ layer (e.g.
1293 ``poky/meta/conf/machine/include``). In this example, the default
1294 tunning file is "cortexa8hf-neon".
1295
1296 .. note::
1297
1298 The
1299 include
1300 statement that pulls in the
1301 conf/machine/include/tune-cortexa8.inc
1302 file provides many tuning possibilities.
1303
1304- ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__: The
1305 formats the OpenEmbedded build system uses during the build when
1306 creating the root filesystem. In this example, four types of images
1307 are supported.
1308
1309- ```EXTRA_IMAGECMD`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGECMD>`__:
1310 Specifies additional options for image creation commands. In this
1311 example, the "-lnp " option is used when creating the
1312 `JFFS2 <https://en.wikipedia.org/wiki/JFFS2>`__ image.
1313
1314- ```WKS_FILE`` <&YOCTO_DOCS_REF_URL;#var-WKS_FILE>`__: The location of
1315 the `Wic kickstart <&YOCTO_DOCS_REF_URL;#ref-kickstart>`__ file used
1316 by the OpenEmbedded build system to create a partitioned image
1317 (image.wic).
1318
1319- ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__:
1320 Specifies packages to install into an image through the
1321 ```image`` <&YOCTO_DOCS_REF_URL;#ref-classes-image>`__ class. Recipes
1322 use the ``IMAGE_INSTALL`` variable.
1323
1324- ``do_image_wic[depends]``: A task that is constructed during the
1325 build. In this example, the task depends on specific tools in order
1326 to create the sysroot when buiding a Wic image.
1327
1328- ```SERIAL_CONSOLES`` <&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES>`__:
1329 Defines a serial console (TTY) to enable using getty. In this case,
1330 the baud rate is "115200" and the device name is "ttyO0".
1331
1332- ```PREFERRED_PROVIDER_virtual/kernel`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__:
1333 Specifies the recipe that provides "virtual/kernel" when more than
1334 one provider is found. In this case, the recipe that provides
1335 "virtual/kernel" is "linux-yocto", which exists in the layer's
1336 ``recipes-kernel/linux`` directory.
1337
1338- ```PREFERRED_VERSION_linux-yocto`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_VERSION>`__:
1339 Defines the version of the recipe used to build the kernel, which is
1340 "5.0" in this case.
1341
1342- ```KERNEL_IMAGETYPE`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE>`__:
1343 The type of kernel to build for the device. In this case, the
1344 OpenEmbedded build system creates a "zImage" image type.
1345
1346- ```KERNEL_DEVICETREE`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_DEVICETREE>`__:
1347 The names of the generated Linux kernel device trees (i.e. the
1348 ``*.dtb``) files. All the device trees for the various BeagleBone
1349 devices are included.
1350
1351- ```KERNEL_EXTRA_ARGS`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_EXTRA_ARGS>`__:
1352 Additional ``make`` command-line arguments the OpenEmbedded build
1353 system passes on when compiling the kernel. In this example,
1354 "LOADADDR=${UBOOT_ENTRYPOINT}" is passed as a command-line argument.
1355
1356- ```SPL_BINARY`` <&YOCTO_DOCS_REF_URL;#var-SPL_BINARY>`__: Defines the
1357 Secondary Program Loader (SPL) binary type. In this case, the SPL
1358 binary is set to "MLO", which stands for Multimedia card LOader.
1359
1360 The BeagleBone development board requires an SPL to boot and that SPL
1361 file type must be MLO. Consequently, the machine configuration needs
1362 to define ``SPL_BINARY`` as "MLO".
1363
1364 .. note::
1365
1366 For more information on how the SPL variables are used, see the
1367 u-boot.inc
1368 include file.
1369
1370- ```UBOOT_*`` <&YOCTO_DOCS_REF_URL;#var-UBOOT_ENTRYPOINT>`__: Defines
1371 various U-Boot configurations needed to build a U-Boot image. In this
1372 example, a U-Boot image is required to boot the BeagleBone device.
1373 See the following variables for more information:
1374
1375 - ```UBOOT_SUFFIX`` <&YOCTO_DOCS_REF_URL;#var-UBOOT_SUFFIX>`__:
1376 Points to the generated U-Boot extension.
1377
1378 - ```UBOOT_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-UBOOT_MACHINE>`__:
1379 Specifies the value passed on the make command line when building
1380 a U-Boot image.
1381
1382 - ```UBOOT_ENTRYPOINT`` <&YOCTO_DOCS_REF_URL;#var-UBOOT_ENTRYPOINT>`__:
1383 Specifies the entry point for the U-Boot image.
1384
1385 - ```UBOOT_LOADADDRESS`` <&YOCTO_DOCS_REF_URL;#var-UBOOT_LOADADDRESS>`__:
1386 Specifies the load address for the U-Boot image.
1387
1388- ```MACHINE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES>`__:
1389 Specifies the list of hardware features the BeagleBone device is
1390 capable of supporting. In this case, the device supports "usbgadget
1391 usbhost vfat alsa".
1392
1393- ```IMAGE_BOOT_FILES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_BOOT_FILES>`__:
1394 Files installed into the device's boot partition when preparing the
1395 image using the Wic tool with the ``bootimg-partition`` or
1396 ``bootimg-efi`` source plugin.
1397
1398BSP Kernel Recipe Example
1399-------------------------
1400
1401The kernel recipe used to build the kernel image for the BeagleBone
1402device was established in the machine configuration:
1403PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
1404PREFERRED_VERSION_linux-yocto ?= "5.0%" The
1405``meta-yocto-bsp/recipes-kernel/linux`` directory in the layer contains
1406metadata used to build the kernel. In this case, a kernel append file
1407(i.e. ``linux-yocto_5.0.bbappend``) is used to override an established
1408kernel recipe (i.e. ``linux-yocto_5.0.bb``), which is located in
1409` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/linux>`__.
1410
1411Following is the contents of the append file: KBRANCH_genericx86 =
1412"v5.0/standard/base" KBRANCH_genericx86-64 = "v5.0/standard/base"
1413KBRANCH_edgerouter = "v5.0/standard/edgerouter" KBRANCH_beaglebone-yocto
1414= "v5.0/standard/beaglebone" KMACHINE_genericx86 ?= "common-pc"
1415KMACHINE_genericx86-64 ?= "common-pc-64" KMACHINE_beaglebone-yocto ?=
1416"beaglebone" SRCREV_machine_genericx86 ?=
1417"3df4aae6074e94e794e27fe7f17451d9353cdf3d" SRCREV_machine_genericx86-64
1418?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d" SRCREV_machine_edgerouter
1419?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d"
1420SRCREV_machine_beaglebone-yocto ?=
1421"3df4aae6074e94e794e27fe7f17451d9353cdf3d" COMPATIBLE_MACHINE_genericx86
1422= "genericx86" COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
1423COMPATIBLE_MACHINE_edgerouter = "edgerouter"
1424COMPATIBLE_MACHINE_beaglebone-yocto = "beaglebone-yocto"
1425LINUX_VERSION_genericx86 = "5.0.3" LINUX_VERSION_genericx86-64 = "5.0.3"
1426LINUX_VERSION_edgerouter = "5.0.3" LINUX_VERSION_beaglebone-yocto =
1427"5.0.3" This particular append file works for all the machines that are
1428part of the ``meta-yocto-bsp`` layer. The relevant statements are
1429appended with the "beaglebone-yocto" string. The OpenEmbedded build
1430system uses these statements to override similar statements in the
1431kernel recipe:
1432
1433- ```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__: Identifies the
1434 kernel branch that is validated, patched, and configured during the
1435 build.
1436
1437- ```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__: Identifies the
1438 machine name as known by the kernel, which is sometimes a different
1439 name than what is known by the OpenEmbedded build system.
1440
1441- ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__: Identifies the
1442 revision of the source code used to build the image.
1443
1444- ```COMPATIBLE_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE>`__:
1445 A regular expression that resolves to one or more target machines
1446 with which the recipe is compatible.
1447
1448- ```LINUX_VERSION`` <&YOCTO_DOCS_REF_URL;#var-LINUX_VERSION>`__: The
1449 Linux version from kernel.org used by the OpenEmbedded build system
1450 to build the kernel image.
diff --git a/documentation/dev-manual/dev-manual-common-tasks.rst b/documentation/dev-manual/dev-manual-common-tasks.rst
new file mode 100644
index 0000000000..b36c97a6a3
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-common-tasks.rst
@@ -0,0 +1,10227 @@
1************
2Common Tasks
3************
4
5This chapter describes fundamental procedures such as creating layers,
6adding new software packages, extending or customizing images, porting
7work to new hardware (adding a new machine), and so forth. You will find
8that the procedures documented here occur often in the development cycle
9using the Yocto Project.
10
11Understanding and Creating Layers
12=================================
13
14The OpenEmbedded build system supports organizing
15`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ into multiple layers.
16Layers allow you to isolate different types of customizations from each
17other. For introductory information on the Yocto Project Layer Model,
18see the "`The Yocto Project Layer
19Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__" section in
20the Yocto Project Overview and Concepts Manual.
21
22Creating Your Own Layer
23-----------------------
24
25It is very easy to create your own layers to use with the OpenEmbedded
26build system. The Yocto Project ships with tools that speed up creating
27layers. This section describes the steps you perform by hand to create
28layers so that you can better understand them. For information about the
29layer-creation tools, see the "`Creating a New BSP Layer Using the
30``bitbake-layers``
31Script <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
32section in the Yocto Project Board Support Package (BSP) Developer's
33Guide and the "`Creating a General Layer Using the ``bitbake-layers``
34Script <#creating-a-general-layer-using-the-bitbake-layers-script>`__"
35section further down in this manual.
36
37Follow these general steps to create your layer without using tools:
38
391. *Check Existing Layers:* Before creating a new layer, you should be
40 sure someone has not already created a layer containing the Metadata
41 you need. You can see the `OpenEmbedded Metadata
42 Index <http://layers.openembedded.org/layerindex/layers/>`__ for a
43 list of layers from the OpenEmbedded community that can be used in
44 the Yocto Project. You could find a layer that is identical or close
45 to what you need.
46
472. *Create a Directory:* Create the directory for your layer. When you
48 create the layer, be sure to create the directory in an area not
49 associated with the Yocto Project `Source
50 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. the cloned
51 ``poky`` repository).
52
53 While not strictly required, prepend the name of the directory with
54 the string "meta-". For example: meta-mylayer meta-GUI_xyz
55 meta-mymachine With rare exceptions, a layer's name follows this
56 form: meta-root_name Following this layer naming convention can save
57 you trouble later when tools, components, or variables "assume" your
58 layer name begins with "meta-". A notable example is in configuration
59 files as shown in the following step where layer names without the
60 "meta-" string are appended to several variables used in the
61 configuration.
62
633. *Create a Layer Configuration File:* Inside your new layer folder,
64 you need to create a ``conf/layer.conf`` file. It is easiest to take
65 an existing layer configuration file and copy that to your layer's
66 ``conf`` directory and then modify the file as needed.
67
68 The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project
69 `Source
70 Repositories <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf>`__
71 demonstrates the required syntax. For your layer, you need to replace
72 "yoctobsp" with a unique identifier for your layer (e.g. "machinexyz"
73 for a layer named "meta-machinexyz"): # We have a conf and classes
74 directory, add to BBPATH BBPATH .= ":${LAYERDIR}" # We have
75 recipes-\* directories, add to BBFILES BBFILES +=
76 "${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
77 BBFILE_COLLECTIONS += "yoctobsp" BBFILE_PATTERN_yoctobsp =
78 "^${LAYERDIR}/" BBFILE_PRIORITY_yoctobsp = "5" LAYERVERSION_yoctobsp
79 = "4" LAYERSERIES_COMPAT_yoctobsp = "DISTRO_NAME_NO_CAP" Following is
80 an explanation of the layer configuration file:
81
82 - ```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__: Adds the layer's
83 root directory to BitBake's search path. Through the use of the
84 ``BBPATH`` variable, BitBake locates class files (``.bbclass``),
85 configuration files, and files that are included with ``include``
86 and ``require`` statements. For these cases, BitBake uses the
87 first file that matches the name found in ``BBPATH``. This is
88 similar to the way the ``PATH`` variable is used for binaries. It
89 is recommended, therefore, that you use unique class and
90 configuration filenames in your custom layer.
91
92 - ```BBFILES`` <&YOCTO_DOCS_REF_URL;#var-BBFILES>`__: Defines the
93 location for all recipes in the layer.
94
95 - ```BBFILE_COLLECTIONS`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS>`__:
96 Establishes the current layer through a unique identifier that is
97 used throughout the OpenEmbedded build system to refer to the
98 layer. In this example, the identifier "yoctobsp" is the
99 representation for the container layer named "meta-yocto-bsp".
100
101 - ```BBFILE_PATTERN`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN>`__:
102 Expands immediately during parsing to provide the directory of the
103 layer.
104
105 - ```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__:
106 Establishes a priority to use for recipes in the layer when the
107 OpenEmbedded build finds recipes of the same name in different
108 layers.
109
110 - ```LAYERVERSION`` <&YOCTO_DOCS_REF_URL;#var-LAYERVERSION>`__:
111 Establishes a version number for the layer. You can use this
112 version number to specify this exact version of the layer as a
113 dependency when using the
114 ```LAYERDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS>`__
115 variable.
116
117 - ```LAYERDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS>`__:
118 Lists all layers on which this layer depends (if any).
119
120 - ```LAYERSERIES_COMPAT`` <&YOCTO_DOCS_REF_URL;#var-LAYERSERIES_COMPAT>`__:
121 Lists the `Yocto Project <&YOCTO_WIKI_URL;/wiki/Releases>`__
122 releases for which the current version is compatible. This
123 variable is a good way to indicate if your particular layer is
124 current.
125
1264. *Add Content:* Depending on the type of layer, add the content. If
127 the layer adds support for a machine, add the machine configuration
128 in a ``conf/machine/`` file within the layer. If the layer adds
129 distro policy, add the distro configuration in a ``conf/distro/``
130 file within the layer. If the layer introduces new recipes, put the
131 recipes you need in ``recipes-*`` subdirectories within the layer.
132
133 .. note::
134
135 For an explanation of layer hierarchy that is compliant with the
136 Yocto Project, see the "
137 Example Filesystem Layout
138 " section in the Yocto Project Board Support Package (BSP)
139 Developer's Guide.
140
1415. *Optionally Test for Compatibility:* If you want permission to use
142 the Yocto Project Compatibility logo with your layer or application
143 that uses your layer, perform the steps to apply for compatibility.
144 See the "`Making Sure Your Layer is Compatible With Yocto
145 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
146 section for more information.
147
148.. _best-practices-to-follow-when-creating-layers:
149
150Following Best Practices When Creating Layers
151---------------------------------------------
152
153To create layers that are easier to maintain and that will not impact
154builds for other machines, you should consider the information in the
155following list:
156
157- *Avoid "Overlaying" Entire Recipes from Other Layers in Your
158 Configuration:* In other words, do not copy an entire recipe into
159 your layer and then modify it. Rather, use an append file
160 (``.bbappend``) to override only those parts of the original recipe
161 you need to modify.
162
163- *Avoid Duplicating Include Files:* Use append files (``.bbappend``)
164 for each recipe that uses an include file. Or, if you are introducing
165 a new recipe that requires the included file, use the path relative
166 to the original layer directory to refer to the file. For example,
167 use ``require recipes-core/``\ package\ ``/``\ file\ ``.inc`` instead
168 of ``require``\ file\ ``.inc``. If you're finding you have to overlay
169 the include file, it could indicate a deficiency in the include file
170 in the layer to which it originally belongs. If this is the case, you
171 should try to address that deficiency instead of overlaying the
172 include file. For example, you could address this by getting the
173 maintainer of the include file to add a variable or variables to make
174 it easy to override the parts needing to be overridden.
175
176- *Structure Your Layers:* Proper use of overrides within append files
177 and placement of machine-specific files within your layer can ensure
178 that a build is not using the wrong Metadata and negatively impacting
179 a build for a different machine. Following are some examples:
180
181 - *Modify Variables to Support a Different Machine:* Suppose you
182 have a layer named ``meta-one`` that adds support for building
183 machine "one". To do so, you use an append file named
184 ``base-files.bbappend`` and create a dependency on "foo" by
185 altering the ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__
186 variable: DEPENDS = "foo" The dependency is created during any
187 build that includes the layer ``meta-one``. However, you might not
188 want this dependency for all machines. For example, suppose you
189 are building for machine "two" but your ``bblayers.conf`` file has
190 the ``meta-one`` layer included. During the build, the
191 ``base-files`` for machine "two" will also have the dependency on
192 ``foo``.
193
194 To make sure your changes apply only when building machine "one",
195 use a machine override with the ``DEPENDS`` statement: DEPENDS_one
196 = "foo" You should follow the same strategy when using ``_append``
197 and ``_prepend`` operations: DEPENDS_append_one = " foo"
198 DEPENDS_prepend_one = "foo " As an actual example, here's a
199 snippet from the generic kernel include file ``linux-yocto.inc``,
200 wherein the kernel compile and link options are adjusted in the
201 case of a subset of the supported architectures:
202 DEPENDS_append_aarch64 = " libgcc" KERNEL_CC_append_aarch64 = "
203 ${TOOLCHAIN_OPTIONS}" KERNEL_LD_append_aarch64 = "
204 ${TOOLCHAIN_OPTIONS}" DEPENDS_append_nios2 = " libgcc"
205 KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
206 KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
207 DEPENDS_append_arc = " libgcc" KERNEL_CC_append_arc = "
208 ${TOOLCHAIN_OPTIONS}" KERNEL_LD_append_arc = "
209 ${TOOLCHAIN_OPTIONS}" KERNEL_FEATURES_append_qemuall="
210 features/debug/printk.scc"
211
212 .. note::
213
214 Avoiding "+=" and "=+" and using machine-specific
215 \_append
216 and
217 \_prepend
218 operations is recommended as well.
219
220 - *Place Machine-Specific Files in Machine-Specific Locations:* When
221 you have a base recipe, such as ``base-files.bb``, that contains a
222 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement to a
223 file, you can use an append file to cause the build to use your
224 own version of the file. For example, an append file in your layer
225 at ``meta-one/recipes-core/base-files/base-files.bbappend`` could
226 extend ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
227 using
228 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
229 as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" The
230 build for machine "one" will pick up your machine-specific file as
231 long as you have the file in
232 ``meta-one/recipes-core/base-files/base-files/``. However, if you
233 are building for a different machine and the ``bblayers.conf``
234 file includes the ``meta-one`` layer and the location of your
235 machine-specific file is the first location where that file is
236 found according to ``FILESPATH``, builds for all machines will
237 also use that machine-specific file.
238
239 You can make sure that a machine-specific file is used for a
240 particular machine by putting the file in a subdirectory specific
241 to the machine. For example, rather than placing the file in
242 ``meta-one/recipes-core/base-files/base-files/`` as shown above,
243 put it in ``meta-one/recipes-core/base-files/base-files/one/``.
244 Not only does this make sure the file is used only when building
245 for machine "one", but the build process locates the file more
246 quickly.
247
248 In summary, you need to place all files referenced from
249 ``SRC_URI`` in a machine-specific subdirectory within the layer in
250 order to restrict those files to machine-specific builds.
251
252- *Perform Steps to Apply for Yocto Project Compatibility:* If you want
253 permission to use the Yocto Project Compatibility logo with your
254 layer or application that uses your layer, perform the steps to apply
255 for compatibility. See the "`Making Sure Your Layer is Compatible
256 With Yocto
257 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
258 section for more information.
259
260- *Follow the Layer Naming Convention:* Store custom layers in a Git
261 repository that use the ``meta-layer_name`` format.
262
263- *Group Your Layers Locally:* Clone your repository alongside other
264 cloned ``meta`` directories from the `Source
265 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
266
267Making Sure Your Layer is Compatible With Yocto Project
268-------------------------------------------------------
269
270When you create a layer used with the Yocto Project, it is advantageous
271to make sure that the layer interacts well with existing Yocto Project
272layers (i.e. the layer is compatible with the Yocto Project). Ensuring
273compatibility makes the layer easy to be consumed by others in the Yocto
274Project community and could allow you permission to use the Yocto
275Project Compatible Logo.
276
277.. note::
278
279 Only Yocto Project member organizations are permitted to use the
280 Yocto Project Compatible Logo. The logo is not available for general
281 use. For information on how to become a Yocto Project member
282 organization, see the
283 Yocto Project Website
284 .
285
286The Yocto Project Compatibility Program consists of a layer application
287process that requests permission to use the Yocto Project Compatibility
288Logo for your layer and application. The process consists of two parts:
289
2901. Successfully passing a script (``yocto-check-layer``) that when run
291 against your layer, tests it against constraints based on experiences
292 of how layers have worked in the real world and where pitfalls have
293 been found. Getting a "PASS" result from the script is required for
294 successful compatibility registration.
295
2962. Completion of an application acceptance form, which you can find at
297 ` <https://www.yoctoproject.org/webform/yocto-project-compatible-registration>`__.
298
299To be granted permission to use the logo, you need to satisfy the
300following:
301
302- Be able to check the box indicating that you got a "PASS" when
303 running the script against your layer.
304
305- Answer "Yes" to the questions on the form or have an acceptable
306 explanation for any questions answered "No".
307
308- Be a Yocto Project Member Organization.
309
310The remainder of this section presents information on the registration
311form and on the ``yocto-check-layer`` script.
312
313Yocto Project Compatible Program Application
314~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
315
316Use the form to apply for your layer's approval. Upon successful
317application, you can use the Yocto Project Compatibility Logo with your
318layer and the application that uses your layer.
319
320To access the form, use this link:
321` <https://www.yoctoproject.org/webform/yocto-project-compatible-registration>`__.
322Follow the instructions on the form to complete your application.
323
324The application consists of the following sections:
325
326- *Contact Information:* Provide your contact information as the fields
327 require. Along with your information, provide the released versions
328 of the Yocto Project for which your layer is compatible.
329
330- *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the
331 items in the checklist. Space exists at the bottom of the form for
332 any explanations for items for which you answered "No".
333
334- *Recommendations:* Provide answers for the questions regarding Linux
335 kernel use and build success.
336
337``yocto-check-layer`` Script
338~~~~~~~~~~~~~~~~~~~~~~~~~~~~
339
340The ``yocto-check-layer`` script provides you a way to assess how
341compatible your layer is with the Yocto Project. You should run this
342script prior to using the form to apply for compatibility as described
343in the previous section. You need to achieve a "PASS" result in order to
344have your application form successfully processed.
345
346The script divides tests into three areas: COMMON, BSP, and DISTRO. For
347example, given a distribution layer (DISTRO), the layer must pass both
348the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP
349layer, the layer must pass the COMMON and BSP set of tests.
350
351To execute the script, enter the following commands from your build
352directory: $ source oe-init-build-env $ yocto-check-layer
353your_layer_directory Be sure to provide the actual directory for your
354layer as part of the command.
355
356Entering the command causes the script to determine the type of layer
357and then to execute a set of specific tests against the layer. The
358following list overviews the test:
359
360- ``common.test_readme``: Tests if a ``README`` file exists in the
361 layer and the file is not empty.
362
363- ``common.test_parse``: Tests to make sure that BitBake can parse the
364 files without error (i.e. ``bitbake -p``).
365
366- ``common.test_show_environment``: Tests that the global or per-recipe
367 environment is in order without errors (i.e. ``bitbake -e``).
368
369- ``common.test_world``: Verifies that ``bitbake world`` works.
370
371- ``common.test_signatures``: Tests to be sure that BSP and DISTRO
372 layers do not come with recipes that change signatures.
373
374- ``common.test_layerseries_compat``: Verifies layer compatibility is
375 set properly.
376
377- ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine
378 configurations.
379
380- ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not
381 set the machine when the layer is added.
382
383- ``bsp.test_machine_world``: Verifies that ``bitbake world`` works
384 regardless of which machine is selected.
385
386- ``bsp.test_machine_signatures``: Verifies that building for a
387 particular machine affects only the signature of tasks specific to
388 that machine.
389
390- ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has
391 distro configurations.
392
393- ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer
394 does not set the distribution when the layer is added.
395
396Enabling Your Layer
397-------------------
398
399Before the OpenEmbedded build system can use your new layer, you need to
400enable it. To enable your layer, simply add your layer's path to the
401``BBLAYERS`` variable in your ``conf/bblayers.conf`` file, which is
402found in the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
403The following example shows how to enable a layer named
404``meta-mylayer``: # POKY_BBLAYERS_CONF_VERSION is increased each time
405build/conf/bblayers.conf # changes incompatibly
406POKY_BBLAYERS_CONF_VERSION = "2" BBPATH = "${TOPDIR}" BBFILES ?= ""
407BBLAYERS ?= " \\ /home/user/poky/meta \\ /home/user/poky/meta-poky \\
408/home/user/poky/meta-yocto-bsp \\ /home/user/poky/meta-mylayer \\ "
409
410BitBake parses each ``conf/layer.conf`` file from the top down as
411specified in the ``BBLAYERS`` variable within the ``conf/bblayers.conf``
412file. During the processing of each ``conf/layer.conf`` file, BitBake
413adds the recipes, classes and configurations contained within the
414particular layer to the source directory.
415
416.. _using-bbappend-files:
417
418Using .bbappend Files in Your Layer
419-----------------------------------
420
421A recipe that appends Metadata to another recipe is called a BitBake
422append file. A BitBake append file uses the ``.bbappend`` file type
423suffix, while the corresponding recipe to which Metadata is being
424appended uses the ``.bb`` file type suffix.
425
426You can use a ``.bbappend`` file in your layer to make additions or
427changes to the content of another layer's recipe without having to copy
428the other layer's recipe into your layer. Your ``.bbappend`` file
429resides in your layer, while the main ``.bb`` recipe file to which you
430are appending Metadata resides in a different layer.
431
432Being able to append information to an existing recipe not only avoids
433duplication, but also automatically applies recipe changes from a
434different layer into your layer. If you were copying recipes, you would
435have to manually merge changes as they occur.
436
437When you create an append file, you must use the same root name as the
438corresponding recipe file. For example, the append file
439``someapp_DISTRO.bbappend`` must apply to ``someapp_DISTRO.bb``. This
440means the original recipe and append file names are version
441number-specific. If the corresponding recipe is renamed to update to a
442newer version, you must also rename and possibly update the
443corresponding ``.bbappend`` as well. During the build process, BitBake
444displays an error on starting if it detects a ``.bbappend`` file that
445does not have a corresponding recipe with a matching name. See the
446```BB_DANGLINGAPPENDS_WARNONLY`` <&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY>`__
447variable for information on how to handle this error.
448
449As an example, consider the main formfactor recipe and a corresponding
450formfactor append file both from the `Source
451Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Here is the main
452formfactor recipe, which is named ``formfactor_0.0.bb`` and located in
453the "meta" layer at ``meta/recipes-bsp/formfactor``: SUMMARY = "Device
454formfactor information" SECTION = "base" LICENSE = "MIT"
455LIC_FILES_CHKSUM =
456"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
457PR = "r45" SRC_URI = "file://config file://machconfig" S = "${WORKDIR}"
458PACKAGE_ARCH = "${MACHINE_ARCH}" INHIBIT_DEFAULT_DEPS = "1" do_install()
459{ # Install file only if it has contents install -d
460${D}${sysconfdir}/formfactor/ install -m 0644 ${S}/config
461${D}${sysconfdir}/formfactor/ if [ -s "${S}/machconfig" ]; then install
462-m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ fi } In the main
463recipe, note the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
464variable, which tells the OpenEmbedded build system where to find files
465during the build.
466
467Following is the append file, which is named ``formfactor_0.0.bbappend``
468and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The
469file is in the layer at ``recipes-bsp/formfactor``:
470FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
471
472By default, the build system uses the
473```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ variable to
474locate files. This append file extends the locations by setting the
475```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
476variable. Setting this variable in the ``.bbappend`` file is the most
477reliable and recommended method for adding directories to the search
478path used by the build system to find files.
479
480The statement in this example extends the directories to include
481``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``,
482which resolves to a directory named ``formfactor`` in the same directory
483in which the append file resides (i.e.
484``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must
485have the supporting directory structure set up that will contain any
486files or patches you will be including from the layer.
487
488Using the immediate expansion assignment operator ``:=`` is important
489because of the reference to ``THISDIR``. The trailing colon character is
490important as it ensures that items in the list remain colon-separated.
491
492.. note::
493
494 BitBake automatically defines the ``THISDIR`` variable. You should
495 never set this variable yourself. Using "_prepend" as part of the
496 ``FILESEXTRAPATHS`` ensures your path will be searched prior to other
497 paths in the final list.
498
499 Also, not all append files add extra files. Many append files simply
500 exist to add build options (e.g. ``systemd``). For these cases, your
501 append file would not even use the ``FILESEXTRAPATHS`` statement.
502
503Prioritizing Your Layer
504-----------------------
505
506Each layer is assigned a priority value. Priority values control which
507layer takes precedence if there are recipe files with the same name in
508multiple layers. For these cases, the recipe file from the layer with a
509higher priority number takes precedence. Priority values also affect the
510order in which multiple ``.bbappend`` files for the same recipe are
511applied. You can either specify the priority manually, or allow the
512build system to calculate it based on the layer's dependencies.
513
514To specify the layer's priority manually, use the
515```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__
516variable and append the layer's root name: BBFILE_PRIORITY_mylayer = "1"
517
518.. note::
519
520 It is possible for a recipe with a lower version number
521 ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ in a layer that has a higher
522 priority to take precedence.
523
524 Also, the layer priority does not currently affect the precedence
525 order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake
526 might address this.
527
528Managing Layers
529---------------
530
531You can use the BitBake layer management tool ``bitbake-layers`` to
532provide a view into the structure of recipes across a multi-layer
533project. Being able to generate output that reports on configured layers
534with their paths and priorities and on ``.bbappend`` files and their
535applicable recipes can help to reveal potential problems.
536
537For help on the BitBake layer management tool, use the following
538command: $ bitbake-layers --help NOTE: Starting bitbake server... usage:
539bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ...
540BitBake layers utility optional arguments: -d, --debug Enable debug
541output -q, --quiet Print only errors -F, --force Force add without
542recipe parse verification --color COLOR Colorize output (where COLOR is
543auto, always, never) -h, --help show this help message and exit
544subcommands: <subcommand> show-layers show current configured layers.
545show-overlayed list overlayed recipes (where the same recipe exists in
546another layer) show-recipes list available recipes, showing the layer
547they are provided by show-appends list bbappend files and recipe files
548they apply to show-cross-depends Show dependencies between recipes that
549cross layer boundaries. add-layer Add one or more layers to
550bblayers.conf. remove-layer Remove one or more layers from
551bblayers.conf. flatten flatten layer configuration into a separate
552output directory. layerindex-fetch Fetches a layer from a layer index
553along with its dependent layers, and adds them to conf/bblayers.conf.
554layerindex-show-depends Find layer dependencies from layer index.
555create-layer Create a basic layer Use bitbake-layers <subcommand> --help
556to get help on a specific command
557
558The following list describes the available commands:
559
560- *``help:``* Displays general help or help on a specified command.
561
562- *``show-layers:``* Shows the current configured layers.
563
564- *``show-overlayed:``* Lists overlayed recipes. A recipe is overlayed
565 when a recipe with the same name exists in another layer that has a
566 higher layer priority.
567
568- *``show-recipes:``* Lists available recipes and the layers that
569 provide them.
570
571- *``show-appends:``* Lists ``.bbappend`` files and the recipe files to
572 which they apply.
573
574- *``show-cross-depends:``* Lists dependency relationships between
575 recipes that cross layer boundaries.
576
577- *``add-layer:``* Adds a layer to ``bblayers.conf``.
578
579- *``remove-layer:``* Removes a layer from ``bblayers.conf``
580
581- *``flatten:``* Flattens the layer configuration into a separate
582 output directory. Flattening your layer configuration builds a
583 "flattened" directory that contains the contents of all layers, with
584 any overlayed recipes removed and any ``.bbappend`` files appended to
585 the corresponding recipes. You might have to perform some manual
586 cleanup of the flattened layer as follows:
587
588 - Non-recipe files (such as patches) are overwritten. The flatten
589 command shows a warning for these files.
590
591 - Anything beyond the normal layer setup has been added to the
592 ``layer.conf`` file. Only the lowest priority layer's
593 ``layer.conf`` is used.
594
595 - Overridden and appended items from ``.bbappend`` files need to be
596 cleaned up. The contents of each ``.bbappend`` end up in the
597 flattened recipe. However, if there are appended or changed
598 variable values, you need to tidy these up yourself. Consider the
599 following example. Here, the ``bitbake-layers`` command adds the
600 line ``#### bbappended ...`` so that you know where the following
601 lines originate: ... DESCRIPTION = "A useful utility" ...
602 EXTRA_OECONF = "--enable-something" ... #### bbappended from
603 meta-anotherlayer #### DESCRIPTION = "Customized utility"
604 EXTRA_OECONF += "--enable-somethingelse" Ideally, you would tidy
605 up these utilities as follows: ... DESCRIPTION = "Customized
606 utility" ... EXTRA_OECONF = "--enable-something
607 --enable-somethingelse" ...
608
609- *``layerindex-fetch``:* Fetches a layer from a layer index, along
610 with its dependent layers, and adds the layers to the
611 ``conf/bblayers.conf`` file.
612
613- *``layerindex-show-depends``:* Finds layer dependencies from the
614 layer index.
615
616- *``create-layer``:* Creates a basic layer.
617
618Creating a General Layer Using the ``bitbake-layers`` Script
619------------------------------------------------------------
620
621The ``bitbake-layers`` script with the ``create-layer`` subcommand
622simplifies creating a new general layer.
623
624.. note::
625
626 - For information on BSP layers, see the "`BSP
627 Layers <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__" section in the Yocto
628 Project Board Specific (BSP) Developer's Guide.
629
630 - In order to use a layer with the OpenEmbedded build system, you
631 need to add the layer to your ``bblayers.conf`` configuration
632 file. See the "`Adding a Layer Using the ``bitbake-layers``
633 Script <#adding-a-layer-using-the-bitbake-layers-script>`__"
634 section for more information.
635
636The default mode of the script's operation with this subcommand is to
637create a layer with the following:
638
639- A layer priority of 6.
640
641- A ``conf`` subdirectory that contains a ``layer.conf`` file.
642
643- A ``recipes-example`` subdirectory that contains a further
644 subdirectory named ``example``, which contains an ``example.bb``
645 recipe file.
646
647- A ``COPYING.MIT``, which is the license statement for the layer. The
648 script assumes you want to use the MIT license, which is typical for
649 most layers, for the contents of the layer itself.
650
651- A ``README`` file, which is a file describing the contents of your
652 new layer.
653
654In its simplest form, you can use the following command form to create a
655layer. The command creates a layer whose name corresponds to
656your_layer_name in the current directory: $ bitbake-layers create-layer
657your_layer_name As an example, the following command creates a layer
658named ``meta-scottrif`` in your home directory: $ cd /usr/home $
659bitbake-layers create-layer meta-scottrif NOTE: Starting bitbake
660server... Add your new layer with 'bitbake-layers add-layer
661meta-scottrif'
662
663If you want to set the priority of the layer to other than the default
664value of "6", you can either use the ``DASHDASHpriority`` option or you
665can edit the
666```BBFILE_PRIORITY`` <&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY>`__ value
667in the ``conf/layer.conf`` after the script creates it. Furthermore, if
668you want to give the example recipe file some name other than the
669default, you can use the ``DASHDASHexample-recipe-name`` option.
670
671The easiest way to see how the ``bitbake-layers create-layer`` command
672works is to experiment with the script. You can also read the usage
673information by entering the following: $ bitbake-layers create-layer
674--help NOTE: Starting bitbake server... usage: bitbake-layers
675create-layer [-h] [--priority PRIORITY] [--example-recipe-name
676EXAMPLERECIPE] layerdir Create a basic layer positional arguments:
677layerdir Layer directory to create optional arguments: -h, --help show
678this help message and exit --priority PRIORITY, -p PRIORITY Layer
679directory to create --example-recipe-name EXAMPLERECIPE, -e
680EXAMPLERECIPE Filename of the example recipe
681
682Adding a Layer Using the ``bitbake-layers`` Script
683--------------------------------------------------
684
685Once you create your general layer, you must add it to your
686``bblayers.conf`` file. Adding the layer to this configuration file
687makes the OpenEmbedded build system aware of your layer so that it can
688search it for metadata.
689
690Add your layer by using the ``bitbake-layers add-layer`` command: $
691bitbake-layers add-layer your_layer_name Here is an example that adds a
692layer named ``meta-scottrif`` to the configuration file. Following the
693command that adds the layer is another ``bitbake-layers`` command that
694shows the layers that are in your ``bblayers.conf`` file: $
695bitbake-layers add-layer meta-scottrif NOTE: Starting bitbake server...
696Parsing recipes: 100%
697\|##########################################################\| Time:
6980:00:49 Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055
699targets, 56 skipped, 0 masked, 0 errors. $ bitbake-layers show-layers
700NOTE: Starting bitbake server... layer path priority
701==========================================================================
702meta /home/scottrif/poky/meta 5 meta-poky /home/scottrif/poky/meta-poky
7035 meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5 workspace
704/home/scottrif/poky/build/workspace 99 meta-scottrif
705/home/scottrif/poky/build/meta-scottrif 6 Adding the layer to this file
706enables the build system to locate the layer during the build.
707
708.. note::
709
710 During a build, the OpenEmbedded build system looks in the layers
711 from the top of the list down to the bottom in that order.
712
713.. _usingpoky-extend-customimage:
714
715Customizing Images
716==================
717
718You can customize images to satisfy particular requirements. This
719section describes several methods and provides guidelines for each.
720
721.. _usingpoky-extend-customimage-localconf:
722
723Customizing Images Using ``local.conf``
724---------------------------------------
725
726Probably the easiest way to customize an image is to add a package by
727way of the ``local.conf`` configuration file. Because it is limited to
728local use, this method generally only allows you to add packages and is
729not as flexible as creating your own customized image. When you add
730packages using local variables this way, you need to realize that these
731variable changes are in effect for every build and consequently affect
732all images, which might not be what you require.
733
734To add a package to your image using the local configuration file, use
735the ``IMAGE_INSTALL`` variable with the ``_append`` operator:
736IMAGE_INSTALL_append = " strace" Use of the syntax is important -
737specifically, the space between the quote and the package name, which is
738``strace`` in this example. This space is required since the ``_append``
739operator does not add the space.
740
741Furthermore, you must use ``_append`` instead of the ``+=`` operator if
742you want to avoid ordering issues. The reason for this is because doing
743so unconditionally appends to the variable and avoids ordering problems
744due to the variable being set in image recipes and ``.bbclass`` files
745with operators like ``?=``. Using ``_append`` ensures the operation
746takes affect.
747
748As shown in its simplest use, ``IMAGE_INSTALL_append`` affects all
749images. It is possible to extend the syntax so that the variable applies
750to a specific image only. Here is an example:
751IMAGE_INSTALL_append_pn-core-image-minimal = " strace" This example adds
752``strace`` to the ``core-image-minimal`` image only.
753
754You can add packages using a similar approach through the
755``CORE_IMAGE_EXTRA_INSTALL`` variable. If you use this variable, only
756``core-image-*`` images are affected.
757
758.. _usingpoky-extend-customimage-imagefeatures:
759
760Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES``
761-------------------------------------------------------------------------------
762
763Another method for customizing your image is to enable or disable
764high-level image features by using the
765```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ and
766```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
767variables. Although the functions for both variables are nearly
768equivalent, best practices dictate using ``IMAGE_FEATURES`` from within
769a recipe and using ``EXTRA_IMAGE_FEATURES`` from within your
770``local.conf`` file, which is found in the `Build
771Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
772
773To understand how these features work, the best reference is
774``meta/classes/core-image.bbclass``. This class lists out the available
775``IMAGE_FEATURES`` of which most map to package groups while some, such
776as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general
777configuration settings.
778
779In summary, the file looks at the contents of the ``IMAGE_FEATURES``
780variable and then maps or configures the feature accordingly. Based on
781this information, the build system automatically adds the appropriate
782packages or configurations to the
783```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__ variable.
784Effectively, you are enabling extra features by extending the class or
785creating a custom class for use with specialized image ``.bb`` files.
786
787Use the ``EXTRA_IMAGE_FEATURES`` variable from within your local
788configuration file. Using a separate area from which to enable features
789with this variable helps you avoid overwriting the features in the image
790recipe that are enabled with ``IMAGE_FEATURES``. The value of
791``EXTRA_IMAGE_FEATURES`` is added to ``IMAGE_FEATURES`` within
792``meta/conf/bitbake.conf``.
793
794To illustrate how you can use these variables to modify your image,
795consider an example that selects the SSH server. The Yocto Project ships
796with two SSH servers you can use with your images: Dropbear and OpenSSH.
797Dropbear is a minimal SSH server appropriate for resource-constrained
798environments, while OpenSSH is a well-known standard SSH server
799implementation. By default, the ``core-image-sato`` image is configured
800to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb``
801images both include OpenSSH. The ``core-image-minimal`` image does not
802contain an SSH server.
803
804You can customize your image and change these defaults. Edit the
805``IMAGE_FEATURES`` variable in your recipe or use the
806``EXTRA_IMAGE_FEATURES`` in your ``local.conf`` file so that it
807configures the image you are working with to include
808``ssh-server-dropbear`` or ``ssh-server-openssh``.
809
810.. note::
811
812 See the "
813 Images
814 " section in the Yocto Project Reference Manual for a complete list
815 of image features that ship with the Yocto Project.
816
817.. _usingpoky-extend-customimage-custombb:
818
819Customizing Images Using Custom .bb Files
820-----------------------------------------
821
822You can also customize an image by creating a custom recipe that defines
823additional software as part of the image. The following example shows
824the form for the two lines you need: IMAGE_INSTALL =
825"packagegroup-core-x11-base package1 package2" inherit core-image
826
827Defining the software using a custom recipe gives you total control over
828the contents of the image. It is important to use the correct names of
829packages in the ``IMAGE_INSTALL`` variable. You must use the
830OpenEmbedded notation and not the Debian notation for the names (e.g.
831``glibc-dev`` instead of ``libc6-dev``).
832
833The other method for creating a custom image is to base it on an
834existing image. For example, if you want to create an image based on
835``core-image-sato`` but add the additional package ``strace`` to the
836image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new
837``.bb`` and add the following line to the end of the copy: IMAGE_INSTALL
838+= "strace"
839
840.. _usingpoky-extend-customimage-customtasks:
841
842Customizing Images Using Custom Package Groups
843----------------------------------------------
844
845For complex custom images, the best approach for customizing an image is
846to create a custom package group recipe that is used to build the image
847or images. A good example of a package group recipe is
848``meta/recipes-core/packagegroups/packagegroup-base.bb``.
849
850If you examine that recipe, you see that the ``PACKAGES`` variable lists
851the package group packages to produce. The ``inherit packagegroup``
852statement sets appropriate default values and automatically adds
853``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each
854package specified in the ``PACKAGES`` statement.
855
856.. note::
857
858 The
859 inherit packagegroup
860 line should be located near the top of the recipe, certainly before
861 the
862 PACKAGES
863 statement.
864
865For each package you specify in ``PACKAGES``, you can use ``RDEPENDS``
866and ``RRECOMMENDS`` entries to provide a list of packages the parent
867task package should contain. You can see examples of these further down
868in the ``packagegroup-base.bb`` recipe.
869
870Here is a short, fabricated example showing the same basic pieces for a
871hypothetical packagegroup defined in ``packagegroup-custom.bb``, where
872the variable ``PN`` is the standard way to abbreviate the reference to
873the full packagegroup name ``packagegroup-custom``: DESCRIPTION = "My
874Custom Package Groups" inherit packagegroup PACKAGES = "\\ ${PN}-apps \\
875${PN}-tools \\ " RDEPENDS_${PN}-apps = "\\ dropbear \\ portmap \\
876psplash" RDEPENDS_${PN}-tools = "\\ oprofile \\ oprofileui-server \\
877lttng-tools" RRECOMMENDS_${PN}-tools = "\\ kernel-module-oprofile"
878
879In the previous example, two package group packages are created with
880their dependencies and their recommended package dependencies listed:
881``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To
882build an image using these package group packages, you need to add
883``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to
884``IMAGE_INSTALL``. For other forms of image dependencies see the other
885areas of this section.
886
887.. _usingpoky-extend-customimage-image-name:
888
889Customizing an Image Hostname
890-----------------------------
891
892By default, the configured hostname (i.e. ``/etc/hostname``) in an image
893is the same as the machine name. For example, if
894```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ equals "qemux86", the
895configured hostname written to ``/etc/hostname`` is "qemux86".
896
897You can customize this name by altering the value of the "hostname"
898variable in the ``base-files`` recipe using either an append file or a
899configuration file. Use the following in an append file:
900hostname="myhostname" Use the following in a configuration file:
901hostname_pn-base-files = "myhostname"
902
903Changing the default value of the variable "hostname" can be useful in
904certain situations. For example, suppose you need to do extensive
905testing on an image and you would like to easily identify the image
906under test from existing images with typical default hostnames. In this
907situation, you could change the default hostname to "testme", which
908results in all the images using the name "testme". Once testing is
909complete and you do not need to rebuild the image for test any longer,
910you can easily reset the default hostname.
911
912Another point of interest is that if you unset the variable, the image
913will have no default hostname in the filesystem. Here is an example that
914unsets the variable in a configuration file: hostname_pn-base-files = ""
915Having no default hostname in the filesystem is suitable for
916environments that use dynamic hostnames such as virtual machines.
917
918.. _new-recipe-writing-a-new-recipe:
919
920Writing a New Recipe
921====================
922
923Recipes (``.bb`` files) are fundamental components in the Yocto Project
924environment. Each software component built by the OpenEmbedded build
925system requires a recipe to define the component. This section describes
926how to create, write, and test a new recipe.
927
928.. note::
929
930 For information on variables that are useful for recipes and for
931 information about recipe naming issues, see the "
932 Required
933 " section of the Yocto Project Reference Manual.
934
935.. _new-recipe-overview:
936
937Overview
938--------
939
940The following figure shows the basic process for creating a new recipe.
941The remainder of the section provides details for the steps.
942
943.. _new-recipe-locate-or-automatically-create-a-base-recipe:
944
945Locate or Automatically Create a Base Recipe
946--------------------------------------------
947
948You can always write a recipe from scratch. However, three choices exist
949that can help you quickly get a start on a new recipe:
950
951- *``devtool add``:* A command that assists in creating a recipe and an
952 environment conducive to development.
953
954- *``recipetool create``:* A command provided by the Yocto Project that
955 automates creation of a base recipe based on the source files.
956
957- *Existing Recipes:* Location and modification of an existing recipe
958 that is similar in function to the recipe you need.
959
960.. note::
961
962 For information on recipe syntax, see the "
963 Recipe Syntax
964 " section.
965
966.. _new-recipe-creating-the-base-recipe-using-devtool:
967
968Creating the Base Recipe Using ``devtool add``
969~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
970
971The ``devtool add`` command uses the same logic for auto-creating the
972recipe as ``recipetool create``, which is listed below. Additionally,
973however, ``devtool add`` sets up an environment that makes it easy for
974you to patch the source and to make changes to the recipe as is often
975necessary when adding a recipe to build a new piece of software to be
976included in a build.
977
978You can find a complete description of the ``devtool add`` command in
979the "`A Closer Look at ``devtool``
980add <&YOCTO_DOCS_SDK_URL;#sdk-a-closer-look-at-devtool-add>`__" section
981in the Yocto Project Application Development and the Extensible Software
982Development Kit (eSDK) manual.
983
984.. _new-recipe-creating-the-base-recipe-using-recipetool:
985
986Creating the Base Recipe Using ``recipetool create``
987~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
988
989``recipetool create`` automates creation of a base recipe given a set of
990source code files. As long as you can extract or point to the source
991files, the tool will construct a recipe and automatically configure all
992pre-build information into the recipe. For example, suppose you have an
993application that builds using Autotools. Creating the base recipe using
994``recipetool`` results in a recipe that has the pre-build dependencies,
995license requirements, and checksums configured.
996
997To run the tool, you just need to be in your `Build
998Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ and have sourced the
999build environment setup script (i.e.
1000```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__).
1001To get help on the tool, use the following command: $ recipetool -h
1002NOTE: Starting bitbake server... usage: recipetool [-d] [-q] [--color
1003COLOR] [-h] <subcommand> ... OpenEmbedded recipe tool options: -d,
1004--debug Enable debug output -q, --quiet Print only errors --color COLOR
1005Colorize output (where COLOR is auto, always, never) -h, --help show
1006this help message and exit subcommands: create Create a new recipe
1007newappend Create a bbappend for the specified target in the specified
1008layer setvar Set a variable within a recipe appendfile Create/update a
1009bbappend to replace a target file appendsrcfiles Create/update a
1010bbappend to add or replace source files appendsrcfile Create/update a
1011bbappend to add or replace a source file Use recipetool <subcommand>
1012--help to get help on a specific command
1013
1014Running ``recipetool create -o`` OUTFILE creates the base recipe and
1015locates it properly in the layer that contains your source files.
1016Following are some syntax examples:
1017
1018Use this syntax to generate a recipe based on source. Once generated,
1019the recipe resides in the existing source code layer: recipetool create
1020-o OUTFILE source Use this syntax to generate a recipe using code that
1021you extract from source. The extracted code is placed in its own layer
1022defined by EXTERNALSRC. recipetool create -o OUTFILE -x EXTERNALSRC
1023source Use this syntax to generate a recipe based on source. The options
1024direct ``recipetool`` to generate debugging information. Once generated,
1025the recipe resides in the existing source code layer: recipetool create
1026-d -o OUTFILE source
1027
1028.. _new-recipe-locating-and-using-a-similar-recipe:
1029
1030Locating and Using a Similar Recipe
1031~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1032
1033Before writing a recipe from scratch, it is often useful to discover
1034whether someone else has already written one that meets (or comes close
1035to meeting) your needs. The Yocto Project and OpenEmbedded communities
1036maintain many recipes that might be candidates for what you are doing.
1037You can find a good central index of these recipes in the `OpenEmbedded
1038Layer Index <http://layers.openembedded.org>`__.
1039
1040Working from an existing recipe or a skeleton recipe is the best way to
1041get started. Here are some points on both methods:
1042
1043- *Locate and modify a recipe that is close to what you want to do:*
1044 This method works when you are familiar with the current recipe
1045 space. The method does not work so well for those new to the Yocto
1046 Project or writing recipes.
1047
1048 Some risks associated with this method are using a recipe that has
1049 areas totally unrelated to what you are trying to accomplish with
1050 your recipe, not recognizing areas of the recipe that you might have
1051 to add from scratch, and so forth. All these risks stem from
1052 unfamiliarity with the existing recipe space.
1053
1054- *Use and modify the following skeleton recipe:* If for some reason
1055 you do not want to use ``recipetool`` and you cannot find an existing
1056 recipe that is close to meeting your needs, you can use the following
1057 structure to provide the fundamental areas of a new recipe.
1058 DESCRIPTION = "" HOMEPAGE = "" LICENSE = "" SECTION = "" DEPENDS = ""
1059 LIC_FILES_CHKSUM = "" SRC_URI = ""
1060
1061.. _new-recipe-storing-and-naming-the-recipe:
1062
1063Storing and Naming the Recipe
1064-----------------------------
1065
1066Once you have your base recipe, you should put it in your own layer and
1067name it appropriately. Locating it correctly ensures that the
1068OpenEmbedded build system can find it when you use BitBake to process
1069the recipe.
1070
1071- *Storing Your Recipe:* The OpenEmbedded build system locates your
1072 recipe through the layer's ``conf/layer.conf`` file and the
1073 ```BBFILES`` <&YOCTO_DOCS_REF_URL;#var-BBFILES>`__ variable. This
1074 variable sets up a path from which the build system can locate
1075 recipes. Here is the typical use: BBFILES +=
1076 "${LAYERDIR}/recipes-*/*/*.bb \\ ${LAYERDIR}/recipes-*/*/*.bbappend"
1077 Consequently, you need to be sure you locate your new recipe inside
1078 your layer such that it can be found.
1079
1080 You can find more information on how layers are structured in the
1081 "`Understanding and Creating
1082 Layers <#understanding-and-creating-layers>`__" section.
1083
1084- *Naming Your Recipe:* When you name your recipe, you need to follow
1085 this naming convention: basename_version.bb Use lower-cased
1086 characters and do not include the reserved suffixes ``-native``,
1087 ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use them
1088 as part of your recipe name unless the string applies). Here are some
1089 examples: cups_1.7.0.bb gawk_4.0.2.bb irssi_0.8.16-rc1.bb
1090
1091.. _new-recipe-running-a-build-on-the-recipe:
1092
1093Running a Build on the Recipe
1094-----------------------------
1095
1096Creating a new recipe is usually an iterative process that requires
1097using BitBake to process the recipe multiple times in order to
1098progressively discover and add information to the recipe file.
1099
1100Assuming you have sourced the build environment setup script (i.e.
1101````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) and you are in
1102the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, use
1103BitBake to process your recipe. All you need to provide is the
1104``basename`` of the recipe as described in the previous section: $
1105bitbake basename
1106
1107During the build, the OpenEmbedded build system creates a temporary work
1108directory for each recipe
1109(``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``)
1110where it keeps extracted source files, log files, intermediate
1111compilation and packaging files, and so forth.
1112
1113The path to the per-recipe temporary work directory depends on the
1114context in which it is being built. The quickest way to find this path
1115is to have BitBake return it by running the following: $ bitbake -e
1116basename \| grep ^WORKDIR= As an example, assume a Source Directory
1117top-level folder named ``poky``, a default Build Directory at
1118``poky/build``, and a ``qemux86-poky-linux`` machine target system.
1119Furthermore, suppose your recipe is named ``foo_1.3.0.bb``. In this
1120case, the work directory the build system uses to build the package
1121would be as follows: poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
1122Inside this directory you can find sub-directories such as ``image``,
1123``packages-split``, and ``temp``. After the build, you can examine these
1124to determine how well the build went.
1125
1126.. note::
1127
1128 You can find log files for each task in the recipe's
1129 temp
1130 directory (e.g.
1131 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp
1132 ). Log files are named
1133 log.
1134 taskname
1135 (e.g.
1136 log.do_configure
1137 ,
1138 log.do_fetch
1139 , and
1140 log.do_compile
1141 ).
1142
1143You can find more information about the build process in "`The Yocto
1144Project Development
1145Environment <&YOCTO_DOCS_OM_URL;#overview-development-environment>`__"
1146chapter of the Yocto Project Overview and Concepts Manual.
1147
1148.. _new-recipe-fetching-code:
1149
1150Fetching Code
1151-------------
1152
1153The first thing your recipe must do is specify how to fetch the source
1154files. Fetching is controlled mainly through the
1155```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. Your recipe
1156must have a ``SRC_URI`` variable that points to where the source is
1157located. For a graphical representation of source locations, see the
1158"`Sources <&YOCTO_DOCS_OM_URL;#sources-dev-environment>`__" section in
1159the Yocto Project Overview and Concepts Manual.
1160
1161The ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ task uses
1162the prefix of each entry in the ``SRC_URI`` variable value to determine
1163which `fetcher <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__ to use to get your
1164source files. It is the ``SRC_URI`` variable that triggers the fetcher.
1165The ```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task uses
1166the variable after source is fetched to apply patches. The OpenEmbedded
1167build system uses
1168```FILESOVERRIDES`` <&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES>`__ for
1169scanning directory locations for local files in ``SRC_URI``.
1170
1171The ``SRC_URI`` variable in your recipe must define each unique location
1172for your source files. It is good practice to not hard-code version
1173numbers in a URL used in ``SRC_URI``. Rather than hard-code these
1174values, use ``${``\ ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__\ ``}``,
1175which causes the fetch process to use the version specified in the
1176recipe filename. Specifying the version in this manner means that
1177upgrading the recipe to a future version is as simple as renaming the
1178recipe to match the new version.
1179
1180Here is a simple example from the
1181``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source
1182comes from a single tarball. Notice the use of the
1183```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ variable: SRC_URI =
1184"https://strace.io/files/${PV}/strace-${PV}.tar.xz \\
1185
1186Files mentioned in ``SRC_URI`` whose names end in a typical archive
1187extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so
1188forth), are automatically extracted during the
1189```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ task. For
1190another example that specifies these types of files, see the
1191"`Autotooled Package <#new-recipe-autotooled-package>`__" section.
1192
1193Another way of specifying source is from an SCM. For Git repositories,
1194you must specify ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ and
1195you should specify ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ to include
1196the revision with ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__. Here
1197is an example from the recipe
1198``meta/recipes-kernel/blktrace/blktrace_git.bb``: SRCREV =
1199"d6918c8832793b4205ed3bfede78c2f915c23385" PR = "r6" PV =
1200"1.0.5+git${SRCPV}" SRC_URI = "git://git.kernel.dk/blktrace.git \\
1201file://ldflags.patch"
1202
1203If your ``SRC_URI`` statement includes URLs pointing to individual files
1204fetched from a remote server other than a version control system,
1205BitBake attempts to verify the files against checksums defined in your
1206recipe to ensure they have not been tampered with or otherwise modified
1207since the recipe was written. Two checksums are used:
1208``SRC_URI[md5sum]`` and ``SRC_URI[sha256sum]``.
1209
1210If your ``SRC_URI`` variable points to more than a single URL (excluding
1211SCM URLs), you need to provide the ``md5`` and ``sha256`` checksums for
1212each URL. For these cases, you provide a name for each URL as part of
1213the ``SRC_URI`` and then reference that name in the subsequent checksum
1214statements. Here is an example combining lines from the files
1215``git.inc`` and ``git_2.24.1.bb``: SRC_URI =
1216"${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \\
1217${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages"
1218SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b"
1219SRC_URI[tarball.sha256sum] =
1220"ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02"
1221SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d"
1222SRC_URI[manpages.sha256sum] =
1223"9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230"
1224
1225Proper values for ``md5`` and ``sha256`` checksums might be available
1226with other signatures on the download page for the upstream source (e.g.
1227``md5``, ``sha1``, ``sha256``, ``GPG``, and so forth). Because the
1228OpenEmbedded build system only deals with ``sha256sum`` and ``md5sum``,
1229you should verify all the signatures you find by hand.
1230
1231If no ``SRC_URI`` checksums are specified when you attempt to build the
1232recipe, or you provide an incorrect checksum, the build will produce an
1233error for each missing or incorrect checksum. As part of the error
1234message, the build system provides the checksum string corresponding to
1235the fetched file. Once you have the correct checksums, you can copy and
1236paste them into your recipe and then run the build again to continue.
1237
1238.. note::
1239
1240 As mentioned, if the upstream source provides signatures for
1241 verifying the downloaded source code, you should verify those
1242 manually before setting the checksum values in the recipe and
1243 continuing with the build.
1244
1245This final example is a bit more complicated and is from the
1246``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The
1247example's ``SRC_URI`` statement identifies multiple files as the source
1248files for the recipe: a tarball, a patch file, a desktop file, and an
1249icon. SRC_URI =
1250"http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \\
1251file://xwc.patch \\ file://rxvt.desktop \\ file://rxvt.png"
1252
1253When you specify local files using the ``file://`` URI protocol, the
1254build system fetches files from the local machine. The path is relative
1255to the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ variable
1256and searches specific directories in a certain order:
1257``${``\ ```BP`` <&YOCTO_DOCS_REF_URL;#var-BP>`__\ ``}``,
1258``${``\ ```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__\ ``}``, and
1259``files``. The directories are assumed to be subdirectories of the
1260directory in which the recipe or append file resides. For another
1261example that specifies these types of files, see the "`Single .c File
1262Package (Hello
1263World!) <#new-recipe-single-c-file-package-hello-world>`__" section.
1264
1265The previous example also specifies a patch file. Patch files are files
1266whose names usually end in ``.patch`` or ``.diff`` but can end with
1267compressed suffixes such as ``diff.gz`` and ``patch.bz2``, for example.
1268The build system automatically applies patches as described in the
1269"`Patching Code <#new-recipe-patching-code>`__" section.
1270
1271.. _new-recipe-unpacking-code:
1272
1273Unpacking Code
1274--------------
1275
1276During the build, the
1277```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ task unpacks
1278the source with ``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``
1279pointing to where it is unpacked.
1280
1281If you are fetching your source files from an upstream source archived
1282tarball and the tarball's internal structure matches the common
1283convention of a top-level subdirectory named
1284``${``\ ```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__\ ``}-${``\ ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__\ ``}``,
1285then you do not need to set ``S``. However, if ``SRC_URI`` specifies to
1286fetch source from an archive that does not use this convention, or from
1287an SCM like Git or Subversion, your recipe needs to define ``S``.
1288
1289If processing your recipe using BitBake successfully unpacks the source
1290files, you need to be sure that the directory pointed to by ``${S}``
1291matches the structure of the source.
1292
1293.. _new-recipe-patching-code:
1294
1295Patching Code
1296-------------
1297
1298Sometimes it is necessary to patch code after it has been fetched. Any
1299files mentioned in ``SRC_URI`` whose names end in ``.patch`` or
1300``.diff`` or compressed versions of these suffixes (e.g. ``diff.gz`` are
1301treated as patches. The
1302```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task
1303automatically applies these patches.
1304
1305The build system should be able to apply patches with the "-p1" option
1306(i.e. one directory level in the path will be stripped off). If your
1307patch needs to have more directory levels stripped off, specify the
1308number of levels using the "striplevel" option in the ``SRC_URI`` entry
1309for the patch. Alternatively, if your patch needs to be applied in a
1310specific subdirectory that is not specified in the patch file, use the
1311"patchdir" option in the entry.
1312
1313As with all local files referenced in
1314```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ using ``file://``,
1315you should place patch files in a directory next to the recipe either
1316named the same as the base name of the recipe
1317(```BP`` <&YOCTO_DOCS_REF_URL;#var-BP>`__ and
1318```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__) or "files".
1319
1320.. _new-recipe-licensing:
1321
1322Licensing
1323---------
1324
1325Your recipe needs to have both the
1326```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ and
1327```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
1328variables:
1329
1330- *``LICENSE``:* This variable specifies the license for the software.
1331 If you do not know the license under which the software you are
1332 building is distributed, you should go to the source code and look
1333 for that information. Typical files containing this information
1334 include ``COPYING``, ``LICENSE``, and ``README`` files. You could
1335 also find the information near the top of a source file. For example,
1336 given a piece of software licensed under the GNU General Public
1337 License version 2, you would set ``LICENSE`` as follows: LICENSE =
1338 "GPLv2"
1339
1340 The licenses you specify within ``LICENSE`` can have any name as long
1341 as you do not use spaces, since spaces are used as separators between
1342 license names. For standard licenses, use the names of the files in
1343 ``meta/files/common-licenses/`` or the ``SPDXLICENSEMAP`` flag names
1344 defined in ``meta/conf/licenses.conf``.
1345
1346- *``LIC_FILES_CHKSUM``:* The OpenEmbedded build system uses this
1347 variable to make sure the license text has not changed. If it has,
1348 the build produces an error and it affords you the chance to figure
1349 it out and correct the problem.
1350
1351 You need to specify all applicable licensing files for the software.
1352 At the end of the configuration step, the build process will compare
1353 the checksums of the files to be sure the text has not changed. Any
1354 differences result in an error with the message containing the
1355 current checksum. For more explanation and examples of how to set the
1356 ``LIC_FILES_CHKSUM`` variable, see the "`Tracking License
1357 Changes <#>`__" section.
1358
1359 To determine the correct checksum string, you can list the
1360 appropriate files in the ``LIC_FILES_CHKSUM`` variable with incorrect
1361 md5 strings, attempt to build the software, and then note the
1362 resulting error messages that will report the correct md5 strings.
1363 See the "`Fetching Code <#new-recipe-fetching-code>`__" section for
1364 additional information.
1365
1366 Here is an example that assumes the software has a ``COPYING`` file:
1367 LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" When you try to build the
1368 software, the build system will produce an error and give you the
1369 correct string that you can substitute into the recipe file for a
1370 subsequent build.
1371
1372.. _new-dependencies:
1373
1374Dependencies
1375------------
1376
1377Most software packages have a short list of other packages that they
1378require, which are called dependencies. These dependencies fall into two
1379main categories: build-time dependencies, which are required when the
1380software is built; and runtime dependencies, which are required to be
1381installed on the target in order for the software to run.
1382
1383Within a recipe, you specify build-time dependencies using the
1384```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable. Although
1385nuances exist, items specified in ``DEPENDS`` should be names of other
1386recipes. It is important that you specify all build-time dependencies
1387explicitly. If you do not, due to the parallel nature of BitBake's
1388execution, you can end up with a race condition where the dependency is
1389present for one task of a recipe (e.g.
1390```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__) and
1391then gone when the next task runs (e.g.
1392```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__).
1393
1394Another consideration is that configure scripts might automatically
1395check for optional dependencies and enable corresponding functionality
1396if those dependencies are found. This behavior means that to ensure
1397deterministic results and thus avoid more race conditions, you need to
1398either explicitly specify these dependencies as well, or tell the
1399configure script explicitly to disable the functionality. If you wish to
1400make a recipe that is more generally useful (e.g. publish the recipe in
1401a layer for others to use), instead of hard-disabling the functionality,
1402you can use the
1403```PACKAGECONFIG`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG>`__ variable
1404to allow functionality and the corresponding dependencies to be enabled
1405and disabled easily by other users of the recipe.
1406
1407Similar to build-time dependencies, you specify runtime dependencies
1408through a variable -
1409```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__, which is
1410package-specific. All variables that are package-specific need to have
1411the name of the package added to the end as an override. Since the main
1412package for a recipe has the same name as the recipe, and the recipe's
1413name can be found through the
1414``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` variable, then
1415you specify the dependencies for the main package by setting
1416``RDEPENDS_${PN}``. If the package were named ``${PN}-tools``, then you
1417would set ``RDEPENDS_${PN}-tools``, and so forth.
1418
1419Some runtime dependencies will be set automatically at packaging time.
1420These dependencies include any shared library dependencies (i.e. if a
1421package "example" contains "libexample" and another package "mypackage"
1422contains a binary that links to "libexample" then the OpenEmbedded build
1423system will automatically add a runtime dependency to "mypackage" on
1424"example"). See the "`Automatically Added Runtime
1425Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
1426section in the Yocto Project Overview and Concepts Manual for further
1427details.
1428
1429.. _new-recipe-configuring-the-recipe:
1430
1431Configuring the Recipe
1432----------------------
1433
1434Most software provides some means of setting build-time configuration
1435options before compilation. Typically, setting these options is
1436accomplished by running a configure script with options, or by modifying
1437a build configuration file.
1438
1439.. note::
1440
1441 As of Yocto Project Release 1.7, some of the core recipes that
1442 package binary configuration scripts now disable the scripts due to
1443 the scripts previously requiring error-prone path substitution. The
1444 OpenEmbedded build system uses
1445 pkg-config
1446 now, which is much more robust. You can find a list of the
1447 \*-config
1448 scripts that are disabled list in the "
1449 Binary Configuration Scripts Disabled
1450 " section in the Yocto Project Reference Manual.
1451
1452A major part of build-time configuration is about checking for
1453build-time dependencies and possibly enabling optional functionality as
1454a result. You need to specify any build-time dependencies for the
1455software you are building in your recipe's
1456```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ value, in terms of
1457other recipes that satisfy those dependencies. You can often find
1458build-time or runtime dependencies described in the software's
1459documentation.
1460
1461The following list provides configuration items of note based on how
1462your software is built:
1463
1464- *Autotools:* If your source files have a ``configure.ac`` file, then
1465 your software is built using Autotools. If this is the case, you just
1466 need to worry about modifying the configuration.
1467
1468 When using Autotools, your recipe needs to inherit the
1469 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class
1470 and your recipe does not have to contain a
1471 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task.
1472 However, you might still want to make some adjustments. For example,
1473 you can set
1474 ```EXTRA_OECONF`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF>`__ or
1475 ```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
1476 to pass any needed configure options that are specific to the recipe.
1477
1478- *CMake:* If your source files have a ``CMakeLists.txt`` file, then
1479 your software is built using CMake. If this is the case, you just
1480 need to worry about modifying the configuration.
1481
1482 When you use CMake, your recipe needs to inherit the
1483 ```cmake`` <&YOCTO_DOCS_REF_URL;#ref-classes-cmake>`__ class and your
1484 recipe does not have to contain a
1485 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task.
1486 You can make some adjustments by setting
1487 ```EXTRA_OECMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE>`__ to
1488 pass any needed configure options that are specific to the recipe.
1489
1490 .. note::
1491
1492 If you need to install one or more custom CMake toolchain files
1493 that are supplied by the application you are building, install the
1494 files to
1495 ${D}${datadir}/cmake/
1496 Modules during
1497 do_install
1498 .
1499
1500- *Other:* If your source files do not have a ``configure.ac`` or
1501 ``CMakeLists.txt`` file, then your software is built using some
1502 method other than Autotools or CMake. If this is the case, you
1503 normally need to provide a
1504 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task
1505 in your recipe unless, of course, there is nothing to configure.
1506
1507 Even if your software is not being built by Autotools or CMake, you
1508 still might not need to deal with any configuration issues. You need
1509 to determine if configuration is even a required step. You might need
1510 to modify a Makefile or some configuration file used for the build to
1511 specify necessary build options. Or, perhaps you might need to run a
1512 provided, custom configure script with the appropriate options.
1513
1514 For the case involving a custom configure script, you would run
1515 ``./configure --help`` and look for the options you need to set.
1516
1517Once configuration succeeds, it is always good practice to look at the
1518``log.do_configure`` file to ensure that the appropriate options have
1519been enabled and no additional build-time dependencies need to be added
1520to ``DEPENDS``. For example, if the configure script reports that it
1521found something not mentioned in ``DEPENDS``, or that it did not find
1522something that it needed for some desired optional functionality, then
1523you would need to add those to ``DEPENDS``. Looking at the log might
1524also reveal items being checked for, enabled, or both that you do not
1525want, or items not being found that are in ``DEPENDS``, in which case
1526you would need to look at passing extra options to the configure script
1527as needed. For reference information on configure options specific to
1528the software you are building, you can consult the output of the
1529``./configure --help`` command within ``${S}`` or consult the software's
1530upstream documentation.
1531
1532.. _new-recipe-using-headers-to-interface-with-devices:
1533
1534Using Headers to Interface with Devices
1535---------------------------------------
1536
1537If your recipe builds an application that needs to communicate with some
1538device or needs an API into a custom kernel, you will need to provide
1539appropriate header files. Under no circumstances should you ever modify
1540the existing
1541``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc`` file.
1542These headers are used to build ``libc`` and must not be compromised
1543with custom or machine-specific header information. If you customize
1544``libc`` through modified headers all other applications that use
1545``libc`` thus become affected.
1546
1547.. note::
1548
1549 Never copy and customize the
1550 libc
1551 header file (i.e.
1552 meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
1553 ).
1554
1555The correct way to interface to a device or custom kernel is to use a
1556separate package that provides the additional headers for the driver or
1557other unique interfaces. When doing so, your application also becomes
1558responsible for creating a dependency on that specific provider.
1559
1560Consider the following:
1561
1562- Never modify ``linux-libc-headers.inc``. Consider that file to be
1563 part of the ``libc`` system, and not something you use to access the
1564 kernel directly. You should access ``libc`` through specific ``libc``
1565 calls.
1566
1567- Applications that must talk directly to devices should either provide
1568 necessary headers themselves, or establish a dependency on a special
1569 headers package that is specific to that driver.
1570
1571For example, suppose you want to modify an existing header that adds I/O
1572control or network support. If the modifications are used by a small
1573number programs, providing a unique version of a header is easy and has
1574little impact. When doing so, bear in mind the guidelines in the
1575previous list.
1576
1577.. note::
1578
1579 If for some reason your changes need to modify the behavior of the
1580 libc
1581 , and subsequently all other applications on the system, use a
1582 .bbappend
1583 to modify the
1584 linux-kernel-headers.inc
1585 file. However, take care to not make the changes machine specific.
1586
1587Consider a case where your kernel is older and you need an older
1588``libc`` ABI. The headers installed by your recipe should still be a
1589standard mainline kernel, not your own custom one.
1590
1591When you use custom kernel headers you need to get them from
1592```STAGING_KERNEL_DIR`` <&YOCTO_DOCS_REF_URL;#var-STAGING_KERNEL_DIR>`__,
1593which is the directory with kernel headers that are required to build
1594out-of-tree modules. Your recipe will also need the following:
1595do_configure[depends] += "virtual/kernel:do_shared_workdir"
1596
1597.. _new-recipe-compilation:
1598
1599Compilation
1600-----------
1601
1602During a build, the ``do_compile`` task happens after source is fetched,
1603unpacked, and configured. If the recipe passes through ``do_compile``
1604successfully, nothing needs to be done.
1605
1606However, if the compile step fails, you need to diagnose the failure.
1607Here are some common issues that cause failures.
1608
1609.. note::
1610
1611 For cases where improper paths are detected for configuration files
1612 or for when libraries/headers cannot be found, be sure you are using
1613 the more robust
1614 pkg-config
1615 . See the note in section "
1616 Configuring the Recipe
1617 " for additional information.
1618
1619- *Parallel build failures:* These failures manifest themselves as
1620 intermittent errors, or errors reporting that a file or directory
1621 that should be created by some other part of the build process could
1622 not be found. This type of failure can occur even if, upon
1623 inspection, the file or directory does exist after the build has
1624 failed, because that part of the build process happened in the wrong
1625 order.
1626
1627 To fix the problem, you need to either satisfy the missing dependency
1628 in the Makefile or whatever script produced the Makefile, or (as a
1629 workaround) set
1630 ```PARALLEL_MAKE`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ to an
1631 empty string: PARALLEL_MAKE = ""
1632
1633 For information on parallel Makefile issues, see the "`Debugging
1634 Parallel Make Races <#debugging-parallel-make-races>`__" section.
1635
1636- *Improper host path usage:* This failure applies to recipes building
1637 for the target or ``nativesdk`` only. The failure occurs when the
1638 compilation process uses improper headers, libraries, or other files
1639 from the host system when cross-compiling for the target.
1640
1641 To fix the problem, examine the ``log.do_compile`` file to identify
1642 the host paths being used (e.g. ``/usr/include``, ``/usr/lib``, and
1643 so forth) and then either add configure options, apply a patch, or do
1644 both.
1645
1646- *Failure to find required libraries/headers:* If a build-time
1647 dependency is missing because it has not been declared in
1648 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__, or because the
1649 dependency exists but the path used by the build process to find the
1650 file is incorrect and the configure step did not detect it, the
1651 compilation process could fail. For either of these failures, the
1652 compilation process notes that files could not be found. In these
1653 cases, you need to go back and add additional options to the
1654 configure script as well as possibly add additional build-time
1655 dependencies to ``DEPENDS``.
1656
1657 Occasionally, it is necessary to apply a patch to the source to
1658 ensure the correct paths are used. If you need to specify paths to
1659 find files staged into the sysroot from other recipes, use the
1660 variables that the OpenEmbedded build system provides (e.g.
1661 ``STAGING_BINDIR``, ``STAGING_INCDIR``, ``STAGING_DATADIR``, and so
1662 forth).
1663
1664.. _new-recipe-installing:
1665
1666Installing
1667----------
1668
1669During ``do_install``, the task copies the built files along with their
1670hierarchy to locations that would mirror their locations on the target
1671device. The installation process copies files from the
1672``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``,
1673``${``\ ```B`` <&YOCTO_DOCS_REF_URL;#var-B>`__\ ``}``, and
1674``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``
1675directories to the ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}``
1676directory to create the structure as it should appear on the target
1677system.
1678
1679How your software is built affects what you must do to be sure your
1680software is installed correctly. The following list describes what you
1681must do for installation depending on the type of build system used by
1682the software being built:
1683
1684- *Autotools and CMake:* If the software your recipe is building uses
1685 Autotools or CMake, the OpenEmbedded build system understands how to
1686 install the software. Consequently, you do not have to have a
1687 ``do_install`` task as part of your recipe. You just need to make
1688 sure the install portion of the build completes with no issues.
1689 However, if you wish to install additional files not already being
1690 installed by ``make install``, you should do this using a
1691 ``do_install_append`` function using the install command as described
1692 in the "Manual" bulleted item later in this list.
1693
1694- *Other (using ``make install``):* You need to define a ``do_install``
1695 function in your recipe. The function should call
1696 ``oe_runmake install`` and will likely need to pass in the
1697 destination directory as well. How you pass that path is dependent on
1698 how the ``Makefile`` being run is written (e.g. ``DESTDIR=${D}``,
1699 ``PREFIX=${D}``, ``INSTALLROOT=${D}``, and so forth).
1700
1701 For an example recipe using ``make install``, see the
1702 "`Makefile-Based Package <#new-recipe-makefile-based-package>`__"
1703 section.
1704
1705- *Manual:* You need to define a ``do_install`` function in your
1706 recipe. The function must first use ``install -d`` to create the
1707 directories under
1708 ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}``. Once the
1709 directories exist, your function can use ``install`` to manually
1710 install the built software into the directories.
1711
1712 You can find more information on ``install`` at
1713 ` <http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html>`__.
1714
1715For the scenarios that do not use Autotools or CMake, you need to track
1716the installation and diagnose and fix any issues until everything
1717installs correctly. You need to look in the default location of
1718``${D}``, which is ``${WORKDIR}/image``, to be sure your files have been
1719installed correctly.
1720
1721.. note::
1722
1723 - During the installation process, you might need to modify some of
1724 the installed files to suit the target layout. For example, you
1725 might need to replace hard-coded paths in an initscript with
1726 values of variables provided by the build system, such as
1727 replacing ``/usr/bin/`` with ``${bindir}``. If you do perform such
1728 modifications during ``do_install``, be sure to modify the
1729 destination file after copying rather than before copying.
1730 Modifying after copying ensures that the build system can
1731 re-execute ``do_install`` if needed.
1732
1733 - ``oe_runmake install``, which can be run directly or can be run
1734 indirectly by the
1735 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ and
1736 ```cmake`` <&YOCTO_DOCS_REF_URL;#ref-classes-cmake>`__ classes,
1737 runs ``make install`` in parallel. Sometimes, a Makefile can have
1738 missing dependencies between targets that can result in race
1739 conditions. If you experience intermittent failures during
1740 ``do_install``, you might be able to work around them by disabling
1741 parallel Makefile installs by adding the following to the recipe:
1742 PARALLEL_MAKEINST = "" See
1743 ```PARALLEL_MAKEINST`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST>`__
1744 for additional information.
1745
1746 - If you need to install one or more custom CMake toolchain files
1747 that are supplied by the application you are building, install the
1748 files to ``${D}${datadir}/cmake/`` Modules during
1749 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__.
1750
1751.. _new-recipe-enabling-system-services:
1752
1753Enabling System Services
1754------------------------
1755
1756If you want to install a service, which is a process that usually starts
1757on boot and runs in the background, then you must include some
1758additional definitions in your recipe.
1759
1760If you are adding services and the service initialization script or the
1761service file itself is not installed, you must provide for that
1762installation in your recipe using a ``do_install_append`` function. If
1763your recipe already has a ``do_install`` function, update the function
1764near its end rather than adding an additional ``do_install_append``
1765function.
1766
1767When you create the installation for your services, you need to
1768accomplish what is normally done by ``make install``. In other words,
1769make sure your installation arranges the output similar to how it is
1770arranged on the target system.
1771
1772The OpenEmbedded build system provides support for starting services two
1773different ways:
1774
1775- *SysVinit:* SysVinit is a system and service manager that manages the
1776 init system used to control the very basic functions of your system.
1777 The init program is the first program started by the Linux kernel
1778 when the system boots. Init then controls the startup, running and
1779 shutdown of all other programs.
1780
1781 To enable a service using SysVinit, your recipe needs to inherit the
1782 ```update-rc.d`` <&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d>`__
1783 class. The class helps facilitate safely installing the package on
1784 the target.
1785
1786 You will need to set the
1787 ```INITSCRIPT_PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES>`__,
1788 ```INITSCRIPT_NAME`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME>`__,
1789 and
1790 ```INITSCRIPT_PARAMS`` <&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS>`__
1791 variables within your recipe.
1792
1793- *systemd:* System Management Daemon (systemd) was designed to replace
1794 SysVinit and to provide enhanced management of services. For more
1795 information on systemd, see the systemd homepage at
1796 ` <http://freedesktop.org/wiki/Software/systemd/>`__.
1797
1798 To enable a service using systemd, your recipe needs to inherit the
1799 ```systemd`` <&YOCTO_DOCS_REF_URL;#ref-classes-systemd>`__ class. See
1800 the ``systemd.bbclass`` file located in your `Source
1801 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. section for
1802 more information.
1803
1804.. _new-recipe-packaging:
1805
1806Packaging
1807---------
1808
1809Successful packaging is a combination of automated processes performed
1810by the OpenEmbedded build system and some specific steps you need to
1811take. The following list describes the process:
1812
1813- *Splitting Files*: The ``do_package`` task splits the files produced
1814 by the recipe into logical components. Even software that produces a
1815 single binary might still have debug symbols, documentation, and
1816 other logical components that should be split out. The ``do_package``
1817 task ensures that files are split up and packaged correctly.
1818
1819- *Running QA Checks*: The
1820 ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__ class adds a
1821 step to the package generation process so that output quality
1822 assurance checks are generated by the OpenEmbedded build system. This
1823 step performs a range of checks to be sure the build's output is free
1824 of common problems that show up during runtime. For information on
1825 these checks, see the
1826 ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__ class and
1827 the "`QA Error and Warning
1828 Messages <&YOCTO_DOCS_REF_URL;#ref-qa-checks>`__" chapter in the
1829 Yocto Project Reference Manual.
1830
1831- *Hand-Checking Your Packages*: After you build your software, you
1832 need to be sure your packages are correct. Examine the
1833 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/packages-split``
1834 directory and make sure files are where you expect them to be. If you
1835 discover problems, you can set
1836 ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__,
1837 ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__,
1838 ``do_install(_append)``, and so forth as needed.
1839
1840- *Splitting an Application into Multiple Packages*: If you need to
1841 split an application into several packages, see the "`Splitting an
1842 Application into Multiple
1843 Packages <#splitting-an-application-into-multiple-packages>`__"
1844 section for an example.
1845
1846- *Installing a Post-Installation Script*: For an example showing how
1847 to install a post-installation script, see the "`Post-Installation
1848 Scripts <#new-recipe-post-installation-scripts>`__" section.
1849
1850- *Marking Package Architecture*: Depending on what your recipe is
1851 building and how it is configured, it might be important to mark the
1852 packages produced as being specific to a particular machine, or to
1853 mark them as not being specific to a particular machine or
1854 architecture at all.
1855
1856 By default, packages apply to any machine with the same architecture
1857 as the target machine. When a recipe produces packages that are
1858 machine-specific (e.g. the
1859 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ value is passed
1860 into the configure script or a patch is applied only for a particular
1861 machine), you should mark them as such by adding the following to the
1862 recipe: PACKAGE_ARCH = "${MACHINE_ARCH}"
1863
1864 On the other hand, if the recipe produces packages that do not
1865 contain anything specific to the target machine or architecture at
1866 all (e.g. recipes that simply package script files or configuration
1867 files), you should use the
1868 ```allarch`` <&YOCTO_DOCS_REF_URL;#ref-classes-allarch>`__ class to
1869 do this for you by adding this to your recipe: inherit allarch
1870 Ensuring that the package architecture is correct is not critical
1871 while you are doing the first few builds of your recipe. However, it
1872 is important in order to ensure that your recipe rebuilds (or does
1873 not rebuild) appropriately in response to changes in configuration,
1874 and to ensure that you get the appropriate packages installed on the
1875 target machine, particularly if you run separate builds for more than
1876 one target machine.
1877
1878.. _new-sharing-files-between-recipes:
1879
1880Sharing Files Between Recipes
1881-----------------------------
1882
1883Recipes often need to use files provided by other recipes on the build
1884host. For example, an application linking to a common library needs
1885access to the library itself and its associated headers. The way this
1886access is accomplished is by populating a sysroot with files. Each
1887recipe has two sysroots in its work directory, one for target files
1888(``recipe-sysroot``) and one for files that are native to the build host
1889(``recipe-sysroot-native``).
1890
1891.. note::
1892
1893 You could find the term "staging" used within the Yocto project
1894 regarding files populating sysroots (e.g. the
1895 STAGING_DIR
1896 variable).
1897
1898Recipes should never populate the sysroot directly (i.e. write files
1899into sysroot). Instead, files should be installed into standard
1900locations during the
1901```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task within
1902the ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}`` directory. The
1903reason for this limitation is that almost all files that populate the
1904sysroot are cataloged in manifests in order to ensure the files can be
1905removed later when a recipe is either modified or removed. Thus, the
1906sysroot is able to remain free from stale files.
1907
1908A subset of the files installed by the
1909```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task are
1910used by the
1911```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
1912task as defined by the the
1913```SYSROOT_DIRS`` <&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS>`__ variable to
1914automatically populate the sysroot. It is possible to modify the list of
1915directories that populate the sysroot. The following example shows how
1916you could add the ``/opt`` directory to the list of directories within a
1917recipe: SYSROOT_DIRS += "/opt"
1918
1919For a more complete description of the
1920```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
1921task and its associated functions, see the
1922```staging`` <&YOCTO_DOCS_REF_URL;#ref-classes-staging>`__ class.
1923
1924.. _metadata-virtual-providers:
1925
1926Using Virtual Providers
1927-----------------------
1928
1929Prior to a build, if you know that several different recipes provide the
1930same functionality, you can use a virtual provider (i.e. ``virtual/*``)
1931as a placeholder for the actual provider. The actual provider is
1932determined at build-time.
1933
1934A common scenario where a virtual provider is used would be for the
1935kernel recipe. Suppose you have three kernel recipes whose
1936```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ values map to ``kernel-big``,
1937``kernel-mid``, and ``kernel-small``. Furthermore, each of these recipes
1938in some way uses a ```PROVIDES`` <&YOCTO_DOCS_REF_URL;#var-PROVIDES>`__
1939statement that essentially identifies itself as being able to provide
1940``virtual/kernel``. Here is one way through the
1941```kernel`` <&YOCTO_DOCS_REF_URL;#ref-classes-kernel>`__ class: PROVIDES
1942+= "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") ==
1943"kernel") else "" }" Any recipe that inherits the ``kernel`` class is
1944going to utilize a ``PROVIDES`` statement that identifies that recipe as
1945being able to provide the ``virtual/kernel`` item.
1946
1947Now comes the time to actually build an image and you need a kernel
1948recipe, but which one? You can configure your build to call out the
1949kernel recipe you want by using the
1950```PREFERRED_PROVIDER`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER>`__
1951variable. As an example, consider the
1952```x86-base.inc`` <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc>`__
1953include file, which is a machine (i.e.
1954```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__) configuration file.
1955This include file is the reason all x86-based machines use the
1956``linux-yocto`` kernel. Here are the relevant lines from the include
1957file: PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto"
1958PREFERRED_VERSION_linux-yocto ??= "4.15%"
1959
1960When you use a virtual provider, you do not have to "hard code" a recipe
1961name as a build dependency. You can use the
1962```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable to state the
1963build is dependent on ``virtual/kernel`` for example: DEPENDS =
1964"virtual/kernel" During the build, the OpenEmbedded build system picks
1965the correct recipe needed for the ``virtual/kernel`` dependency based on
1966the ``PREFERRED_PROVIDER`` variable. If you want to use the small kernel
1967mentioned at the beginning of this section, configure your build as
1968follows: PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small"
1969
1970.. note::
1971
1972 Any recipe that
1973 PROVIDES
1974 a
1975 virtual/\*
1976 item that is ultimately not selected through
1977 PREFERRED_PROVIDER
1978 does not get built. Preventing these recipes from building is usually
1979 the desired behavior since this mechanism's purpose is to select
1980 between mutually exclusive alternative providers.
1981
1982The following lists specific examples of virtual providers:
1983
1984- ``virtual/kernel``: Provides the name of the kernel recipe to use
1985 when building a kernel image.
1986
1987- ``virtual/bootloader``: Provides the name of the bootloader to use
1988 when building an image.
1989
1990- ``virtual/libgbm``: Provides ``gbm.pc``.
1991
1992- ``virtual/egl``: Provides ``egl.pc`` and possibly ``wayland-egl.pc``.
1993
1994- ``virtual/libgl``: Provides ``gl.pc`` (i.e. libGL).
1995
1996- ``virtual/libgles1``: Provides ``glesv1_cm.pc`` (i.e. libGLESv1_CM).
1997
1998- ``virtual/libgles2``: Provides ``glesv2.pc`` (i.e. libGLESv2).
1999
2000Properly Versioning Pre-Release Recipes
2001---------------------------------------
2002
2003Sometimes the name of a recipe can lead to versioning problems when the
2004recipe is upgraded to a final release. For example, consider the
2005``irssi_0.8.16-rc1.bb`` recipe file in the list of example recipes in
2006the "`Storing and Naming the
2007Recipe <#new-recipe-storing-and-naming-the-recipe>`__" section. This
2008recipe is at a release candidate stage (i.e. "rc1"). When the recipe is
2009released, the recipe filename becomes ``irssi_0.8.16.bb``. The version
2010change from ``0.8.16-rc1`` to ``0.8.16`` is seen as a decrease by the
2011build system and package managers, so the resulting packages will not
2012correctly trigger an upgrade.
2013
2014In order to ensure the versions compare properly, the recommended
2015convention is to set ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ within the
2016recipe to "previous_version+current_version". You can use an additional
2017variable so that you can use the current version elsewhere. Here is an
2018example: REALPV = "0.8.16-rc1" PV = "0.8.15+${REALPV}"
2019
2020.. _new-recipe-post-installation-scripts:
2021
2022Post-Installation Scripts
2023-------------------------
2024
2025Post-installation scripts run immediately after installing a package on
2026the target or during image creation when a package is included in an
2027image. To add a post-installation script to a package, add a
2028``pkg_postinst_``\ PACKAGENAME\ ``()`` function to the recipe file
2029(``.bb``) and replace PACKAGENAME with the name of the package you want
2030to attach to the ``postinst`` script. To apply the post-installation
2031script to the main package for the recipe, which is usually what is
2032required, specify
2033``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` in place of
2034PACKAGENAME.
2035
2036A post-installation function has the following structure:
2037pkg_postinst_PACKAGENAME() { # Commands to carry out }
2038
2039The script defined in the post-installation function is called when the
2040root filesystem is created. If the script succeeds, the package is
2041marked as installed.
2042
2043.. note::
2044
2045 Any RPM post-installation script that runs on the target should
2046 return a 0 exit code. RPM does not allow non-zero exit codes for
2047 these scripts, and the RPM package manager will cause the package to
2048 fail installation on the target.
2049
2050Sometimes it is necessary for the execution of a post-installation
2051script to be delayed until the first boot. For example, the script might
2052need to be executed on the device itself. To delay script execution
2053until boot time, you must explicitly mark post installs to defer to the
2054target. You can use ``pkg_postinst_ontarget()`` or call
2055``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``. Any
2056failure of a ``pkg_postinst()`` script (including exit 1) triggers an
2057error during the
2058```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__ task.
2059
2060If you have recipes that use ``pkg_postinst`` function and they require
2061the use of non-standard native tools that have dependencies during
2062rootfs construction, you need to use the
2063```PACKAGE_WRITE_DEPS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_WRITE_DEPS>`__
2064variable in your recipe to list these tools. If you do not use this
2065variable, the tools might be missing and execution of the
2066post-installation script is deferred until first boot. Deferring the
2067script to first boot is undesirable and for read-only rootfs impossible.
2068
2069.. note::
2070
2071 Equivalent support for pre-install, pre-uninstall, and post-uninstall
2072 scripts exist by way of
2073 pkg_preinst
2074 ,
2075 pkg_prerm
2076 , and
2077 pkg_postrm
2078 , respectively. These scrips work in exactly the same way as does
2079 pkg_postinst
2080 with the exception that they run at different times. Also, because of
2081 when they run, they are not applicable to being run at image creation
2082 time like
2083 pkg_postinst
2084 .
2085
2086.. _new-recipe-testing:
2087
2088Testing
2089-------
2090
2091The final step for completing your recipe is to be sure that the
2092software you built runs correctly. To accomplish runtime testing, add
2093the build's output packages to your image and test them on the target.
2094
2095For information on how to customize your image by adding specific
2096packages, see the "`Customizing
2097Images <#usingpoky-extend-customimage>`__" section.
2098
2099.. _new-recipe-testing-examples:
2100
2101Examples
2102--------
2103
2104To help summarize how to write a recipe, this section provides some
2105examples given various scenarios:
2106
2107- Recipes that use local files
2108
2109- Using an Autotooled package
2110
2111- Using a Makefile-based package
2112
2113- Splitting an application into multiple packages
2114
2115- Adding binaries to an image
2116
2117.. _new-recipe-single-c-file-package-hello-world:
2118
2119Single .c File Package (Hello World!)
2120~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2121
2122Building an application from a single file that is stored locally (e.g.
2123under ``files``) requires a recipe that has the file listed in the
2124``SRC_URI`` variable. Additionally, you need to manually write the
2125``do_compile`` and ``do_install`` tasks. The ``S`` variable defines the
2126directory containing the source code, which is set to
2127```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__ in this case - the
2128directory BitBake uses for the build. SUMMARY = "Simple helloworld
2129application" SECTION = "examples" LICENSE = "MIT" LIC_FILES_CHKSUM =
2130"file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
2131SRC_URI = "file://helloworld.c" S = "${WORKDIR}" do_compile() { ${CC}
2132helloworld.c -o helloworld } do_install() { install -d ${D}${bindir}
2133install -m 0755 helloworld ${D}${bindir} }
2134
2135By default, the ``helloworld``, ``helloworld-dbg``, and
2136``helloworld-dev`` packages are built. For information on how to
2137customize the packaging process, see the "`Splitting an Application into
2138Multiple Packages <#splitting-an-application-into-multiple-packages>`__"
2139section.
2140
2141.. _new-recipe-autotooled-package:
2142
2143Autotooled Package
2144~~~~~~~~~~~~~~~~~~
2145
2146Applications that use Autotools such as ``autoconf`` and ``automake``
2147require a recipe that has a source archive listed in ``SRC_URI`` and
2148also inherit the
2149```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class,
2150which contains the definitions of all the steps needed to build an
2151Autotool-based application. The result of the build is automatically
2152packaged. And, if the application uses NLS for localization, packages
2153with local information are generated (one package per language).
2154Following is one example: (``hello_2.3.bb``) SUMMARY = "GNU Helloworld
2155application" SECTION = "examples" LICENSE = "GPLv2+" LIC_FILES_CHKSUM =
2156"file://COPYING;md5=751419260aa954499f7abaabaa882bbe" SRC_URI =
2157"${GNU_MIRROR}/hello/hello-${PV}.tar.gz" inherit autotools gettext
2158
2159The variable ``LIC_FILES_CHKSUM`` is used to track source license
2160changes as described in the "`Tracking License
2161Changes <#usingpoky-configuring-LIC_FILES_CHKSUM>`__" section in the
2162Yocto Project Overview and Concepts Manual. You can quickly create
2163Autotool-based recipes in a manner similar to the previous example.
2164
2165.. _new-recipe-makefile-based-package:
2166
2167Makefile-Based Package
2168~~~~~~~~~~~~~~~~~~~~~~
2169
2170Applications that use GNU ``make`` also require a recipe that has the
2171source archive listed in ``SRC_URI``. You do not need to add a
2172``do_compile`` step since by default BitBake starts the ``make`` command
2173to compile the application. If you need additional ``make`` options, you
2174should store them in the
2175```EXTRA_OEMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE>`__ or
2176```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
2177variables. BitBake passes these options into the GNU ``make``
2178invocation. Note that a ``do_install`` task is still required.
2179Otherwise, BitBake runs an empty ``do_install`` task by default.
2180
2181Some applications might require extra parameters to be passed to the
2182compiler. For example, the application might need an additional header
2183path. You can accomplish this by adding to the ``CFLAGS`` variable. The
2184following example shows this: CFLAGS_prepend = "-I ${S}/include "
2185
2186In the following example, ``mtd-utils`` is a makefile-based package:
2187SUMMARY = "Tools for managing memory technology devices" SECTION =
2188"base" DEPENDS = "zlib lzo e2fsprogs util-linux" HOMEPAGE =
2189"http://www.linux-mtd.infradead.org/" LICENSE = "GPLv2+"
2190LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3
2191\\
2192file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
2193# Use the latest version at 26 Oct, 2013 SRCREV =
2194"9f107132a6a073cce37434ca9cda6917dd8d866b" SRC_URI =
2195"git://git.infradead.org/mtd-utils.git \\
2196file://add-exclusion-to-mkfs-jffs2-git-2.patch \\ " PV =
2197"1.5.1+git${SRCPV}" S = "${WORKDIR}/git" EXTRA_OEMAKE = "'CC=${CC}'
2198'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include
2199-DWITHOUT_XATTR' 'BUILDDIR=${S}'" do_install () { oe_runmake install
2200DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir}
2201INCLUDEDIR=${includedir} } PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs
2202mtd-utils-misc" FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2
2203${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool"
2204FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*"
2205FILES_mtd-utils-misc = "${sbindir}/nftl\* ${sbindir}/ftl\*
2206${sbindir}/rfd\* ${sbindir}/doc\* ${sbindir}/serve_image
2207${sbindir}/recv_image" PARALLEL_MAKE = "" BBCLASSEXTEND = "native"
2208
2209Splitting an Application into Multiple Packages
2210~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2211
2212You can use the variables ``PACKAGES`` and ``FILES`` to split an
2213application into multiple packages.
2214
2215Following is an example that uses the ``libxpm`` recipe. By default,
2216this recipe generates a single package that contains the library along
2217with a few binaries. You can modify the recipe to split the binaries
2218into separate packages: require xorg-lib-common.inc SUMMARY = "Xpm: X
2219Pixmap extension library" LICENSE = "BSD" LIC_FILES_CHKSUM =
2220"file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" DEPENDS +=
2221"libxext libsm libxt" PE = "1" XORG_PN = "libXpm" PACKAGES =+ "sxpm
2222cxpm" FILES_cxpm = "${bindir}/cxpm" FILES_sxpm = "${bindir}/sxpm"
2223
2224In the previous example, we want to ship the ``sxpm`` and ``cxpm``
2225binaries in separate packages. Since ``bindir`` would be packaged into
2226the main ``PN`` package by default, we prepend the ``PACKAGES`` variable
2227so additional package names are added to the start of list. This results
2228in the extra ``FILES_*`` variables then containing information that
2229define which files and directories go into which packages. Files
2230included by earlier packages are skipped by latter packages. Thus, the
2231main ``PN`` package does not include the above listed files.
2232
2233Packaging Externally Produced Binaries
2234~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2235
2236Sometimes, you need to add pre-compiled binaries to an image. For
2237example, suppose that binaries for proprietary code exist, which are
2238created by a particular division of a company. Your part of the company
2239needs to use those binaries as part of an image that you are building
2240using the OpenEmbedded build system. Since you only have the binaries
2241and not the source code, you cannot use a typical recipe that expects to
2242fetch the source specified in
2243```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ and then compile it.
2244
2245One method is to package the binaries and then install them as part of
2246the image. Generally, it is not a good idea to package binaries since,
2247among other things, it can hinder the ability to reproduce builds and
2248could lead to compatibility problems with ABI in the future. However,
2249sometimes you have no choice.
2250
2251The easiest solution is to create a recipe that uses the
2252```bin_package`` <&YOCTO_DOCS_REF_URL;#ref-classes-bin-package>`__ class
2253and to be sure that you are using default locations for build artifacts.
2254In most cases, the ``bin_package`` class handles "skipping" the
2255configure and compile steps as well as sets things up to grab packages
2256from the appropriate area. In particular, this class sets ``noexec`` on
2257both the ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__
2258and ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ tasks,
2259sets ``FILES_${PN}`` to "/" so that it picks up all files, and sets up a
2260```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task, which
2261effectively copies all files from ``${S}`` to ``${D}``. The
2262``bin_package`` class works well when the files extracted into ``${S}``
2263are already laid out in the way they should be laid out on the target.
2264For more information on these variables, see the
2265```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__,
2266```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__,
2267```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__, and
2268```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__ variables in the Yocto Project
2269Reference Manual's variable glossary.
2270
2271.. note::
2272
2273 - Using ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ is a good
2274 idea even for components distributed in binary form, and is often
2275 necessary for shared libraries. For a shared library, listing the
2276 library dependencies in ``DEPENDS`` makes sure that the libraries
2277 are available in the staging sysroot when other recipes link
2278 against the library, which might be necessary for successful
2279 linking.
2280
2281 - Using ``DEPENDS`` also allows runtime dependencies between
2282 packages to be added automatically. See the "`Automatically Added
2283 Runtime
2284 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
2285 section in the Yocto Project Overview and Concepts Manual for more
2286 information.
2287
2288If you cannot use the ``bin_package`` class, you need to be sure you are
2289doing the following:
2290
2291- Create a recipe where the
2292 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ and
2293 ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ tasks do
2294 nothing: It is usually sufficient to just not define these tasks in
2295 the recipe, because the default implementations do nothing unless a
2296 Makefile is found in
2297 ``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``.
2298
2299 If ``${S}`` might contain a Makefile, or if you inherit some class
2300 that replaces ``do_configure`` and ``do_compile`` with custom
2301 versions, then you can use the
2302 ``[``\ ```noexec`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__\ ``]``
2303 flag to turn the tasks into no-ops, as follows: do_configure[noexec]
2304 = "1" do_compile[noexec] = "1" Unlike
2305 ```deleting the tasks`` <&YOCTO_DOCS_BB_URL;#deleting-a-task>`__,
2306 using the flag preserves the dependency chain from the
2307 ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__,
2308 ```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__, and
2309 ```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ tasks to the
2310 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task.
2311
2312- Make sure your ``do_install`` task installs the binaries
2313 appropriately.
2314
2315- Ensure that you set up ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__
2316 (usually
2317 ``FILES_${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``) to
2318 point to the files you have installed, which of course depends on
2319 where you have installed them and whether those files are in
2320 different locations than the defaults.
2321
2322Following Recipe Style Guidelines
2323---------------------------------
2324
2325When writing recipes, it is good to conform to existing style
2326guidelines. The `OpenEmbedded
2327Styleguide <http://www.openembedded.org/wiki/Styleguide>`__ wiki page
2328provides rough guidelines for preferred recipe style.
2329
2330It is common for existing recipes to deviate a bit from this style.
2331However, aiming for at least a consistent style is a good idea. Some
2332practices, such as omitting spaces around ``=`` operators in assignments
2333or ordering recipe components in an erratic way, are widely seen as poor
2334style.
2335
2336Recipe Syntax
2337-------------
2338
2339Understanding recipe file syntax is important for writing recipes. The
2340following list overviews the basic items that make up a BitBake recipe
2341file. For more complete BitBake syntax descriptions, see the "`Syntax
2342and Operators <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata>`__"
2343chapter of the BitBake User Manual.
2344
2345- *Variable Assignments and Manipulations:* Variable assignments allow
2346 a value to be assigned to a variable. The assignment can be static
2347 text or might include the contents of other variables. In addition to
2348 the assignment, appending and prepending operations are also
2349 supported.
2350
2351 The following example shows some of the ways you can use variables in
2352 recipes: S = "${WORKDIR}/postfix-${PV}" CFLAGS += "-DNO_ASM"
2353 SRC_URI_append = " file://fixup.patch"
2354
2355- *Functions:* Functions provide a series of actions to be performed.
2356 You usually use functions to override the default implementation of a
2357 task function or to complement a default function (i.e. append or
2358 prepend to an existing function). Standard functions use ``sh`` shell
2359 syntax, although access to OpenEmbedded variables and internal
2360 methods are also available.
2361
2362 The following is an example function from the ``sed`` recipe:
2363 do_install () { autotools_do_install install -d ${D}${base_bindir} mv
2364 ${D}${bindir}/sed ${D}${base_bindir}/sed rmdir ${D}${bindir}/ } It is
2365 also possible to implement new functions that are called between
2366 existing tasks as long as the new functions are not replacing or
2367 complementing the default functions. You can implement functions in
2368 Python instead of shell. Both of these options are not seen in the
2369 majority of recipes.
2370
2371- *Keywords:* BitBake recipes use only a few keywords. You use keywords
2372 to include common functions (``inherit``), load parts of a recipe
2373 from other files (``include`` and ``require``) and export variables
2374 to the environment (``export``).
2375
2376 The following example shows the use of some of these keywords: export
2377 POSTCONF = "${STAGING_BINDIR}/postconf" inherit autoconf require
2378 otherfile.inc
2379
2380- *Comments (#):* Any lines that begin with the hash character (``#``)
2381 are treated as comment lines and are ignored: # This is a comment
2382
2383This next list summarizes the most important and most commonly used
2384parts of the recipe syntax. For more information on these parts of the
2385syntax, you can reference the `Syntax and
2386Operators <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata>`__ chapter
2387in the BitBake User Manual.
2388
2389- *Line Continuation (\):* Use the backward slash (``\``) character to
2390 split a statement over multiple lines. Place the slash character at
2391 the end of the line that is to be continued on the next line: VAR =
2392 "A really long \\ line"
2393
2394 .. note::
2395
2396 You cannot have any characters including spaces or tabs after the
2397 slash character.
2398
2399- *Using Variables (${VARNAME}):* Use the ``${VARNAME}`` syntax to
2400 access the contents of a variable: SRC_URI =
2401 "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
2402
2403 .. note::
2404
2405 It is important to understand that the value of a variable
2406 expressed in this form does not get substituted automatically. The
2407 expansion of these expressions happens on-demand later (e.g.
2408 usually when a function that makes reference to the variable
2409 executes). This behavior ensures that the values are most
2410 appropriate for the context in which they are finally used. On the
2411 rare occasion that you do need the variable expression to be
2412 expanded immediately, you can use the
2413 :=
2414 operator instead of
2415 =
2416 when you make the assignment, but this is not generally needed.
2417
2418- *Quote All Assignments ("value"):* Use double quotes around values in
2419 all variable assignments (e.g. ``"value"``). Following is an example:
2420 VAR1 = "${OTHERVAR}" VAR2 = "The version is ${PV}"
2421
2422- *Conditional Assignment (?=):* Conditional assignment is used to
2423 assign a value to a variable, but only when the variable is currently
2424 unset. Use the question mark followed by the equal sign (``?=``) to
2425 make a "soft" assignment used for conditional assignment. Typically,
2426 "soft" assignments are used in the ``local.conf`` file for variables
2427 that are allowed to come through from the external environment.
2428
2429 Here is an example where ``VAR1`` is set to "New value" if it is
2430 currently empty. However, if ``VAR1`` has already been set, it
2431 remains unchanged: VAR1 ?= "New value" In this next example, ``VAR1``
2432 is left with the value "Original value": VAR1 = "Original value" VAR1
2433 ?= "New value"
2434
2435- *Appending (+=):* Use the plus character followed by the equals sign
2436 (``+=``) to append values to existing variables.
2437
2438 .. note::
2439
2440 This operator adds a space between the existing content of the
2441 variable and the new content.
2442
2443 Here is an example: SRC_URI += "file://fix-makefile.patch"
2444
2445- *Prepending (=+):* Use the equals sign followed by the plus character
2446 (``=+``) to prepend values to existing variables.
2447
2448 .. note::
2449
2450 This operator adds a space between the new content and the
2451 existing content of the variable.
2452
2453 Here is an example: VAR =+ "Starts"
2454
2455- *Appending (_append):* Use the ``_append`` operator to append values
2456 to existing variables. This operator does not add any additional
2457 space. Also, the operator is applied after all the ``+=``, and ``=+``
2458 operators have been applied and after all ``=`` assignments have
2459 occurred.
2460
2461 The following example shows the space being explicitly added to the
2462 start to ensure the appended value is not merged with the existing
2463 value: SRC_URI_append = " file://fix-makefile.patch" You can also use
2464 the ``_append`` operator with overrides, which results in the actions
2465 only being performed for the specified target or machine:
2466 SRC_URI_append_sh4 = " file://fix-makefile.patch"
2467
2468- *Prepending (_prepend):* Use the ``_prepend`` operator to prepend
2469 values to existing variables. This operator does not add any
2470 additional space. Also, the operator is applied after all the ``+=``,
2471 and ``=+`` operators have been applied and after all ``=``
2472 assignments have occurred.
2473
2474 The following example shows the space being explicitly added to the
2475 end to ensure the prepended value is not merged with the existing
2476 value: CFLAGS_prepend = "-I${S}/myincludes " You can also use the
2477 ``_prepend`` operator with overrides, which results in the actions
2478 only being performed for the specified target or machine:
2479 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
2480
2481- *Overrides:* You can use overrides to set a value conditionally,
2482 typically based on how the recipe is being built. For example, to set
2483 the ```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable's
2484 value to "standard/base" for any target
2485 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__, except for
2486 qemuarm where it should be set to "standard/arm-versatile-926ejs",
2487 you would do the following: KBRANCH = "standard/base" KBRANCH_qemuarm
2488 = "standard/arm-versatile-926ejs" Overrides are also used to separate
2489 alternate values of a variable in other situations. For example, when
2490 setting variables such as
2491 ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ and
2492 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ that are
2493 specific to individual packages produced by a recipe, you should
2494 always use an override that specifies the name of the package.
2495
2496- *Indentation:* Use spaces for indentation rather than than tabs. For
2497 shell functions, both currently work. However, it is a policy
2498 decision of the Yocto Project to use tabs in shell functions. Realize
2499 that some layers have a policy to use spaces for all indentation.
2500
2501- *Using Python for Complex Operations:* For more advanced processing,
2502 it is possible to use Python code during variable assignments (e.g.
2503 search and replacement on a variable).
2504
2505 You indicate Python code using the ``${@python_code}`` syntax for the
2506 variable assignment: SRC_URI =
2507 "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.',
2508 '')}.tgz
2509
2510- *Shell Function Syntax:* Write shell functions as if you were writing
2511 a shell script when you describe a list of actions to take. You
2512 should ensure that your script works with a generic ``sh`` and that
2513 it does not require any ``bash`` or other shell-specific
2514 functionality. The same considerations apply to various system
2515 utilities (e.g. ``sed``, ``grep``, ``awk``, and so forth) that you
2516 might wish to use. If in doubt, you should check with multiple
2517 implementations - including those from BusyBox.
2518
2519.. _platdev-newmachine:
2520
2521Adding a New Machine
2522====================
2523
2524Adding a new machine to the Yocto Project is a straightforward process.
2525This section describes how to add machines that are similar to those
2526that the Yocto Project already supports.
2527
2528.. note::
2529
2530 Although well within the capabilities of the Yocto Project, adding a
2531 totally new architecture might require changes to
2532 gcc/glibc
2533 and to the site information, which is beyond the scope of this
2534 manual.
2535
2536For a complete example that shows how to add a new machine, see the
2537"`Creating a New BSP Layer Using the ``bitbake-layers``
2538Script <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
2539section in the Yocto Project Board Support Package (BSP) Developer's
2540Guide.
2541
2542.. _platdev-newmachine-conffile:
2543
2544Adding the Machine Configuration File
2545-------------------------------------
2546
2547To add a new machine, you need to add a new machine configuration file
2548to the layer's ``conf/machine`` directory. This configuration file
2549provides details about the device you are adding.
2550
2551The OpenEmbedded build system uses the root name of the machine
2552configuration file to reference the new machine. For example, given a
2553machine configuration file named ``crownbay.conf``, the build system
2554recognizes the machine as "crownbay".
2555
2556The most important variables you must set in your machine configuration
2557file or include from a lower-level configuration file are as follows:
2558
2559- ``TARGET_ARCH`` (e.g. "arm")
2560
2561- ``PREFERRED_PROVIDER_virtual/kernel``
2562
2563- ``MACHINE_FEATURES`` (e.g. "apm screen wifi")
2564
2565You might also need these variables:
2566
2567- ``SERIAL_CONSOLES`` (e.g. "115200;ttyS0 115200;ttyS1")
2568
2569- ``KERNEL_IMAGETYPE`` (e.g. "zImage")
2570
2571- ``IMAGE_FSTYPES`` (e.g. "tar.gz jffs2")
2572
2573You can find full details on these variables in the reference section.
2574You can leverage existing machine ``.conf`` files from
2575``meta-yocto-bsp/conf/machine/``.
2576
2577.. _platdev-newmachine-kernel:
2578
2579Adding a Kernel for the Machine
2580-------------------------------
2581
2582The OpenEmbedded build system needs to be able to build a kernel for the
2583machine. You need to either create a new kernel recipe for this machine,
2584or extend an existing kernel recipe. You can find several kernel recipe
2585examples in the Source Directory at ``meta/recipes-kernel/linux`` that
2586you can use as references.
2587
2588If you are creating a new kernel recipe, normal recipe-writing rules
2589apply for setting up a ``SRC_URI``. Thus, you need to specify any
2590necessary patches and set ``S`` to point at the source code. You need to
2591create a ``do_configure`` task that configures the unpacked kernel with
2592a ``defconfig`` file. You can do this by using a ``make defconfig``
2593command or, more commonly, by copying in a suitable ``defconfig`` file
2594and then running ``make oldconfig``. By making use of ``inherit kernel``
2595and potentially some of the ``linux-*.inc`` files, most other
2596functionality is centralized and the defaults of the class normally work
2597well.
2598
2599If you are extending an existing kernel recipe, it is usually a matter
2600of adding a suitable ``defconfig`` file. The file needs to be added into
2601a location similar to ``defconfig`` files used for other machines in a
2602given kernel recipe. A possible way to do this is by listing the file in
2603the ``SRC_URI`` and adding the machine to the expression in
2604``COMPATIBLE_MACHINE``: COMPATIBLE_MACHINE = '(qemux86|qemumips)' For
2605more information on ``defconfig`` files, see the "`Changing the
2606Configuration <&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration>`__"
2607section in the Yocto Project Linux Kernel Development Manual.
2608
2609.. _platdev-newmachine-formfactor:
2610
2611Adding a Formfactor Configuration File
2612--------------------------------------
2613
2614A formfactor configuration file provides information about the target
2615hardware for which the image is being built and information that the
2616build system cannot obtain from other sources such as the kernel. Some
2617examples of information contained in a formfactor configuration file
2618include framebuffer orientation, whether or not the system has a
2619keyboard, the positioning of the keyboard in relation to the screen, and
2620the screen resolution.
2621
2622The build system uses reasonable defaults in most cases. However, if
2623customization is necessary, you need to create a ``machconfig`` file in
2624the ``meta/recipes-bsp/formfactor/files`` directory. This directory
2625contains directories for specific machines such as ``qemuarm`` and
2626``qemux86``. For information about the settings available and the
2627defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file
2628found in the same area.
2629
2630Following is an example for "qemuarm" machine: HAVE_TOUCHSCREEN=1
2631HAVE_KEYBOARD=1 DISPLAY_CAN_ROTATE=0 DISPLAY_ORIENTATION=0
2632#DISPLAY_WIDTH_PIXELS=640 #DISPLAY_HEIGHT_PIXELS=480 #DISPLAY_BPP=16
2633DISPLAY_DPI=150 DISPLAY_SUBPIXEL_ORDER=vrgb
2634
2635.. _gs-upgrading-recipes:
2636
2637Upgrading Recipes
2638=================
2639
2640Over time, upstream developers publish new versions for software built
2641by layer recipes. It is recommended to keep recipes up-to-date with
2642upstream version releases.
2643
2644While several methods exist that allow you upgrade a recipe, you might
2645consider checking on the upgrade status of a recipe first. You can do so
2646using the ``devtool check-upgrade-status`` command. See the "`Checking
2647on the Upgrade Status of a
2648Recipe <&YOCTO_DOCS_REF_URL;#devtool-checking-on-the-upgrade-status-of-a-recipe>`__"
2649section in the Yocto Project Reference Manual for more information.
2650
2651The remainder of this section describes three ways you can upgrade a
2652recipe. You can use the Automated Upgrade Helper (AUH) to set up
2653automatic version upgrades. Alternatively, you can use
2654``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
2655you can manually upgrade a recipe by editing the recipe itself.
2656
2657.. _gs-using-the-auto-upgrade-helper:
2658
2659Using the Auto Upgrade Helper (AUH)
2660-----------------------------------
2661
2662The AUH utility works in conjunction with the OpenEmbedded build system
2663in order to automatically generate upgrades for recipes based on new
2664versions being published upstream. Use AUH when you want to create a
2665service that performs the upgrades automatically and optionally sends
2666you an email with the results.
2667
2668AUH allows you to update several recipes with a single use. You can also
2669optionally perform build and integration tests using images with the
2670results saved to your hard drive and emails of results optionally sent
2671to recipe maintainers. Finally, AUH creates Git commits with appropriate
2672commit messages in the layer's tree for the changes made to recipes.
2673
2674.. note::
2675
2676 Conditions do exist when you should not use AUH to upgrade recipes
2677 and you should instead use either
2678 devtool upgrade
2679 or upgrade your recipes manually:
2680
2681 - When AUH cannot complete the upgrade sequence. This situation
2682 usually results because custom patches carried by the recipe
2683 cannot be automatically rebased to the new version. In this case,
2684 ``devtool upgrade`` allows you to manually resolve conflicts.
2685
2686 - When for any reason you want fuller control over the upgrade
2687 process. For example, when you want special arrangements for
2688 testing.
2689
2690The following steps describe how to set up the AUH utility:
2691
26921. *Be Sure the Development Host is Set Up:* You need to be sure that
2693 your development host is set up to use the Yocto Project. For
2694 information on how to set up your host, see the "`Preparing the Build
2695 Host <#dev-preparing-the-build-host>`__" section.
2696
26972. *Make Sure Git is Configured:* The AUH utility requires Git to be
2698 configured because AUH uses Git to save upgrades. Thus, you must have
2699 Git user and email configured. The following command shows your
2700 configurations: $ git config --list If you do not have the user and
2701 email configured, you can use the following commands to do so: $ git
2702 config --global user.name some_name $ git config --global user.email
2703 username@domain.com
2704
27053. *Clone the AUH Repository:* To use AUH, you must clone the repository
2706 onto your development host. The following command uses Git to create
2707 a local copy of the repository on your system: $ git clone
2708 git://git.yoctoproject.org/auto-upgrade-helper Cloning into
2709 'auto-upgrade-helper'... remote: Counting objects: 768, done. remote:
2710 Compressing objects: 100% (300/300), done. remote: Total 768 (delta
2711 499), reused 703 (delta 434) Receiving objects: 100% (768/768),
2712 191.47 KiB \| 98.00 KiB/s, done. Resolving deltas: 100% (499/499),
2713 done. Checking connectivity... done. AUH is not part of the
2714 `OpenEmbedded-Core (OE-Core) <&YOCTO_DOCS_REF_URL;#oe-core>`__ or
2715 `Poky <&YOCTO_DOCS_REF_URL;#poky>`__ repositories.
2716
27174. *Create a Dedicated Build Directory:* Run the
2718 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__
2719 script to create a fresh build directory that you use exclusively for
2720 running the AUH utility: $ cd ~/poky $ source oe-init-build-env
2721 your_AUH_build_directory Re-using an existing build directory and its
2722 configurations is not recommended as existing settings could cause
2723 AUH to fail or behave undesirably.
2724
27255. *Make Configurations in Your Local Configuration File:* Several
2726 settings need to exist in the ``local.conf`` file in the build
2727 directory you just created for AUH. Make these following
2728 configurations:
2729
2730 - If you want to enable `Build
2731 History <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__,
2732 which is optional, you need the following lines in the
2733 ``conf/local.conf`` file: INHERIT =+ "buildhistory"
2734 BUILDHISTORY_COMMIT = "1" With this configuration and a successful
2735 upgrade, a build history "diff" file appears in the
2736 ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
2737 your build directory.
2738
2739 - If you want to enable testing through the
2740 ```testimage`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__
2741 class, which is optional, you need to have the following set in
2742 your ``conf/local.conf`` file: INHERIT += "testimage"
2743
2744 .. note::
2745
2746 If your distro does not enable by default ptest, which Poky
2747 does, you need the following in your
2748 local.conf
2749 file:
2750 ::
2751
2752 DISTRO_FEATURES_append = " ptest"
2753
2754
27556. *Optionally Start a vncserver:* If you are running in a server
2756 without an X11 session, you need to start a vncserver: $ vncserver :1
2757 $ export DISPLAY=:1
2758
27597. *Create and Edit an AUH Configuration File:* You need to have the
2760 ``upgrade-helper/upgrade-helper.conf`` configuration file in your
2761 build directory. You can find a sample configuration file in the `AUH
2762 source
2763 repository <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/>`__.
2764
2765 Read through the sample file and make configurations as needed. For
2766 example, if you enabled build history in your ``local.conf`` as
2767 described earlier, you must enable it in ``upgrade-helper.conf``.
2768
2769 Also, if you are using the default ``maintainers.inc`` file supplied
2770 with Poky and located in ``meta-yocto`` and you do not set a
2771 "maintainers_whitelist" or "global_maintainer_override" in the
2772 ``upgrade-helper.conf`` configuration, and you specify "-e all" on
2773 the AUH command-line, the utility automatically sends out emails to
2774 all the default maintainers. Please avoid this.
2775
2776This next set of examples describes how to use the AUH:
2777
2778- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
2779 following form: $ upgrade-helper.py recipe_name For example, this
2780 command upgrades the ``xmodmap`` recipe: $ upgrade-helper.py xmodmap
2781
2782- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
2783 specific recipe to a particular version, use the following form: $
2784 upgrade-helper.py recipe_name -t version For example, this command
2785 upgrades the ``xmodmap`` recipe to version 1.2.3: $ upgrade-helper.py
2786 xmodmap -t 1.2.3
2787
2788- *Upgrading all Recipes to the Latest Versions and Suppressing Email
2789 Notifications:* To upgrade all recipes to their most recent versions
2790 and suppress the email notifications, use the following command: $
2791 upgrade-helper.py all
2792
2793- *Upgrading all Recipes to the Latest Versions and Send Email
2794 Notifications:* To upgrade all recipes to their most recent versions
2795 and send email messages to maintainers for each attempted recipe as
2796 well as a status email, use the following command: $
2797 upgrade-helper.py -e all
2798
2799Once you have run the AUH utility, you can find the results in the AUH
2800build directory: ${BUILDDIR}/upgrade-helper/timestamp The AUH utility
2801also creates recipe update commits from successful upgrade attempts in
2802the layer tree.
2803
2804You can easily set up to run the AUH utility on a regular basis by using
2805a cron job. See the
2806```weeklyjob.sh`` <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh>`__
2807file distributed with the utility for an example.
2808
2809.. _gs-using-devtool-upgrade:
2810
2811Using ``devtool upgrade``
2812-------------------------
2813
2814As mentioned earlier, an alternative method for upgrading recipes to
2815newer versions is to use
2816```devtool upgrade`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__.
2817You can read about ``devtool upgrade`` in general in the "`Use
2818``devtool upgrade`` to Create a Version of the Recipe that Supports a
2819Newer Version of the
2820Software <&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software>`__"
2821section in the Yocto Project Application Development and the Extensible
2822Software Development Kit (eSDK) Manual.
2823
2824To see all the command-line options available with ``devtool upgrade``,
2825use the following help command: $ devtool upgrade -h
2826
2827If you want to find out what version a recipe is currently at upstream
2828without any attempt to upgrade your local version of the recipe, you can
2829use the following command: $ devtool latest-version recipe_name
2830
2831As mentioned in the previous section describing AUH, ``devtool upgrade``
2832works in a less-automated manner than AUH. Specifically,
2833``devtool upgrade`` only works on a single recipe that you name on the
2834command line, cannot perform build and integration testing using images,
2835and does not automatically generate commits for changes in the source
2836tree. Despite all these "limitations", ``devtool upgrade`` updates the
2837recipe file to the new upstream version and attempts to rebase custom
2838patches contained by the recipe as needed.
2839
2840.. note::
2841
2842 AUH uses much of
2843 devtool upgrade
2844 behind the scenes making AUH somewhat of a "wrapper" application for
2845 devtool upgrade
2846 .
2847
2848A typical scenario involves having used Git to clone an upstream
2849repository that you use during build operations. Because you are (or
2850have) built the recipe in the past, the layer is likely added to your
2851configuration already. If for some reason, the layer is not added, you
2852could add it easily using the
2853```bitbake-layers`` <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__
2854script. For example, suppose you use the ``nano.bb`` recipe from the
2855``meta-oe`` layer in the ``meta-openembedded`` repository. For this
2856example, assume that the layer has been cloned into following area:
2857/home/scottrif/meta-openembedded The following command from your `Build
2858Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ adds the layer to
2859your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``): $
2860bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe NOTE:
2861Starting bitbake server... Parsing recipes: 100%
2862\|##########################################\| Time: 0:00:55 Parsing of
28631431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56
2864skipped, 0 masked, 0 errors. Removing 12 recipes from the x86_64
2865sysroot: 100% \|##############\| Time: 0:00:00 Removing 1 recipes from
2866the x86_64_i586 sysroot: 100% \|##########\| Time: 0:00:00 Removing 5
2867recipes from the i586 sysroot: 100% \|#################\| Time: 0:00:00
2868Removing 5 recipes from the qemux86 sysroot: 100% \|##############\|
2869Time: 0:00:00 For this example, assume that the ``nano.bb`` recipe that
2870is upstream has a 2.9.3 version number. However, the version in the
2871local repository is 2.7.4. The following command from your build
2872directory automatically upgrades the recipe for you:
2873
2874.. note::
2875
2876 Using the
2877 -V
2878 option is not necessary. Omitting the version number causes
2879 devtool upgrade
2880 to upgrade the recipe to the most recent version.
2881
2882$ devtool upgrade nano -V 2.9.3 NOTE: Starting bitbake server... NOTE:
2883Creating workspace layer in /home/scottrif/poky/build/workspace Parsing
2884recipes: 100% \|##########################################\| Time:
28850:00:46 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040
2886targets, 56 skipped, 0 masked, 0 errors. NOTE: Extracting current
2887version source... NOTE: Resolving any missing task queue dependencies .
2888. . NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE:
2889Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun
2890and all succeeded. Adding changed files: 100%
2891\|#####################################\| Time: 0:00:00 NOTE: Upgraded
2892source extracted to /home/scottrif/poky/build/workspace/sources/nano
2893NOTE: New recipe is
2894/home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
2895Continuing with this example, you can use ``devtool build`` to build the
2896newly upgraded recipe: $ devtool build nano NOTE: Starting bitbake
2897server... Loading cache: 100%
2898\|################################################################################################\|
2899Time: 0:00:01 Loaded 2040 entries from dependency cache. Parsing
2900recipes: 100%
2901\|##############################################################################################\|
2902Time: 0:00:00 Parsing of 1432 .bb files complete (1431 cached, 1
2903parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. NOTE: Resolving
2904any missing task queue dependencies . . . NOTE: Executing SetScene Tasks
2905NOTE: Executing RunQueue Tasks NOTE: nano: compiling from external
2906source tree /home/scottrif/poky/build/workspace/sources/nano NOTE: Tasks
2907Summary: Attempted 520 tasks of which 304 didn't need to be rerun and
2908all succeeded. Within the ``devtool upgrade`` workflow, opportunity
2909exists to deploy and test your rebuilt software. For this example,
2910however, running ``devtool finish`` cleans up the workspace once the
2911source in your workspace is clean. This usually means using Git to stage
2912and submit commits for the changes generated by the upgrade process.
2913
2914Once the tree is clean, you can clean things up in this example with the
2915following command from the ``${BUILDDIR}/workspace/sources/nano``
2916directory: $ devtool finish nano meta-oe NOTE: Starting bitbake
2917server... Loading cache: 100%
2918\|################################################################################################\|
2919Time: 0:00:00 Loaded 2040 entries from dependency cache. Parsing
2920recipes: 100%
2921\|##############################################################################################\|
2922Time: 0:00:01 Parsing of 1432 .bb files complete (1431 cached, 1
2923parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. NOTE: Adding new
2924patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch NOTE:
2925Updating recipe nano_2.9.3.bb NOTE: Removing file
2926/home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
2927NOTE: Moving recipe file to
2928/home/scottrif/meta-openembedded/meta-oe/recipes-support/nano NOTE:
2929Leaving source tree /home/scottrif/poky/build/workspace/sources/nano
2930as-is; if you no longer need it then please delete it manually Using the
2931``devtool finish`` command cleans up the workspace and creates a patch
2932file based on your commits. The tool puts all patch files back into the
2933source directory in a sub-directory named ``nano`` in this case.
2934
2935.. _dev-manually-upgrading-a-recipe:
2936
2937Manually Upgrading a Recipe
2938---------------------------
2939
2940If for some reason you choose not to upgrade recipes using the `Auto
2941Upgrade Helper (AUH) <#gs-using-the-auto-upgrade-helper>`__ or by using
2942```devtool upgrade`` <#gs-using-devtool-upgrade>`__, you can manually
2943edit the recipe files to upgrade the versions.
2944
2945.. note::
2946
2947 Manually updating multiple recipes scales poorly and involves many
2948 steps. The recommendation to upgrade recipe versions is through AUH
2949 or
2950 devtool upgrade
2951 , both of which automate some steps and provide guidance for others
2952 needed for the manual process.
2953
2954To manually upgrade recipe versions, follow these general steps:
2955
29561. *Change the Version:* Rename the recipe such that the version (i.e.
2957 the ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ part of the recipe name)
2958 changes appropriately. If the version is not part of the recipe name,
2959 change the value as it is set for ``PV`` within the recipe itself.
2960
29612. *Update ``SRCREV`` if Needed:* If the source code your recipe builds
2962 is fetched from Git or some other version control system, update
2963 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ to point to the
2964 commit hash that matches the new version.
2965
29663. *Build the Software:* Try to build the recipe using BitBake. Typical
2967 build failures include the following:
2968
2969 - License statements were updated for the new version. For this
2970 case, you need to review any changes to the license and update the
2971 values of ```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ and
2972 ```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
2973 as needed.
2974
2975 .. note::
2976
2977 License changes are often inconsequential. For example, the
2978 license text's copyright year might have changed.
2979
2980 - Custom patches carried by the older version of the recipe might
2981 fail to apply to the new version. For these cases, you need to
2982 review the failures. Patches might not be necessary for the new
2983 version of the software if the upgraded version has fixed those
2984 issues. If a patch is necessary and failing, you need to rebase it
2985 into the new version.
2986
29874. *Optionally Attempt to Build for Several Architectures:* Once you
2988 successfully build the new software for a given architecture, you
2989 could test the build for other architectures by changing the
2990 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable and
2991 rebuilding the software. This optional step is especially important
2992 if the recipe is to be released publicly.
2993
29945. *Check the Upstream Change Log or Release Notes:* Checking both these
2995 reveals if new features exist that could break
2996 backwards-compatibility. If so, you need to take steps to mitigate or
2997 eliminate that situation.
2998
29996. *Optionally Create a Bootable Image and Test:* If you want, you can
3000 test the new software by booting it onto actual hardware.
3001
30027. *Create a Commit with the Change in the Layer Repository:* After all
3003 builds work and any testing is successful, you can create commits for
3004 any changes in the layer holding your upgraded recipe.
3005
3006.. _finding-the-temporary-source-code:
3007
3008Finding Temporary Source Code
3009=============================
3010
3011You might find it helpful during development to modify the temporary
3012source code used by recipes to build packages. For example, suppose you
3013are developing a patch and you need to experiment a bit to figure out
3014your solution. After you have initially built the package, you can
3015iteratively tweak the source code, which is located in the `Build
3016Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, and then you can
3017force a re-compile and quickly test your altered code. Once you settle
3018on a solution, you can then preserve your changes in the form of
3019patches.
3020
3021During a build, the unpacked temporary source code used by recipes to
3022build packages is available in the Build Directory as defined by the
3023```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ variable. Below is the default
3024value for the ``S`` variable as defined in the
3025``meta/conf/bitbake.conf`` configuration file in the `Source
3026Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__: S =
3027"${WORKDIR}/${BP}" You should be aware that many recipes override the
3028``S`` variable. For example, recipes that fetch their source from Git
3029usually set ``S`` to ``${WORKDIR}/git``.
3030
3031.. note::
3032
3033 The
3034 BP
3035 represents the base recipe name, which consists of the name and
3036 version:
3037 ::
3038
3039 BP = "${BPN}-${PV}"
3040
3041
3042The path to the work directory for the recipe
3043(```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__) is defined as
3044follows:
3045${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} The
3046actual directory depends on several things:
3047
3048- ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__: The top-level build
3049 output directory.
3050
3051- ```MULTIMACH_TARGET_SYS`` <&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS>`__:
3052 The target system identifier.
3053
3054- ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__: The recipe name.
3055
3056- ```EXTENDPE`` <&YOCTO_DOCS_REF_URL;#var-EXTENDPE>`__: The epoch - (if
3057 ```PE`` <&YOCTO_DOCS_REF_URL;#var-PE>`__ is not specified, which is
3058 usually the case for most recipes, then ``EXTENDPE`` is blank).
3059
3060- ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The recipe version.
3061
3062- ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__: The recipe revision.
3063
3064As an example, assume a Source Directory top-level folder named
3065``poky``, a default Build Directory at ``poky/build``, and a
3066``qemux86-poky-linux`` machine target system. Furthermore, suppose your
3067recipe is named ``foo_1.3.0.bb``. In this case, the work directory the
3068build system uses to build the package would be as follows:
3069poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
3070
3071.. _using-a-quilt-workflow:
3072
3073Using Quilt in Your Workflow
3074============================
3075
3076`Quilt <http://savannah.nongnu.org/projects/quilt>`__ is a powerful tool
3077that allows you to capture source code changes without having a clean
3078source tree. This section outlines the typical workflow you can use to
3079modify source code, test changes, and then preserve the changes in the
3080form of a patch all using Quilt.
3081
3082.. note::
3083
3084 With regard to preserving changes to source files, if you clean a
3085 recipe or have
3086 rm_work
3087 enabled, the
3088 devtool
3089 workflow
3090 as described in the Yocto Project Application Development and the
3091 Extensible Software Development Kit (eSDK) manual is a safer
3092 development flow than the flow that uses Quilt.
3093
3094Follow these general steps:
3095
30961. *Find the Source Code:* Temporary source code used by the
3097 OpenEmbedded build system is kept in the `Build
3098 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. See the
3099 "`Finding Temporary Source
3100 Code <#finding-the-temporary-source-code>`__" section to learn how to
3101 locate the directory that has the temporary source code for a
3102 particular package.
3103
31042. *Change Your Working Directory:* You need to be in the directory that
3105 has the temporary source code. That directory is defined by the
3106 ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ variable.
3107
31083. *Create a New Patch:* Before modifying source code, you need to
3109 create a new patch. To create a new patch file, use ``quilt new`` as
3110 below: $ quilt new my_changes.patch
3111
31124. *Notify Quilt and Add Files:* After creating the patch, you need to
3113 notify Quilt about the files you plan to edit. You notify Quilt by
3114 adding the files to the patch you just created: $ quilt add file1.c
3115 file2.c file3.c
3116
31175. *Edit the Files:* Make your changes in the source code to the files
3118 you added to the patch.
3119
31206. *Test Your Changes:* Once you have modified the source code, the
3121 easiest way to test your changes is by calling the ``do_compile``
3122 task as shown in the following example: $ bitbake -c compile -f
3123 package The ``-f`` or ``--force`` option forces the specified task to
3124 execute. If you find problems with your code, you can just keep
3125 editing and re-testing iteratively until things work as expected.
3126
3127 .. note::
3128
3129 All the modifications you make to the temporary source code
3130 disappear once you run the
3131 do_clean
3132 or
3133 do_cleanall
3134 tasks using BitBake (i.e.
3135 bitbake -c clean
3136 package
3137 and
3138 bitbake -c cleanall
3139 package
3140 ). Modifications will also disappear if you use the
3141 rm_work
3142 feature as described in the "
3143 Conserving Disk Space During Builds
3144 " section.
3145
31467. *Generate the Patch:* Once your changes work as expected, you need to
3147 use Quilt to generate the final patch that contains all your
3148 modifications. $ quilt refresh At this point, the
3149 ``my_changes.patch`` file has all your edits made to the ``file1.c``,
3150 ``file2.c``, and ``file3.c`` files.
3151
3152 You can find the resulting patch file in the ``patches/``
3153 subdirectory of the source (``S``) directory.
3154
31558. *Copy the Patch File:* For simplicity, copy the patch file into a
3156 directory named ``files``, which you can create in the same directory
3157 that holds the recipe (``.bb``) file or the append (``.bbappend``)
3158 file. Placing the patch here guarantees that the OpenEmbedded build
3159 system will find the patch. Next, add the patch into the ``SRC_URI``
3160 of the recipe. Here is an example: SRC_URI +=
3161 "file://my_changes.patch"
3162
3163.. _platdev-appdev-devshell:
3164
3165Using a Development Shell
3166=========================
3167
3168When debugging certain commands or even when just editing packages,
3169``devshell`` can be a useful tool. When you invoke ``devshell``, all
3170tasks up to and including
3171```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ are run for the
3172specified target. Then, a new terminal is opened and you are placed in
3173``${``\ ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__\ ``}``, the source
3174directory. In the new terminal, all the OpenEmbedded build-related
3175environment variables are still defined so you can use commands such as
3176``configure`` and ``make``. The commands execute just as if the
3177OpenEmbedded build system were executing them. Consequently, working
3178this way can be helpful when debugging a build or preparing software to
3179be used with the OpenEmbedded build system.
3180
3181Following is an example that uses ``devshell`` on a target named
3182``matchbox-desktop``: $ bitbake matchbox-desktop -c devshell
3183
3184This command spawns a terminal with a shell prompt within the
3185OpenEmbedded build environment. The
3186```OE_TERMINAL`` <&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL>`__ variable
3187controls what type of shell is opened.
3188
3189For spawned terminals, the following occurs:
3190
3191- The ``PATH`` variable includes the cross-toolchain.
3192
3193- The ``pkgconfig`` variables find the correct ``.pc`` files.
3194
3195- The ``configure`` command finds the Yocto Project site files as well
3196 as any other necessary files.
3197
3198Within this environment, you can run configure or compile commands as if
3199they were being run by the OpenEmbedded build system itself. As noted
3200earlier, the working directory also automatically changes to the Source
3201Directory (```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__).
3202
3203To manually run a specific task using ``devshell``, run the
3204corresponding ``run.*`` script in the
3205``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/temp``
3206directory (e.g., ``run.do_configure.``\ pid). If a task's script does
3207not exist, which would be the case if the task was skipped by way of the
3208sstate cache, you can create the task by first running it outside of the
3209``devshell``: $ bitbake -c task
3210
3211.. note::
3212
3213 - Execution of a task's ``run.*`` script and BitBake's execution of
3214 a task are identical. In other words, running the script re-runs
3215 the task just as it would be run using the ``bitbake -c`` command.
3216
3217 - Any ``run.*`` file that does not have a ``.pid`` extension is a
3218 symbolic link (symlink) to the most recent version of that file.
3219
3220Remember, that the ``devshell`` is a mechanism that allows you to get
3221into the BitBake task execution environment. And as such, all commands
3222must be called just as BitBake would call them. That means you need to
3223provide the appropriate options for cross-compilation and so forth as
3224applicable.
3225
3226When you are finished using ``devshell``, exit the shell or close the
3227terminal window.
3228
3229.. note::
3230
3231 - It is worth remembering that when using ``devshell`` you need to
3232 use the full compiler name such as ``arm-poky-linux-gnueabi-gcc``
3233 instead of just using ``gcc``. The same applies to other
3234 applications such as ``binutils``, ``libtool`` and so forth.
3235 BitBake sets up environment variables such as ``CC`` to assist
3236 applications, such as ``make`` to find the correct tools.
3237
3238 - It is also worth noting that ``devshell`` still works over X11
3239 forwarding and similar situations.
3240
3241.. _platdev-appdev-devpyshell:
3242
3243Using a Development Python Shell
3244================================
3245
3246Similar to working within a development shell as described in the
3247previous section, you can also spawn and work within an interactive
3248Python development shell. When debugging certain commands or even when
3249just editing packages, ``devpyshell`` can be a useful tool. When you
3250invoke ``devpyshell``, all tasks up to and including
3251```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ are run for the
3252specified target. Then a new terminal is opened. Additionally, key
3253Python objects and code are available in the same way they are to
3254BitBake tasks, in particular, the data store 'd'. So, commands such as
3255the following are useful when exploring the data store and running
3256functions: pydevshell> d.getVar("STAGING_DIR")
3257'/media/build1/poky/build/tmp/sysroots' pydevshell>
3258d.getVar("STAGING_DIR") '${TMPDIR}/sysroots' pydevshell> d.setVar("FOO",
3259"bar") pydevshell> d.getVar("FOO") 'bar' pydevshell> d.delVar("FOO")
3260pydevshell> d.getVar("FOO") pydevshell> bb.build.exec_func("do_unpack",
3261d) pydevshell> The commands execute just as if the OpenEmbedded build
3262system were executing them. Consequently, working this way can be
3263helpful when debugging a build or preparing software to be used with the
3264OpenEmbedded build system.
3265
3266Following is an example that uses ``devpyshell`` on a target named
3267``matchbox-desktop``: $ bitbake matchbox-desktop -c devpyshell
3268
3269This command spawns a terminal and places you in an interactive Python
3270interpreter within the OpenEmbedded build environment. The
3271```OE_TERMINAL`` <&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL>`__ variable
3272controls what type of shell is opened.
3273
3274When you are finished using ``devpyshell``, you can exit the shell
3275either by using Ctrl+d or closing the terminal window.
3276
3277.. _dev-building:
3278
3279Building
3280========
3281
3282This section describes various build procedures. For example, the steps
3283needed for a simple build, a target that uses multiple configurations,
3284building an image for more than one machine, and so forth.
3285
3286.. _dev-building-a-simple-image:
3287
3288Building a Simple Image
3289-----------------------
3290
3291In the development environment, you need to build an image whenever you
3292change hardware support, add or change system libraries, or add or
3293change services that have dependencies. Several methods exist that allow
3294you to build an image within the Yocto Project. This section presents
3295the basic steps you need to build a simple image using BitBake from a
3296build host running Linux.
3297
3298.. note::
3299
3300 - For information on how to build an image using
3301 `Toaster <&YOCTO_DOCS_REF_URL;#toaster-term>`__, see the `Toaster
3302 User Manual <&YOCTO_DOCS_TOAST_URL;>`__.
3303
3304 - For information on how to use ``devtool`` to build images, see the
3305 "`Using ``devtool`` in Your SDK
3306 Workflow <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__"
3307 section in the Yocto Project Application Development and the
3308 Extensible Software Development Kit (eSDK) manual.
3309
3310 - For a quick example on how to build an image using the
3311 OpenEmbedded build system, see the `Yocto Project Quick
3312 Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
3313
3314The build process creates an entire Linux distribution from source and
3315places it in your `Build
3316Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ under
3317``tmp/deploy/images``. For detailed information on the build process
3318using BitBake, see the
3319"`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__" section in the
3320Yocto Project Overview and Concepts Manual.
3321
3322The following figure and list overviews the build process:
3323
33241. *Set up Your Host Development System to Support Development Using the
3325 Yocto Project*: See the "`Setting Up to Use the Yocto
3326 Project <#dev-manual-start>`__" section for options on how to get a
3327 build host ready to use the Yocto Project.
3328
33292. *Initialize the Build Environment:* Initialize the build environment
3330 by sourcing the build environment script (i.e.
3331 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__): $ source
3332 OE_INIT_FILE [build_dir]
3333
3334 When you use the initialization script, the OpenEmbedded build system
3335 uses ``build`` as the default Build Directory in your current work
3336 directory. You can use a build_dir argument with the script to
3337 specify a different build directory.
3338
3339 .. note::
3340
3341 A common practice is to use a different Build Directory for
3342 different targets. For example,
3343 ~/build/x86
3344 for a
3345 qemux86
3346 target, and
3347 ~/build/arm
3348 for a
3349 qemuarm
3350 target.
3351
33523. *Make Sure Your ``local.conf`` File is Correct:* Ensure the
3353 ``conf/local.conf`` configuration file, which is found in the Build
3354 Directory, is set up how you want it. This file defines many aspects
3355 of the build environment including the target machine architecture
3356 through the ``MACHINE`` variable, the packaging format used during
3357 the build
3358 (```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__),
3359 and a centralized tarball download directory through the
3360 ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ variable.
3361
33624. *Build the Image:* Build the image using the ``bitbake`` command: $
3363 bitbake target
3364
3365 .. note::
3366
3367 For information on BitBake, see the
3368 BitBake User Manual
3369 .
3370
3371 The target is the name of the recipe you want to build. Common
3372 targets are the images in ``meta/recipes-core/images``,
3373 ``meta/recipes-sato/images``, and so forth all found in the `Source
3374 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Or, the target
3375 can be the name of a recipe for a specific piece of software such as
3376 BusyBox. For more details about the images the OpenEmbedded build
3377 system supports, see the
3378 "`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
3379 Project Reference Manual.
3380
3381 As an example, the following command builds the
3382 ``core-image-minimal`` image: $ bitbake core-image-minimal Once an
3383 image has been built, it often needs to be installed. The images and
3384 kernels built by the OpenEmbedded build system are placed in the
3385 Build Directory in ``tmp/deploy/images``. For information on how to
3386 run pre-built images such as ``qemux86`` and ``qemuarm``, see the
3387 `Yocto Project Application Development and the Extensible Software
3388 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual. For
3389 information about how to install these images, see the documentation
3390 for your particular board or machine.
3391
3392.. _dev-building-images-for-multiple-targets-using-multiple-configurations:
3393
3394Building Images for Multiple Targets Using Multiple Configurations
3395------------------------------------------------------------------
3396
3397You can use a single ``bitbake`` command to build multiple images or
3398packages for different targets where each image or package requires a
3399different configuration (multiple configuration builds). The builds, in
3400this scenario, are sometimes referred to as "multiconfigs", and this
3401section uses that term throughout.
3402
3403This section describes how to set up for multiple configuration builds
3404and how to account for cross-build dependencies between the
3405multiconfigs.
3406
3407.. _dev-setting-up-and-running-a-multiple-configuration-build:
3408
3409Setting Up and Running a Multiple Configuration Build
3410~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3411
3412To accomplish a multiple configuration build, you must define each
3413target's configuration separately using a parallel configuration file in
3414the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, and you
3415must follow a required file hierarchy. Additionally, you must enable the
3416multiple configuration builds in your ``local.conf`` file.
3417
3418Follow these steps to set up and execute multiple configuration builds:
3419
3420- *Create Separate Configuration Files*: You need to create a single
3421 configuration file for each build target (each multiconfig).
3422 Minimally, each configuration file must define the machine and the
3423 temporary directory BitBake uses for the build. Suggested practice
3424 dictates that you do not overlap the temporary directories used
3425 during the builds. However, it is possible that you can share the
3426 temporary directory
3427 (```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__). For example,
3428 consider a scenario with two different multiconfigs for the same
3429 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__: "qemux86" built
3430 for two distributions such as "poky" and "poky-lsb". In this case,
3431 you might want to use the same ``TMPDIR``.
3432
3433 Here is an example showing the minimal statements needed in a
3434 configuration file for a "qemux86" target whose temporary build
3435 directory is ``tmpmultix86``: MACHINE="qemux86"
3436 TMPDIR="${TOPDIR}/tmpmultix86"
3437
3438 The location for these multiconfig configuration files is specific.
3439 They must reside in the current build directory in a sub-directory of
3440 ``conf`` named ``multiconfig``. Following is an example that defines
3441 two configuration files for the "x86" and "arm" multiconfigs:
3442
3443 The reason for this required file hierarchy is because the ``BBPATH``
3444 variable is not constructed until the layers are parsed.
3445 Consequently, using the configuration file as a pre-configuration
3446 file is not possible unless it is located in the current working
3447 directory.
3448
3449- *Add the BitBake Multi-configuration Variable to the Local
3450 Configuration File*: Use the
3451 ```BBMULTICONFIG`` <&YOCTO_DOCS_REF_URL;#var-BBMULTICONFIG>`__
3452 variable in your ``conf/local.conf`` configuration file to specify
3453 each multiconfig. Continuing with the example from the previous
3454 figure, the ``BBMULTICONFIG`` variable needs to enable two
3455 multiconfigs: "x86" and "arm" by specifying each configuration file:
3456 BBMULTICONFIG = "x86 arm"
3457
3458 .. note::
3459
3460 A "default" configuration already exists by definition. This
3461 configuration is named: "" (i.e. empty string) and is defined by
3462 the variables coming from your
3463 local.conf
3464 file. Consequently, the previous example actually adds two
3465 additional configurations to your build: "arm" and "x86" along
3466 with "".
3467
3468- *Launch BitBake*: Use the following BitBake command form to launch
3469 the multiple configuration build: $ bitbake
3470 [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ] For
3471 the example in this section, the following command applies: $ bitbake
3472 mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
3473 The previous BitBake command builds a ``core-image-minimal`` image
3474 that is configured through the ``x86.conf`` configuration file, a
3475 ``core-image-sato`` image that is configured through the ``arm.conf``
3476 configuration file and a ``core-image-base`` that is configured
3477 through your ``local.conf`` configuration file.
3478
3479.. note::
3480
3481 Support for multiple configuration builds in the Yocto Project DISTRO
3482 (DISTRO_NAME) Release does not include Shared State (sstate)
3483 optimizations. Consequently, if a build uses the same object twice
3484 in, for example, two different
3485 TMPDIR
3486 directories, the build either loads from an existing sstate cache for
3487 that build at the start or builds the object fresh.
3488
3489.. _dev-enabling-multiple-configuration-build-dependencies:
3490
3491Enabling Multiple Configuration Build Dependencies
3492~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3493
3494Sometimes dependencies can exist between targets (multiconfigs) in a
3495multiple configuration build. For example, suppose that in order to
3496build a ``core-image-sato`` image for an "x86" multiconfig, the root
3497filesystem of an "arm" multiconfig must exist. This dependency is
3498essentially that the
3499```do_image`` <&YOCTO_DOCS_REF_URL;#ref-tasks-image>`__ task in the
3500``core-image-sato`` recipe depends on the completion of the
3501```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__ task of the
3502``core-image-minimal`` recipe.
3503
3504To enable dependencies in a multiple configuration build, you must
3505declare the dependencies in the recipe using the following statement
3506form: task_or_package[mcdepends] =
3507"mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
3508To better show how to use this statement, consider the example scenario
3509from the first paragraph of this section. The following statement needs
3510to be added to the recipe that builds the ``core-image-sato`` image:
3511do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" In this
3512example, the from_multiconfig is "x86". The to_multiconfig is "arm". The
3513task on which the ``do_image`` task in the recipe depends is the
3514``do_rootfs`` task from the ``core-image-minimal`` recipe associated
3515with the "arm" multiconfig.
3516
3517Once you set up this dependency, you can build the "x86" multiconfig
3518using a BitBake command as follows: $ bitbake mc:x86:core-image-sato
3519This command executes all the tasks needed to create the
3520``core-image-sato`` image for the "x86" multiconfig. Because of the
3521dependency, BitBake also executes through the ``do_rootfs`` task for the
3522"arm" multiconfig build.
3523
3524Having a recipe depend on the root filesystem of another build might not
3525seem that useful. Consider this change to the statement in the
3526``core-image-sato`` recipe: do_image[mcdepends] =
3527"mc:x86:arm:core-image-minimal:do_image" In this case, BitBake must
3528create the ``core-image-minimal`` image for the "arm" build since the
3529"x86" build depends on it.
3530
3531Because "x86" and "arm" are enabled for multiple configuration builds
3532and have separate configuration files, BitBake places the artifacts for
3533each build in the respective temporary build directories (i.e.
3534```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__).
3535
3536.. _building-an-initramfs-image:
3537
3538Building an Initial RAM Filesystem (initramfs) Image
3539----------------------------------------------------
3540
3541An initial RAM filesystem (initramfs) image provides a temporary root
3542filesystem used for early system initialization (e.g. loading of modules
3543needed to locate and mount the "real" root filesystem).
3544
3545.. note::
3546
3547 The initramfs image is the successor of initial RAM disk (initrd). It
3548 is a "copy in and out" (cpio) archive of the initial filesystem that
3549 gets loaded into memory during the Linux startup process. Because
3550 Linux uses the contents of the archive during initialization, the
3551 initramfs image needs to contain all of the device drivers and tools
3552 needed to mount the final root filesystem.
3553
3554Follow these steps to create an initramfs image:
3555
35561. *Create the initramfs Image Recipe:* You can reference the
3557 ``core-image-minimal-initramfs.bb`` recipe found in the
3558 ``meta/recipes-core`` directory of the `Source
3559 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ as an example
3560 from which to work.
3561
35622. *Decide if You Need to Bundle the initramfs Image Into the Kernel
3563 Image:* If you want the initramfs image that is built to be bundled
3564 in with the kernel image, set the
3565 ```INITRAMFS_IMAGE_BUNDLE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE>`__
3566 variable to "1" in your ``local.conf`` configuration file and set the
3567 ```INITRAMFS_IMAGE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE>`__
3568 variable in the recipe that builds the kernel image.
3569
3570 .. note::
3571
3572 It is recommended that you do bundle the initramfs image with the
3573 kernel image to avoid circular dependencies between the kernel
3574 recipe and the initramfs recipe should the initramfs image include
3575 kernel modules.
3576
3577 Setting the ``INITRAMFS_IMAGE_BUNDLE`` flag causes the initramfs
3578 image to be unpacked into the ``${B}/usr/`` directory. The unpacked
3579 initramfs image is then passed to the kernel's ``Makefile`` using the
3580 ```CONFIG_INITRAMFS_SOURCE`` <&YOCTO_DOCS_REF_URL;#var-CONFIG_INITRAMFS_SOURCE>`__
3581 variable, allowing the initramfs image to be built into the kernel
3582 normally.
3583
3584 .. note::
3585
3586 If you choose to not bundle the initramfs image with the kernel
3587 image, you are essentially using an
3588 Initial RAM Disk (initrd)
3589 . Creating an initrd is handled primarily through the
3590 INITRD_IMAGE
3591 ,
3592 INITRD_LIVE
3593 , and
3594 INITRD_IMAGE_LIVE
3595 variables. For more information, see the
3596 image-live.bbclass
3597 file.
3598
35993. *Optionally Add Items to the initramfs Image Through the initramfs
3600 Image Recipe:* If you add items to the initramfs image by way of its
3601 recipe, you should use
3602 ```PACKAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL>`__
3603 rather than
3604 ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__.
3605 ``PACKAGE_INSTALL`` gives more direct control of what is added to the
3606 image as compared to the defaults you might not necessarily want that
3607 are set by the ```image`` <&YOCTO_DOCS_REF_URL;#ref-classes-image>`__
3608 or ```core-image`` <&YOCTO_DOCS_REF_URL;#ref-classes-core-image>`__
3609 classes.
3610
36114. *Build the Kernel Image and the initramfs Image:* Build your kernel
3612 image using BitBake. Because the initramfs image recipe is a
3613 dependency of the kernel image, the initramfs image is built as well
3614 and bundled with the kernel image if you used the
3615 ```INITRAMFS_IMAGE_BUNDLE`` <&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE>`__
3616 variable described earlier.
3617
3618Building a Tiny System
3619----------------------
3620
3621Very small distributions have some significant advantages such as
3622requiring less on-die or in-package memory (cheaper), better performance
3623through efficient cache usage, lower power requirements due to less
3624memory, faster boot times, and reduced development overhead. Some
3625real-world examples where a very small distribution gives you distinct
3626advantages are digital cameras, medical devices, and small headless
3627systems.
3628
3629This section presents information that shows you how you can trim your
3630distribution to even smaller sizes than the ``poky-tiny`` distribution,
3631which is around 5 Mbytes, that can be built out-of-the-box using the
3632Yocto Project.
3633
3634.. _tiny-system-overview:
3635
3636Overview
3637~~~~~~~~
3638
3639The following list presents the overall steps you need to consider and
3640perform to create distributions with smaller root filesystems, achieve
3641faster boot times, maintain your critical functionality, and avoid
3642initial RAM disks:
3643
3644- `Determine your goals and guiding
3645 principles. <#goals-and-guiding-principles>`__
3646
3647- `Understand what contributes to your image
3648 size. <#understand-what-gives-your-image-size>`__
3649
3650- `Reduce the size of the root
3651 filesystem. <#trim-the-root-filesystem>`__
3652
3653- `Reduce the size of the kernel. <#trim-the-kernel>`__
3654
3655- `Eliminate packaging
3656 requirements. <#remove-package-management-requirements>`__
3657
3658- `Look for other ways to minimize
3659 size. <#look-for-other-ways-to-minimize-size>`__
3660
3661- `Iterate on the process. <#iterate-on-the-process>`__
3662
3663Goals and Guiding Principles
3664~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3665
3666Before you can reach your destination, you need to know where you are
3667going. Here is an example list that you can use as a guide when creating
3668very small distributions:
3669
3670- Determine how much space you need (e.g. a kernel that is 1 Mbyte or
3671 less and a root filesystem that is 3 Mbytes or less).
3672
3673- Find the areas that are currently taking 90% of the space and
3674 concentrate on reducing those areas.
3675
3676- Do not create any difficult "hacks" to achieve your goals.
3677
3678- Leverage the device-specific options.
3679
3680- Work in a separate layer so that you keep changes isolated. For
3681 information on how to create layers, see the "`Understanding and
3682 Creating Layers <#understanding-and-creating-layers>`__" section.
3683
3684.. _understand-what-gives-your-image-size:
3685
3686Understand What Contributes to Your Image Size
3687~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3688
3689It is easiest to have something to start with when creating your own
3690distribution. You can use the Yocto Project out-of-the-box to create the
3691``poky-tiny`` distribution. Ultimately, you will want to make changes in
3692your own distribution that are likely modeled after ``poky-tiny``.
3693
3694.. note::
3695
3696 To use
3697 poky-tiny
3698 in your build, set the
3699 DISTRO
3700 variable in your
3701 local.conf
3702 file to "poky-tiny" as described in the "
3703 Creating Your Own Distribution
3704 " section.
3705
3706Understanding some memory concepts will help you reduce the system size.
3707Memory consists of static, dynamic, and temporary memory. Static memory
3708is the TEXT (code), DATA (initialized data in the code), and BSS
3709(uninitialized data) sections. Dynamic memory represents memory that is
3710allocated at runtime: stacks, hash tables, and so forth. Temporary
3711memory is recovered after the boot process. This memory consists of
3712memory used for decompressing the kernel and for the ``__init__``
3713functions.
3714
3715To help you see where you currently are with kernel and root filesystem
3716sizes, you can use two tools found in the `Source
3717Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ in the
3718``scripts/tiny/`` directory:
3719
3720- ``ksize.py``: Reports component sizes for the kernel build objects.
3721
3722- ``dirsize.py``: Reports component sizes for the root filesystem.
3723
3724This next tool and command help you organize configuration fragments and
3725view file dependencies in a human-readable form:
3726
3727- ``merge_config.sh``: Helps you manage configuration files and
3728 fragments within the kernel. With this tool, you can merge individual
3729 configuration fragments together. The tool allows you to make
3730 overrides and warns you of any missing configuration options. The
3731 tool is ideal for allowing you to iterate on configurations, create
3732 minimal configurations, and create configuration files for different
3733 machines without having to duplicate your process.
3734
3735 The ``merge_config.sh`` script is part of the Linux Yocto kernel Git
3736 repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``,
3737 ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig``
3738 directory.
3739
3740 For more information on configuration fragments, see the "`Creating
3741 Configuration
3742 Fragments <&YOCTO_DOCS_KERNEL_DEV_URL;#creating-config-fragments>`__"
3743 section in the Yocto Project Linux Kernel Development Manual.
3744
3745- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command
3746 with these options brings up a Dependency Explorer from which you can
3747 view file dependencies. Understanding these dependencies allows you
3748 to make informed decisions when cutting out various pieces of the
3749 kernel and root filesystem.
3750
3751Trim the Root Filesystem
3752~~~~~~~~~~~~~~~~~~~~~~~~
3753
3754The root filesystem is made up of packages for booting, libraries, and
3755applications. To change things, you can configure how the packaging
3756happens, which changes the way you build them. You can also modify the
3757filesystem itself or select a different filesystem.
3758
3759First, find out what is hogging your root filesystem by running the
3760``dirsize.py`` script from your root directory: $ cd
3761root-directory-of-image $ dirsize.py 100000 > dirsize-100k.log $ cat
3762dirsize-100k.log You can apply a filter to the script to ignore files
3763under a certain size. The previous example filters out any files below
3764100 Kbytes. The sizes reported by the tool are uncompressed, and thus
3765will be smaller by a relatively constant factor in a compressed root
3766filesystem. When you examine your log file, you can focus on areas of
3767the root filesystem that take up large amounts of memory.
3768
3769You need to be sure that what you eliminate does not cripple the
3770functionality you need. One way to see how packages relate to each other
3771is by using the Dependency Explorer UI with the BitBake command: $ cd
3772image-directory $ bitbake -u taskexp -g image Use the interface to
3773select potential packages you wish to eliminate and see their dependency
3774relationships.
3775
3776When deciding how to reduce the size, get rid of packages that result in
3777minimal impact on the feature set. For example, you might not need a VGA
3778display. Or, you might be able to get by with ``devtmpfs`` and ``mdev``
3779instead of ``udev``.
3780
3781Use your ``local.conf`` file to make changes. For example, to eliminate
3782``udev`` and ``glib``, set the following in the local configuration
3783file: VIRTUAL-RUNTIME_dev_manager = ""
3784
3785Finally, you should consider exactly the type of root filesystem you
3786need to meet your needs while also reducing its size. For example,
3787consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an
3788``initramfs`` using ``initramfs``. Be aware that ``ext3`` requires a 1
3789Mbyte journal. If you are okay with running read-only, you do not need
3790this journal.
3791
3792.. note::
3793
3794 After each round of elimination, you need to rebuild your system and
3795 then use the tools to see the effects of your reductions.
3796
3797Trim the Kernel
3798~~~~~~~~~~~~~~~
3799
3800The kernel is built by including policies for hardware-independent
3801aspects. What subsystems do you enable? For what architecture are you
3802building? Which drivers do you build by default?
3803
3804.. note::
3805
3806 You can modify the kernel source if you want to help with boot time.
3807
3808Run the ``ksize.py`` script from the top-level Linux build directory to
3809get an idea of what is making up the kernel: $ cd
3810top-level-linux-build-directory $ ksize.py > ksize.log $ cat ksize.log
3811When you examine the log, you will see how much space is taken up with
3812the built-in ``.o`` files for drivers, networking, core kernel files,
3813filesystem, sound, and so forth. The sizes reported by the tool are
3814uncompressed, and thus will be smaller by a relatively constant factor
3815in a compressed kernel image. Look to reduce the areas that are large
3816and taking up around the "90% rule."
3817
3818To examine, or drill down, into any particular area, use the ``-d``
3819option with the script: $ ksize.py -d > ksize.log Using this option
3820breaks out the individual file information for each area of the kernel
3821(e.g. drivers, networking, and so forth).
3822
3823Use your log file to see what you can eliminate from the kernel based on
3824features you can let go. For example, if you are not going to need
3825sound, you do not need any drivers that support sound.
3826
3827After figuring out what to eliminate, you need to reconfigure the kernel
3828to reflect those changes during the next build. You could run
3829``menuconfig`` and make all your changes at once. However, that makes it
3830difficult to see the effects of your individual eliminations and also
3831makes it difficult to replicate the changes for perhaps another target
3832device. A better method is to start with no configurations using
3833``allnoconfig``, create configuration fragments for individual changes,
3834and then manage the fragments into a single configuration file using
3835``merge_config.sh``. The tool makes it easy for you to iterate using the
3836configuration change and build cycle.
3837
3838Each time you make configuration changes, you need to rebuild the kernel
3839and check to see what impact your changes had on the overall size.
3840
3841Remove Package Management Requirements
3842~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3843
3844Packaging requirements add size to the image. One way to reduce the size
3845of the image is to remove all the packaging requirements from the image.
3846This reduction includes both removing the package manager and its unique
3847dependencies as well as removing the package management data itself.
3848
3849To eliminate all the packaging requirements for an image, be sure that
3850"package-management" is not part of your
3851```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__
3852statement for the image. When you remove this feature, you are removing
3853the package manager as well as its dependencies from the root
3854filesystem.
3855
3856Look for Other Ways to Minimize Size
3857~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3858
3859Depending on your particular circumstances, other areas that you can
3860trim likely exist. The key to finding these areas is through tools and
3861methods described here combined with experimentation and iteration. Here
3862are a couple of areas to experiment with:
3863
3864- ``glibc``: In general, follow this process:
3865
3866 1. Remove ``glibc`` features from
3867 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
3868 that you think you do not need.
3869
3870 2. Build your distribution.
3871
3872 3. If the build fails due to missing symbols in a package, determine
3873 if you can reconfigure the package to not need those features. For
3874 example, change the configuration to not support wide character
3875 support as is done for ``ncurses``. Or, if support for those
3876 characters is needed, determine what ``glibc`` features provide
3877 the support and restore the configuration.
3878
3879 4. Rebuild and repeat the process.
3880
3881- ``busybox``: For BusyBox, use a process similar as described for
3882 ``glibc``. A difference is you will need to boot the resulting system
3883 to see if you are able to do everything you expect from the running
3884 system. You need to be sure to integrate configuration fragments into
3885 Busybox because BusyBox handles its own core features and then allows
3886 you to add configuration fragments on top.
3887
3888Iterate on the Process
3889~~~~~~~~~~~~~~~~~~~~~~
3890
3891If you have not reached your goals on system size, you need to iterate
3892on the process. The process is the same. Use the tools and see just what
3893is taking up 90% of the root filesystem and the kernel. Decide what you
3894can eliminate without limiting your device beyond what you need.
3895
3896Depending on your system, a good place to look might be Busybox, which
3897provides a stripped down version of Unix tools in a single, executable
3898file. You might be able to drop virtual terminal services or perhaps
3899ipv6.
3900
3901Building Images for More than One Machine
3902-----------------------------------------
3903
3904A common scenario developers face is creating images for several
3905different machines that use the same software environment. In this
3906situation, it is tempting to set the tunings and optimization flags for
3907each build specifically for the targeted hardware (i.e. "maxing out" the
3908tunings). Doing so can considerably add to build times and package feed
3909maintenance collectively for the machines. For example, selecting tunes
3910that are extremely specific to a CPU core used in a system might enable
3911some micro optimizations in GCC for that particular system but would
3912otherwise not gain you much of a performance difference across the other
3913systems as compared to using a more general tuning across all the builds
3914(e.g. setting ```DEFAULTTUNE`` <&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE>`__
3915specifically for each machine's build). Rather than "max out" each
3916build's tunings, you can take steps that cause the OpenEmbedded build
3917system to reuse software across the various machines where it makes
3918sense.
3919
3920If build speed and package feed maintenance are considerations, you
3921should consider the points in this section that can help you optimize
3922your tunings to best consider build times and package feed maintenance.
3923
3924- *Share the Build Directory:* If at all possible, share the
3925 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ across builds. The
3926 Yocto Project supports switching between different
3927 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ values in the same
3928 ``TMPDIR``. This practice is well supported and regularly used by
3929 developers when building for multiple machines. When you use the same
3930 ``TMPDIR`` for multiple machine builds, the OpenEmbedded build system
3931 can reuse the existing native and often cross-recipes for multiple
3932 machines. Thus, build time decreases.
3933
3934 .. note::
3935
3936 If
3937 DISTRO
3938 settings change or fundamental configuration settings such as the
3939 filesystem layout, you need to work with a clean
3940 TMPDIR
3941 . Sharing
3942 TMPDIR
3943 under these circumstances might work but since it is not
3944 guaranteed, you should use a clean
3945 TMPDIR
3946 .
3947
3948- *Enable the Appropriate Package Architecture:* By default, the
3949 OpenEmbedded build system enables three levels of package
3950 architectures: "all", "tune" or "package", and "machine". Any given
3951 recipe usually selects one of these package architectures (types) for
3952 its output. Depending for what a given recipe creates packages,
3953 making sure you enable the appropriate package architecture can
3954 directly impact the build time.
3955
3956 A recipe that just generates scripts can enable "all" architecture
3957 because there are no binaries to build. To specifically enable "all"
3958 architecture, be sure your recipe inherits the
3959 ```allarch`` <&YOCTO_DOCS_REF_URL;#ref-classes-allarch>`__ class.
3960 This class is useful for "all" architectures because it configures
3961 many variables so packages can be used across multiple architectures.
3962
3963 If your recipe needs to generate packages that are machine-specific
3964 or when one of the build or runtime dependencies is already
3965 machine-architecture dependent, which makes your recipe also
3966 machine-architecture dependent, make sure your recipe enables the
3967 "machine" package architecture through the
3968 ```MACHINE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH>`__
3969 variable: PACKAGE_ARCH = "${MACHINE_ARCH}" When you do not
3970 specifically enable a package architecture through the
3971 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__, The
3972 OpenEmbedded build system defaults to the
3973 ```TUNE_PKGARCH`` <&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH>`__ setting:
3974 PACKAGE_ARCH = "${TUNE_PKGARCH}"
3975
3976- *Choose a Generic Tuning File if Possible:* Some tunes are more
3977 generic and can run on multiple targets (e.g. an ``armv5`` set of
3978 packages could run on ``armv6`` and ``armv7`` processors in most
3979 cases). Similarly, ``i486`` binaries could work on ``i586`` and
3980 higher processors. You should realize, however, that advances on
3981 newer processor versions would not be used.
3982
3983 If you select the same tune for several different machines, the
3984 OpenEmbedded build system reuses software previously built, thus
3985 speeding up the overall build time. Realize that even though a new
3986 sysroot for each machine is generated, the software is not recompiled
3987 and only one package feed exists.
3988
3989- *Manage Granular Level Packaging:* Sometimes cases exist where
3990 injecting another level of package architecture beyond the three
3991 higher levels noted earlier can be useful. For example, consider how
3992 NXP (formerly Freescale) allows for the easy reuse of binary packages
3993 in their layer
3994 ```meta-freescale`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/>`__.
3995 In this example, the
3996 ```fsl-dynamic-packagearch`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>`__
3997 class shares GPU packages for i.MX53 boards because all boards share
3998 the AMD GPU. The i.MX6-based boards can do the same because all
3999 boards share the Vivante GPU. This class inspects the BitBake
4000 datastore to identify if the package provides or depends on one of
4001 the sub-architecture values. If so, the class sets the
4002 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__ value
4003 based on the ``MACHINE_SUBARCH`` value. If the package does not
4004 provide or depend on one of the sub-architecture values but it
4005 matches a value in the machine-specific filter, it sets
4006 ```MACHINE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH>`__. This
4007 behavior reduces the number of packages built and saves build time by
4008 reusing binaries.
4009
4010- *Use Tools to Debug Issues:* Sometimes you can run into situations
4011 where software is being rebuilt when you think it should not be. For
4012 example, the OpenEmbedded build system might not be using shared
4013 state between machines when you think it should be. These types of
4014 situations are usually due to references to machine-specific
4015 variables such as ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__,
4016 ```SERIAL_CONSOLES`` <&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES>`__,
4017 ```XSERVER`` <&YOCTO_DOCS_REF_URL;#var-XSERVER>`__,
4018 ```MACHINE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES>`__,
4019 and so forth in code that is supposed to only be tune-specific or
4020 when the recipe depends
4021 (```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__,
4022 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__,
4023 ```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__,
4024 ```RSUGGESTS`` <&YOCTO_DOCS_REF_URL;#var-RSUGGESTS>`__, and so forth)
4025 on some other recipe that already has
4026 ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__ defined
4027 as "${MACHINE_ARCH}".
4028
4029 .. note::
4030
4031 Patches to fix any issues identified are most welcome as these
4032 issues occasionally do occur.
4033
4034 For such cases, you can use some tools to help you sort out the
4035 situation:
4036
4037 - *``sstate-diff-machines.sh``:* You can find this tool in the
4038 ``scripts`` directory of the Source Repositories. See the comments
4039 in the script for information on how to use the tool.
4040
4041 - *BitBake's "-S printdiff" Option:* Using this option causes
4042 BitBake to try to establish the closest signature match it can
4043 (e.g. in the shared state cache) and then run ``bitbake-diffsigs``
4044 over the matches to determine the stamps and delta where these two
4045 stamp trees diverge.
4046
4047Building Software from an External Source
4048-----------------------------------------
4049
4050By default, the OpenEmbedded build system uses the `Build
4051Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ when building source
4052code. The build process involves fetching the source files, unpacking
4053them, and then patching them if necessary before the build takes place.
4054
4055Situations exist where you might want to build software from source
4056files that are external to and thus outside of the OpenEmbedded build
4057system. For example, suppose you have a project that includes a new BSP
4058with a heavily customized kernel. And, you want to minimize exposing the
4059build system to the development team so that they can focus on their
4060project and maintain everyone's workflow as much as possible. In this
4061case, you want a kernel source directory on the development machine
4062where the development occurs. You want the recipe's
4063```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable to point to
4064the external directory and use it as is, not copy it.
4065
4066To build from software that comes from an external source, all you need
4067to do is inherit the
4068```externalsrc`` <&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc>`__ class
4069and then set the
4070```EXTERNALSRC`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC>`__ variable to
4071point to your external source code. Here are the statements to put in
4072your ``local.conf`` file: INHERIT += "externalsrc"
4073EXTERNALSRC_pn-myrecipe = "path-to-your-source-tree"
4074
4075This next example shows how to accomplish the same thing by setting
4076``EXTERNALSRC`` in the recipe itself or in the recipe's append file:
4077EXTERNALSRC = "path" EXTERNALSRC_BUILD = "path"
4078
4079.. note::
4080
4081 In order for these settings to take effect, you must globally or
4082 locally inherit the
4083 externalsrc
4084 class.
4085
4086By default, ``externalsrc.bbclass`` builds the source code in a
4087directory separate from the external source directory as specified by
4088```EXTERNALSRC`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC>`__. If you need
4089to have the source built in the same directory in which it resides, or
4090some other nominated directory, you can set
4091```EXTERNALSRC_BUILD`` <&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD>`__
4092to point to that directory: EXTERNALSRC_BUILD_pn-myrecipe =
4093"path-to-your-source-tree"
4094
4095Replicating a Build Offline
4096---------------------------
4097
4098It can be useful to take a "snapshot" of upstream sources used in a
4099build and then use that "snapshot" later to replicate the build offline.
4100To do so, you need to first prepare and populate your downloads
4101directory your "snapshot" of files. Once your downloads directory is
4102ready, you can use it at any time and from any machine to replicate your
4103build.
4104
4105Follow these steps to populate your Downloads directory:
4106
41071. *Create a Clean Downloads Directory:* Start with an empty downloads
4108 directory (```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__). You
4109 start with an empty downloads directory by either removing the files
4110 in the existing directory or by setting ``DL_DIR`` to point to either
4111 an empty location or one that does not yet exist.
4112
41132. *Generate Tarballs of the Source Git Repositories:* Edit your
4114 ``local.conf`` configuration file as follows: DL_DIR =
4115 "/home/your-download-dir/" BB_GENERATE_MIRROR_TARBALLS = "1" During
4116 the fetch process in the next step, BitBake gathers the source files
4117 and creates tarballs in the directory pointed to by ``DL_DIR``. See
4118 the
4119 ```BB_GENERATE_MIRROR_TARBALLS`` <&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS>`__
4120 variable for more information.
4121
41223. *Populate Your Downloads Directory Without Building:* Use BitBake to
4123 fetch your sources but inhibit the build: $ bitbake target
4124 --runonly=fetch The downloads directory (i.e. ``${DL_DIR}``) now has
4125 a "snapshot" of the source files in the form of tarballs, which can
4126 be used for the build.
4127
41284. *Optionally Remove Any Git or other SCM Subdirectories From the
4129 Downloads Directory:* If you want, you can clean up your downloads
4130 directory by removing any Git or other Source Control Management
4131 (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs
4132 already contain these subdirectories.
4133
4134Once your downloads directory has everything it needs regarding source
4135files, you can create your "own-mirror" and build your target.
4136Understand that you can use the files to build the target offline from
4137any machine and at any time.
4138
4139Follow these steps to build your target using the files in the downloads
4140directory:
4141
41421. *Using Local Files Only:* Inside your ``local.conf`` file, add the
4143 ```SOURCE_MIRROR_URL`` <&YOCTO_DOCS_REF_URL;#var-SOURCE_MIRROR_URL>`__
4144 variable, inherit the
4145 ```own-mirrors`` <&YOCTO_DOCS_REF_URL;#ref-classes-own-mirrors>`__
4146 class, and use the
4147 ```BB_NO_NETWORK`` <&YOCTO_DOCS_BB_URL;#var-bb-BB_NO_NETWORK>`__
4148 variable to your ``local.conf``. SOURCE_MIRROR_URL ?=
4149 "file:///home/your-download-dir/" INHERIT += "own-mirrors"
4150 BB_NO_NETWORK = "1" The ``SOURCE_MIRROR_URL`` and ``own-mirror``
4151 class set up the system to use the downloads directory as your "own
4152 mirror". Using the ``BB_NO_NETWORK`` variable makes sure that
4153 BitBake's fetching process in step 3 stays local, which means files
4154 from your "own-mirror" are used.
4155
41562. *Start With a Clean Build:* You can start with a clean build by
4157 removing the
4158 ``${``\ ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__\ ``}``
4159 directory or using a new `Build
4160 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
4161
41623. *Build Your Target:* Use BitBake to build your target: $ bitbake
4163 target The build completes using the known local "snapshot" of source
4164 files from your mirror. The resulting tarballs for your "snapshot" of
4165 source files are in the downloads directory.
4166
4167 .. note::
4168
4169 The offline build does not work if recipes attempt to find the
4170 latest version of software by setting
4171 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ to
4172 ``${``\ ```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__\ ``}``:
4173 SRCREV = "${AUTOREV}" When a recipe sets ``SRCREV`` to
4174 ``${AUTOREV}``, the build system accesses the network in an
4175 attempt to determine the latest version of software from the SCM.
4176 Typically, recipes that use ``AUTOREV`` are custom or modified
4177 recipes. Recipes that reside in public repositories usually do not
4178 use ``AUTOREV``.
4179
4180 If you do have recipes that use ``AUTOREV``, you can take steps to
4181 still use the recipes in an offline build. Do the following:
4182
4183 1. Use a configuration generated by enabling `build
4184 history <#maintaining-build-output-quality>`__.
4185
4186 2. Use the ``buildhistory-collect-srcrevs`` command to collect the
4187 stored ``SRCREV`` values from the build's history. For more
4188 information on collecting these values, see the "`Build History
4189 Package Information <#build-history-package-information>`__"
4190 section.
4191
4192 3. Once you have the correct source revisions, you can modify
4193 those recipes to to set ``SRCREV`` to specific versions of the
4194 software.
4195
4196Speeding Up a Build
4197===================
4198
4199Build time can be an issue. By default, the build system uses simple
4200controls to try and maximize build efficiency. In general, the default
4201settings for all the following variables result in the most efficient
4202build times when dealing with single socket systems (i.e. a single CPU).
4203If you have multiple CPUs, you might try increasing the default values
4204to gain more speed. See the descriptions in the glossary for each
4205variable for more information:
4206
4207- ```BB_NUMBER_THREADS``: <&YOCTO_DOCS_REF_URL;#var-BB_NUMBER_THREADS>`__
4208 The maximum number of threads BitBake simultaneously executes.
4209
4210- ```BB_NUMBER_PARSE_THREADS``: <&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS>`__
4211 The number of threads BitBake uses during parsing.
4212
4213- ```PARALLEL_MAKE``: <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ Extra
4214 options passed to the ``make`` command during the
4215 ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ task in
4216 order to specify parallel compilation on the local build host.
4217
4218- ```PARALLEL_MAKEINST``: <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST>`__
4219 Extra options passed to the ``make`` command during the
4220 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task in
4221 order to specify parallel installation on the local build host.
4222
4223As mentioned, these variables all scale to the number of processor cores
4224available on the build system. For single socket systems, this
4225auto-scaling ensures that the build system fundamentally takes advantage
4226of potential parallel operations during the build based on the build
4227machine's capabilities.
4228
4229Following are additional factors that can affect build speed:
4230
4231- File system type: The file system type that the build is being
4232 performed on can also influence performance. Using ``ext4`` is
4233 recommended as compared to ``ext2`` and ``ext3`` due to ``ext4``
4234 improved features such as extents.
4235
4236- Disabling the updating of access time using ``noatime``: The
4237 ``noatime`` mount option prevents the build system from updating file
4238 and directory access times.
4239
4240- Setting a longer commit: Using the "commit=" mount option increases
4241 the interval in seconds between disk cache writes. Changing this
4242 interval from the five second default to something longer increases
4243 the risk of data loss but decreases the need to write to the disk,
4244 thus increasing the build performance.
4245
4246- Choosing the packaging backend: Of the available packaging backends,
4247 IPK is the fastest. Additionally, selecting a singular packaging
4248 backend also helps.
4249
4250- Using ``tmpfs`` for ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__
4251 as a temporary file system: While this can help speed up the build,
4252 the benefits are limited due to the compiler using ``-pipe``. The
4253 build system goes to some lengths to avoid ``sync()`` calls into the
4254 file system on the principle that if there was a significant failure,
4255 the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__
4256 contents could easily be rebuilt.
4257
4258- Inheriting the
4259 ```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ class:
4260 Inheriting this class has shown to speed up builds due to
4261 significantly lower amounts of data stored in the data cache as well
4262 as on disk. Inheriting this class also makes cleanup of
4263 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ faster, at the
4264 expense of being easily able to dive into the source code. File
4265 system maintainers have recommended that the fastest way to clean up
4266 large numbers of files is to reformat partitions rather than delete
4267 files due to the linear nature of partitions. This, of course,
4268 assumes you structure the disk partitions and file systems in a way
4269 that this is practical.
4270
4271Aside from the previous list, you should keep some trade offs in mind
4272that can help you speed up the build:
4273
4274- Remove items from
4275 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
4276 that you might not need.
4277
4278- Exclude debug symbols and other debug information: If you do not need
4279 these symbols and other debug information, disabling the ``*-dbg``
4280 package generation can speed up the build. You can disable this
4281 generation by setting the
4282 ```INHIBIT_PACKAGE_DEBUG_SPLIT`` <&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_DEBUG_SPLIT>`__
4283 variable to "1".
4284
4285- Disable static library generation for recipes derived from
4286 ``autoconf`` or ``libtool``: Following is an example showing how to
4287 disable static libraries and still provide an override to handle
4288 exceptions: STATICLIBCONF = "--disable-static"
4289 STATICLIBCONF_sqlite3-native = "" EXTRA_OECONF += "${STATICLIBCONF}"
4290
4291 .. note::
4292
4293 - Some recipes need static libraries in order to work correctly
4294 (e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides,
4295 as in the previous example, account for these kinds of
4296 exceptions.
4297
4298 - Some packages have packaging code that assumes the presence of
4299 the static libraries. If so, you might need to exclude them as
4300 well.
4301
4302.. _platdev-working-with-libraries:
4303
4304Working With Libraries
4305======================
4306
4307Libraries are an integral part of your system. This section describes
4308some common practices you might find helpful when working with libraries
4309to build your system:
4310
4311- `How to include static library
4312 files <#including-static-library-files>`__
4313
4314- `How to use the Multilib feature to combine multiple versions of
4315 library files into a single
4316 image <#combining-multiple-versions-library-files-into-one-image>`__
4317
4318- `How to install multiple versions of the same library in parallel on
4319 the same
4320 system <#installing-multiple-versions-of-the-same-library>`__
4321
4322Including Static Library Files
4323------------------------------
4324
4325If you are building a library and the library offers static linking, you
4326can control which static library files (``*.a`` files) get included in
4327the built library.
4328
4329The ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ and
4330```FILES_*`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ variables in the
4331``meta/conf/bitbake.conf`` configuration file define how files installed
4332by the ``do_install`` task are packaged. By default, the ``PACKAGES``
4333variable includes ``${PN}-staticdev``, which represents all static
4334library files.
4335
4336.. note::
4337
4338 Some previously released versions of the Yocto Project defined the
4339 static library files through
4340 ${PN}-dev
4341 .
4342
4343Following is part of the BitBake configuration file, where you can see
4344how the static library files are defined: PACKAGE_BEFORE_PN ?= ""
4345PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
4346${PACKAGE_BEFORE_PN} ${PN}" PACKAGES_DYNAMIC = "^${PN}-locale-.*" FILES
4347= "" FILES_${PN} = "${bindir}/\* ${sbindir}/\* ${libexecdir}/\*
4348${libdir}/lib*${SOLIBS} \\ ${sysconfdir} ${sharedstatedir}
4349${localstatedir} \\ ${base_bindir}/\* ${base_sbindir}/\* \\
4350${base_libdir}/*${SOLIBS} \\ ${base_prefix}/lib/udev/rules.d
4351${prefix}/lib/udev/rules.d \\ ${datadir}/${BPN} ${libdir}/${BPN}/\* \\
4352${datadir}/pixmaps ${datadir}/applications \\ ${datadir}/idl
4353${datadir}/omf ${datadir}/sounds \\ ${libdir}/bonobo/servers"
4354FILES_${PN}-bin = "${bindir}/\* ${sbindir}/*" FILES_${PN}-doc =
4355"${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \\
4356${datadir}/gnome/help" SECTION_${PN}-doc = "doc" FILES_SOLIBSDEV ?=
4357"${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
4358FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \\
4359${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \\
4360${datadir}/aclocal ${base_libdir}/*.o \\ ${libdir}/${BPN}/*.la
4361${base_libdir}/*.la" SECTION_${PN}-dev = "devel" ALLOW_EMPTY_${PN}-dev =
4362"1" RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" FILES_${PN}-staticdev
4363= "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
4364SECTION_${PN}-staticdev = "devel" RDEPENDS_${PN}-staticdev = "${PN}-dev
4365(= ${EXTENDPKGV})"
4366
4367.. _combining-multiple-versions-library-files-into-one-image:
4368
4369Combining Multiple Versions of Library Files into One Image
4370-----------------------------------------------------------
4371
4372The build system offers the ability to build libraries with different
4373target optimizations or architecture formats and combine these together
4374into one system image. You can link different binaries in the image
4375against the different libraries as needed for specific use cases. This
4376feature is called "Multilib."
4377
4378An example would be where you have most of a system compiled in 32-bit
4379mode using 32-bit libraries, but you have something large, like a
4380database engine, that needs to be a 64-bit application and uses 64-bit
4381libraries. Multilib allows you to get the best of both 32-bit and 64-bit
4382libraries.
4383
4384While the Multilib feature is most commonly used for 32 and 64-bit
4385differences, the approach the build system uses facilitates different
4386target optimizations. You could compile some binaries to use one set of
4387libraries and other binaries to use a different set of libraries. The
4388libraries could differ in architecture, compiler options, or other
4389optimizations.
4390
4391Several examples exist in the ``meta-skeleton`` layer found in the
4392`Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__:
4393
4394- ``conf/multilib-example.conf`` configuration file
4395
4396- ``conf/multilib-example2.conf`` configuration file
4397
4398- ``recipes-multilib/images/core-image-multilib-example.bb`` recipe
4399
4400Preparing to Use Multilib
4401~~~~~~~~~~~~~~~~~~~~~~~~~
4402
4403User-specific requirements drive the Multilib feature. Consequently,
4404there is no one "out-of-the-box" configuration that likely exists to
4405meet your needs.
4406
4407In order to enable Multilib, you first need to ensure your recipe is
4408extended to support multiple libraries. Many standard recipes are
4409already extended and support multiple libraries. You can check in the
4410``meta/conf/multilib.conf`` configuration file in the `Source
4411Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ to see how this is
4412done using the
4413```BBCLASSEXTEND`` <&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND>`__ variable.
4414Eventually, all recipes will be covered and this list will not be
4415needed.
4416
4417For the most part, the Multilib class extension works automatically to
4418extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where
4419``MLPREFIX`` is the particular multilib (e.g. "lib32-" or "lib64-").
4420Standard variables such as
4421```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__,
4422```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__,
4423```RPROVIDES`` <&YOCTO_DOCS_REF_URL;#var-RPROVIDES>`__,
4424```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__,
4425```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__, and
4426```PACKAGES_DYNAMIC`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC>`__ are
4427automatically extended by the system. If you are extending any manual
4428code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure
4429those names are extended correctly. This automatic extension code
4430resides in ``multilib.bbclass``.
4431
4432Using Multilib
4433~~~~~~~~~~~~~~
4434
4435After you have set up the recipes, you need to define the actual
4436combination of multiple libraries you want to build. You accomplish this
4437through your ``local.conf`` configuration file in the `Build
4438Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. An example
4439configuration would be as follows: MACHINE = "qemux86-64" require
4440conf/multilib.conf MULTILIBS = "multilib:lib32"
4441DEFAULTTUNE_virtclass-multilib-lib32 = "x86" IMAGE_INSTALL_append = "
4442lib32-glib-2.0" This example enables an additional library named
4443``lib32`` alongside the normal target packages. When combining these
4444"lib32" alternatives, the example uses "x86" for tuning. For information
4445on this particular tuning, see
4446``meta/conf/machine/include/ia32/arch-ia32.inc``.
4447
4448The example then includes ``lib32-glib-2.0`` in all the images, which
4449illustrates one method of including a multiple library dependency. You
4450can use a normal image build to include this dependency, for example: $
4451bitbake core-image-sato You can also build Multilib packages
4452specifically with a command like this: $ bitbake lib32-glib-2.0
4453
4454Additional Implementation Details
4455~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4456
4457Generic implementation details as well as details that are specific to
4458package management systems exist. Following are implementation details
4459that exist regardless of the package management system:
4460
4461- The typical convention used for the class extension code as used by
4462 Multilib assumes that all package names specified in
4463 ```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ that contain
4464 ``${PN}`` have ``${PN}`` at the start of the name. When that
4465 convention is not followed and ``${PN}`` appears at the middle or the
4466 end of a name, problems occur.
4467
4468- The ```TARGET_VENDOR`` <&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR>`__
4469 value under Multilib will be extended to "-vendormlmultilib" (e.g.
4470 "-pokymllib32" for a "lib32" Multilib with Poky). The reason for this
4471 slightly unwieldy contraction is that any "-" characters in the
4472 vendor string presently break Autoconf's ``config.sub``, and other
4473 separators are problematic for different reasons.
4474
4475For the RPM Package Management System, the following implementation
4476details exist:
4477
4478- A unique architecture is defined for the Multilib packages, along
4479 with creating a unique deploy folder under ``tmp/deploy/rpm`` in the
4480 `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. For
4481 example, consider ``lib32`` in a ``qemux86-64`` image. The possible
4482 architectures in the system are "all", "qemux86_64",
4483 "lib32_qemux86_64", and "lib32_x86".
4484
4485- The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM
4486 packaging. The naming for a normal RPM package and a Multilib RPM
4487 package in a ``qemux86-64`` system resolves to something similar to
4488 ``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``,
4489 respectively.
4490
4491- When installing a Multilib image, the RPM backend first installs the
4492 base image and then installs the Multilib libraries.
4493
4494- The build system relies on RPM to resolve the identical files in the
4495 two (or more) Multilib packages.
4496
4497For the IPK Package Management System, the following implementation
4498details exist:
4499
4500- The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK
4501 packaging. The naming for a normal RPM package and a Multilib IPK
4502 package in a ``qemux86-64`` system resolves to something like
4503 ``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw_x86.ipk``,
4504 respectively.
4505
4506- The IPK deploy folder is not modified with ``${MLPREFIX}`` because
4507 packages with and without the Multilib feature can exist in the same
4508 folder due to the ``${PN}`` differences.
4509
4510- IPK defines a sanity check for Multilib installation using certain
4511 rules for file comparison, overridden, etc.
4512
4513Installing Multiple Versions of the Same Library
4514------------------------------------------------
4515
4516Situations can exist where you need to install and use multiple versions
4517of the same library on the same system at the same time. These
4518situations almost always exist when a library API changes and you have
4519multiple pieces of software that depend on the separate versions of the
4520library. To accommodate these situations, you can install multiple
4521versions of the same library in parallel on the same system.
4522
4523The process is straightforward as long as the libraries use proper
4524versioning. With properly versioned libraries, all you need to do to
4525individually specify the libraries is create separate, appropriately
4526named recipes where the ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ part of
4527the name includes a portion that differentiates each library version
4528(e.g.the major part of the version number). Thus, instead of having a
4529single recipe that loads one version of a library (e.g. ``clutter``),
4530you provide multiple recipes that result in different versions of the
4531libraries you want. As an example, the following two recipes would allow
4532the two separate versions of the ``clutter`` library to co-exist on the
4533same system: clutter-1.6_1.6.20.bb clutter-1.8_1.8.4.bb Additionally, if
4534you have other recipes that depend on a given library, you need to use
4535the ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable to
4536create the dependency. Continuing with the same example, if you want to
4537have a recipe depend on the 1.8 version of the ``clutter`` library, use
4538the following in your recipe: DEPENDS = "clutter-1.8"
4539
4540Using x32 psABI
4541===============
4542
4543x32 processor-specific Application Binary Interface (`x32
4544psABI <https://software.intel.com/en-us/node/628948>`__) is a native
454532-bit processor-specific ABI for Intel 64 (x86-64) architectures. An
4546ABI defines the calling conventions between functions in a processing
4547environment. The interface determines what registers are used and what
4548the sizes are for various C data types.
4549
4550Some processing environments prefer using 32-bit applications even when
4551running on Intel 64-bit platforms. Consider the i386 psABI, which is a
4552very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not
4553provide efficient use and access of the Intel 64-bit processor
4554resources, leaving the system underutilized. Now consider the x86_64
4555psABI. This ABI is newer and uses 64-bits for data sizes and program
4556pointers. The extra bits increase the footprint size of the programs,
4557libraries, and also increases the memory and file system size
4558requirements. Executing under the x32 psABI enables user programs to
4559utilize CPU and system resources more efficiently while keeping the
4560memory footprint of the applications low. Extra bits are used for
4561registers but not for addressing mechanisms.
4562
4563The Yocto Project supports the final specifications of x32 psABI as
4564follows:
4565
4566- You can create packages and images in x32 psABI format on x86_64
4567 architecture targets.
4568
4569- You can successfully build recipes with the x32 toolchain.
4570
4571- You can create and boot ``core-image-minimal`` and
4572 ``core-image-sato`` images.
4573
4574- RPM Package Manager (RPM) support exists for x32 binaries.
4575
4576- Support for large images exists.
4577
4578To use the x32 psABI, you need to edit your ``conf/local.conf``
4579configuration file as follows: MACHINE = "qemux86-64" DEFAULTTUNE =
4580"x86-64-x32" baselib = "${@d.getVar('BASE_LIB_tune-' +
4581(d.getVar('DEFAULTTUNE') \\ or 'INVALID')) or 'lib'}" Once you have set
4582up your configuration file, use BitBake to build an image that supports
4583the x32 psABI. Here is an example: $ bitbake core-image-sato
4584
4585Enabling GObject Introspection Support
4586======================================
4587
4588`GObject
4589introspection <https://wiki.gnome.org/Projects/GObjectIntrospection>`__
4590is the standard mechanism for accessing GObject-based software from
4591runtime environments. GObject is a feature of the GLib library that
4592provides an object framework for the GNOME desktop and related software.
4593GObject Introspection adds information to GObject that allows objects
4594created within it to be represented across different programming
4595languages. If you want to construct GStreamer pipelines using Python, or
4596control UPnP infrastructure using Javascript and GUPnP, GObject
4597introspection is the only way to do it.
4598
4599This section describes the Yocto Project support for generating and
4600packaging GObject introspection data. GObject introspection data is a
4601description of the API provided by libraries built on top of GLib
4602framework, and, in particular, that framework's GObject mechanism.
4603GObject Introspection Repository (GIR) files go to ``-dev`` packages,
4604``typelib`` files go to main packages as they are packaged together with
4605libraries that are introspected.
4606
4607The data is generated when building such a library, by linking the
4608library with a small executable binary that asks the library to describe
4609itself, and then executing the binary and processing its output.
4610
4611Generating this data in a cross-compilation environment is difficult
4612because the library is produced for the target architecture, but its
4613code needs to be executed on the build host. This problem is solved with
4614the OpenEmbedded build system by running the code through QEMU, which
4615allows precisely that. Unfortunately, QEMU does not always work
4616perfectly as mentioned in the "`Known Issues <#known-issues>`__"
4617section.
4618
4619Enabling the Generation of Introspection Data
4620---------------------------------------------
4621
4622Enabling the generation of introspection data (GIR files) in your
4623library package involves the following:
4624
46251. Inherit the
4626 ```gobject-introspection`` <&YOCTO_DOCS_REF_URL;#ref-classes-gobject-introspection>`__
4627 class.
4628
46292. Make sure introspection is not disabled anywhere in the recipe or
4630 from anything the recipe includes. Also, make sure that
4631 "gobject-introspection-data" is not in
4632 ```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__
4633 and that "qemu-usermode" is not in
4634 ```MACHINE_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED>`__.
4635 If either of these conditions exist, nothing will happen.
4636
46373. Try to build the recipe. If you encounter build errors that look like
4638 something is unable to find ``.so`` libraries, check where these
4639 libraries are located in the source tree and add the following to the
4640 recipe: GIR_EXTRA_LIBS_PATH = "${B}/something/.libs"
4641
4642 .. note::
4643
4644 See recipes in the
4645 oe-core
4646 repository that use that
4647 GIR_EXTRA_LIBS_PATH
4648 variable as an example.
4649
46504. Look for any other errors, which probably mean that introspection
4651 support in a package is not entirely standard, and thus breaks down
4652 in a cross-compilation environment. For such cases, custom-made fixes
4653 are needed. A good place to ask and receive help in these cases is
4654 the `Yocto Project mailing
4655 lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__.
4656
4657.. note::
4658
4659 Using a library that no longer builds against the latest Yocto
4660 Project release and prints introspection related errors is a good
4661 candidate for the previous procedure.
4662
4663Disabling the Generation of Introspection Data
4664----------------------------------------------
4665
4666You might find that you do not want to generate introspection data. Or,
4667perhaps QEMU does not work on your build host and target architecture
4668combination. If so, you can use either of the following methods to
4669disable GIR file generations:
4670
4671- Add the following to your distro configuration:
4672 DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
4673 Adding this statement disables generating introspection data using
4674 QEMU but will still enable building introspection tools and libraries
4675 (i.e. building them does not require the use of QEMU).
4676
4677- Add the following to your machine configuration:
4678 MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode" Adding this
4679 statement disables the use of QEMU when building packages for your
4680 machine. Currently, this feature is used only by introspection
4681 recipes and has the same effect as the previously described option.
4682
4683 .. note::
4684
4685 Future releases of the Yocto Project might have other features
4686 affected by this option.
4687
4688If you disable introspection data, you can still obtain it through other
4689means such as copying the data from a suitable sysroot, or by generating
4690it on the target hardware. The OpenEmbedded build system does not
4691currently provide specific support for these techniques.
4692
4693Testing that Introspection Works in an Image
4694--------------------------------------------
4695
4696Use the following procedure to test if generating introspection data is
4697working in an image:
4698
46991. Make sure that "gobject-introspection-data" is not in
4700 ```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__
4701 and that "qemu-usermode" is not in
4702 ```MACHINE_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED>`__.
4703
47042. Build ``core-image-sato``.
4705
47063. Launch a Terminal and then start Python in the terminal.
4707
47084. Enter the following in the terminal: >>> from gi.repository import
4709 GLib >>> GLib.get_host_name()
4710
47115. For something a little more advanced, enter the following:
4712 http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html
4713
4714Known Issues
4715------------
4716
4717The following know issues exist for GObject Introspection Support:
4718
4719- ``qemu-ppc64`` immediately crashes. Consequently, you cannot build
4720 introspection data on that architecture.
4721
4722- x32 is not supported by QEMU. Consequently, introspection data is
4723 disabled.
4724
4725- musl causes transient GLib binaries to crash on assertion failures.
4726 Consequently, generating introspection data is disabled.
4727
4728- Because QEMU is not able to run the binaries correctly, introspection
4729 is disabled for some specific packages under specific architectures
4730 (e.g. ``gcr``, ``libsecret``, and ``webkit``).
4731
4732- QEMU usermode might not work properly when running 64-bit binaries
4733 under 32-bit host machines. In particular, "qemumips64" is known to
4734 not work under i686.
4735
4736.. _dev-optionally-using-an-external-toolchain:
4737
4738Optionally Using an External Toolchain
4739======================================
4740
4741You might want to use an external toolchain as part of your development.
4742If this is the case, the fundamental steps you need to accomplish are as
4743follows:
4744
4745- Understand where the installed toolchain resides. For cases where you
4746 need to build the external toolchain, you would need to take separate
4747 steps to build and install the toolchain.
4748
4749- Make sure you add the layer that contains the toolchain to your
4750 ``bblayers.conf`` file through the
4751 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable.
4752
4753- Set the ``EXTERNAL_TOOLCHAIN`` variable in your ``local.conf`` file
4754 to the location in which you installed the toolchain.
4755
4756A good example of an external toolchain used with the Yocto Project is
4757Mentor Graphics Sourcery G++ Toolchain. You can see information on how
4758to use that particular layer in the ``README`` file at
4759` <http://github.com/MentorEmbedded/meta-sourcery/>`__. You can find
4760further information by reading about the
4761```TCMODE`` <&YOCTO_DOCS_REF_URL;#var-TCMODE>`__ variable in the Yocto
4762Project Reference Manual's variable glossary.
4763
4764Creating Partitioned Images Using Wic
4765=====================================
4766
4767Creating an image for a particular hardware target using the
4768OpenEmbedded build system does not necessarily mean you can boot that
4769image as is on your device. Physical devices accept and boot images in
4770various ways depending on the specifics of the device. Usually,
4771information about the hardware can tell you what image format the device
4772requires. Should your device require multiple partitions on an SD card,
4773flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to
4774create the properly partitioned image.
4775
4776The ``wic`` command generates partitioned images from existing
4777OpenEmbedded build artifacts. Image generation is driven by partitioning
4778commands contained in an Openembedded kickstart file (``.wks``)
4779specified either directly on the command line or as one of a selection
4780of canned kickstart files as shown with the ``wic list images`` command
4781in the "`Using an Existing Kickstart
4782File <#using-a-provided-kickstart-file>`__" section. When you apply the
4783command to a given set of build artifacts, the result is an image or set
4784of images that can be directly written onto media and used on a
4785particular system.
4786
4787.. note::
4788
4789 For a kickstart file reference, see the "
4790 OpenEmbedded Kickstart (
4791 .wks
4792 ) Reference
4793 " Chapter in the Yocto Project Reference Manual.
4794
4795The ``wic`` command and the infrastructure it is based on is by
4796definition incomplete. The purpose of the command is to allow the
4797generation of customized images, and as such, was designed to be
4798completely extensible through a plugin interface. See the "`Using the
4799Wic PlugIn Interface <#wic-using-the-wic-plugin-interface>`__" section
4800for information on these plugins.
4801
4802This section provides some background information on Wic, describes what
4803you need to have in place to run the tool, provides instruction on how
4804to use the Wic utility, provides information on using the Wic plugins
4805interface, and provides several examples that show how to use Wic.
4806
4807.. _wic-background:
4808
4809Background
4810----------
4811
4812This section provides some background on the Wic utility. While none of
4813this information is required to use Wic, you might find it interesting.
4814
4815- The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The
4816 "oe" diphthong in "oeic" was promoted to the letter "w", because
4817 "oeic" is both difficult to remember and to pronounce.
4818
4819- Wic is loosely based on the Meego Image Creator (``mic``) framework.
4820 The Wic implementation has been heavily modified to make direct use
4821 of OpenEmbedded build artifacts instead of package installation and
4822 configuration, which are already incorporated within the OpenEmbedded
4823 artifacts.
4824
4825- Wic is a completely independent standalone utility that initially
4826 provides easier-to-use and more flexible replacements for an existing
4827 functionality in OE-Core's
4828 ```image-live`` <&YOCTO_DOCS_REF_URL;#ref-classes-image-live>`__
4829 class. The difference between Wic and those examples is that with Wic
4830 the functionality of those scripts is implemented by a
4831 general-purpose partitioning language, which is based on Redhat
4832 kickstart syntax.
4833
4834.. _wic-requirements:
4835
4836Requirements
4837------------
4838
4839In order to use the Wic utility with the OpenEmbedded Build system, your
4840system needs to meet the following requirements:
4841
4842- The Linux distribution on your development host must support the
4843 Yocto Project. See the "`Supported Linux
4844 Distributions <&YOCTO_DOCS_REF_URL;#detailed-supported-distros>`__"
4845 section in the Yocto Project Reference Manual for the list of
4846 distributions that support the Yocto Project.
4847
4848- The standard system utilities, such as ``cp``, must be installed on
4849 your development host system.
4850
4851- You must have sourced the build environment setup script (i.e.
4852 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) found in the
4853 `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
4854
4855- You need to have the build artifacts already available, which
4856 typically means that you must have already created an image using the
4857 Openembedded build system (e.g. ``core-image-minimal``). While it
4858 might seem redundant to generate an image in order to create an image
4859 using Wic, the current version of Wic requires the artifacts in the
4860 form generated by the OpenEmbedded build system.
4861
4862- You must build several native tools, which are built to run on the
4863 build system: $ bitbake parted-native dosfstools-native mtools-native
4864
4865- Include "wic" as part of the
4866 ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__
4867 variable.
4868
4869- Include the name of the `wic kickstart
4870 file <&YOCTO_DOCS_REF_URL;#openembedded-kickstart-wks-reference>`__
4871 as part of the ```WKS_FILE`` <&YOCTO_DOCS_REF_URL;#var-WKS_FILE>`__
4872 variable
4873
4874.. _wic-getting-help:
4875
4876Getting Help
4877------------
4878
4879You can get general help for the ``wic`` command by entering the ``wic``
4880command by itself or by entering the command with a help argument as
4881follows: $ wic -h $ wic --help $ wic help
4882
4883Currently, Wic supports seven commands: ``cp``, ``create``, ``help``,
4884``list``, ``ls``, ``rm``, and ``write``. You can get help for all these
4885commands except "help" by using the following form: $ wic help command
4886For example, the following command returns help for the ``write``
4887command: $ wic help write
4888
4889Wic supports help for three topics: ``overview``, ``plugins``, and
4890``kickstart``. You can get help for any topic using the following form:
4891$ wic help topic For example, the following returns overview help for
4892Wic: $ wic help overview
4893
4894One additional level of help exists for Wic. You can get help on
4895individual images through the ``list`` command. You can use the ``list``
4896command to return the available Wic images as follows: $ wic list images
4897genericx86 Create an EFI disk image for genericx86\* beaglebone-yocto
4898Create SD card image for Beaglebone edgerouter Create SD card image for
4899Edgerouter qemux86-directdisk Create a qemu machine 'pcbios' direct disk
4900image directdisk-gpt Create a 'pcbios' direct disk image mkefidisk
4901Create an EFI disk image directdisk Create a 'pcbios' direct disk image
4902systemd-bootdisk Create an EFI disk image with systemd-boot mkhybridiso
4903Create a hybrid ISO image sdimage-bootpart Create SD card image with a
4904boot partition directdisk-multi-rootfs Create multi rootfs image using
4905rootfs plugin directdisk-bootloader-config Create a 'pcbios' direct disk
4906image with custom bootloader config Once you know the list of available
4907Wic images, you can use ``help`` with the command to get help on a
4908particular image. For example, the following command returns help on the
4909"beaglebone-yocto" image: $ wic list beaglebone-yocto help Creates a
4910partitioned SD card image for Beaglebone. Boot files are located in the
4911first vfat partition.
4912
4913Operational Modes
4914-----------------
4915
4916You can use Wic in two different modes, depending on how much control
4917you need for specifying the Openembedded build artifacts that are used
4918for creating the image: Raw and Cooked:
4919
4920- *Raw Mode:* You explicitly specify build artifacts through Wic
4921 command-line arguments.
4922
4923- *Cooked Mode:* The current
4924 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ setting and image
4925 name are used to automatically locate and provide the build
4926 artifacts. You just supply a kickstart file and the name of the image
4927 from which to use artifacts.
4928
4929Regardless of the mode you use, you need to have the build artifacts
4930ready and available.
4931
4932Raw Mode
4933~~~~~~~~
4934
4935Running Wic in raw mode allows you to specify all the partitions through
4936the ``wic`` command line. The primary use for raw mode is if you have
4937built your kernel outside of the Yocto Project `Build
4938Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. In other words, you
4939can point to arbitrary kernel, root filesystem locations, and so forth.
4940Contrast this behavior with cooked mode where Wic looks in the Build
4941Directory (e.g. ``tmp/deploy/images/``\ machine).
4942
4943The general form of the ``wic`` command in raw mode is: $ wic create
4944wks_file options ... Where: wks_file: An OpenEmbedded kickstart file.
4945You can provide your own custom file or use a file from a set of
4946existing files as described by further options. optional arguments: -h,
4947--help show this help message and exit -o OUTDIR, --outdir OUTDIR name
4948of directory to create image in -e IMAGE_NAME, --image-name IMAGE_NAME
4949name of the image to use the artifacts from e.g. core- image-sato -r
4950ROOTFS_DIR, --rootfs-dir ROOTFS_DIR path to the /rootfs dir to use as
4951the .wks rootfs source -b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR path to
4952the dir containing the boot artifacts (e.g. /EFI or /syslinux dirs) to
4953use as the .wks bootimg source -k KERNEL_DIR, --kernel-dir KERNEL_DIR
4954path to the dir containing the kernel to use in the .wks bootimg -n
4955NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT path to the native
4956sysroot containing the tools to use to build the image -s,
4957--skip-build-check skip the build check -f, --build-rootfs build rootfs
4958-c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz} compress image with
4959specified compressor -m, --bmap generate .bmap --no-fstab-update Do not
4960change fstab file. -v VARS_DIR, --vars VARS_DIR directory with
4961<image>.env files that store bitbake variables -D, --debug output debug
4962information
4963
4964.. note::
4965
4966 You do not need root privileges to run Wic. In fact, you should not
4967 run as root when using the utility.
4968
4969Cooked Mode
4970~~~~~~~~~~~
4971
4972Running Wic in cooked mode leverages off artifacts in the Build
4973Directory. In other words, you do not have to specify kernel or root
4974filesystem locations as part of the command. All you need to provide is
4975a kickstart file and the name of the image from which to use artifacts
4976by using the "-e" option. Wic looks in the Build Directory (e.g.
4977``tmp/deploy/images/``\ machine) for artifacts.
4978
4979The general form of the ``wic`` command using Cooked Mode is as follows:
4980$ wic create wks_file -e IMAGE_NAME Where: wks_file: An OpenEmbedded
4981kickstart file. You can provide your own custom file or use a file from
4982a set of existing files provided with the Yocto Project release.
4983required argument: -e IMAGE_NAME, --image-name IMAGE_NAME name of the
4984image to use the artifacts from e.g. core- image-sato
4985
4986.. _using-a-provided-kickstart-file:
4987
4988Using an Existing Kickstart File
4989--------------------------------
4990
4991If you do not want to create your own kickstart file, you can use an
4992existing file provided by the Wic installation. As shipped, kickstart
4993files can be found in the Yocto Project `Source
4994Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ in the
4995following two locations: poky/meta-yocto-bsp/wic
4996poky/scripts/lib/wic/canned-wks Use the following command to list the
4997available kickstart files: $ wic list images genericx86 Create an EFI
4998disk image for genericx86\* beaglebone-yocto Create SD card image for
4999Beaglebone edgerouter Create SD card image for Edgerouter
5000qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
5001directdisk-gpt Create a 'pcbios' direct disk image mkefidisk Create an
5002EFI disk image directdisk Create a 'pcbios' direct disk image
5003systemd-bootdisk Create an EFI disk image with systemd-boot mkhybridiso
5004Create a hybrid ISO image sdimage-bootpart Create SD card image with a
5005boot partition directdisk-multi-rootfs Create multi rootfs image using
5006rootfs plugin directdisk-bootloader-config Create a 'pcbios' direct disk
5007image with custom bootloader config When you use an existing file, you
5008do not have to use the ``.wks`` extension. Here is an example in Raw
5009Mode that uses the ``directdisk`` file: $ wic create directdisk -r
5010rootfs_dir -b bootimg_dir \\ -k kernel_dir -n native_sysroot
5011
5012Here are the actual partition language commands used in the
5013``genericx86.wks`` file to generate an image: # short-description:
5014Create an EFI disk image for genericx86\* # long-description: Creates a
5015partitioned EFI disk image for genericx86\* machines part /boot --source
5016bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos
5017--active --align 1024 part / --source rootfs --ondisk sda --fstype=ext4
5018--label platform --align 1024 --use-uuid part swap --ondisk sda --size
501944 --label swap1 --fstype=swap bootloader --ptable gpt --timeout=5
5020--append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
5021
5022.. _wic-using-the-wic-plugin-interface:
5023
5024Using the Wic Plugin Interface
5025------------------------------
5026
5027You can extend and specialize Wic functionality by using Wic plugins.
5028This section explains the Wic plugin interface.
5029
5030.. note::
5031
5032 Wic plugins consist of "source" and "imager" plugins. Imager plugins
5033 are beyond the scope of this section.
5034
5035Source plugins provide a mechanism to customize partition content during
5036the Wic image generation process. You can use source plugins to map
5037values that you specify using ``--source`` commands in kickstart files
5038(i.e. ``*.wks``) to a plugin implementation used to populate a given
5039partition.
5040
5041.. note::
5042
5043 If you use plugins that have build-time dependencies (e.g. native
5044 tools, bootloaders, and so forth) when building a Wic image, you need
5045 to specify those dependencies using the
5046 WKS_FILE_DEPENDS
5047 variable.
5048
5049Source plugins are subclasses defined in plugin files. As shipped, the
5050Yocto Project provides several plugin files. You can see the source
5051plugin files that ship with the Yocto Project
5052`here <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source>`__.
5053Each of these plugin files contains source plugins that are designed to
5054populate a specific Wic image partition.
5055
5056Source plugins are subclasses of the ``SourcePlugin`` class, which is
5057defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example,
5058the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py``
5059file is a subclass of the ``SourcePlugin`` class, which is found in the
5060``pluginbase.py`` file.
5061
5062You can also implement source plugins in a layer outside of the Source
5063Repositories (external layer). To do so, be sure that your plugin files
5064are located in a directory whose path is
5065``scripts/lib/wic/plugins/source/`` within your external layer. When the
5066plugin files are located there, the source plugins they contain are made
5067available to Wic.
5068
5069When the Wic implementation needs to invoke a partition-specific
5070implementation, it looks for the plugin with the same name as the
5071``--source`` parameter used in the kickstart file given to that
5072partition. For example, if the partition is set up using the following
5073command in a kickstart file: part /boot --source bootimg-pcbios --ondisk
5074sda --label boot --active --align 1024 The methods defined as class
5075members of the matching source plugin (i.e. ``bootimg-pcbios``) in the
5076``bootimg-pcbios.py`` plugin file are used.
5077
5078To be more concrete, here is the corresponding plugin definition from
5079the ``bootimg-pcbios.py`` file for the previous command along with an
5080example method called by the Wic implementation when it needs to prepare
5081a partition using an implementation-specific function: . . . class
5082BootimgPcbiosPlugin(SourcePlugin): """ Create MBR boot partition and
5083install syslinux on it. """ name = 'bootimg-pcbios' . . . @classmethod
5084def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
5085oe_builddir, bootimg_dir, kernel_dir, rootfs_dir, native_sysroot): """
5086Called to do the actual content population for a partition i.e. it
5087'prepares' the partition to be incorporated into the image. In this
5088case, prepare content for legacy bios boot partition. """ . . . If a
5089subclass (plugin) itself does not implement a particular function, Wic
5090locates and uses the default version in the superclass. It is for this
5091reason that all source plugins are derived from the ``SourcePlugin``
5092class.
5093
5094The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines
5095a set of methods that source plugins can implement or override. Any
5096plugins (subclass of ``SourcePlugin``) that do not implement a
5097particular method inherit the implementation of the method from the
5098``SourcePlugin`` class. For more information, see the ``SourcePlugin``
5099class in the ``pluginbase.py`` file for details:
5100
5101The following list describes the methods implemented in the
5102``SourcePlugin`` class:
5103
5104- *``do_prepare_partition()``:* Called to populate a partition with
5105 actual content. In other words, the method prepares the final
5106 partition image that is incorporated into the disk image.
5107
5108- *``do_configure_partition()``:* Called before
5109 ``do_prepare_partition()`` to create custom configuration files for a
5110 partition (e.g. syslinux or grub configuration files).
5111
5112- *``do_install_disk()``:* Called after all partitions have been
5113 prepared and assembled into a disk image. This method provides a hook
5114 to allow finalization of a disk image (e.g. writing an MBR).
5115
5116- *``do_stage_partition()``:* Special content-staging hook called
5117 before ``do_prepare_partition()``. This method is normally empty.
5118
5119 Typically, a partition just uses the passed-in parameters (e.g. the
5120 unmodified value of ``bootimg_dir``). However, in some cases, things
5121 might need to be more tailored. As an example, certain files might
5122 additionally need to be taken from ``bootimg_dir + /boot``. This hook
5123 allows those files to be staged in a customized fashion.
5124
5125 .. note::
5126
5127 get_bitbake_var()
5128 allows you to access non-standard variables that you might want to
5129 use for this behavior.
5130
5131You can extend the source plugin mechanism. To add more hooks, create
5132more source plugin methods within ``SourcePlugin`` and the corresponding
5133derived subclasses. The code that calls the plugin methods uses the
5134``plugin.get_source_plugin_methods()`` function to find the method or
5135methods needed by the call. Retrieval of those methods is accomplished
5136by filling up a dict with keys that contain the method names of
5137interest. On success, these will be filled in with the actual methods.
5138See the Wic implementation for examples and details.
5139
5140.. _wic-usage-examples:
5141
5142Examples
5143--------
5144
5145This section provides several examples that show how to use the Wic
5146utility. All the examples assume the list of requirements in the
5147"`Requirements <#wic-requirements>`__" section have been met. The
5148examples assume the previously generated image is
5149``core-image-minimal``.
5150
5151.. _generate-an-image-using-a-provided-kickstart-file:
5152
5153Generate an Image using an Existing Kickstart File
5154~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5155
5156This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart
5157file: $ wic create mkefidisk -e core-image-minimal INFO: Building
5158wic-tools... . . . INFO: The new image(s) can be found here:
5159./mkefidisk-201804191017-sda.direct The following build artifacts were
5160used to create the image(s): ROOTFS_DIR:
5161/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5162BOOTIMG_DIR:
5163/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5164KERNEL_DIR:
5165/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5166NATIVE_SYSROOT:
5167/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5168INFO: The image(s) were created using OE kickstart file:
5169/home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
5170The previous example shows the easiest way to create an image by running
5171in cooked mode and supplying a kickstart file and the "-e" option to
5172point to the existing build artifacts. Your ``local.conf`` file needs to
5173have the ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable set
5174to the machine you are using, which is "qemux86" in this example.
5175
5176Once the image builds, the output provides image location, artifact use,
5177and kickstart file information.
5178
5179.. note::
5180
5181 You should always verify the details provided in the output to make
5182 sure that the image was indeed created exactly as expected.
5183
5184Continuing with the example, you can now write the image from the Build
5185Directory onto a USB stick, or whatever media for which you built your
5186image, and boot from the media. You can write the image by using
5187``bmaptool`` or ``dd``: $ oe-run-native bmaptool copy
5188mkefidisk-201804191017-sda.direct /dev/sdX or $ sudo dd
5189if=mkefidisk-201804191017-sda.direct of=/dev/sdX
5190
5191.. note::
5192
5193 For more information on how to use the
5194 bmaptool
5195 to flash a device with an image, see the "
5196 Flashing Images Using
5197 bmaptool
5198 " section.
5199
5200Using a Modified Kickstart File
5201~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5202
5203Because partitioned image creation is driven by the kickstart file, it
5204is easy to affect image creation by changing the parameters in the file.
5205This next example demonstrates that through modification of the
5206``directdisk-gpt`` kickstart file.
5207
5208As mentioned earlier, you can use the command ``wic list images`` to
5209show the list of existing kickstart files. The directory in which the
5210``directdisk-gpt.wks`` file resides is
5211``scripts/lib/image/canned-wks/``, which is located in the `Source
5212Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
5213Because available files reside in this directory, you can create and add
5214your own custom files to the directory. Subsequent use of the
5215``wic list images`` command would then include your kickstart files.
5216
5217In this example, the existing ``directdisk-gpt`` file already does most
5218of what is needed. However, for the hardware in this example, the image
5219will need to boot from ``sdb`` instead of ``sda``, which is what the
5220``directdisk-gpt`` kickstart file uses.
5221
5222The example begins by making a copy of the ``directdisk-gpt.wks`` file
5223in the ``scripts/lib/image/canned-wks`` directory and then by changing
5224the lines that specify the target disk from which to boot. $ cp
5225/home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \\
5226/home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5227Next, the example modifies the ``directdisksdb-gpt.wks`` file and
5228changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The
5229example changes the following two lines and leaves the remaining lines
5230untouched: part /boot --source bootimg-pcbios --ondisk sdb --label boot
5231--active --align 1024 part / --source rootfs --ondisk sdb --fstype=ext4
5232--label platform --align 1024 --use-uuid Once the lines are changed, the
5233example generates the ``directdisksdb-gpt`` image. The command points
5234the process at the ``core-image-minimal`` artifacts for the Next Unit of
5235Computing (nuc) ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ the
5236``local.conf``. $ wic create directdisksdb-gpt -e core-image-minimal
5237INFO: Building wic-tools... . . . Initialising tasks: 100%
5238\|#######################################\| Time: 0:00:01 NOTE:
5239Executing SetScene Tasks NOTE: Executing RunQueue Tasks NOTE: Tasks
5240Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and
5241all succeeded. INFO: Creating image(s)... INFO: The new image(s) can be
5242found here: ./directdisksdb-gpt-201710090938-sdb.direct The following
5243build artifacts were used to create the image(s): ROOTFS_DIR:
5244/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5245BOOTIMG_DIR:
5246/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5247KERNEL_DIR:
5248/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5249NATIVE_SYSROOT:
5250/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5251INFO: The image(s) were created using OE kickstart file:
5252/home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5253Continuing with the example, you can now directly ``dd`` the image to a
5254USB stick, or whatever media for which you built your image, and boot
5255the resulting media: $ sudo dd
5256if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb 140966+0
5257records in 140966+0 records out 72174592 bytes (72 MB, 69 MiB) copied,
525878.0282 s, 925 kB/s $ sudo eject /dev/sdb
5259
5260Using a Modified Kickstart File and Running in Raw Mode
5261~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5262
5263This next example manually specifies each build artifact (runs in Raw
5264Mode) and uses a modified kickstart file. The example also uses the
5265``-o`` option to cause Wic to create the output somewhere other than the
5266default output directory, which is the current directory: $ wic create
5267/home/stephano/my_yocto/test.wks -o /home/stephano/testwic \\
5268--rootfs-dir
5269/home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs
5270\\ --bootimg-dir
5271/home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5272\\ --kernel-dir
5273/home/stephano/build/master/build/tmp/deploy/images/qemux86 \\
5274--native-sysroot
5275/home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
5276INFO: Creating image(s)... INFO: The new image(s) can be found here:
5277/home/stephano/testwic/test-201710091445-sdb.direct The following build
5278artifacts were used to create the image(s): ROOTFS_DIR:
5279/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5280BOOTIMG_DIR:
5281/home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5282KERNEL_DIR:
5283/home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5284NATIVE_SYSROOT:
5285/home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5286INFO: The image(s) were created using OE kickstart file:
5287/home/stephano/my_yocto/test.wks For this example,
5288```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ did not have to be
5289specified in the ``local.conf`` file since the artifact is manually
5290specified.
5291
5292Using Wic to Manipulate an Image
5293~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5294
5295Wic image manipulation allows you to shorten turnaround time during
5296image development. For example, you can use Wic to delete the kernel
5297partition of a Wic image and then insert a newly built kernel. This
5298saves you time from having to rebuild the entire image each time you
5299modify the kernel.
5300
5301.. note::
5302
5303 In order to use Wic to manipulate a Wic image as in this example,
5304 your development machine must have the
5305 mtools
5306 package installed.
5307
5308The following example examines the contents of the Wic image, deletes
5309the existing kernel, and then inserts a new kernel:
5310
53111. *List the Partitions:* Use the ``wic ls`` command to list all the
5312 partitions in the Wic image: $ wic ls
5313 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic Num Start
5314 End Size Fstype 1 1048576 25041919 23993344 fat16 2 25165824 72157183
5315 46991360 ext4 The previous output shows two partitions in the
5316 ``core-image-minimal-qemux86.wic`` image.
5317
53182. *Examine a Particular Partition:* Use the ``wic ls`` command again
5319 but in a different form to examine a particular partition.
5320
5321 .. note::
5322
5323 You can get command usage on any Wic command using the following
5324 form:
5325 ::
5326
5327 $ wic help command
5328
5329
5330 For example, the following command shows you the various ways to
5331 use the
5332 wic ls
5333 command:
5334 ::
5335
5336 $ wic help ls
5337
5338
5339 The following command shows what is in Partition one: $ wic ls
5340 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1 Volume in
5341 drive : is boot Volume Serial Number is E894-1809 Directory for ::/
5342 libcom32 c32 186500 2017-10-09 16:06 libutil c32 24148 2017-10-09
5343 16:06 syslinux cfg 220 2017-10-09 16:06 vesamenu c32 27104 2017-10-09
5344 16:06 vmlinuz 6904608 2017-10-09 16:06 5 files 7 142 580 bytes 16 582
5345 656 bytes free The previous output shows five files, with the
5346 ``vmlinuz`` being the kernel.
5347
5348 .. note::
5349
5350 If you see the following error, you need to update or create a
5351 ~/.mtoolsrc
5352 file and be sure to have the line “mtools_skip_check=1“ in the
5353 file. Then, run the Wic command again:
5354 ::
5355
5356 ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
5357 output: Total number of sectors (47824) not a multiple of sectors per track (32)!
5358 Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
5359
5360
53613. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the
5362 ``vmlinuz`` file (kernel): $ wic rm
5363 tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
5364
53654. *Add In the New Kernel:* Use the ``wic cp`` command to add the
5366 updated kernel to the Wic image. Depending on how you built your
5367 kernel, it could be in different places. If you used ``devtool`` and
5368 an SDK to build your kernel, it resides in the ``tmp/work`` directory
5369 of the extensible SDK. If you used ``make`` to build the kernel, the
5370 kernel will be in the ``workspace/sources`` area.
5371
5372 The following example assumes ``devtool`` was used to build the
5373 kernel: cp
5374 ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage
5375 \\
5376 ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
5377 Once the new kernel is added back into the image, you can use the
5378 ``dd`` command or ```bmaptool`` <#flashing-images-using-bmaptool>`__
5379 to flash your wic image onto an SD card or USB stick and test your
5380 target.
5381
5382 .. note::
5383
5384 Using
5385 bmaptool
5386 is generally 10 to 20 times faster than using
5387 dd
5388 .
5389
5390Flashing Images Using ``bmaptool``
5391==================================
5392
5393A fast and easy way to flash an image to a bootable device is to use
5394Bmaptool, which is integrated into the OpenEmbedded build system.
5395Bmaptool is a generic tool that creates a file's block map (bmap) and
5396then uses that map to copy the file. As compared to traditional tools
5397such as dd or cp, Bmaptool can copy (or flash) large files like raw
5398system image files much faster.
5399
5400.. note::
5401
5402 - If you are using Ubuntu or Debian distributions, you can install
5403 the ``bmap-tools`` package using the following command and then
5404 use the tool without specifying ``PATH`` even from the root
5405 account: $ sudo apt-get install bmap-tools
5406
5407 - If you are unable to install the ``bmap-tools`` package, you will
5408 need to build Bmaptool before using it. Use the following command:
5409 $ bitbake bmap-tools-native
5410
5411Following, is an example that shows how to flash a Wic image. Realize
5412that while this example uses a Wic image, you can use Bmaptool to flash
5413any type of image. Use these steps to flash an image using Bmaptool:
5414
54151. *Update your ``local.conf`` File:* You need to have the following set
5416 in your ``local.conf`` file before building your image: IMAGE_FSTYPES
5417 += "wic wic.bmap"
5418
54192. *Get Your Image:* Either have your image ready (pre-built with the
5420 ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__
5421 setting previously mentioned) or take the step to build the image: $
5422 bitbake image
5423
54243. *Flash the Device:* Flash the device with the image by using Bmaptool
5425 depending on your particular setup. The following commands assume the
5426 image resides in the Build Directory's ``deploy/images/`` area:
5427
5428 - If you have write access to the media, use this command form: $
5429 oe-run-native bmap-tools-native bmaptool copy
5430 build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
5431
5432 - If you do not have write access to the media, set your permissions
5433 first and then use the same command form: $ sudo chmod 666
5434 /dev/sdX $ oe-run-native bmap-tools-native bmaptool copy
5435 build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
5436
5437For help on the ``bmaptool`` command, use the following command: $
5438bmaptool --help
5439
5440Making Images More Secure
5441=========================
5442
5443Security is of increasing concern for embedded devices. Consider the
5444issues and problems discussed in just this sampling of work found across
5445the Internet:
5446
5447- *"*\ `Security Risks of Embedded
5448 Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"*
5449 by Bruce Schneier
5450
5451- *"*\ `Internet Census
5452 2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna
5453 Botnet
5454
5455- *"*\ `Security Issues for Embedded
5456 Devices <http://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"*
5457 by Jake Edge
5458
5459When securing your image is of concern, there are steps, tools, and
5460variables that you can consider to help you reach the security goals you
5461need for your particular device. Not all situations are identical when
5462it comes to making an image secure. Consequently, this section provides
5463some guidance and suggestions for consideration when you want to make
5464your image more secure.
5465
5466.. note::
5467
5468 Because the security requirements and risks are different for every
5469 type of device, this section cannot provide a complete reference on
5470 securing your custom OS. It is strongly recommended that you also
5471 consult other sources of information on embedded Linux system
5472 hardening and on security.
5473
5474General Considerations
5475----------------------
5476
5477General considerations exist that help you create more secure images.
5478You should consider the following suggestions to help make your device
5479more secure:
5480
5481- Scan additional code you are adding to the system (e.g. application
5482 code) by using static analysis tools. Look for buffer overflows and
5483 other potential security problems.
5484
5485- Pay particular attention to the security for any web-based
5486 administration interface.
5487
5488 Web interfaces typically need to perform administrative functions and
5489 tend to need to run with elevated privileges. Thus, the consequences
5490 resulting from the interface's security becoming compromised can be
5491 serious. Look for common web vulnerabilities such as
5492 cross-site-scripting (XSS), unvalidated inputs, and so forth.
5493
5494 As with system passwords, the default credentials for accessing a
5495 web-based interface should not be the same across all devices. This
5496 is particularly true if the interface is enabled by default as it can
5497 be assumed that many end-users will not change the credentials.
5498
5499- Ensure you can update the software on the device to mitigate
5500 vulnerabilities discovered in the future. This consideration
5501 especially applies when your device is network-enabled.
5502
5503- Ensure you remove or disable debugging functionality before producing
5504 the final image. For information on how to do this, see the
5505 "`Considerations Specific to the OpenEmbedded Build
5506 System <#considerations-specific-to-the-openembedded-build-system>`__"
5507 section.
5508
5509- Ensure you have no network services listening that are not needed.
5510
5511- Remove any software from the image that is not needed.
5512
5513- Enable hardware support for secure boot functionality when your
5514 device supports this functionality.
5515
5516Security Flags
5517--------------
5518
5519The Yocto Project has security flags that you can enable that help make
5520your build output more secure. The security flags are in the
5521``meta/conf/distro/include/security_flags.inc`` file in your `Source
5522Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
5523
5524.. note::
5525
5526 Depending on the recipe, certain security flags are enabled and
5527 disabled by default.
5528
5529Use the following line in your ``local.conf`` file or in your custom
5530distribution configuration file to enable the security compiler and
5531linker flags for your build: require
5532conf/distro/include/security_flags.inc
5533
5534Considerations Specific to the OpenEmbedded Build System
5535--------------------------------------------------------
5536
5537You can take some steps that are specific to the OpenEmbedded build
5538system to make your images more secure:
5539
5540- Ensure "debug-tweaks" is not one of your selected
5541 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__.
5542 When creating a new project, the default is to provide you with an
5543 initial ``local.conf`` file that enables this feature using the
5544 ```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
5545 variable with the line: EXTRA_IMAGE_FEATURES = "debug-tweaks" To
5546 disable that feature, simply comment out that line in your
5547 ``local.conf`` file, or make sure ``IMAGE_FEATURES`` does not contain
5548 "debug-tweaks" before producing your final image. Among other things,
5549 leaving this in place sets the root password as blank, which makes
5550 logging in for debugging or inspection easy during development but
5551 also means anyone can easily log in during production.
5552
5553- It is possible to set a root password for the image and also to set
5554 passwords for any extra users you might add (e.g. administrative or
5555 service type users). When you set up passwords for multiple images or
5556 users, you should not duplicate passwords.
5557
5558 To set up passwords, use the
5559 ```extrausers`` <&YOCTO_DOCS_REF_URL;#ref-classes-extrausers>`__
5560 class, which is the preferred method. For an example on how to set up
5561 both root and user passwords, see the
5562 "```extrausers.bbclass`` <&YOCTO_DOCS_REF_URL;#ref-classes-extrausers>`__"
5563 section.
5564
5565 .. note::
5566
5567 When adding extra user accounts or setting a root password, be
5568 cautious about setting the same password on every device. If you
5569 do this, and the password you have set is exposed, then every
5570 device is now potentially compromised. If you need this access but
5571 want to ensure security, consider setting a different, random
5572 password for each device. Typically, you do this as a separate
5573 step after you deploy the image onto the device.
5574
5575- Consider enabling a Mandatory Access Control (MAC) framework such as
5576 SMACK or SELinux and tuning it appropriately for your device's usage.
5577 You can find more information in the
5578 ```meta-selinux`` <http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/>`__
5579 layer.
5580
5581Tools for Hardening Your Image
5582------------------------------
5583
5584The Yocto Project provides tools for making your image more secure. You
5585can find these tools in the ``meta-security`` layer of the `Yocto
5586Project Source Repositories <&YOCTO_GIT_URL;>`__.
5587
5588Creating Your Own Distribution
5589==============================
5590
5591When you build an image using the Yocto Project and do not alter any
5592distribution `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__, you are
5593creating a Poky distribution. If you wish to gain more control over
5594package alternative selections, compile-time options, and other
5595low-level configurations, you can create your own distribution.
5596
5597To create your own distribution, the basic steps consist of creating
5598your own distribution layer, creating your own distribution
5599configuration file, and then adding any needed code and Metadata to the
5600layer. The following steps provide some more detail:
5601
5602- *Create a layer for your new distro:* Create your distribution layer
5603 so that you can keep your Metadata and code for the distribution
5604 separate. It is strongly recommended that you create and use your own
5605 layer for configuration and code. Using your own layer as compared to
5606 just placing configurations in a ``local.conf`` configuration file
5607 makes it easier to reproduce the same build configuration when using
5608 multiple build machines. See the "`Creating a General Layer Using the
5609 ``bitbake-layers``
5610 Script <#creating-a-general-layer-using-the-bitbake-layers-script>`__"
5611 section for information on how to quickly set up a layer.
5612
5613- *Create the distribution configuration file:* The distribution
5614 configuration file needs to be created in the ``conf/distro``
5615 directory of your layer. You need to name it using your distribution
5616 name (e.g. ``mydistro.conf``).
5617
5618 .. note::
5619
5620 The
5621 DISTRO
5622 variable in your
5623 local.conf
5624 file determines the name of your distribution.
5625
5626 You can split out parts of your configuration file into include files
5627 and then "require" them from within your distribution configuration
5628 file. Be sure to place the include files in the
5629 ``conf/distro/include`` directory of your layer. A common example
5630 usage of include files would be to separate out the selection of
5631 desired version and revisions for individual recipes.
5632
5633 Your configuration file needs to set the following required
5634 variables: ```DISTRO_NAME`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME>`__
5635 ```DISTRO_VERSION`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION>`__
5636 These following variables are optional and you typically set them
5637 from the distribution configuration file:
5638 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
5639 ```DISTRO_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS>`__
5640 ```DISTRO_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS>`__
5641 ```TCLIBC`` <&YOCTO_DOCS_REF_URL;#var-TCLIBC>`__
5642
5643 .. tip::
5644
5645 If you want to base your distribution configuration file on the
5646 very basic configuration from OE-Core, you can use
5647 conf/distro/defaultsetup.conf
5648 as a reference and just include variables that differ as compared
5649 to
5650 defaultsetup.conf
5651 . Alternatively, you can create a distribution configuration file
5652 from scratch using the
5653 defaultsetup.conf
5654 file or configuration files from other distributions such as Poky
5655 or Angstrom as references.
5656
5657- *Provide miscellaneous variables:* Be sure to define any other
5658 variables for which you want to create a default or enforce as part
5659 of the distribution configuration. You can include nearly any
5660 variable from the ``local.conf`` file. The variables you use are not
5661 limited to the list in the previous bulleted item.
5662
5663- *Point to Your distribution configuration file:* In your
5664 ``local.conf`` file in the `Build
5665 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, set your
5666 ```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__ variable to point to
5667 your distribution's configuration file. For example, if your
5668 distribution's configuration file is named ``mydistro.conf``, then
5669 you point to it as follows: DISTRO = "mydistro"
5670
5671- *Add more to the layer if necessary:* Use your layer to hold other
5672 information needed for the distribution:
5673
5674 - Add recipes for installing distro-specific configuration files
5675 that are not already installed by another recipe. If you have
5676 distro-specific configuration files that are included by an
5677 existing recipe, you should add an append file (``.bbappend``) for
5678 those. For general information and recommendations on how to add
5679 recipes to your layer, see the "`Creating Your Own
5680 Layer <#creating-your-own-layer>`__" and "`Following Best
5681 Practices When Creating
5682 Layers <#best-practices-to-follow-when-creating-layers>`__"
5683 sections.
5684
5685 - Add any image recipes that are specific to your distribution.
5686
5687 - Add a ``psplash`` append file for a branded splash screen. For
5688 information on append files, see the "`Using .bbappend Files in
5689 Your Layer <#using-bbappend-files>`__" section.
5690
5691 - Add any other append files to make custom changes that are
5692 specific to individual recipes.
5693
5694Creating a Custom Template Configuration Directory
5695==================================================
5696
5697If you are producing your own customized version of the build system for
5698use by other users, you might want to customize the message shown by the
5699setup script or you might want to change the template configuration
5700files (i.e. ``local.conf`` and ``bblayers.conf``) that are created in a
5701new build directory.
5702
5703The OpenEmbedded build system uses the environment variable
5704``TEMPLATECONF`` to locate the directory from which it gathers
5705configuration information that ultimately ends up in the `Build
5706Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ ``conf`` directory.
5707By default, ``TEMPLATECONF`` is set as follows in the ``poky``
5708repository: TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} This is the
5709directory used by the build system to find templates from which to build
5710some key configuration files. If you look at this directory, you will
5711see the ``bblayers.conf.sample``, ``local.conf.sample``, and
5712``conf-notes.txt`` files. The build system uses these files to form the
5713respective ``bblayers.conf`` file, ``local.conf`` file, and display the
5714list of BitBake targets when running the setup script.
5715
5716To override these default configuration files with configurations you
5717want used within every new Build Directory, simply set the
5718``TEMPLATECONF`` variable to your directory. The ``TEMPLATECONF``
5719variable is set in the ``.templateconf`` file, which is in the top-level
5720`Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ folder
5721(e.g. ``poky``). Edit the ``.templateconf`` so that it can locate your
5722directory.
5723
5724Best practices dictate that you should keep your template configuration
5725directory in your custom distribution layer. For example, suppose you
5726have a layer named ``meta-mylayer`` located in your home directory and
5727you want your template configuration directory named ``myconf``.
5728Changing the ``.templateconf`` as follows causes the OpenEmbedded build
5729system to look in your directory and base its configuration files on the
5730``*.sample`` configuration files it finds. The final configuration files
5731(i.e. ``local.conf`` and ``bblayers.conf`` ultimately still end up in
5732your Build Directory, but they are based on your ``*.sample`` files.
5733TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf}
5734
5735Aside from the ``*.sample`` configuration files, the ``conf-notes.txt``
5736also resides in the default ``meta-poky/conf`` directory. The script
5737that sets up the build environment (i.e.
5738````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__) uses this file to
5739display BitBake targets as part of the script output. Customizing this
5740``conf-notes.txt`` file is a good way to make sure your list of custom
5741targets appears as part of the script's output.
5742
5743Here is the default list of targets displayed as a result of running
5744either of the setup scripts: You can now run 'bitbake <target>' Common
5745targets are: core-image-minimal core-image-sato meta-toolchain
5746meta-ide-support
5747
5748Changing the listed common targets is as easy as editing your version of
5749``conf-notes.txt`` in your custom template configuration directory and
5750making sure you have ``TEMPLATECONF`` set to your directory.
5751
5752.. _dev-saving-memory-during-a-build:
5753
5754Conserving Disk Space During Builds
5755===================================
5756
5757To help conserve disk space during builds, you can add the following
5758statement to your project's ``local.conf`` configuration file found in
5759the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT
5760+= "rm_work" Adding this statement deletes the work directory used for
5761building a recipe once the recipe is built. For more information on
5762"rm_work", see the
5763```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ class in the
5764Yocto Project Reference Manual.
5765
5766Working with Packages
5767=====================
5768
5769This section describes a few tasks that involve packages:
5770
5771- `Excluding packages from an
5772 image <#excluding-packages-from-an-image>`__
5773
5774- `Incrementing a binary package
5775 version <#incrementing-a-binary-package-version>`__
5776
5777- `Handling optional module
5778 packaging <#handling-optional-module-packaging>`__
5779
5780- `Using runtime package
5781 management <#using-runtime-package-management>`__
5782
5783- `Generating and using signed
5784 packages <#generating-and-using-signed-packages>`__
5785
5786- `Setting up and running package test
5787 (ptest) <#testing-packages-with-ptest>`__
5788
5789- `Creating node package manager (NPM)
5790 packages <#creating-node-package-manager-npm-packages>`__
5791
5792- `Adding custom metadata to
5793 packages <#adding-custom-metadata-to-packages>`__
5794
5795Excluding Packages from an Image
5796--------------------------------
5797
5798You might find it necessary to prevent specific packages from being
5799installed into an image. If so, you can use several variables to direct
5800the build system to essentially ignore installing recommended packages
5801or to not install a package at all.
5802
5803The following list introduces variables you can use to prevent packages
5804from being installed into your image. Each of these variables only works
5805with IPK and RPM package types. Support for Debian packages does not
5806exist. Also, you can use these variables from your ``local.conf`` file
5807or attach them to a specific image recipe by using a recipe name
5808override. For more detail on the variables, see the descriptions in the
5809Yocto Project Reference Manual's glossary chapter.
5810
5811- ```BAD_RECOMMENDATIONS`` <&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS>`__:
5812 Use this variable to specify "recommended-only" packages that you do
5813 not want installed.
5814
5815- ```NO_RECOMMENDATIONS`` <&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS>`__:
5816 Use this variable to prevent all "recommended-only" packages from
5817 being installed.
5818
5819- ```PACKAGE_EXCLUDE`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE>`__:
5820 Use this variable to prevent specific packages from being installed
5821 regardless of whether they are "recommended-only" or not. You need to
5822 realize that the build process could fail with an error when you
5823 prevent the installation of a package whose presence is required by
5824 an installed package.
5825
5826.. _incrementing-a-binary-package-version:
5827
5828Incrementing a Package Version
5829------------------------------
5830
5831This section provides some background on how binary package versioning
5832is accomplished and presents some of the services, variables, and
5833terminology involved.
5834
5835In order to understand binary package versioning, you need to consider
5836the following:
5837
5838- Binary Package: The binary package that is eventually built and
5839 installed into an image.
5840
5841- Binary Package Version: The binary package version is composed of two
5842 components - a version and a revision.
5843
5844 .. note::
5845
5846 Technically, a third component, the "epoch" (i.e.
5847 PE
5848 ) is involved but this discussion for the most part ignores
5849 PE
5850 .
5851
5852 The version and revision are taken from the
5853 ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ and
5854 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ variables, respectively.
5855
5856- ``PV``: The recipe version. ``PV`` represents the version of the
5857 software being packaged. Do not confuse ``PV`` with the binary
5858 package version.
5859
5860- ``PR``: The recipe revision.
5861
5862- ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__: The OpenEmbedded
5863 build system uses this string to help define the value of ``PV`` when
5864 the source code revision needs to be included in it.
5865
5866- `PR Service <https://wiki.yoctoproject.org/wiki/PR_Service>`__: A
5867 network-based service that helps automate keeping package feeds
5868 compatible with existing package manager applications such as RPM,
5869 APT, and OPKG.
5870
5871Whenever the binary package content changes, the binary package version
5872must change. Changing the binary package version is accomplished by
5873changing or "bumping" the ``PR`` and/or ``PV`` values. Increasing these
5874values occurs one of two ways:
5875
5876- Automatically using a Package Revision Service (PR Service).
5877
5878- Manually incrementing the ``PR`` and/or ``PV`` variables.
5879
5880Given a primary challenge of any build system and its users is how to
5881maintain a package feed that is compatible with existing package manager
5882applications such as RPM, APT, and OPKG, using an automated system is
5883much preferred over a manual system. In either system, the main
5884requirement is that binary package version numbering increases in a
5885linear fashion and that a number of version components exist that
5886support that linear progression. For information on how to ensure
5887package revisioning remains linear, see the "`Automatically Incrementing
5888a Binary Package Revision
5889Number <#automatically-incrementing-a-binary-package-revision-number>`__"
5890section.
5891
5892The following three sections provide related information on the PR
5893Service, the manual method for "bumping" ``PR`` and/or ``PV``, and on
5894how to ensure binary package revisioning remains linear.
5895
5896Working With a PR Service
5897~~~~~~~~~~~~~~~~~~~~~~~~~
5898
5899As mentioned, attempting to maintain revision numbers in the
5900`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ is error prone, inaccurate,
5901and causes problems for people submitting recipes. Conversely, the PR
5902Service automatically generates increasing numbers, particularly the
5903revision field, which removes the human element.
5904
5905.. note::
5906
5907 For additional information on using a PR Service, you can see the
5908 PR Service
5909 wiki page.
5910
5911The Yocto Project uses variables in order of decreasing priority to
5912facilitate revision numbering (i.e.
5913```PE`` <&YOCTO_DOCS_REF_URL;#var-PE>`__,
5914```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__, and
5915```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ for epoch, version, and
5916revision, respectively). The values are highly dependent on the policies
5917and procedures of a given distribution and package feed.
5918
5919Because the OpenEmbedded build system uses
5920"`signatures <&YOCTO_DOCS_OM_URL;#overview-checksums>`__", which are
5921unique to a given build, the build system knows when to rebuild
5922packages. All the inputs into a given task are represented by a
5923signature, which can trigger a rebuild when different. Thus, the build
5924system itself does not rely on the ``PR``, ``PV``, and ``PE`` numbers to
5925trigger a rebuild. The signatures, however, can be used to generate
5926these values.
5927
5928The PR Service works with both ``OEBasic`` and ``OEBasicHash``
5929generators. The value of ``PR`` bumps when the checksum changes and the
5930different generator mechanisms change signatures under different
5931circumstances.
5932
5933As implemented, the build system includes values from the PR Service
5934into the ``PR`` field as an addition using the form "``.x``" so ``r0``
5935becomes ``r0.1``, ``r0.2`` and so forth. This scheme allows existing
5936``PR`` values to be used for whatever reasons, which include manual
5937``PR`` bumps, should it be necessary.
5938
5939By default, the PR Service is not enabled or running. Thus, the packages
5940generated are just "self consistent". The build system adds and removes
5941packages and there are no guarantees about upgrade paths but images will
5942be consistent and correct with the latest changes.
5943
5944The simplest form for a PR Service is for it to exist for a single host
5945development system that builds the package feed (building system). For
5946this scenario, you can enable a local PR Service by setting
5947```PRSERV_HOST`` <&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST>`__ in your
5948``local.conf`` file in the `Build
5949Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: PRSERV_HOST =
5950"localhost:0" Once the service is started, packages will automatically
5951get increasing ``PR`` values and BitBake takes care of starting and
5952stopping the server.
5953
5954If you have a more complex setup where multiple host development systems
5955work against a common, shared package feed, you have a single PR Service
5956running and it is connected to each building system. For this scenario,
5957you need to start the PR Service using the ``bitbake-prserv`` command:
5958bitbake-prserv --host ip --port port --start In addition to
5959hand-starting the service, you need to update the ``local.conf`` file of
5960each building system as described earlier so each system points to the
5961server and port.
5962
5963It is also recommended you use build history, which adds some sanity
5964checks to binary package versions, in conjunction with the server that
5965is running the PR Service. To enable build history, add the following to
5966each building system's ``local.conf`` file: # It is recommended to
5967activate "buildhistory" for testing the PR service INHERIT +=
5968"buildhistory" BUILDHISTORY_COMMIT = "1" For information on build
5969history, see the "`Maintaining Build Output
5970Quality <#maintaining-build-output-quality>`__" section.
5971
5972.. note::
5973
5974 The OpenEmbedded build system does not maintain ``PR`` information as
5975 part of the shared state (sstate) packages. If you maintain an sstate
5976 feed, its expected that either all your building systems that
5977 contribute to the sstate feed use a shared PR Service, or you do not
5978 run a PR Service on any of your building systems. Having some systems
5979 use a PR Service while others do not leads to obvious problems.
5980
5981 For more information on shared state, see the "`Shared State
5982 Cache <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__" section in the
5983 Yocto Project Overview and Concepts Manual.
5984
5985Manually Bumping PR
5986~~~~~~~~~~~~~~~~~~~
5987
5988The alternative to setting up a PR Service is to manually "bump" the
5989```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ variable.
5990
5991If a committed change results in changing the package output, then the
5992value of the PR variable needs to be increased (or "bumped") as part of
5993that commit. For new recipes you should add the ``PR`` variable and set
5994its initial value equal to "r0", which is the default. Even though the
5995default value is "r0", the practice of adding it to a new recipe makes
5996it harder to forget to bump the variable when you make changes to the
5997recipe in future.
5998
5999If you are sharing a common ``.inc`` file with multiple recipes, you can
6000also use the ``INC_PR`` variable to ensure that the recipes sharing the
6001``.inc`` file are rebuilt when the ``.inc`` file itself is changed. The
6002``.inc`` file must set ``INC_PR`` (initially to "r0"), and all recipes
6003referring to it should set ``PR`` to "${INC_PR}.0" initially,
6004incrementing the last number when the recipe is changed. If the ``.inc``
6005file is changed then its ``INC_PR`` should be incremented.
6006
6007When upgrading the version of a binary package, assuming the ``PV``
6008changes, the ``PR`` variable should be reset to "r0" (or "${INC_PR}.0"
6009if you are using ``INC_PR``).
6010
6011Usually, version increases occur only to binary packages. However, if
6012for some reason ``PV`` changes but does not increase, you can increase
6013the ``PE`` variable (Package Epoch). The ``PE`` variable defaults to
6014"0".
6015
6016Binary package version numbering strives to follow the `Debian Version
6017Field Policy
6018Guidelines <http://www.debian.org/doc/debian-policy/ch-controlfields.html>`__.
6019These guidelines define how versions are compared and what "increasing"
6020a version means.
6021
6022.. _automatically-incrementing-a-binary-package-revision-number:
6023
6024Automatically Incrementing a Package Version Number
6025~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6026
6027When fetching a repository, BitBake uses the
6028```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ variable to determine
6029the specific source code revision from which to build. You set the
6030``SRCREV`` variable to
6031```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__ to cause the
6032OpenEmbedded build system to automatically use the latest revision of
6033the software: SRCREV = "${AUTOREV}"
6034
6035Furthermore, you need to reference ``SRCPV`` in ``PV`` in order to
6036automatically update the version whenever the revision of the source
6037code changes. Here is an example: PV = "1.0+git${SRCPV}" The
6038OpenEmbedded build system substitutes ``SRCPV`` with the following:
6039AUTOINC+source_code_revision The build system replaces the ``AUTOINC``
6040with a number. The number used depends on the state of the PR Service:
6041
6042- If PR Service is enabled, the build system increments the number,
6043 which is similar to the behavior of
6044 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__. This behavior results in
6045 linearly increasing package versions, which is desirable. Here is an
6046 example: hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6047 hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk
6048
6049- If PR Service is not enabled, the build system replaces the
6050 ``AUTOINC`` placeholder with zero (i.e. "0"). This results in
6051 changing the package version since the source revision is included.
6052 However, package versions are not increased linearly. Here is an
6053 example: hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6054 hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk
6055
6056In summary, the OpenEmbedded build system does not track the history of
6057binary package versions for this purpose. ``AUTOINC``, in this case, is
6058comparable to ``PR``. If PR server is not enabled, ``AUTOINC`` in the
6059package version is simply replaced by "0". If PR server is enabled, the
6060build system keeps track of the package versions and bumps the number
6061when the package revision changes.
6062
6063Handling Optional Module Packaging
6064----------------------------------
6065
6066Many pieces of software split functionality into optional modules (or
6067plugins) and the plugins that are built might depend on configuration
6068options. To avoid having to duplicate the logic that determines what
6069modules are available in your recipe or to avoid having to package each
6070module by hand, the OpenEmbedded build system provides functionality to
6071handle module packaging dynamically.
6072
6073To handle optional module packaging, you need to do two things:
6074
6075- Ensure the module packaging is actually done.
6076
6077- Ensure that any dependencies on optional modules from other recipes
6078 are satisfied by your recipe.
6079
6080Making Sure the Packaging is Done
6081~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6082
6083To ensure the module packaging actually gets done, you use the
6084``do_split_packages`` function within the ``populate_packages`` Python
6085function in your recipe. The ``do_split_packages`` function searches for
6086a pattern of files or directories under a specified path and creates a
6087package for each one it finds by appending to the
6088```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ variable and
6089setting the appropriate values for ``FILES_packagename``,
6090``RDEPENDS_packagename``, ``DESCRIPTION_packagename``, and so forth.
6091Here is an example from the ``lighttpd`` recipe: python
6092populate_packages_prepend () { lighttpd_libdir = d.expand('${libdir}')
6093do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$',
6094'lighttpd-module-%s', 'Lighttpd module for %s', extra_depends='') } The
6095previous example specifies a number of things in the call to
6096``do_split_packages``.
6097
6098- A directory within the files installed by your recipe through
6099 ``do_install`` in which to search.
6100
6101- A regular expression used to match module files in that directory. In
6102 the example, note the parentheses () that mark the part of the
6103 expression from which the module name should be derived.
6104
6105- A pattern to use for the package names.
6106
6107- A description for each package.
6108
6109- An empty string for ``extra_depends``, which disables the default
6110 dependency on the main ``lighttpd`` package. Thus, if a file in
6111 ``${libdir}`` called ``mod_alias.so`` is found, a package called
6112 ``lighttpd-module-alias`` is created for it and the
6113 ```DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-DESCRIPTION>`__ is set to
6114 "Lighttpd module for alias".
6115
6116Often, packaging modules is as simple as the previous example. However,
6117more advanced options exist that you can use within
6118``do_split_packages`` to modify its behavior. And, if you need to, you
6119can add more logic by specifying a hook function that is called for each
6120package. It is also perfectly acceptable to call ``do_split_packages``
6121multiple times if you have more than one set of modules to package.
6122
6123For more examples that show how to use ``do_split_packages``, see the
6124``connman.inc`` file in the ``meta/recipes-connectivity/connman/``
6125directory of the ``poky`` `source
6126repository <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__. You can
6127also find examples in ``meta/classes/kernel.bbclass``.
6128
6129Following is a reference that shows ``do_split_packages`` mandatory and
6130optional arguments: Mandatory arguments root The path in which to search
6131file_regex Regular expression to match searched files. Use parentheses
6132() to mark the part of this expression that should be used to derive the
6133module name (to be substituted where %s is used in other function
6134arguments as noted below) output_pattern Pattern to use for the package
6135names. Must include %s. description Description to set for each package.
6136Must include %s. Optional arguments postinst Postinstall script to use
6137for all packages (as a string) recursive True to perform a recursive
6138search - default False hook A hook function to be called for every
6139match. The function will be called with the following arguments (in the
6140order listed): f Full path to the file/directory match pkg The package
6141name file_regex As above output_pattern As above modulename The module
6142name derived using file_regex extra_depends Extra runtime dependencies
6143(RDEPENDS) to be set for all packages. The default value of None causes
6144a dependency on the main package (${PN}) - if you do not want this, pass
6145empty string '' for this parameter. aux_files_pattern Extra item(s) to
6146be added to FILES for each package. Can be a single string item or a
6147list of strings for multiple items. Must include %s. postrm postrm
6148script to use for all packages (as a string) allow_dirs True to allow
6149directories to be matched - default False prepend If True, prepend
6150created packages to PACKAGES instead of the default False which appends
6151them match_path match file_regex on the whole relative path to the root
6152rather than just the file name aux_files_pattern_verbatim Extra item(s)
6153to be added to FILES for each package, using the actual derived module
6154name rather than converting it to something legal for a package name.
6155Can be a single string item or a list of strings for multiple items.
6156Must include %s. allow_links True to allow symlinks to be matched -
6157default False summary Summary to set for each package. Must include %s;
6158defaults to description if not set.
6159
6160Satisfying Dependencies
6161~~~~~~~~~~~~~~~~~~~~~~~
6162
6163The second part for handling optional module packaging is to ensure that
6164any dependencies on optional modules from other recipes are satisfied by
6165your recipe. You can be sure these dependencies are satisfied by using
6166the ```PACKAGES_DYNAMIC`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC>`__
6167variable. Here is an example that continues with the ``lighttpd`` recipe
6168shown earlier: PACKAGES_DYNAMIC = "lighttpd-module-.*" The name
6169specified in the regular expression can of course be anything. In this
6170example, it is ``lighttpd-module-`` and is specified as the prefix to
6171ensure that any ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ and
6172```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ on a package
6173name starting with the prefix are satisfied during build time. If you
6174are using ``do_split_packages`` as described in the previous section,
6175the value you put in ``PACKAGES_DYNAMIC`` should correspond to the name
6176pattern specified in the call to ``do_split_packages``.
6177
6178Using Runtime Package Management
6179--------------------------------
6180
6181During a build, BitBake always transforms a recipe into one or more
6182packages. For example, BitBake takes the ``bash`` recipe and produces a
6183number of packages (e.g. ``bash``, ``bash-bashbug``,
6184``bash-completion``, ``bash-completion-dbg``, ``bash-completion-dev``,
6185``bash-completion-extra``, ``bash-dbg``, and so forth). Not all
6186generated packages are included in an image.
6187
6188In several situations, you might need to update, add, remove, or query
6189the packages on a target device at runtime (i.e. without having to
6190generate a new image). Examples of such situations include:
6191
6192- You want to provide in-the-field updates to deployed devices (e.g.
6193 security updates).
6194
6195- You want to have a fast turn-around development cycle for one or more
6196 applications that run on your device.
6197
6198- You want to temporarily install the "debug" packages of various
6199 applications on your device so that debugging can be greatly improved
6200 by allowing access to symbols and source debugging.
6201
6202- You want to deploy a more minimal package selection of your device
6203 but allow in-the-field updates to add a larger selection for
6204 customization.
6205
6206In all these situations, you have something similar to a more
6207traditional Linux distribution in that in-field devices are able to
6208receive pre-compiled packages from a server for installation or update.
6209Being able to install these packages on a running, in-field device is
6210what is termed "runtime package management".
6211
6212In order to use runtime package management, you need a host or server
6213machine that serves up the pre-compiled packages plus the required
6214metadata. You also need package manipulation tools on the target. The
6215build machine is a likely candidate to act as the server. However, that
6216machine does not necessarily have to be the package server. The build
6217machine could push its artifacts to another machine that acts as the
6218server (e.g. Internet-facing). In fact, doing so is advantageous for a
6219production environment as getting the packages away from the development
6220system's build directory prevents accidental overwrites.
6221
6222A simple build that targets just one device produces more than one
6223package database. In other words, the packages produced by a build are
6224separated out into a couple of different package groupings based on
6225criteria such as the target's CPU architecture, the target board, or the
6226C library used on the target. For example, a build targeting the
6227``qemux86`` device produces the following three package databases:
6228``noarch``, ``i586``, and ``qemux86``. If you wanted your ``qemux86``
6229device to be aware of all the packages that were available to it, you
6230would need to point it to each of these databases individually. In a
6231similar way, a traditional Linux distribution usually is configured to
6232be aware of a number of software repositories from which it retrieves
6233packages.
6234
6235Using runtime package management is completely optional and not required
6236for a successful build or deployment in any way. But if you want to make
6237use of runtime package management, you need to do a couple things above
6238and beyond the basics. The remainder of this section describes what you
6239need to do.
6240
6241.. _runtime-package-management-build:
6242
6243Build Considerations
6244~~~~~~~~~~~~~~~~~~~~
6245
6246This section describes build considerations of which you need to be
6247aware in order to provide support for runtime package management.
6248
6249When BitBake generates packages, it needs to know what format or formats
6250to use. In your configuration, you use the
6251```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
6252variable to specify the format:
6253
62541. Open the ``local.conf`` file inside your `Build
6255 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (e.g.
6256 ``~/poky/build/conf/local.conf``).
6257
62582. Select the desired package format as follows: PACKAGE_CLASSES ?=
6259 “package_packageformat” where packageformat can be "ipk", "rpm",
6260 "deb", or "tar" which are the supported package formats.
6261
6262 .. note::
6263
6264 Because the Yocto Project supports four different package formats,
6265 you can set the variable with more than one argument. However, the
6266 OpenEmbedded build system only uses the first argument when
6267 creating an image or Software Development Kit (SDK).
6268
6269If you would like your image to start off with a basic package database
6270containing the packages in your current build as well as to have the
6271relevant tools available on the target for runtime package management,
6272you can include "package-management" in the
6273```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__
6274variable. Including "package-management" in this configuration variable
6275ensures that when the image is assembled for your target, the image
6276includes the currently-known package databases as well as the
6277target-specific tools required for runtime package management to be
6278performed on the target. However, this is not strictly necessary. You
6279could start your image off without any databases but only include the
6280required on-target package tool(s). As an example, you could include
6281"opkg" in your
6282```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__ variable
6283if you are using the IPK package format. You can then initialize your
6284target's package database(s) later once your image is up and running.
6285
6286Whenever you perform any sort of build step that can potentially
6287generate a package or modify existing package, it is always a good idea
6288to re-generate the package index after the build by using the following
6289command: $ bitbake package-index It might be tempting to build the
6290package and the package index at the same time with a command such as
6291the following: $ bitbake some-package package-index Do not do this as
6292BitBake does not schedule the package index for after the completion of
6293the package you are building. Consequently, you cannot be sure of the
6294package index including information for the package you just built.
6295Thus, be sure to run the package update step separately after building
6296any packages.
6297
6298You can use the
6299```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6300```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6301and
6302```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6303variables to pre-configure target images to use a package feed. If you
6304do not define these variables, then manual steps as described in the
6305subsequent sections are necessary to configure the target. You should
6306set these variables before building the image in order to produce a
6307correctly configured image.
6308
6309When your build is complete, your packages reside in the
6310``${TMPDIR}/deploy/packageformat`` directory. For example, if
6311``${``\ ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__\ ``}`` is
6312``tmp`` and your selected package type is RPM, then your RPM packages
6313are available in ``tmp/deploy/rpm``.
6314
6315.. _runtime-package-management-server:
6316
6317Host or Server Machine Setup
6318~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6319
6320Although other protocols are possible, a server using HTTP typically
6321serves packages. If you want to use HTTP, then set up and configure a
6322web server such as Apache 2, lighttpd, or SimpleHTTPServer on the
6323machine serving the packages.
6324
6325To keep things simple, this section describes how to set up a
6326SimpleHTTPServer web server to share package feeds from the developer's
6327machine. Although this server might not be the best for a production
6328environment, the setup is simple and straight forward. Should you want
6329to use a different server more suited for production (e.g. Apache 2,
6330Lighttpd, or Nginx), take the appropriate steps to do so.
6331
6332From within the build directory where you have built an image based on
6333your packaging choice (i.e. the
6334```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
6335setting), simply start the server. The following example assumes a build
6336directory of ``~/poky/build/tmp/deploy/rpm`` and a ``PACKAGE_CLASSES``
6337setting of "package_rpm": $ cd ~/poky/build/tmp/deploy/rpm $ python -m
6338SimpleHTTPServer
6339
6340.. _runtime-package-management-target:
6341
6342Target Setup
6343~~~~~~~~~~~~
6344
6345Setting up the target differs depending on the package management
6346system. This section provides information for RPM, IPK, and DEB.
6347
6348.. _runtime-package-management-target-rpm:
6349
6350Using RPM
6351^^^^^^^^^
6352
6353The `Dandified Packaging
6354Tool <https://en.wikipedia.org/wiki/DNF_(software)>`__ (DNF) performs
6355runtime package management of RPM packages. In order to use DNF for
6356runtime package management, you must perform an initial setup on the
6357target machine for cases where the ``PACKAGE_FEED_*`` variables were not
6358set as part of the image that is running on the target. This means if
6359you built your image and did not not use these variables as part of the
6360build and your image is now running on the target, you need to perform
6361the steps in this section if you want to use runtime package management.
6362
6363.. note::
6364
6365 For information on the
6366 PACKAGE_FEED_\*
6367 variables, see
6368 PACKAGE_FEED_ARCHS
6369 ,
6370 PACKAGE_FEED_BASE_PATHS
6371 , and
6372 PACKAGE_FEED_URIS
6373 in the Yocto Project Reference Manual variables glossary.
6374
6375On the target, you must inform DNF that package databases are available.
6376You do this by creating a file named
6377``/etc/yum.repos.d/oe-packages.repo`` and defining the ``oe-packages``.
6378
6379As an example, assume the target is able to use the following package
6380databases: ``all``, ``i586``, and ``qemux86`` from a server named
6381``my.server``. The specifics for setting up the web server are up to
6382you. The critical requirement is that the URIs in the target repository
6383configuration point to the correct remote location for the feeds.
6384
6385.. note::
6386
6387 For development purposes, you can point the web server to the build
6388 system's
6389 deploy
6390 directory. However, for production use, it is better to copy the
6391 package directories to a location outside of the build area and use
6392 that location. Doing so avoids situations where the build system
6393 overwrites or changes the
6394 deploy
6395 directory.
6396
6397When telling DNF where to look for the package databases, you must
6398declare individual locations per architecture or a single location used
6399for all architectures. You cannot do both:
6400
6401- *Create an Explicit List of Architectures:* Define individual base
6402 URLs to identify where each package database is located:
6403 [oe-packages] baseurl=http://my.server/rpm/i586
6404 http://my.server/rpm/qemux86 http://my.server/rpm/all This example
6405 informs DNF about individual package databases for all three
6406 architectures.
6407
6408- *Create a Single (Full) Package Index:* Define a single base URL that
6409 identifies where a full package database is located: [oe-packages]
6410 baseurl=http://my.server/rpm This example informs DNF about a single
6411 package database that contains all the package index information for
6412 all supported architectures.
6413
6414Once you have informed DNF where to find the package databases, you need
6415to fetch them: # dnf makecache DNF is now able to find, install, and
6416upgrade packages from the specified repository or repositories.
6417
6418.. note::
6419
6420 See the
6421 DNF documentation
6422 for additional information.
6423
6424.. _runtime-package-management-target-ipk:
6425
6426Using IPK
6427^^^^^^^^^
6428
6429The ``opkg`` application performs runtime package management of IPK
6430packages. You must perform an initial setup for ``opkg`` on the target
6431machine if the
6432```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6433```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6434and
6435```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6436variables have not been set or the target image was built before the
6437variables were set.
6438
6439The ``opkg`` application uses configuration files to find available
6440package databases. Thus, you need to create a configuration file inside
6441the ``/etc/opkg/`` direction, which informs ``opkg`` of any repository
6442you want to use.
6443
6444As an example, suppose you are serving packages from a ``ipk/``
6445directory containing the ``i586``, ``all``, and ``qemux86`` databases
6446through an HTTP server named ``my.server``. On the target, create a
6447configuration file (e.g. ``my_repo.conf``) inside the ``/etc/opkg/``
6448directory containing the following: src/gz all http://my.server/ipk/all
6449src/gz i586 http://my.server/ipk/i586 src/gz qemux86
6450http://my.server/ipk/qemux86 Next, instruct ``opkg`` to fetch the
6451repository information: # opkg update The ``opkg`` application is now
6452able to find, install, and upgrade packages from the specified
6453repository.
6454
6455.. _runtime-package-management-target-deb:
6456
6457Using DEB
6458^^^^^^^^^
6459
6460The ``apt`` application performs runtime package management of DEB
6461packages. This application uses a source list file to find available
6462package databases. You must perform an initial setup for ``apt`` on the
6463target machine if the
6464```PACKAGE_FEED_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS>`__,
6465```PACKAGE_FEED_BASE_PATHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS>`__,
6466and
6467```PACKAGE_FEED_URIS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS>`__
6468variables have not been set or the target image was built before the
6469variables were set.
6470
6471To inform ``apt`` of the repository you want to use, you might create a
6472list file (e.g. ``my_repo.list``) inside the
6473``/etc/apt/sources.list.d/`` directory. As an example, suppose you are
6474serving packages from a ``deb/`` directory containing the ``i586``,
6475``all``, and ``qemux86`` databases through an HTTP server named
6476``my.server``. The list file should contain: deb
6477http://my.server/deb/all ./ deb http://my.server/deb/i586 ./ deb
6478http://my.server/deb/qemux86 ./ Next, instruct the ``apt`` application
6479to fetch the repository information: # apt-get update After this step,
6480``apt`` is able to find, install, and upgrade packages from the
6481specified repository.
6482
6483Generating and Using Signed Packages
6484------------------------------------
6485
6486In order to add security to RPM packages used during a build, you can
6487take steps to securely sign them. Once a signature is verified, the
6488OpenEmbedded build system can use the package in the build. If security
6489fails for a signed package, the build system aborts the build.
6490
6491This section describes how to sign RPM packages during a build and how
6492to use signed package feeds (repositories) when doing a build.
6493
6494Signing RPM Packages
6495~~~~~~~~~~~~~~~~~~~~
6496
6497To enable signing RPM packages, you must set up the following
6498configurations in either your ``local.config`` or ``distro.config``
6499file: # Inherit sign_rpm.bbclass to enable signing functionality INHERIT
6500+= " sign_rpm" # Define the GPG key that will be used for signing.
6501RPM_GPG_NAME = "key_name" # Provide passphrase for the key
6502RPM_GPG_PASSPHRASE = "passphrase"
6503
6504.. note::
6505
6506 Be sure to supply appropriate values for both
6507 key_name
6508 and
6509 passphrase
6510
6511Aside from the ``RPM_GPG_NAME`` and ``RPM_GPG_PASSPHRASE`` variables in
6512the previous example, two optional variables related to signing exist:
6513
6514- *``GPG_BIN``:* Specifies a ``gpg`` binary/wrapper that is executed
6515 when the package is signed.
6516
6517- *``GPG_PATH``:* Specifies the ``gpg`` home directory used when the
6518 package is signed.
6519
6520Processing Package Feeds
6521~~~~~~~~~~~~~~~~~~~~~~~~
6522
6523In addition to being able to sign RPM packages, you can also enable
6524signed package feeds for IPK and RPM packages.
6525
6526The steps you need to take to enable signed package feed use are similar
6527to the steps used to sign RPM packages. You must define the following in
6528your ``local.config`` or ``distro.config`` file: INHERIT +=
6529"sign_package_feed" PACKAGE_FEED_GPG_NAME = "key_name"
6530PACKAGE_FEED_GPG_PASSPHRASE_FILE = "path_to_file_containing_passphrase"
6531For signed package feeds, the passphrase must exist in a separate file,
6532which is pointed to by the ``PACKAGE_FEED_GPG_PASSPHRASE_FILE``
6533variable. Regarding security, keeping a plain text passphrase out of the
6534configuration is more secure.
6535
6536Aside from the ``PACKAGE_FEED_GPG_NAME`` and
6537``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` variables, three optional variables
6538related to signed package feeds exist:
6539
6540- *``GPG_BIN``:* Specifies a ``gpg`` binary/wrapper that is executed
6541 when the package is signed.
6542
6543- *``GPG_PATH``:* Specifies the ``gpg`` home directory used when the
6544 package is signed.
6545
6546- *``PACKAGE_FEED_GPG_SIGNATURE_TYPE``:* Specifies the type of ``gpg``
6547 signature. This variable applies only to RPM and IPK package feeds.
6548 Allowable values for the ``PACKAGE_FEED_GPG_SIGNATURE_TYPE`` are
6549 "ASC", which is the default and specifies ascii armored, and "BIN",
6550 which specifies binary.
6551
6552Testing Packages With ptest
6553---------------------------
6554
6555A Package Test (ptest) runs tests against packages built by the
6556OpenEmbedded build system on the target machine. A ptest contains at
6557least two items: the actual test, and a shell script (``run-ptest``)
6558that starts the test. The shell script that starts the test must not
6559contain the actual test - the script only starts the test. On the other
6560hand, the test can be anything from a simple shell script that runs a
6561binary and checks the output to an elaborate system of test binaries and
6562data files.
6563
6564The test generates output in the format used by Automake: result:
6565testname where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
6566the testname can be any identifying string.
6567
6568For a list of Yocto Project recipes that are already enabled with ptest,
6569see the `Ptest <https://wiki.yoctoproject.org/wiki/Ptest>`__ wiki page.
6570
6571.. note::
6572
6573 A recipe is "ptest-enabled" if it inherits the
6574 ptest
6575 class.
6576
6577Adding ptest to Your Build
6578~~~~~~~~~~~~~~~~~~~~~~~~~~
6579
6580To add package testing to your build, add the
6581```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__ and
6582```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
6583variables to your ``local.conf`` file, which is found in the `Build
6584Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__:
6585DISTRO_FEATURES_append = " ptest" EXTRA_IMAGE_FEATURES += "ptest-pkgs"
6586Once your build is complete, the ptest files are installed into the
6587``/usr/lib/package/ptest`` directory within the image, where ``package``
6588is the name of the package.
6589
6590Running ptest
6591~~~~~~~~~~~~~
6592
6593The ``ptest-runner`` package installs a shell script that loops through
6594all installed ptest test suites and runs them in sequence. Consequently,
6595you might want to add this package to your image.
6596
6597Getting Your Package Ready
6598~~~~~~~~~~~~~~~~~~~~~~~~~~
6599
6600In order to enable a recipe to run installed ptests on target hardware,
6601you need to prepare the recipes that build the packages you want to
6602test. Here is what you have to do for each recipe:
6603
6604- *Be sure the recipe inherits
6605 the*\ ```ptest`` <&YOCTO_DOCS_REF_URL;#ref-classes-ptest>`__\ *class:*
6606 Include the following line in each recipe: inherit ptest
6607
6608- *Create ``run-ptest``:* This script starts your test. Locate the
6609 script where you will refer to it using
6610 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__. Here is an
6611 example that starts a test for ``dbus``: #!/bin/sh cd test make -k
6612 runtest-TESTS
6613
6614- *Ensure dependencies are met:* If the test adds build or runtime
6615 dependencies that normally do not exist for the package (such as
6616 requiring "make" to run the test suite), use the
6617 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ and
6618 ```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ variables in
6619 your recipe in order for the package to meet the dependencies. Here
6620 is an example where the package has a runtime dependency on "make":
6621 RDEPENDS_${PN}-ptest += "make"
6622
6623- *Add a function to build the test suite:* Not many packages support
6624 cross-compilation of their test suites. Consequently, you usually
6625 need to add a cross-compilation function to the package.
6626
6627 Many packages based on Automake compile and run the test suite by
6628 using a single command such as ``make check``. However, the host
6629 ``make check`` builds and runs on the same computer, while
6630 cross-compiling requires that the package is built on the host but
6631 executed for the target architecture (though often, as in the case
6632 for ptest, the execution occurs on the host). The built version of
6633 Automake that ships with the Yocto Project includes a patch that
6634 separates building and execution. Consequently, packages that use the
6635 unaltered, patched version of ``make check`` automatically
6636 cross-compiles.
6637
6638 Regardless, you still must add a ``do_compile_ptest`` function to
6639 build the test suite. Add a function similar to the following to your
6640 recipe: do_compile_ptest() { oe_runmake buildtest-TESTS }
6641
6642- *Ensure special configurations are set:* If the package requires
6643 special configurations prior to compiling the test code, you must
6644 insert a ``do_configure_ptest`` function into the recipe.
6645
6646- *Install the test suite:* The ``ptest`` class automatically copies
6647 the file ``run-ptest`` to the target and then runs make
6648 ``install-ptest`` to run the tests. If this is not enough, you need
6649 to create a ``do_install_ptest`` function and make sure it gets
6650 called after the "make install-ptest" completes.
6651
6652Creating Node Package Manager (NPM) Packages
6653--------------------------------------------
6654
6655`NPM <https://en.wikipedia.org/wiki/Npm_(software)>`__ is a package
6656manager for the JavaScript programming language. The Yocto Project
6657supports the NPM `fetcher <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__. You can
6658use this fetcher in combination with
6659```devtool`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__ to create
6660recipes that produce NPM packages.
6661
6662Two workflows exist that allow you to create NPM packages using
6663``devtool``: the NPM registry modules method and the NPM project code
6664method.
6665
6666.. note::
6667
6668 While it is possible to create NPM recipes manually, using
6669 devtool
6670 is far simpler.
6671
6672Additionally, some requirements and caveats exist.
6673
6674.. _npm-package-creation-requirements:
6675
6676Requirements and Caveats
6677~~~~~~~~~~~~~~~~~~~~~~~~
6678
6679You need to be aware of the following before using ``devtool`` to create
6680NPM packages:
6681
6682- Of the two methods that you can use ``devtool`` to create NPM
6683 packages, the registry approach is slightly simpler. However, you
6684 might consider the project approach because you do not have to
6685 publish your module in the NPM registry
6686 (```npm-registry`` <https://docs.npmjs.com/misc/registry>`__), which
6687 is NPM's public registry.
6688
6689- Be familiar with
6690 ```devtool`` <&YOCTO_DOCS_REF_URL;#ref-devtool-reference>`__.
6691
6692- The NPM host tools need the native ``nodejs-npm`` package, which is
6693 part of the OpenEmbedded environment. You need to get the package by
6694 cloning the ` <https://github.com/openembedded/meta-openembedded>`__
6695 repository out of GitHub. Be sure to add the path to your local copy
6696 to your ``bblayers.conf`` file.
6697
6698- ``devtool`` cannot detect native libraries in module dependencies.
6699 Consequently, you must manually add packages to your recipe.
6700
6701- While deploying NPM packages, ``devtool`` cannot determine which
6702 dependent packages are missing on the target (e.g. the node runtime
6703 ``nodejs``). Consequently, you need to find out what files are
6704 missing and be sure they are on the target.
6705
6706- Although you might not need NPM to run your node package, it is
6707 useful to have NPM on your target. The NPM package name is
6708 ``nodejs-npm``.
6709
6710.. _npm-using-the-registry-modules-method:
6711
6712Using the Registry Modules Method
6713~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6714
6715This section presents an example that uses the ``cute-files`` module,
6716which is a file browser web application.
6717
6718.. note::
6719
6720 You must know the
6721 cute-files
6722 module version.
6723
6724The first thing you need to do is use ``devtool`` and the NPM fetcher to
6725create the recipe: $ devtool add
6726"npm://registry.npmjs.org;package=cute-files;version=1.0.2" The
6727``devtool add`` command runs ``recipetool create`` and uses the same
6728fetch URI to download each dependency and capture license details where
6729possible. The result is a generated recipe.
6730
6731The recipe file is fairly simple and contains every license that
6732``recipetool`` finds and includes the licenses in the recipe's
6733```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
6734variables. You need to examine the variables and look for those with
6735"unknown" in the ```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__
6736field. You need to track down the license information for "unknown"
6737modules and manually add the information to the recipe.
6738
6739``recipetool`` creates a "shrinkwrap" file for your recipe. Shrinkwrap
6740files capture the version of all dependent modules. Many packages do not
6741provide shrinkwrap files. ``recipetool`` create a shrinkwrap file as it
6742runs.
6743
6744.. note::
6745
6746 A package is created for each sub-module. This policy is the only
6747 practical way to have the licenses for all of the dependencies
6748 represented in the license manifest of the image.
6749
6750The ``devtool edit-recipe`` command lets you take a look at the recipe:
6751$ devtool edit-recipe cute-files SUMMARY = "Turn any folder on your
6752computer into a cute file browser, available on the local network."
6753LICENSE = "MIT & ISC & Unknown" LIC_FILES_CHKSUM =
6754"file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \\
6755file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502
6756\\
6757file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99
6758\\ ... SRC_URI = " \\
6759npm://registry.npmjs.org/;package=cute-files;version=${PV} \\
6760npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \\ " S = "${WORKDIR}/npm"
6761inherit npm LICENSE_${PN} = "MIT" LICENSE_${PN}-accepts = "MIT"
6762LICENSE_${PN}-array-flatten = "MIT" ... LICENSE_${PN}-vary = "MIT" Three
6763key points exist in the previous example:
6764
6765- ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ uses the NPM
6766 scheme so that the NPM fetcher is used.
6767
6768- ``recipetool`` collects all the license information. If a
6769 sub-module's license is unavailable, the sub-module's name appears in
6770 the comments.
6771
6772- The ``inherit npm`` statement causes the
6773 ```npm`` <&YOCTO_DOCS_REF_URL;#ref-classes-npm>`__ class to package
6774 up all the modules.
6775
6776You can run the following command to build the ``cute-files`` package: $
6777devtool build cute-files Remember that ``nodejs`` must be installed on
6778the target before your package.
6779
6780Assuming 192.168.7.2 for the target's IP address, use the following
6781command to deploy your package: $ devtool deploy-target -s cute-files
6782root@192.168.7.2 Once the package is installed on the target, you can
6783test the application:
6784
6785.. note::
6786
6787 Because of a know issue, you cannot simply run
6788 cute-files
6789 as you would if you had run
6790 npm install
6791 .
6792
6793$ cd /usr/lib/node_modules/cute-files $ node cute-files.js On a browser,
6794go to ``http://192.168.7.2:3000`` and you see the following:
6795
6796You can find the recipe in ``workspace/recipes/cute-files``. You can use
6797the recipe in any layer you choose.
6798
6799.. _npm-using-the-npm-projects-method:
6800
6801Using the NPM Projects Code Method
6802~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6803
6804Although it is useful to package modules already in the NPM registry,
6805adding ``node.js`` projects under development is a more common developer
6806use case.
6807
6808This section covers the NPM projects code method, which is very similar
6809to the "registry" approach described in the previous section. In the NPM
6810projects method, you provide ``devtool`` with an URL that points to the
6811source files.
6812
6813Replicating the same example, (i.e. ``cute-files``) use the following
6814command: $ devtool add https://github.com/martinaglv/cute-files.git The
6815recipe this command generates is very similar to the recipe created in
6816the previous section. However, the ``SRC_URI`` looks like the following:
6817SRC_URI = " \\ git://github.com/martinaglv/cute-files.git;protocol=https
6818\\ npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \\ " In this example,
6819the main module is taken from the Git repository and dependents are
6820taken from the NPM registry. Other than those differences, the recipe is
6821basically the same between the two methods. You can build and deploy the
6822package exactly as described in the previous section that uses the
6823registry modules method.
6824
6825Adding custom metadata to packages
6826----------------------------------
6827
6828The variable
6829```PACKAGE_ADD_METADATA`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ADD_METADATA>`__
6830can be used to add additional metadata to packages. This is reflected in
6831the package control/spec file. To take the ipk format for example, the
6832CONTROL file stored inside would contain the additional metadata as
6833additional lines.
6834
6835The variable can be used in multiple ways, including using suffixes to
6836set it for a specific package type and/or package. Note that the order
6837of precedence is the same as this list:
6838
6839- ``PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>``
6840
6841- ``PACKAGE_ADD_METADATA_<PKGTYPE>``
6842
6843- ``PACKAGE_ADD_METADATA_<PN>``
6844
6845- ``PACKAGE_ADD_METADATA``
6846
6847<PKGTYPE> is a parameter and expected to be a distinct name of specific
6848package type:
6849
6850- IPK for .ipk packages
6851
6852- DEB for .deb packages
6853
6854- RPM for .rpm packages
6855
6856<PN> is a parameter and expected to be a package name.
6857
6858The variable can contain multiple [one-line] metadata fields separated
6859by the literal sequence '\n'. The separator can be redefined using the
6860variable flag ``separator``.
6861
6862The following is an example that adds two custom fields for ipk
6863packages: PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup:
6864Applications/Spreadsheets"
6865
6866Efficiently Fetching Source Files During a Build
6867================================================
6868
6869The OpenEmbedded build system works with source files located through
6870the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. When
6871you build something using BitBake, a big part of the operation is
6872locating and downloading all the source tarballs. For images,
6873downloading all the source for various packages can take a significant
6874amount of time.
6875
6876This section shows you how you can use mirrors to speed up fetching
6877source files and how you can pre-fetch files all of which leads to more
6878efficient use of resources and time.
6879
6880Setting up Effective Mirrors
6881----------------------------
6882
6883A good deal that goes into a Yocto Project build is simply downloading
6884all of the source tarballs. Maybe you have been working with another
6885build system (OpenEmbedded or Angstrom) for which you have built up a
6886sizable directory of source tarballs. Or, perhaps someone else has such
6887a directory for which you have read access. If so, you can save time by
6888adding statements to your configuration file so that the build process
6889checks local directories first for existing tarballs before checking the
6890Internet.
6891
6892Here is an efficient way to set it up in your ``local.conf`` file:
6893SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" INHERIT +=
6894"own-mirrors" BB_GENERATE_MIRROR_TARBALLS = "1" # BB_NO_NETWORK = "1"
6895
6896In the previous example, the
6897```BB_GENERATE_MIRROR_TARBALLS`` <&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS>`__
6898variable causes the OpenEmbedded build system to generate tarballs of
6899the Git repositories and store them in the
6900```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ directory. Due to
6901performance reasons, generating and storing these tarballs is not the
6902build system's default behavior.
6903
6904You can also use the
6905```PREMIRRORS`` <&YOCTO_DOCS_REF_URL;#var-PREMIRRORS>`__ variable. For
6906an example, see the variable's glossary entry in the Yocto Project
6907Reference Manual.
6908
6909Getting Source Files and Suppressing the Build
6910----------------------------------------------
6911
6912Another technique you can use to ready yourself for a successive string
6913of build operations, is to pre-fetch all the source files without
6914actually starting a build. This technique lets you work through any
6915download issues and ultimately gathers all the source files into your
6916download directory
6917```build/downloads`` <&YOCTO_DOCS_REF_URL;#structure-build-downloads>`__,
6918which is located with ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__.
6919
6920Use the following BitBake command form to fetch all the necessary
6921sources without starting the build: $ bitbake target --runall=fetch This
6922variation of the BitBake command guarantees that you have all the
6923sources for that BitBake target should you disconnect from the Internet
6924and want to do the build later offline.
6925
6926Selecting an Initialization Manager
6927===================================
6928
6929By default, the Yocto Project uses SysVinit as the initialization
6930manager. However, support also exists for systemd, which is a full
6931replacement for init with parallel starting of services, reduced shell
6932overhead and other features that are used by many distributions.
6933
6934Within the system, SysVinit treats system components as services. These
6935services are maintained as shell scripts stored in the ``/etc/init.d/``
6936directory. Services organize into different run levels. This
6937organization is maintained by putting links to the services in the
6938``/etc/rcN.d/`` directories, where N/ is one of the following options:
6939"S", "0", "1", "2", "3", "4", "5", or "6".
6940
6941.. note::
6942
6943 Each runlevel has a dependency on the previous runlevel. This
6944 dependency allows the services to work properly.
6945
6946In comparison, systemd treats components as units. Using units is a
6947broader concept as compared to using a service. A unit includes several
6948different types of entities. Service is one of the types of entities.
6949The runlevel concept in SysVinit corresponds to the concept of a target
6950in systemd, where target is also a type of supported unit.
6951
6952In a SysVinit-based system, services load sequentially (i.e. one by one)
6953during and parallelization is not supported. With systemd, services
6954start in parallel. Needless to say, the method can have an impact on
6955system startup performance.
6956
6957If you want to use SysVinit, you do not have to do anything. But, if you
6958want to use systemd, you must take some steps as described in the
6959following sections.
6960
6961Using systemd Exclusively
6962-------------------------
6963
6964Set these variables in your distribution configuration file as follows:
6965DISTRO_FEATURES_append = " systemd" VIRTUAL-RUNTIME_init_manager =
6966"systemd" You can also prevent the SysVinit distribution feature from
6967being automatically enabled as follows:
6968DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" Doing so removes any
6969redundant SysVinit scripts.
6970
6971To remove initscripts from your image altogether, set this variable
6972also: VIRTUAL-RUNTIME_initscripts = ""
6973
6974For information on the backfill variable, see
6975```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__.
6976
6977Using systemd for the Main Image and Using SysVinit for the Rescue Image
6978------------------------------------------------------------------------
6979
6980Set these variables in your distribution configuration file as follows:
6981DISTRO_FEATURES_append = " systemd" VIRTUAL-RUNTIME_init_manager =
6982"systemd" Doing so causes your main image to use the
6983``packagegroup-core-boot.bb`` recipe and systemd. The rescue/minimal
6984image cannot use this package group. However, it can install SysVinit
6985and the appropriate packages will have support for both systemd and
6986SysVinit.
6987
6988.. _selecting-dev-manager:
6989
6990Selecting a Device Manager
6991==========================
6992
6993The Yocto Project provides multiple ways to manage the device manager
6994(``/dev``):
6995
6996- *Persistent and Pre-Populated\ ``/dev``:* For this case, the ``/dev``
6997 directory is persistent and the required device nodes are created
6998 during the build.
6999
7000- *Use ``devtmpfs`` with a Device Manager:* For this case, the ``/dev``
7001 directory is provided by the kernel as an in-memory file system and
7002 is automatically populated by the kernel at runtime. Additional
7003 configuration of device nodes is done in user space by a device
7004 manager like ``udev`` or ``busybox-mdev``.
7005
7006.. _static-dev-management:
7007
7008Using Persistent and Pre-Populated\ ``/dev``
7009--------------------------------------------
7010
7011To use the static method for device population, you need to set the
7012```USE_DEVFS`` <&YOCTO_DOCS_REF_URL;#var-USE_DEVFS>`__ variable to "0"
7013as follows: USE_DEVFS = "0"
7014
7015The content of the resulting ``/dev`` directory is defined in a Device
7016Table file. The
7017```IMAGE_DEVICE_TABLES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES>`__
7018variable defines the Device Table to use and should be set in the
7019machine or distro configuration file. Alternatively, you can set this
7020variable in your ``local.conf`` configuration file.
7021
7022If you do not define the ``IMAGE_DEVICE_TABLES`` variable, the default
7023``device_table-minimal.txt`` is used: IMAGE_DEVICE_TABLES =
7024"device_table-mymachine.txt"
7025
7026The population is handled by the ``makedevs`` utility during image
7027creation:
7028
7029.. _devtmpfs-dev-management:
7030
7031Using ``devtmpfs`` and a Device Manager
7032---------------------------------------
7033
7034To use the dynamic method for device population, you need to use (or be
7035sure to set) the ```USE_DEVFS`` <&YOCTO_DOCS_REF_URL;#var-USE_DEVFS>`__
7036variable to "1", which is the default: USE_DEVFS = "1" With this
7037setting, the resulting ``/dev`` directory is populated by the kernel
7038using ``devtmpfs``. Make sure the corresponding kernel configuration
7039variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux
7040kernel.
7041
7042All devices created by ``devtmpfs`` will be owned by ``root`` and have
7043permissions ``0600``.
7044
7045To have more control over the device nodes, you can use a device manager
7046like ``udev`` or ``busybox-mdev``. You choose the device manager by
7047defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or
7048distro configuration file. Alternatively, you can set this variable in
7049your ``local.conf`` configuration file: VIRTUAL-RUNTIME_dev_manager =
7050"udev" # Some alternative values # VIRTUAL-RUNTIME_dev_manager =
7051"busybox-mdev" # VIRTUAL-RUNTIME_dev_manager = "systemd"
7052
7053.. _platdev-appdev-srcrev:
7054
7055Using an External SCM
7056=====================
7057
7058If you're working on a recipe that pulls from an external Source Code
7059Manager (SCM), it is possible to have the OpenEmbedded build system
7060notice new recipe changes added to the SCM and then build the resulting
7061packages that depend on the new recipes by using the latest versions.
7062This only works for SCMs from which it is possible to get a sensible
7063revision number for changes. Currently, you can do this with Apache
7064Subversion (SVN), Git, and Bazaar (BZR) repositories.
7065
7066To enable this behavior, the ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__ of
7067the recipe needs to reference
7068```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__. Here is an example: PV =
7069"1.2.3+git${SRCPV}" Then, you can add the following to your
7070``local.conf``: SRCREV_pn-PN = "${AUTOREV}"
7071```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__ is the name of the recipe for
7072which you want to enable automatic source revision updating.
7073
7074If you do not want to update your local configuration file, you can add
7075the following directly to the recipe to finish enabling the feature:
7076SRCREV = "${AUTOREV}"
7077
7078The Yocto Project provides a distribution named ``poky-bleeding``, whose
7079configuration file contains the line: require
7080conf/distro/include/poky-floating-revisions.inc This line pulls in the
7081listed include file that contains numerous lines of exactly that form:
7082#SRCREV_pn-opkg-native ?= "${AUTOREV}" #SRCREV_pn-opkg-sdk ?=
7083"${AUTOREV}" #SRCREV_pn-opkg ?= "${AUTOREV}"
7084#SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" #SRCREV_pn-opkg-utils ?=
7085"${AUTOREV}" SRCREV_pn-gconf-dbus ?= "${AUTOREV}"
7086SRCREV_pn-matchbox-common ?= "${AUTOREV}" SRCREV_pn-matchbox-config-gtk
7087?= "${AUTOREV}" SRCREV_pn-matchbox-desktop ?= "${AUTOREV}"
7088SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" SRCREV_pn-matchbox-panel-2
7089?= "${AUTOREV}" SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}"
7090SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" SRCREV_pn-matchbox-wm ?=
7091"${AUTOREV}" SRCREV_pn-settings-daemon ?= "${AUTOREV}"
7092SRCREV_pn-screenshot ?= "${AUTOREV}" . . . These lines allow you to
7093experiment with building a distribution that tracks the latest
7094development source for numerous packages.
7095
7096.. note::
7097
7098 The
7099 poky-bleeding
7100 distribution is not tested on a regular basis. Keep this in mind if
7101 you use it.
7102
7103Creating a Read-Only Root Filesystem
7104====================================
7105
7106Suppose, for security reasons, you need to disable your target device's
7107root filesystem's write permissions (i.e. you need a read-only root
7108filesystem). Or, perhaps you are running the device's operating system
7109from a read-only storage device. For either case, you can customize your
7110image for that behavior.
7111
7112.. note::
7113
7114 Supporting a read-only root filesystem requires that the system and
7115 applications do not try to write to the root filesystem. You must
7116 configure all parts of the target system to write elsewhere, or to
7117 gracefully fail in the event of attempting to write to the root
7118 filesystem.
7119
7120Creating the Root Filesystem
7121----------------------------
7122
7123To create the read-only root filesystem, simply add the
7124"read-only-rootfs" feature to your image, normally in one of two ways.
7125The first way is to add the "read-only-rootfs" image feature in the
7126image's recipe file via the ``IMAGE_FEATURES`` variable: IMAGE_FEATURES
7127+= "read-only-rootfs" As an alternative, you can add the same feature
7128from within your build directory's ``local.conf`` file with the
7129associated ``EXTRA_IMAGE_FEATURES`` variable, as in:
7130EXTRA_IMAGE_FEATURES = "read-only-rootfs"
7131
7132For more information on how to use these variables, see the
7133"`Customizing Images Using Custom ``IMAGE_FEATURES`` and
7134``EXTRA_IMAGE_FEATURES`` <#usingpoky-extend-customimage-imagefeatures>`__"
7135section. For information on the variables, see
7136```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ and
7137```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__.
7138
7139Post-Installation Scripts
7140-------------------------
7141
7142It is very important that you make sure all post-Installation
7143(``pkg_postinst``) scripts for packages that are installed into the
7144image can be run at the time when the root filesystem is created during
7145the build on the host system. These scripts cannot attempt to run during
7146first-boot on the target device. With the "read-only-rootfs" feature
7147enabled, the build system checks during root filesystem creation to make
7148sure all post-installation scripts succeed. If any of these scripts
7149still need to be run after the root filesystem is created, the build
7150immediately fails. These build-time checks ensure that the build fails
7151rather than the target device fails later during its initial boot
7152operation.
7153
7154Most of the common post-installation scripts generated by the build
7155system for the out-of-the-box Yocto Project are engineered so that they
7156can run during root filesystem creation (e.g. post-installation scripts
7157for caching fonts). However, if you create and add custom scripts, you
7158need to be sure they can be run during this file system creation.
7159
7160Here are some common problems that prevent post-installation scripts
7161from running during root filesystem creation:
7162
7163- *Not using $D in front of absolute paths:* The build system defines
7164 ``$``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__ when the root
7165 filesystem is created. Furthermore, ``$D`` is blank when the script
7166 is run on the target device. This implies two purposes for ``$D``:
7167 ensuring paths are valid in both the host and target environments,
7168 and checking to determine which environment is being used as a method
7169 for taking appropriate actions.
7170
7171- *Attempting to run processes that are specific to or dependent on the
7172 target architecture:* You can work around these attempts by using
7173 native tools, which run on the host system, to accomplish the same
7174 tasks, or by alternatively running the processes under QEMU, which
7175 has the ``qemu_run_binary`` function. For more information, see the
7176 ```qemu`` <&YOCTO_DOCS_REF_URL;#ref-classes-qemu>`__ class.
7177
7178Areas With Write Access
7179-----------------------
7180
7181With the "read-only-rootfs" feature enabled, any attempt by the target
7182to write to the root filesystem at runtime fails. Consequently, you must
7183make sure that you configure processes and applications that attempt
7184these types of writes do so to directories with write access (e.g.
7185``/tmp`` or ``/var/run``).
7186
7187Maintaining Build Output Quality
7188================================
7189
7190Many factors can influence the quality of a build. For example, if you
7191upgrade a recipe to use a new version of an upstream software package or
7192you experiment with some new configuration options, subtle changes can
7193occur that you might not detect until later. Consider the case where
7194your recipe is using a newer version of an upstream package. In this
7195case, a new version of a piece of software might introduce an optional
7196dependency on another library, which is auto-detected. If that library
7197has already been built when the software is building, the software will
7198link to the built library and that library will be pulled into your
7199image along with the new software even if you did not want the library.
7200
7201The ```buildhistory`` <&YOCTO_DOCS_REF_URL;#ref-classes-buildhistory>`__
7202class exists to help you maintain the quality of your build output. You
7203can use the class to highlight unexpected and possibly unwanted changes
7204in the build output. When you enable build history, it records
7205information about the contents of each package and image and then
7206commits that information to a local Git repository where you can examine
7207the information.
7208
7209The remainder of this section describes the following:
7210
7211- How you can enable and disable build history
7212
7213- How to understand what the build history contains
7214
7215- How to limit the information used for build history
7216
7217- How to examine the build history from both a command-line and web
7218 interface
7219
7220Enabling and Disabling Build History
7221------------------------------------
7222
7223Build history is disabled by default. To enable it, add the following
7224``INHERIT`` statement and set the
7225```BUILDHISTORY_COMMIT`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT>`__
7226variable to "1" at the end of your ``conf/local.conf`` file found in the
7227`Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
7228"buildhistory" BUILDHISTORY_COMMIT = "1" Enabling build history as
7229previously described causes the OpenEmbedded build system to collect
7230build output information and commit it as a single commit to a local
7231`Git <&YOCTO_DOCS_OM_URL;#git>`__ repository.
7232
7233.. note::
7234
7235 Enabling build history increases your build times slightly,
7236 particularly for images, and increases the amount of disk space used
7237 during the build.
7238
7239You can disable build history by removing the previous statements from
7240your ``conf/local.conf`` file.
7241
7242Understanding What the Build History Contains
7243---------------------------------------------
7244
7245Build history information is kept in
7246``${``\ ```TOPDIR`` <&YOCTO_DOCS_REF_URL;#var-TOPDIR>`__\ ``}/buildhistory``
7247in the Build Directory as defined by the
7248```BUILDHISTORY_DIR`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_DIR>`__
7249variable. The following is an example abbreviated listing:
7250
7251At the top level, a ``metadata-revs`` file exists that lists the
7252revisions of the repositories for the enabled layers when the build was
7253produced. The rest of the data splits into separate ``packages``,
7254``images`` and ``sdk`` directories, the contents of which are described
7255as follows.
7256
7257Build History Package Information
7258~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7259
7260The history for each package contains a text file that has name-value
7261pairs with information about the package. For example,
7262``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
7263contains the following: PV = 1.22.1 PR = r32 RPROVIDES = RDEPENDS =
7264glibc (>= 2.20) update-alternatives-opkg RRECOMMENDS = busybox-syslog
7265busybox-udhcpc update-rc.d PKGSIZE = 540168 FILES = /usr/bin/\*
7266/usr/sbin/\* /usr/lib/busybox/\* /usr/lib/lib*.so.\* \\ /etc /com /var
7267/bin/\* /sbin/\* /lib/*.so.\* /lib/udev/rules.d \\ /usr/lib/udev/rules.d
7268/usr/share/busybox /usr/lib/busybox/\* \\ /usr/share/pixmaps
7269/usr/share/applications /usr/share/idl \\ /usr/share/omf
7270/usr/share/sounds /usr/lib/bonobo/servers FILELIST = /bin/busybox
7271/bin/busybox.nosuid /bin/busybox.suid /bin/sh \\
7272/etc/busybox.links.nosuid /etc/busybox.links.suid Most of these
7273name-value pairs correspond to variables used to produce the package.
7274The exceptions are ``FILELIST``, which is the actual list of files in
7275the package, and ``PKGSIZE``, which is the total size of files in the
7276package in bytes.
7277
7278A file also exists that corresponds to the recipe from which the package
7279came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``): PV
7280= 1.22.1 PR = r32 DEPENDS = initscripts kern-tools-native
7281update-rc.d-native \\ virtual/i586-poky-linux-compilerlibs
7282virtual/i586-poky-linux-gcc \\ virtual/libc virtual/update-alternatives
7283PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \\
7284busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \\
7285busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
7286
7287Finally, for those recipes fetched from a version control system (e.g.,
7288Git), a file exists that lists source revisions that are specified in
7289the recipe and lists the actual revisions used during the build. Listed
7290and actual revisions might differ when
7291```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ is set to
7292${```AUTOREV`` <&YOCTO_DOCS_REF_URL;#var-AUTOREV>`__}. Here is an
7293example assuming
7294``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``):
7295# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
7296SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" #
7297SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" SRCREV_meta =
7298"a227f20eff056e511d504b2e490f3774ab260d6f" You can use the
7299``buildhistory-collect-srcrevs`` command with the ``-a`` option to
7300collect the stored ``SRCREV`` values from build history and report them
7301in a format suitable for use in global configuration (e.g.,
7302``local.conf`` or a distro include file) to override floating
7303``AUTOREV`` values to a fixed set of revisions. Here is some example
7304output from this command: $ buildhistory-collect-srcrevs -a #
7305i586-poky-linux SRCREV_pn-glibc =
7306"b8079dd0d360648e4e8de48656c5c38972621072" SRCREV_pn-glibc-initial =
7307"b8079dd0d360648e4e8de48656c5c38972621072" SRCREV_pn-opkg-utils =
7308"53274f087565fd45d8452c5367997ba6a682a37a" SRCREV_pn-kmod =
7309"fd56638aed3fe147015bfa10ed4a5f7491303cb4" # x86_64-linux
7310SRCREV_pn-gtk-doc-stub-native =
7311"1dea266593edb766d6d898c79451ef193eb17cfa" SRCREV_pn-dtc-native =
7312"65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" SRCREV_pn-update-rc.d-native
7313= "eca680ddf28d024954895f59a241a622dd575c11"
7314SRCREV_glibc_pn-cross-localedef-native =
7315"b8079dd0d360648e4e8de48656c5c38972621072"
7316SRCREV_localedef_pn-cross-localedef-native =
7317"c833367348d39dad7ba018990bfdaffaec8e9ed3" SRCREV_pn-prelink-native =
7318"faa069deec99bf61418d0bab831c83d7c1b797ca" SRCREV_pn-opkg-utils-native =
7319"53274f087565fd45d8452c5367997ba6a682a37a" SRCREV_pn-kern-tools-native =
7320"23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" SRCREV_pn-kmod-native =
7321"fd56638aed3fe147015bfa10ed4a5f7491303cb4" # qemux86-poky-linux
7322SRCREV_machine_pn-linux-yocto =
7323"38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" SRCREV_meta_pn-linux-yocto =
7324"a227f20eff056e511d504b2e490f3774ab260d6f" # all-poky-linux
7325SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
7326
7327.. note::
7328
7329 Here are some notes on using the
7330 buildhistory-collect-srcrevs
7331 command:
7332
7333 - By default, only values where the ``SRCREV`` was not hardcoded
7334 (usually when ``AUTOREV`` is used) are reported. Use the ``-a``
7335 option to see all ``SRCREV`` values.
7336
7337 - The output statements might not have any effect if overrides are
7338 applied elsewhere in the build system configuration. Use the
7339 ``-f`` option to add the ``forcevariable`` override to each output
7340 line if you need to work around this restriction.
7341
7342 - The script does apply special handling when building for multiple
7343 machines. However, the script does place a comment before each set
7344 of values that specifies which triplet to which they belong as
7345 previously shown (e.g., ``i586-poky-linux``).
7346
7347Build History Image Information
7348~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7349
7350The files produced for each image are as follows:
7351
7352- ``image-files:`` A directory containing selected files from the root
7353 filesystem. The files are defined by
7354 ```BUILDHISTORY_IMAGE_FILES`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_IMAGE_FILES>`__.
7355
7356- ``build-id.txt:`` Human-readable information about the build
7357 configuration and metadata source revisions. This file contains the
7358 full build header as printed by BitBake.
7359
7360- ``*.dot:`` Dependency graphs for the image that are compatible with
7361 ``graphviz``.
7362
7363- ``files-in-image.txt:`` A list of files in the image with
7364 permissions, owner, group, size, and symlink information.
7365
7366- ``image-info.txt:`` A text file containing name-value pairs with
7367 information about the image. See the following listing example for
7368 more information.
7369
7370- ``installed-package-names.txt:`` A list of installed packages by name
7371 only.
7372
7373- ``installed-package-sizes.txt:`` A list of installed packages ordered
7374 by size.
7375
7376- ``installed-packages.txt:`` A list of installed packages with full
7377 package filenames.
7378
7379.. note::
7380
7381 Installed package information is able to be gathered and produced
7382 even if package management is disabled for the final image.
7383
7384Here is an example of ``image-info.txt``: DISTRO = poky DISTRO_VERSION =
73851.7 USER_CLASSES = buildstats image-mklibs image-prelink IMAGE_CLASSES =
7386image_types IMAGE_FEATURES = debug-tweaks IMAGE_LINGUAS = IMAGE_INSTALL
7387= packagegroup-core-boot run-postinsts BAD_RECOMMENDATIONS =
7388NO_RECOMMENDATIONS = PACKAGE_EXCLUDE = ROOTFS_POSTPROCESS_COMMAND =
7389write_package_manifest; license_create_manifest; \\ write_image_manifest
7390; buildhistory_list_installed_image ; \\
7391buildhistory_get_image_installed ; ssh_allow_empty_password; \\
7392postinst_enable_logging; rootfs_update_timestamp ;
7393ssh_disable_dns_lookup ; IMAGE_POSTPROCESS_COMMAND =
7394buildhistory_get_imageinfo ; IMAGESIZE = 6900 Other than ``IMAGESIZE``,
7395which is the total size of the files in the image in Kbytes, the
7396name-value pairs are variables that may have influenced the content of
7397the image. This information is often useful when you are trying to
7398determine why a change in the package or file listings has occurred.
7399
7400Using Build History to Gather Image Information Only
7401~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7402
7403As you can see, build history produces image information, including
7404dependency graphs, so you can see why something was pulled into the
7405image. If you are just interested in this information and not interested
7406in collecting specific package or SDK information, you can enable
7407writing only image information without any history by adding the
7408following to your ``conf/local.conf`` file found in the `Build
7409Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
7410"buildhistory" BUILDHISTORY_COMMIT = "0" BUILDHISTORY_FEATURES = "image"
7411Here, you set the
7412```BUILDHISTORY_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_FEATURES>`__
7413variable to use the image feature only.
7414
7415Build History SDK Information
7416~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7417
7418Build history collects similar information on the contents of SDKs (e.g.
7419``bitbake -c populate_sdk imagename``) as compared to information it
7420collects for images. Furthermore, this information differs depending on
7421whether an extensible or standard SDK is being produced.
7422
7423The following list shows the files produced for SDKs:
7424
7425- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
7426 owner, group, size, and symlink information. This list includes both
7427 the host and target parts of the SDK.
7428
7429- ``sdk-info.txt:`` A text file containing name-value pairs with
7430 information about the SDK. See the following listing example for more
7431 information.
7432
7433- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
7434 with information about task group sizes (e.g. ``do_populate_sysroot``
7435 tasks have a total size). The ``sstate-task-sizes.txt`` file exists
7436 only when an extensible SDK is created.
7437
7438- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
7439 with information for the shared-state packages and sizes in the SDK.
7440 The ``sstate-package-sizes.txt`` file exists only when an extensible
7441 SDK is created.
7442
7443- ``sdk-files:`` A folder that contains copies of the files mentioned
7444 in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
7445 Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
7446 specific to the extensible SDK although you can set it differently if
7447 you would like to pull in specific files from the standard SDK.
7448
7449 The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
7450 ``conf/auto.conf``, ``conf/locked-sigs.inc``, and
7451 ``conf/devtool.conf``. Thus, for an extensible SDK, these files get
7452 copied into the ``sdk-files`` directory.
7453
7454- The following information appears under each of the ``host`` and
7455 ``target`` directories for the portions of the SDK that run on the
7456 host and on the target, respectively:
7457
7458 .. note::
7459
7460 The following files for the most part are empty when producing an
7461 extensible SDK because this type of SDK is not constructed from
7462 packages as is the standard SDK.
7463
7464 - ``depends.dot:`` Dependency graph for the SDK that is compatible
7465 with ``graphviz``.
7466
7467 - ``installed-package-names.txt:`` A list of installed packages by
7468 name only.
7469
7470 - ``installed-package-sizes.txt:`` A list of installed packages
7471 ordered by size.
7472
7473 - ``installed-packages.txt:`` A list of installed packages with full
7474 package filenames.
7475
7476Here is an example of ``sdk-info.txt``: DISTRO = poky DISTRO_VERSION =
74771.3+snapshot-20130327 SDK_NAME = poky-glibc-i686-arm SDK_VERSION =
74781.3+snapshot SDKMACHINE = SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
7479BAD_RECOMMENDATIONS = SDKSIZE = 352712 Other than ``SDKSIZE``, which is
7480the total size of the files in the SDK in Kbytes, the name-value pairs
7481are variables that might have influenced the content of the SDK. This
7482information is often useful when you are trying to determine why a
7483change in the package or file listings has occurred.
7484
7485Examining Build History Information
7486~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7487
7488You can examine build history output from the command line or from a web
7489interface.
7490
7491To see any changes that have occurred (assuming you have
7492```BUILDHISTORY_COMMIT`` <&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT>`__\ `` = "1"``),
7493you can simply use any Git command that allows you to view the history
7494of a repository. Here is one method: $ git log -p You need to realize,
7495however, that this method does show changes that are not significant
7496(e.g. a package's size changing by a few bytes).
7497
7498A command-line tool called ``buildhistory-diff`` does exist, though,
7499that queries the Git repository and prints just the differences that
7500might be significant in human-readable form. Here is an example: $
7501~/poky/poky/scripts/buildhistory-diff . HEAD^ Changes to
7502images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
7503/etc/anotherpkg.conf was added /sbin/anotherpkg was added \*
7504(installed-package-names.txt): \* anotherpkg was added Changes to
7505images/qemux86_64/glibc/core-image-minimal
7506(installed-package-names.txt): anotherpkg was added
7507packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" \* PR
7508changed from "r0" to "r1" \* PV changed from "0.1.10" to "0.1.12"
7509packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to
7510144381 (+30%) \* PR changed from "r0" to "r1" \* PV changed from
7511"0.1.10" to "0.1.12"
7512
7513.. note::
7514
7515 The
7516 buildhistory-diff
7517 tool requires the
7518 GitPython
7519 package. Be sure to install it using Pip3 as follows:
7520 ::
7521
7522 $ pip3 install GitPython --user
7523
7524
7525 Alternatively, you can install
7526 python3-git
7527 using the appropriate distribution package manager (e.g.
7528 apt-get
7529 ,
7530 dnf
7531 , or
7532 zipper
7533 ).
7534
7535To see changes to the build history using a web interface, follow the
7536instruction in the ``README`` file here.
7537` <http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/>`__.
7538
7539Here is a sample screenshot of the interface:
7540
7541Performing Automated Runtime Testing
7542====================================
7543
7544The OpenEmbedded build system makes available a series of automated
7545tests for images to verify runtime functionality. You can run these
7546tests on either QEMU or actual target hardware. Tests are written in
7547Python making use of the ``unittest`` module, and the majority of them
7548run commands on the target system over SSH. This section describes how
7549you set up the environment to use these tests, run available tests, and
7550write and add your own tests.
7551
7552For information on the test and QA infrastructure available within the
7553Yocto Project, see the "`Testing and Quality
7554Assurance <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__"
7555section in the Yocto Project Reference Manual.
7556
7557Enabling Tests
7558--------------
7559
7560Depending on whether you are planning to run tests using QEMU or on the
7561hardware, you have to take different steps to enable the tests. See the
7562following subsections for information on how to enable both types of
7563tests.
7564
7565.. _qemu-image-enabling-tests:
7566
7567Enabling Runtime Tests on QEMU
7568~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7569
7570In order to run tests, you need to do the following:
7571
7572- *Set up to avoid interaction with ``sudo`` for networking:* To
7573 accomplish this, you must do one of the following:
7574
7575 - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all
7576 commands or just for ``runqemu-ifup``. You must provide the full
7577 path as that can change if you are using multiple clones of the
7578 source repository.
7579
7580 .. note::
7581
7582 On some distributions, you also need to comment out "Defaults
7583 requiretty" in
7584 /etc/sudoers
7585 .
7586
7587 - Manually configure a tap interface for your system.
7588
7589 - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which
7590 should generate a list of tap devices. This is the option
7591 typically chosen for Autobuilder-type environments.
7592
7593 .. note::
7594
7595 - Be sure to use an absolute path when calling this script
7596 with sudo.
7597
7598 - The package recipe ``qemu-helper-native`` is required to run
7599 this script. Build the package using the following command:
7600 $ bitbake qemu-helper-native
7601
7602- *Set the ``DISPLAY`` variable:* You need to set this variable so that
7603 you have an X server available (e.g. start ``vncserver`` for a
7604 headless machine).
7605
7606- *Be sure your host's firewall accepts incoming connections from
7607 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an
7608 HTTP server on a random high number port, which is used to serve
7609 files to the target. The DNF module serves
7610 ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands.
7611 That means your host's firewall must accept incoming connections from
7612 192.168.7.0/24, which is the default IP range used for tap devices by
7613 ``runqemu``.
7614
7615- *Be sure your host has the correct packages installed:* Depending
7616 your host's distribution, you need to have the following packages
7617 installed:
7618
7619 - Ubuntu and Debian: ``sysstat`` and ``iproute2``
7620
7621 - OpenSUSE: ``sysstat`` and ``iproute2``
7622
7623 - Fedora: ``sysstat`` and ``iproute``
7624
7625 - CentOS: ``sysstat`` and ``iproute``
7626
7627Once you start running the tests, the following happens:
7628
76291. A copy of the root filesystem is written to ``${WORKDIR}/testimage``.
7630
76312. The image is booted under QEMU using the standard ``runqemu`` script.
7632
76333. A default timeout of 500 seconds occurs to allow for the boot process
7634 to reach the login prompt. You can change the timeout period by
7635 setting
7636 ```TEST_QEMUBOOT_TIMEOUT`` <&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT>`__
7637 in the ``local.conf`` file.
7638
76394. Once the boot process is reached and the login prompt appears, the
7640 tests run. The full boot log is written to
7641 ``${WORKDIR}/testimage/qemu_boot_log``.
7642
76435. Each test module loads in the order found in ``TEST_SUITES``. You can
7644 find the full output of the commands run over SSH in
7645 ``${WORKDIR}/testimgage/ssh_target_log``.
7646
76476. If no failures occur, the task running the tests ends successfully.
7648 You can find the output from the ``unittest`` in the task log at
7649 ``${WORKDIR}/temp/log.do_testimage``.
7650
7651.. _hardware-image-enabling-tests:
7652
7653Enabling Runtime Tests on Hardware
7654~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7655
7656The OpenEmbedded build system can run tests on real hardware, and for
7657certain devices it can also deploy the image to be tested onto the
7658device beforehand.
7659
7660For automated deployment, a "master image" is installed onto the
7661hardware once as part of setup. Then, each time tests are to be run, the
7662following occurs:
7663
76641. The master image is booted into and used to write the image to be
7665 tested to a second partition.
7666
76672. The device is then rebooted using an external script that you need to
7668 provide.
7669
76703. The device boots into the image to be tested.
7671
7672When running tests (independent of whether the image has been deployed
7673automatically or not), the device is expected to be connected to a
7674network on a pre-determined IP address. You can either use static IP
7675addresses written into the image, or set the image to use DHCP and have
7676your DHCP server on the test network assign a known IP address based on
7677the MAC address of the device.
7678
7679In order to run tests on hardware, you need to set ``TEST_TARGET`` to an
7680appropriate value. For QEMU, you do not have to change anything, the
7681default value is "qemu". For running tests on hardware, the following
7682options exist:
7683
7684- *"simpleremote":* Choose "simpleremote" if you are going to run tests
7685 on a target system that is already running the image to be tested and
7686 is available on the network. You can use "simpleremote" in
7687 conjunction with either real hardware or an image running within a
7688 separately started QEMU or any other virtual machine manager.
7689
7690- *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is
7691 an EFI-based machine with ``systemd-boot`` as bootloader and
7692 ``core-image-testmaster`` (or something similar) is installed. Also,
7693 your hardware under test must be in a DHCP-enabled network that gives
7694 it the same IP address for each reboot.
7695
7696 If you choose "SystemdbootTarget", there are additional requirements
7697 and considerations. See the "`Selecting
7698 SystemdbootTarget <#selecting-systemdboottarget>`__" section, which
7699 follows, for more information.
7700
7701- *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying
7702 images and running tests on the BeagleBone "Black" or original
7703 "White" hardware. For information on how to use these tests, see the
7704 comments at the top of the BeagleBoneTarget
7705 ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file.
7706
7707- *"EdgeRouterTarget":* Choose "EdgeRouterTarget" is you are deploying
7708 images and running tests on the Ubiquiti Networks EdgeRouter Lite.
7709 For information on how to use these tests, see the comments at the
7710 top of the EdgeRouterTarget
7711 ``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file.
7712
7713- *"GrubTarget":* Choose the "supports deploying images and running
7714 tests on any generic PC that boots using GRUB. For information on how
7715 to use these tests, see the comments at the top of the GrubTarget
7716 ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file.
7717
7718- *"your-target":* Create your own custom target if you want to run
7719 tests when you are deploying images and running tests on a custom
7720 machine within your BSP layer. To do this, you need to add a Python
7721 unit that defines the target class under ``lib/oeqa/controllers/``
7722 within your layer. You must also provide an empty ``__init__.py``.
7723 For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``.
7724
7725Selecting SystemdbootTarget
7726~~~~~~~~~~~~~~~~~~~~~~~~~~~
7727
7728If you did not set ``TEST_TARGET`` to "SystemdbootTarget", then you do
7729not need any information in this section. You can skip down to the
7730"`Running Tests <#qemu-image-running-tests>`__" section.
7731
7732If you did set ``TEST_TARGET`` to "SystemdbootTarget", you also need to
7733perform a one-time setup of your master image by doing the following:
7734
77351. *Set ``EFI_PROVIDER``:* Be sure that ``EFI_PROVIDER`` is as follows:
7736 EFI_PROVIDER = "systemd-boot"
7737
77382. *Build the master image:* Build the ``core-image-testmaster`` image.
7739 The ``core-image-testmaster`` recipe is provided as an example for a
7740 "master" image and you can customize the image recipe as you would
7741 any other recipe.
7742
7743 Here are the image recipe requirements:
7744
7745 - Inherits ``core-image`` so that kernel modules are installed.
7746
7747 - Installs normal linux utilities not busybox ones (e.g. ``bash``,
7748 ``coreutils``, ``tar``, ``gzip``, and ``kmod``).
7749
7750 - Uses a custom Initial RAM Disk (initramfs) image with a custom
7751 installer. A normal image that you can install usually creates a
7752 single rootfs partition. This image uses another installer that
7753 creates a specific partition layout. Not all Board Support
7754 Packages (BSPs) can use an installer. For such cases, you need to
7755 manually create the following partition layout on the target:
7756
7757 - First partition mounted under ``/boot``, labeled "boot".
7758
7759 - The main rootfs partition where this image gets installed,
7760 which is mounted under ``/``.
7761
7762 - Another partition labeled "testrootfs" where test images get
7763 deployed.
7764
77653. *Install image:* Install the image that you just built on the target
7766 system.
7767
7768The final thing you need to do when setting ``TEST_TARGET`` to
7769"SystemdbootTarget" is to set up the test image:
7770
77711. *Set up your ``local.conf`` file:* Make sure you have the following
7772 statements in your ``local.conf`` file: IMAGE_FSTYPES += "tar.gz"
7773 INHERIT += "testimage" TEST_TARGET = "SystemdbootTarget"
7774 TEST_TARGET_IP = "192.168.2.3"
7775
77762. *Build your test image:* Use BitBake to build the image: $ bitbake
7777 core-image-sato
7778
7779Power Control
7780~~~~~~~~~~~~~
7781
7782For most hardware targets other than "simpleremote", you can control
7783power:
7784
7785- You can use ``TEST_POWERCONTROL_CMD`` together with
7786 ``TEST_POWERCONTROL_EXTRA_ARGS`` as a command that runs on the host
7787 and does power cycling. The test code passes one argument to that
7788 command: off, on or cycle (off then on). Here is an example that
7789 could appear in your ``local.conf`` file: TEST_POWERCONTROL_CMD =
7790 "powercontrol.exp test 10.11.12.1 nuc1" In this example, the expect
7791 script does the following: ssh test@10.11.12.1 "pyctl nuc1 arg" It
7792 then runs a Python script that controls power for a label called
7793 ``nuc1``.
7794
7795 .. note::
7796
7797 You need to customize
7798 TEST_POWERCONTROL_CMD
7799 and
7800 TEST_POWERCONTROL_EXTRA_ARGS
7801 for your own setup. The one requirement is that it accepts "on",
7802 "off", and "cycle" as the last argument.
7803
7804- When no command is defined, it connects to the device over SSH and
7805 uses the classic reboot command to reboot the device. Classic reboot
7806 is fine as long as the machine actually reboots (i.e. the SSH test
7807 has not failed). It is useful for scenarios where you have a simple
7808 setup, typically with a single board, and where some manual
7809 interaction is okay from time to time.
7810
7811If you have no hardware to automatically perform power control but still
7812wish to experiment with automated hardware testing, you can use the
7813dialog-power-control script that shows a dialog prompting you to perform
7814the required power action. This script requires either KDialog or Zenity
7815to be installed. To use this script, set the
7816```TEST_POWERCONTROL_CMD`` <&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD>`__
7817variable as follows: TEST_POWERCONTROL_CMD =
7818"${COREBASE}/scripts/contrib/dialog-power-control"
7819
7820Serial Console Connection
7821~~~~~~~~~~~~~~~~~~~~~~~~~
7822
7823For test target classes requiring a serial console to interact with the
7824bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget),
7825you need to specify a command to use to connect to the serial console of
7826the target machine by using the
7827```TEST_SERIALCONTROL_CMD`` <&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD>`__
7828variable and optionally the
7829```TEST_SERIALCONTROL_EXTRA_ARGS`` <&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS>`__
7830variable.
7831
7832These cases could be a serial terminal program if the machine is
7833connected to a local serial port, or a ``telnet`` or ``ssh`` command
7834connecting to a remote console server. Regardless of the case, the
7835command simply needs to connect to the serial console and forward that
7836connection to standard input and output as any normal terminal program
7837does. For example, to use the picocom terminal program on serial device
7838``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:
7839TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" For local
7840devices where the serial port device disappears when the device reboots,
7841an additional "serdevtry" wrapper script is provided. To use this
7842wrapper, simply prefix the terminal command with
7843``${COREBASE}/scripts/contrib/serdevtry``: TEST_SERIALCONTROL_CMD =
7844"${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0"
7845
7846.. _qemu-image-running-tests:
7847
7848Running Tests
7849-------------
7850
7851You can start the tests automatically or manually:
7852
7853- *Automatically running tests:* To run the tests automatically after
7854 the OpenEmbedded build system successfully creates an image, first
7855 set the
7856 ```TESTIMAGE_AUTO`` <&YOCTO_DOCS_REF_URL;#var-TESTIMAGE_AUTO>`__
7857 variable to "1" in your ``local.conf`` file in the `Build
7858 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: TESTIMAGE_AUTO =
7859 "1" Next, build your image. If the image successfully builds, the
7860 tests run: bitbake core-image-sato
7861
7862- *Manually running tests:* To manually run the tests, first globally
7863 inherit the
7864 ```testimage`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__ class
7865 by editing your ``local.conf`` file: INHERIT += "testimage" Next, use
7866 BitBake to run the tests: bitbake -c testimage image
7867
7868All test files reside in ``meta/lib/oeqa/runtime`` in the `Source
7869Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. A test name maps
7870directly to a Python module. Each test module may contain a number of
7871individual tests. Tests are usually grouped together by the area tested
7872(e.g tests for systemd reside in ``meta/lib/oeqa/runtime/systemd.py``).
7873
7874You can add tests to any layer provided you place them in the proper
7875area and you extend ```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__ in
7876the ``local.conf`` file as normal. Be sure that tests reside in
7877``layer/lib/oeqa/runtime``.
7878
7879.. note::
7880
7881 Be sure that module names do not collide with module names used in
7882 the default set of test modules in
7883 meta/lib/oeqa/runtime
7884 .
7885
7886You can change the set of tests run by appending or overriding
7887```TEST_SUITES`` <&YOCTO_DOCS_REF_URL;#var-TEST_SUITES>`__ variable in
7888``local.conf``. Each name in ``TEST_SUITES`` represents a required test
7889for the image. Test modules named within ``TEST_SUITES`` cannot be
7890skipped even if a test is not suitable for an image (e.g. running the
7891RPM tests on an image without ``rpm``). Appending "auto" to
7892``TEST_SUITES`` causes the build system to try to run all tests that are
7893suitable for the image (i.e. each test module may elect to skip itself).
7894
7895The order you list tests in ``TEST_SUITES`` is important and influences
7896test dependencies. Consequently, tests that depend on other tests should
7897be added after the test on which they depend. For example, since the
7898``ssh`` test depends on the ``ping`` test, "ssh" needs to come after
7899"ping" in the list. The test class provides no re-ordering or dependency
7900handling.
7901
7902.. note::
7903
7904 Each module can have multiple classes with multiple test methods.
7905 And, Python
7906 unittest
7907 rules apply.
7908
7909Here are some things to keep in mind when running tests:
7910
7911- The default tests for the image are defined as:
7912 DEFAULT_TEST_SUITES_pn-image = "ping ssh df connman syslog xorg scp
7913 vnc date rpm dnf dmesg"
7914
7915- Add your own test to the list of the by using the following:
7916 TEST_SUITES_append = " mytest"
7917
7918- Run a specific list of tests as follows: TEST_SUITES = "test1 test2
7919 test3" Remember, order is important. Be sure to place a test that is
7920 dependent on another test later in the order.
7921
7922Exporting Tests
7923---------------
7924
7925You can export tests so that they can run independently of the build
7926system. Exporting tests is required if you want to be able to hand the
7927test execution off to a scheduler. You can only export tests that are
7928defined in ```TEST_SUITES`` <&YOCTO_DOCS_REF_URL;#var-TEST_SUITES>`__.
7929
7930If your image is already built, make sure the following are set in your
7931``local.conf`` file: INHERIT +="testexport" TEST_TARGET_IP =
7932"IP-address-for-the-test-target" TEST_SERVER_IP =
7933"IP-address-for-the-test-server" You can then export the tests with the
7934following BitBake command form: $ bitbake image -c testexport Exporting
7935the tests places them in the `Build
7936Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in
7937``tmp/testexport/``\ image, which is controlled by the
7938``TEST_EXPORT_DIR`` variable.
7939
7940You can now run the tests outside of the build environment: $ cd
7941tmp/testexport/image $ ./runexported.py testdata.json
7942
7943Here is a complete example that shows IP addresses and uses the
7944``core-image-sato`` image: INHERIT +="testexport" TEST_TARGET_IP =
7945"192.168.7.2" TEST_SERVER_IP = "192.168.7.1" Use BitBake to export the
7946tests: $ bitbake core-image-sato -c testexport Run the tests outside of
7947the build environment using the following: $ cd
7948tmp/testexport/core-image-sato $ ./runexported.py testdata.json
7949
7950.. _qemu-image-writing-new-tests:
7951
7952Writing New Tests
7953-----------------
7954
7955As mentioned previously, all new test files need to be in the proper
7956place for the build system to find them. New tests for additional
7957functionality outside of the core should be added to the layer that adds
7958the functionality, in ``layer/lib/oeqa/runtime`` (as long as
7959```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__ is extended in the
7960layer's ``layer.conf`` file as normal). Just remember the following:
7961
7962- Filenames need to map directly to test (module) names.
7963
7964- Do not use module names that collide with existing core tests.
7965
7966- Minimally, an empty ``__init__.py`` file must exist in the runtime
7967 directory.
7968
7969To create a new test, start by copying an existing module (e.g.
7970``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use
7971code from ``meta/lib/oeqa/utils``, which are helper classes.
7972
7973.. note::
7974
7975 Structure shell commands such that you rely on them and they return a
7976 single code for success. Be aware that sometimes you will need to
7977 parse the output. See the
7978 df.py
7979 and
7980 date.py
7981 modules for examples.
7982
7983You will notice that all test classes inherit ``oeRuntimeTest``, which
7984is found in ``meta/lib/oetest.py``. This base class offers some helper
7985attributes, which are described in the following sections:
7986
7987.. _qemu-image-writing-tests-class-methods:
7988
7989Class Methods
7990~~~~~~~~~~~~~
7991
7992Class methods are as follows:
7993
7994- *``hasPackage(pkg)``:* Returns "True" if ``pkg`` is in the installed
7995 package list of the image, which is based on the manifest file that
7996 is generated during the ``do_rootfs`` task.
7997
7998- *``hasFeature(feature)``:* Returns "True" if the feature is in
7999 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__ or
8000 ```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__.
8001
8002.. _qemu-image-writing-tests-class-attributes:
8003
8004Class Attributes
8005~~~~~~~~~~~~~~~~
8006
8007Class attributes are as follows:
8008
8009- *``pscmd``:* Equals "ps -ef" if ``procps`` is installed in the image.
8010 Otherwise, ``pscmd`` equals "ps" (busybox).
8011
8012- *``tc``:* The called test context, which gives access to the
8013 following attributes:
8014
8015 - *``d``:* The BitBake datastore, which allows you to use stuff such
8016 as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``.
8017
8018 - *``testslist`` and ``testsrequired``:* Used internally. The tests
8019 do not need these.
8020
8021 - *``filesdir``:* The absolute path to
8022 ``meta/lib/oeqa/runtime/files``, which contains helper files for
8023 tests meant for copying on the target such as small files written
8024 in C for compilation.
8025
8026 - *``target``:* The target controller object used to deploy and
8027 start an image on a particular target (e.g. Qemu, SimpleRemote,
8028 and SystemdbootTarget). Tests usually use the following:
8029
8030 - *``ip``:* The target's IP address.
8031
8032 - *``server_ip``:* The host's IP address, which is usually used
8033 by the DNF test suite.
8034
8035 - *``run(cmd, timeout=None)``:* The single, most used method.
8036 This command is a wrapper for: ``ssh root@host "cmd"``. The
8037 command returns a tuple: (status, output), which are what their
8038 names imply - the return code of "cmd" and whatever output it
8039 produces. The optional timeout argument represents the number
8040 of seconds the test should wait for "cmd" to return. If the
8041 argument is "None", the test uses the default instance's
8042 timeout period, which is 300 seconds. If the argument is "0",
8043 the test runs until the command returns.
8044
8045 - *``copy_to(localpath, remotepath)``:*
8046 ``scp localpath root@ip:remotepath``.
8047
8048 - *``copy_from(remotepath, localpath)``:*
8049 ``scp root@host:remotepath localpath``.
8050
8051.. _qemu-image-writing-tests-instance-attributes:
8052
8053Instance Attributes
8054~~~~~~~~~~~~~~~~~~~
8055
8056A single instance attribute exists, which is ``target``. The ``target``
8057instance attribute is identical to the class attribute of the same name,
8058which is described in the previous section. This attribute exists as
8059both an instance and class attribute so tests can use
8060``self.target.run(cmd)`` in instance methods instead of
8061``oeRuntimeTest.tc.target.run(cmd)``.
8062
8063Installing Packages in the DUT Without the Package Manager
8064----------------------------------------------------------
8065
8066When a test requires a package built by BitBake, it is possible to
8067install that package. Installing the package does not require a package
8068manager be installed in the device under test (DUT). It does, however,
8069require an SSH connection and the target must be using the
8070``sshcontrol`` class.
8071
8072.. note::
8073
8074 This method uses
8075 scp
8076 to copy files from the host to the target, which causes permissions
8077 and special attributes to be lost.
8078
8079A JSON file is used to define the packages needed by a test. This file
8080must be in the same path as the file used to define the tests.
8081Furthermore, the filename must map directly to the test module name with
8082a ``.json`` extension.
8083
8084The JSON file must include an object with the test name as keys of an
8085object or an array. This object (or array of objects) uses the following
8086data:
8087
8088- "pkg" - A mandatory string that is the name of the package to be
8089 installed.
8090
8091- "rm" - An optional boolean, which defaults to "false", that specifies
8092 to remove the package after the test.
8093
8094- "extract" - An optional boolean, which defaults to "false", that
8095 specifies if the package must be extracted from the package format.
8096 When set to "true", the package is not automatically installed into
8097 the DUT.
8098
8099Following is an example JSON file that handles test "foo" installing
8100package "bar" and test "foobar" installing packages "foo" and "bar".
8101Once the test is complete, the packages are removed from the DUT. {
8102"foo": { "pkg": "bar" }, "foobar": [ { "pkg": "foo", "rm": true }, {
8103"pkg": "bar", "rm": true } ] }
8104
8105.. _usingpoky-debugging-tools-and-techniques:
8106
8107Debugging Tools and Techniques
8108==============================
8109
8110The exact method for debugging build failures depends on the nature of
8111the problem and on the system's area from which the bug originates.
8112Standard debugging practices such as comparison against the last known
8113working version with examination of the changes and the re-application
8114of steps to identify the one causing the problem are valid for the Yocto
8115Project just as they are for any other system. Even though it is
8116impossible to detail every possible potential failure, this section
8117provides some general tips to aid in debugging given a variety of
8118situations.
8119
8120.. note::
8121
8122 A useful feature for debugging is the error reporting tool.
8123 Configuring the Yocto Project to use this tool causes the
8124 OpenEmbedded build system to produce error reporting commands as part
8125 of the console output. You can enter the commands after the build
8126 completes to log error information into a common database, that can
8127 help you figure out what might be going wrong. For information on how
8128 to enable and use this feature, see the "
8129 Using the Error Reporting Tool
8130 " section.
8131
8132The following list shows the debugging topics in the remainder of this
8133section:
8134
8135- "`Viewing Logs from Failed
8136 Tasks <#dev-debugging-viewing-logs-from-failed-tasks>`__" describes
8137 how to find and view logs from tasks that failed during the build
8138 process.
8139
8140- "`Viewing Variable
8141 Values <#dev-debugging-viewing-variable-values>`__" describes how to
8142 use the BitBake ``-e`` option to examine variable values after a
8143 recipe has been parsed.
8144
8145- "`Viewing Package Information with
8146 ``oe-pkgdata-util`` <#viewing-package-information-with-oe-pkgdata-util>`__"
8147 describes how to use the ``oe-pkgdata-util`` utility to query
8148 ```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__ and
8149 display package-related information for built packages.
8150
8151- "`Viewing Dependencies Between Recipes and
8152 Tasks <#dev-viewing-dependencies-between-recipes-and-tasks>`__"
8153 describes how to use the BitBake ``-g`` option to display recipe
8154 dependency information used during the build.
8155
8156- "`Viewing Task Variable
8157 Dependencies <#dev-viewing-task-variable-dependencies>`__" describes
8158 how to use the ``bitbake-dumpsig`` command in conjunction with key
8159 subdirectories in the `Build
8160 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ to determine
8161 variable dependencies.
8162
8163- "`Running Specific Tasks <#dev-debugging-taskrunning>`__" describes
8164 how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``)
8165 to run specific tasks in the build chain. It can be useful to run
8166 tasks "out-of-order" when trying isolate build issues.
8167
8168- "`General BitBake Problems <#dev-debugging-bitbake>`__" describes how
8169 to use BitBake's ``-D`` debug output option to reveal more about what
8170 BitBake is doing during the build.
8171
8172- "`Building with No Dependencies <#dev-debugging-buildfile>`__"
8173 describes how to use the BitBake ``-b`` option to build a recipe
8174 while ignoring dependencies.
8175
8176- "`Recipe Logging Mechanisms <#recipe-logging-mechanisms>`__"
8177 describes how to use the many recipe logging functions to produce
8178 debugging output and report errors and warnings.
8179
8180- "`Debugging Parallel Make Races <#debugging-parallel-make-races>`__"
8181 describes how to debug situations where the build consists of several
8182 parts that are run simultaneously and when the output or result of
8183 one part is not ready for use with a different part of the build that
8184 depends on that output.
8185
8186- "`Debugging With the GNU Project Debugger (GDB)
8187 Remotely <#platdev-gdb-remotedebug>`__" describes how to use GDB to
8188 allow you to examine running programs, which can help you fix
8189 problems.
8190
8191- "`Debugging with the GNU Project Debugger (GDB) on the
8192 Target <#debugging-with-the-gnu-project-debugger-gdb-on-the-target>`__"
8193 describes how to use GDB directly on target hardware for debugging.
8194
8195- "`Other Debugging Tips <#dev-other-debugging-others>`__" describes
8196 miscellaneous debugging tips that can be useful.
8197
8198.. _dev-debugging-viewing-logs-from-failed-tasks:
8199
8200Viewing Logs from Failed Tasks
8201------------------------------
8202
8203You can find the log for a task in the file
8204``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/temp/log.do_``\ taskname.
8205For example, the log for the
8206```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ task of the
8207QEMU minimal image for the x86 machine (``qemux86``) might be in
8208``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``.
8209To see the commands `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ ran
8210to generate a log, look at the corresponding ``run.do_``\ taskname file
8211in the same directory.
8212
8213``log.do_``\ taskname and ``run.do_``\ taskname are actually symbolic
8214links to ``log.do_``\ taskname\ ``.``\ pid and
8215``log.run_``\ taskname\ ``.``\ pid, where pid is the PID the task had
8216when it ran. The symlinks always point to the files corresponding to the
8217most recent run.
8218
8219.. _dev-debugging-viewing-variable-values:
8220
8221Viewing Variable Values
8222-----------------------
8223
8224Sometimes you need to know the value of a variable as a result of
8225BitBake's parsing step. This could be because some unexpected behavior
8226occurred in your project. Perhaps an attempt to `modify a
8227variable <&YOCTO_DOCS_BB_URL;#modifying-existing-variables>`__ did not
8228work out as expected.
8229
8230BitBake's ``-e`` option is used to display variable values after
8231parsing. The following command displays the variable values after the
8232configuration files (i.e. ``local.conf``, ``bblayers.conf``,
8233``bitbake.conf`` and so forth) have been parsed: $ bitbake -e The
8234following command displays variable values after a specific recipe has
8235been parsed. The variables include those from the configuration as well:
8236$ bitbake -e recipename
8237
8238.. note::
8239
8240 Each recipe has its own private set of variables (datastore).
8241 Internally, after parsing the configuration, a copy of the resulting
8242 datastore is made prior to parsing each recipe. This copying implies
8243 that variables set in one recipe will not be visible to other
8244 recipes.
8245
8246 Likewise, each task within a recipe gets a private datastore based on
8247 the recipe datastore, which means that variables set within one task
8248 will not be visible to other tasks.
8249
8250In the output of ``bitbake -e``, each variable is preceded by a
8251description of how the variable got its value, including temporary
8252values that were later overriden. This description also includes
8253variable flags (varflags) set on the variable. The output can be very
8254helpful during debugging.
8255
8256Variables that are exported to the environment are preceded by
8257``export`` in the output of ``bitbake -e``. See the following example:
8258export CC="i586-poky-linux-gcc -m32 -march=i586
8259--sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86"
8260
8261In addition to variable values, the output of the ``bitbake -e`` and
8262``bitbake -e`` recipe commands includes the following information:
8263
8264- The output starts with a tree listing all configuration files and
8265 classes included globally, recursively listing the files they include
8266 or inherit in turn. Much of the behavior of the OpenEmbedded build
8267 system (including the behavior of the `normal recipe build
8268 tasks <&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks>`__) is
8269 implemented in the
8270 ```base`` <&YOCTO_DOCS_REF_URL;#ref-classes-base>`__ class and the
8271 classes it inherits, rather than being built into BitBake itself.
8272
8273- After the variable values, all functions appear in the output. For
8274 shell functions, variables referenced within the function body are
8275 expanded. If a function has been modified using overrides or using
8276 override-style operators like ``_append`` and ``_prepend``, then the
8277 final assembled function body appears in the output.
8278
8279Viewing Package Information with ``oe-pkgdata-util``
8280----------------------------------------------------
8281
8282You can use the ``oe-pkgdata-util`` command-line utility to query
8283```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__ and display
8284various package-related information. When you use the utility, you must
8285use it to view information on packages that have already been built.
8286
8287Following are a few of the available ``oe-pkgdata-util`` subcommands.
8288
8289.. note::
8290
8291 You can use the standard \* and ? globbing wildcards as part of
8292 package names and paths.
8293
8294- ``oe-pkgdata-util list-pkgs [``\ pattern\ ``]``: Lists all packages
8295 that have been built, optionally limiting the match to packages that
8296 match pattern.
8297
8298- ``oe-pkgdata-util list-pkg-files ``\ package\ `` ...``: Lists the
8299 files and directories contained in the given packages.
8300
8301 .. note::
8302
8303 A different way to view the contents of a package is to look at
8304 the
8305 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/packages-split``
8306 directory of the recipe that generates the package. This directory
8307 is created by the
8308 ```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ task
8309 and has one subdirectory for each package the recipe generates,
8310 which contains the files stored in that package.
8311
8312 If you want to inspect the ``${WORKDIR}/packages-split``
8313 directory, make sure that
8314 ```rm_work`` <&YOCTO_DOCS_REF_URL;#ref-classes-rm-work>`__ is not
8315 enabled when you build the recipe.
8316
8317- ``oe-pkgdata-util find-path ``\ path\ `` ...``: Lists the names of
8318 the packages that contain the given paths. For example, the following
8319 tells us that ``/usr/share/man/man1/make.1`` is contained in the
8320 ``make-doc`` package: $ oe-pkgdata-util find-path
8321 /usr/share/man/man1/make.1 make-doc: /usr/share/man/man1/make.1
8322
8323- ``oe-pkgdata-util lookup-recipe ``\ package\ `` ...``: Lists the name
8324 of the recipes that produce the given packages.
8325
8326For more information on the ``oe-pkgdata-util`` command, use the help
8327facility: $ oe-pkgdata-util DASHDASHhelp $ oe-pkgdata-util subcommand
8328--help
8329
8330.. _dev-viewing-dependencies-between-recipes-and-tasks:
8331
8332Viewing Dependencies Between Recipes and Tasks
8333----------------------------------------------
8334
8335Sometimes it can be hard to see why BitBake wants to build other recipes
8336before the one you have specified. Dependency information can help you
8337understand why a recipe is built.
8338
8339To generate dependency information for a recipe, run the following
8340command: $ bitbake -g recipename This command writes the following files
8341in the current directory:
8342
8343- ``pn-buildlist``: A list of recipes/targets involved in building
8344 recipename. "Involved" here means that at least one task from the
8345 recipe needs to run when building recipename from scratch. Targets
8346 that are in
8347 ```ASSUME_PROVIDED`` <&YOCTO_DOCS_REF_URL;#var-ASSUME_PROVIDED>`__
8348 are not listed.
8349
8350- ``task-depends.dot``: A graph showing dependencies between tasks.
8351
8352The graphs are in
8353`DOT <https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29>`__
8354format and can be converted to images (e.g. using the ``dot`` tool from
8355`Graphviz <http://www.graphviz.org/>`__).
8356
8357.. note::
8358
8359 - DOT files use a plain text format. The graphs generated using the
8360 ``bitbake -g`` command are often so large as to be difficult to
8361 read without special pruning (e.g. with Bitbake's ``-I`` option)
8362 and processing. Despite the form and size of the graphs, the
8363 corresponding ``.dot`` files can still be possible to read and
8364 provide useful information.
8365
8366 As an example, the ``task-depends.dot`` file contains lines such
8367 as the following: "libxslt.do_configure" ->
8368 "libxml2.do_populate_sysroot" The above example line reveals that
8369 the
8370 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__
8371 task in ``libxslt`` depends on the
8372 ```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
8373 task in ``libxml2``, which is a normal
8374 ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ dependency
8375 between the two recipes.
8376
8377 - For an example of how ``.dot`` files can be processed, see the
8378 ``scripts/contrib/graph-tool`` Python script, which finds and
8379 displays paths between graph nodes.
8380
8381You can use a different method to view dependency information by using
8382the following command: $ bitbake -g -u taskexp recipename This command
8383displays a GUI window from which you can view build-time and runtime
8384dependencies for the recipes involved in building recipename.
8385
8386.. _dev-viewing-task-variable-dependencies:
8387
8388Viewing Task Variable Dependencies
8389----------------------------------
8390
8391As mentioned in the "`Checksums
8392(Signatures) <&YOCTO_DOCS_BB_URL;#checksums>`__" section of the BitBake
8393User Manual, BitBake tries to automatically determine what variables a
8394task depends on so that it can rerun the task if any values of the
8395variables change. This determination is usually reliable. However, if
8396you do things like construct variable names at runtime, then you might
8397have to manually declare dependencies on those variables using
8398``vardeps`` as described in the "`Variable
8399Flags <&YOCTO_DOCS_BB_URL;#variable-flags>`__" section of the BitBake
8400User Manual.
8401
8402If you are unsure whether a variable dependency is being picked up
8403automatically for a given task, you can list the variable dependencies
8404BitBake has determined by doing the following:
8405
84061. Build the recipe containing the task: $ bitbake recipename
8407
84082. Inside the ```STAMPS_DIR`` <&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR>`__
8409 directory, find the signature data (``sigdata``) file that
8410 corresponds to the task. The ``sigdata`` files contain a pickled
8411 Python database of all the metadata that went into creating the input
8412 checksum for the task. As an example, for the
8413 ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ task of the
8414 ``db`` recipe, the ``sigdata`` file might be found in the following
8415 location:
8416 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
8417 For tasks that are accelerated through the shared state
8418 (`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__) cache, an
8419 additional ``siginfo`` file is written into
8420 ```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ along with
8421 the cached task output. The ``siginfo`` files contain exactly the
8422 same information as ``sigdata`` files.
8423
84243. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here
8425 is an example: $ bitbake-dumpsig
8426 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
8427 In the output of the above command, you will find a line like the
8428 following, which lists all the (inferred) variable dependencies for
8429 the task. This list also includes indirect dependencies from
8430 variables depending on other variables, recursively. Task
8431 dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]',
8432 'SRC_URI[sha256sum]', 'base_do_fetch']
8433
8434 .. note::
8435
8436 Functions (e.g.
8437 base_do_fetch
8438 ) also count as variable dependencies. These functions in turn
8439 depend on the variables they reference.
8440
8441 The output of ``bitbake-dumpsig`` also includes the value each
8442 variable had, a list of dependencies for each variable, and
8443 ```BB_HASHBASE_WHITELIST`` <&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST>`__
8444 information.
8445
8446There is also a ``bitbake-diffsigs`` command for comparing two
8447``siginfo`` or ``sigdata`` files. This command can be helpful when
8448trying to figure out what changed between two versions of a task. If you
8449call ``bitbake-diffsigs`` with just one file, the command behaves like
8450``bitbake-dumpsig``.
8451
8452You can also use BitBake to dump out the signature construction
8453information without executing tasks by using either of the following
8454BitBake command-line options: DASHDASHdump-signatures=SIGNATURE_HANDLER
8455-S SIGNATURE_HANDLER
8456
8457.. note::
8458
8459 Two common values for
8460 SIGNATURE_HANDLER
8461 are "none" and "printdiff", which dump only the signature or compare
8462 the dumped signature with the cached one, respectively.
8463
8464Using BitBake with either of these options causes BitBake to dump out
8465``sigdata`` files in the ``stamps`` directory for every task it would
8466have executed instead of building the specified target package.
8467
8468.. _dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task:
8469
8470Viewing Metadata Used to Create the Input Signature of a Shared State Task
8471--------------------------------------------------------------------------
8472
8473Seeing what metadata went into creating the input signature of a shared
8474state (sstate) task can be a useful debugging aid. This information is
8475available in signature information (``siginfo``) files in
8476```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__. For
8477information on how to view and interpret information in ``siginfo``
8478files, see the "`Viewing Task Variable
8479Dependencies <#dev-viewing-task-variable-dependencies>`__" section.
8480
8481For conceptual information on shared state, see the "`Shared
8482State <&YOCTO_DOCS_OM_URL;#shared-state>`__" section in the Yocto
8483Project Overview and Concepts Manual.
8484
8485.. _dev-invalidating-shared-state-to-force-a-task-to-run:
8486
8487Invalidating Shared State to Force a Task to Run
8488------------------------------------------------
8489
8490The OpenEmbedded build system uses
8491`checksums <&YOCTO_DOCS_OM_URL;#overview-checksums>`__ and `shared
8492state <&YOCTO_DOCS_OM_URL;#shared-state>`__ cache to avoid unnecessarily
8493rebuilding tasks. Collectively, this scheme is known as "shared state
8494code."
8495
8496As with all schemes, this one has some drawbacks. It is possible that
8497you could make implicit changes to your code that the checksum
8498calculations do not take into account. These implicit changes affect a
8499task's output but do not trigger the shared state code into rebuilding a
8500recipe. Consider an example during which a tool changes its output.
8501Assume that the output of ``rpmdeps`` changes. The result of the change
8502should be that all the ``package`` and ``package_write_rpm`` shared
8503state cache items become invalid. However, because the change to the
8504output is external to the code and therefore implicit, the associated
8505shared state cache items do not become invalidated. In this case, the
8506build process uses the cached items rather than running the task again.
8507Obviously, these types of implicit changes can cause problems.
8508
8509To avoid these problems during the build, you need to understand the
8510effects of any changes you make. Realize that changes you make directly
8511to a function are automatically factored into the checksum calculation.
8512Thus, these explicit changes invalidate the associated area of shared
8513state cache. However, you need to be aware of any implicit changes that
8514are not obvious changes to the code and could affect the output of a
8515given task.
8516
8517When you identify an implicit change, you can easily take steps to
8518invalidate the cache and force the tasks to run. The steps you can take
8519are as simple as changing a function's comments in the source code. For
8520example, to invalidate package shared state files, change the comment
8521statements of
8522```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ or the
8523comments of one of the functions it calls. Even though the change is
8524purely cosmetic, it causes the checksum to be recalculated and forces
8525the build system to run the task again.
8526
8527.. note::
8528
8529 For an example of a commit that makes a cosmetic change to invalidate
8530 shared state, see this
8531 commit
8532 .
8533
8534.. _dev-debugging-taskrunning:
8535
8536Running Specific Tasks
8537----------------------
8538
8539Any given recipe consists of a set of tasks. The standard BitBake
8540behavior in most cases is: ``do_fetch``, ``do_unpack``, ``do_patch``,
8541``do_configure``, ``do_compile``, ``do_install``, ``do_package``,
8542``do_package_write_*``, and ``do_build``. The default task is
8543``do_build`` and any tasks on which it depends build first. Some tasks,
8544such as ``do_devshell``, are not part of the default build chain. If you
8545wish to run a task that is not part of the default build chain, you can
8546use the ``-c`` option in BitBake. Here is an example: $ bitbake
8547matchbox-desktop -c devshell
8548
8549The ``-c`` option respects task dependencies, which means that all other
8550tasks (including tasks from other recipes) that the specified task
8551depends on will be run before the task. Even when you manually specify a
8552task to run with ``-c``, BitBake will only run the task if it considers
8553it "out of date". See the "`Stamp Files and the Rerunning of
8554Tasks <&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks>`__"
8555section in the Yocto Project Overview and Concepts Manual for how
8556BitBake determines whether a task is "out of date".
8557
8558If you want to force an up-to-date task to be rerun (e.g. because you
8559made manual modifications to the recipe's
8560```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__ that you want to try
8561out), then you can use the ``-f`` option.
8562
8563.. note::
8564
8565 The reason
8566 -f
8567 is never required when running the
8568 do_devshell
8569 task is because the
8570 [
8571 nostamp
8572 ]
8573 variable flag is already set for the task.
8574
8575The following example shows one way you can use the ``-f`` option: $
8576bitbake matchbox-desktop . . make some changes to the source code in the
8577work directory . . $ bitbake matchbox-desktop -c compile -f $ bitbake
8578matchbox-desktop
8579
8580This sequence first builds and then recompiles ``matchbox-desktop``. The
8581last command reruns all tasks (basically the packaging tasks) after the
8582compile. BitBake recognizes that the ``do_compile`` task was rerun and
8583therefore understands that the other tasks also need to be run again.
8584
8585Another, shorter way to rerun a task and all `normal recipe build
8586tasks <&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks>`__ that depend on
8587it is to use the ``-C`` option.
8588
8589.. note::
8590
8591 This option is upper-cased and is separate from the
8592 -c
8593 option, which is lower-cased.
8594
8595Using this option invalidates the given task and then runs the
8596```do_build`` <&YOCTO_DOCS_REF_URL;#ref-tasks-build>`__ task, which is
8597the default task if no task is given, and the tasks on which it depends.
8598You could replace the final two commands in the previous example with
8599the following single command: $ bitbake matchbox-desktop -C compile
8600Internally, the ``-f`` and ``-C`` options work by tainting (modifying)
8601the input checksum of the specified task. This tainting indirectly
8602causes the task and its dependent tasks to be rerun through the normal
8603task dependency mechanisms.
8604
8605.. note::
8606
8607 BitBake explicitly keeps track of which tasks have been tainted in
8608 this fashion, and will print warnings such as the following for
8609 builds involving such tasks:
8610 ::
8611
8612 WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run
8613
8614
8615 The purpose of the warning is to let you know that the work directory
8616 and build output might not be in the clean state they would be in for
8617 a "normal" build, depending on what actions you took. To get rid of
8618 such warnings, you can remove the work directory and rebuild the
8619 recipe, as follows:
8620 ::
8621
8622 $ bitbake matchbox-desktop -c clean
8623 $ bitbake matchbox-desktop
8624
8625
8626You can view a list of tasks in a given package by running the
8627``do_listtasks`` task as follows: $ bitbake matchbox-desktop -c
8628listtasks The results appear as output to the console and are also in
8629the file ``${WORKDIR}/temp/log.do_listtasks``.
8630
8631.. _dev-debugging-bitbake:
8632
8633General BitBake Problems
8634------------------------
8635
8636You can see debug output from BitBake by using the ``-D`` option. The
8637debug output gives more information about what BitBake is doing and the
8638reason behind it. Each ``-D`` option you use increases the logging
8639level. The most common usage is ``-DDD``.
8640
8641The output from ``bitbake -DDD -v`` targetname can reveal why BitBake
8642chose a certain version of a package or why BitBake picked a certain
8643provider. This command could also help you in a situation where you
8644think BitBake did something unexpected.
8645
8646.. _dev-debugging-buildfile:
8647
8648Building with No Dependencies
8649-----------------------------
8650
8651To build a specific recipe (``.bb`` file), you can use the following
8652command form: $ bitbake -b somepath/somerecipe.bb This command form does
8653not check for dependencies. Consequently, you should use it only when
8654you know existing dependencies have been met.
8655
8656.. note::
8657
8658 You can also specify fragments of the filename. In this case, BitBake
8659 checks for a unique match.
8660
8661Recipe Logging Mechanisms
8662-------------------------
8663
8664The Yocto Project provides several logging functions for producing
8665debugging output and reporting errors and warnings. For Python
8666functions, the following logging functions exist. All of these functions
8667log to ``${T}/log.do_``\ task, and can also log to standard output
8668(stdout) with the right settings:
8669
8670- ``bb.plain(``\ msg\ ``)``: Writes msg as is to the log while also
8671 logging to stdout.
8672
8673- ``bb.note(``\ msg\ ``)``: Writes "NOTE: msg" to the log. Also logs to
8674 stdout if BitBake is called with "-v".
8675
8676- ``bb.debug(``\ level\ ``, ``\ msg\ ``)``: Writes "DEBUG: msg" to the
8677 log. Also logs to stdout if the log level is greater than or equal to
8678 level. See the "`-D <&YOCTO_DOCS_BB_URL;#usage-and-syntax>`__" option
8679 in the BitBake User Manual for more information.
8680
8681- ``bb.warn(``\ msg\ ``)``: Writes "WARNING: msg" to the log while also
8682 logging to stdout.
8683
8684- ``bb.error(``\ msg\ ``)``: Writes "ERROR: msg" to the log while also
8685 logging to standard out (stdout).
8686
8687 .. note::
8688
8689 Calling this function does not cause the task to fail.
8690
8691- ``bb.fatal(``\ msg\ ``)``: This logging function is similar to
8692 ``bb.error(``\ msg\ ``)`` but also causes the calling task to fail.
8693
8694 .. note::
8695
8696 bb.fatal()
8697 raises an exception, which means you do not need to put a "return"
8698 statement after the function.
8699
8700The same logging functions are also available in shell functions, under
8701the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``,
8702and ``bbfatal``. The
8703```logging`` <&YOCTO_DOCS_REF_URL;#ref-classes-logging>`__ class
8704implements these functions. See that class in the ``meta/classes``
8705folder of the `Source
8706Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ for information.
8707
8708Logging With Python
8709~~~~~~~~~~~~~~~~~~~
8710
8711When creating recipes using Python and inserting code that handles build
8712logs, keep in mind the goal is to have informative logs while keeping
8713the console as "silent" as possible. Also, if you want status messages
8714in the log, use the "debug" loglevel.
8715
8716Following is an example written in Python. The code handles logging for
8717a function that determines the number of tasks needed to be run. See the
8718"```do_listtasks`` <&YOCTO_DOCS_REF_URL;#ref-tasks-listtasks>`__"
8719section for additional information: python do_listtasks() { bb.debug(2,
8720"Starting to figure out the task list") if noteworthy_condition:
8721bb.note("There are 47 tasks to run") bb.debug(2, "Got to point xyz") if
8722warning_trigger: bb.warn("Detected warning_trigger, this might be a
8723problem later.") if recoverable_error: bb.error("Hit recoverable_error,
8724you really need to fix this!") if fatal_error: bb.fatal("fatal_error
8725detected, unable to print the task list") bb.plain("The tasks present
8726are abc") bb.debug(2, "Finished figuring out the tasklist") }
8727
8728Logging With Bash
8729~~~~~~~~~~~~~~~~~
8730
8731When creating recipes using Bash and inserting code that handles build
8732logs, you have the same goals - informative with minimal console output.
8733The syntax you use for recipes written in Bash is similar to that of
8734recipes written in Python described in the previous section.
8735
8736Following is an example written in Bash. The code logs the progress of
8737the ``do_my_function`` function. do_my_function() { bbdebug 2 "Running
8738do_my_function" if [ exceptional_condition ]; then bbnote "Hit
8739exceptional_condition" fi bbdebug 2 "Got to point xyz" if [
8740warning_trigger ]; then bbwarn "Detected warning_trigger, this might
8741cause a problem later." fi if [ recoverable_error ]; then bberror "Hit
8742recoverable_error, correcting" fi if [ fatal_error ]; then bbfatal
8743"fatal_error detected" fi bbdebug 2 "Completed do_my_function" }
8744
8745Debugging Parallel Make Races
8746-----------------------------
8747
8748A parallel ``make`` race occurs when the build consists of several parts
8749that are run simultaneously and a situation occurs when the output or
8750result of one part is not ready for use with a different part of the
8751build that depends on that output. Parallel make races are annoying and
8752can sometimes be difficult to reproduce and fix. However, some simple
8753tips and tricks exist that can help you debug and fix them. This section
8754presents a real-world example of an error encountered on the Yocto
8755Project autobuilder and the process used to fix it.
8756
8757.. note::
8758
8759 If you cannot properly fix a
8760 make
8761 race condition, you can work around it by clearing either the
8762 PARALLEL_MAKE
8763 or
8764 PARALLEL_MAKEINST
8765 variables.
8766
8767The Failure
8768~~~~~~~~~~~
8769
8770For this example, assume that you are building an image that depends on
8771the "neard" package. And, during the build, BitBake runs into problems
8772and creates the following output.
8773
8774.. note::
8775
8776 This example log file has longer lines artificially broken to make
8777 the listing easier to read.
8778
8779If you examine the output or the log file, you see the failure during
8780``make``: \| DEBUG: SITE files ['endian-little', 'bit-32',
8781'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
8782\| DEBUG: Executing shell function do_compile \| NOTE: make -j 16 \|
8783make --no-print-directory all-am \| /bin/mkdir -p include/near \|
8784/bin/mkdir -p include/near \| /bin/mkdir -p include/near \| ln -s
8785/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87860.14-r0/neard-0.14/include/types.h include/near/types.h \| ln -s
8787/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87880.14-r0/neard-0.14/include/log.h include/near/log.h \| ln -s
8789/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87900.14-r0/neard-0.14/include/plugin.h include/near/plugin.h \| /bin/mkdir
8791-p include/near \| /bin/mkdir -p include/near \| /bin/mkdir -p
8792include/near \| ln -s
8793/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87940.14-r0/neard-0.14/include/tag.h include/near/tag.h \| /bin/mkdir -p
8795include/near \| ln -s
8796/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
87970.14-r0/neard-0.14/include/adapter.h include/near/adapter.h \|
8798/bin/mkdir -p include/near \| ln -s
8799/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88000.14-r0/neard-0.14/include/ndef.h include/near/ndef.h \| ln -s
8801/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88020.14-r0/neard-0.14/include/tlv.h include/near/tlv.h \| /bin/mkdir -p
8803include/near \| /bin/mkdir -p include/near \| ln -s
8804/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88050.14-r0/neard-0.14/include/setting.h include/near/setting.h \|
8806/bin/mkdir -p include/near \| /bin/mkdir -p include/near \| /bin/mkdir
8807-p include/near \| ln -s
8808/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88090.14-r0/neard-0.14/include/device.h include/near/device.h \| ln -s
8810/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88110.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h \| ln -s
8812/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88130.14-r0/neard-0.14/include/snep.h include/near/snep.h \| ln -s
8814/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88150.14-r0/neard-0.14/include/version.h include/near/version.h \| ln -s
8816/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
88170.14-r0/neard-0.14/include/dbus.h include/near/dbus.h \|
8818./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h
8819\| i586-poky-linux-gcc -m32 -march=i586
8820--sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/
8821build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src
8822-I./gdbus -I/home/pokybuild/
8823yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0
8824-I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/
8825lib/glib-2.0/include
8826-I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/
8827tmp/sysroots/qemux86/usr/include/dbus-1.0
8828-I/home/pokybuild/yocto-autobuilder/yocto-slave/
8829nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include
8830-I/home/pokybuild/yocto-autobuilder/
8831yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3
8832-DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\"
8833-DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types
8834-c -o tools/snep-send.o tools/snep-send.c \| In file included from
8835tools/snep-send.c:16:0: \| tools/../src/near.h:41:23: fatal error:
8836near/dbus.h: No such file or directory \| #include <near/dbus.h> \| ^ \|
8837compilation terminated. \| make[1]: \**\* [tools/snep-send.o] Error 1 \|
8838make[1]: \**\* Waiting for unfinished jobs.... \| make: \**\* [all]
8839Error 2 \| ERROR: oe_runmake failed
8840
8841Reproducing the Error
8842~~~~~~~~~~~~~~~~~~~~~
8843
8844Because race conditions are intermittent, they do not manifest
8845themselves every time you do the build. In fact, most times the build
8846will complete without problems even though the potential race condition
8847exists. Thus, once the error surfaces, you need a way to reproduce it.
8848
8849In this example, compiling the "neard" package is causing the problem.
8850So the first thing to do is build "neard" locally. Before you start the
8851build, set the
8852```PARALLEL_MAKE`` <&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE>`__ variable
8853in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a
8854high value for ``PARALLEL_MAKE`` increases the chances of the race
8855condition showing up: $ bitbake neard
8856
8857Once the local build for "neard" completes, start a ``devshell`` build:
8858$ bitbake neard -c devshell For information on how to use a
8859``devshell``, see the "`Using a Development
8860Shell <#platdev-appdev-devshell>`__" section.
8861
8862In the ``devshell``, do the following: $ make clean $ make
8863tools/snep-send.o The ``devshell`` commands cause the failure to clearly
8864be visible. In this case, a missing dependency exists for the "neard"
8865Makefile target. Here is some abbreviated, sample output with the
8866missing dependency clearly visible at the end: i586-poky-linux-gcc -m32
8867-march=i586 --sysroot=/home/scott-lenovo/...... . . . tools/snep-send.c
8868In file included from tools/snep-send.c:16:0: tools/../src/near.h:41:23:
8869fatal error: near/dbus.h: No such file or directory #include
8870<near/dbus.h> ^ compilation terminated. make: \**\* [tools/snep-send.o]
8871Error 1 $
8872
8873Creating a Patch for the Fix
8874~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8875
8876Because there is a missing dependency for the Makefile target, you need
8877to patch the ``Makefile.am`` file, which is generated from
8878``Makefile.in``. You can use Quilt to create the patch: $ quilt new
8879parallelmake.patch Patch patches/parallelmake.patch is now on top $
8880quilt add Makefile.am File Makefile.am added to patch
8881patches/parallelmake.patch For more information on using Quilt, see the
8882"`Using Quilt in Your Workflow <#using-a-quilt-workflow>`__" section.
8883
8884At this point you need to make the edits to ``Makefile.am`` to add the
8885missing dependency. For our example, you have to add the following line
8886to the file: tools/snep-send.$(OBJEXT): include/near/dbus.h
8887
8888Once you have edited the file, use the ``refresh`` command to create the
8889patch: $ quilt refresh Refreshed patch patches/parallelmake.patch Once
8890the patch file exists, you need to add it back to the originating recipe
8891folder. Here is an example assuming a top-level `Source
8892Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ named ``poky``: $
8893cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard
8894The final thing you need to do to implement the fix in the build is to
8895update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the
8896```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement includes
8897the patch file. The recipe file is in the folder above the patch. Here
8898is what the edited ``SRC_URI`` statement would look like: SRC_URI =
8899"${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \\
8900file://neard.in \\ file://neard.service.in \\ file://parallelmake.patch
8901\\ "
8902
8903With the patch complete and moved to the correct folder and the
8904``SRC_URI`` statement updated, you can exit the ``devshell``: $ exit
8905
8906Testing the Build
8907~~~~~~~~~~~~~~~~~
8908
8909With everything in place, you can get back to trying the build again
8910locally: $ bitbake neard This build should succeed.
8911
8912Now you can open up a ``devshell`` again and repeat the clean and make
8913operations as follows: $ bitbake neard -c devshell $ make clean $ make
8914tools/snep-send.o The build should work without issue.
8915
8916As with all solved problems, if they originated upstream, you need to
8917submit the fix for the recipe in OE-Core and upstream so that the
8918problem is taken care of at its source. See the "`Submitting a Change to
8919the Yocto Project <#how-to-submit-a-change>`__" section for more
8920information.
8921
8922.. _platdev-gdb-remotedebug:
8923
8924Debugging With the GNU Project Debugger (GDB) Remotely
8925------------------------------------------------------
8926
8927GDB allows you to examine running programs, which in turn helps you to
8928understand and fix problems. It also allows you to perform post-mortem
8929style analysis of program crashes. GDB is available as a package within
8930the Yocto Project and is installed in SDK images by default. See the
8931"`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the Yocto
8932Project Reference Manual for a description of these images. You can find
8933information on GDB at ` <http://sourceware.org/gdb/>`__.
8934
8935.. note::
8936
8937 For best results, install debug (
8938 -dbg
8939 ) packages for the applications you are going to debug. Doing so
8940 makes extra debug symbols available that give you more meaningful
8941 output.
8942
8943Sometimes, due to memory or disk space constraints, it is not possible
8944to use GDB directly on the remote target to debug applications. These
8945constraints arise because GDB needs to load the debugging information
8946and the binaries of the process being debugged. Additionally, GDB needs
8947to perform many computations to locate information such as function
8948names, variable names and values, stack traces and so forth - even
8949before starting the debugging process. These extra computations place
8950more load on the target system and can alter the characteristics of the
8951program being debugged.
8952
8953To help get past the previously mentioned constraints, you can use
8954gdbserver, which runs on the remote target and does not load any
8955debugging information from the debugged process. Instead, a GDB instance
8956processes the debugging information that is run on a remote computer -
8957the host GDB. The host GDB then sends control commands to gdbserver to
8958make it stop or start the debugged program, as well as read or write
8959memory regions of that debugged program. All the debugging information
8960loaded and processed as well as all the heavy debugging is done by the
8961host GDB. Offloading these processes gives the gdbserver running on the
8962target a chance to remain small and fast.
8963
8964Because the host GDB is responsible for loading the debugging
8965information and for doing the necessary processing to make actual
8966debugging happen, you have to make sure the host can access the
8967unstripped binaries complete with their debugging information and also
8968be sure the target is compiled with no optimizations. The host GDB must
8969also have local access to all the libraries used by the debugged
8970program. Because gdbserver does not need any local debugging
8971information, the binaries on the remote target can remain stripped.
8972However, the binaries must also be compiled without optimization so they
8973match the host's binaries.
8974
8975To remain consistent with GDB documentation and terminology, the binary
8976being debugged on the remote target machine is referred to as the
8977"inferior" binary. For documentation on GDB see the `GDB
8978site <http://sourceware.org/gdb/documentation/>`__.
8979
8980The following steps show you how to debug using the GNU project
8981debugger.
8982
89831. *Configure your build system to construct the companion debug
8984 filesystem:*
8985
8986 In your ``local.conf`` file, set the following: IMAGE_GEN_DEBUGFS =
8987 "1" IMAGE_FSTYPES_DEBUGFS = "tar.bz2" These options cause the
8988 OpenEmbedded build system to generate a special companion filesystem
8989 fragment, which contains the matching source and debug symbols to
8990 your deployable filesystem. The build system does this by looking at
8991 what is in the deployed filesystem, and pulling the corresponding
8992 ``-dbg`` packages.
8993
8994 The companion debug filesystem is not a complete filesystem, but only
8995 contains the debug fragments. This filesystem must be combined with
8996 the full filesystem for debugging. Subsequent steps in this procedure
8997 show how to combine the partial filesystem with the full filesystem.
8998
89992. *Configure the system to include gdbserver in the target filesystem:*
9000
9001 Make the following addition in either your ``local.conf`` file or in
9002 an image recipe: IMAGE_INSTALL_append = “ gdbserver" The change makes
9003 sure the ``gdbserver`` package is included.
9004
90053. *Build the environment:*
9006
9007 Use the following command to construct the image and the companion
9008 Debug Filesystem: $ bitbake image Build the cross GDB component and
9009 make it available for debugging. Build the SDK that matches the
9010 image. Building the SDK is best for a production build that can be
9011 used later for debugging, especially during long term maintenance: $
9012 bitbake -c populate_sdk image
9013
9014 Alternatively, you can build the minimal toolchain components that
9015 match the target. Doing so creates a smaller than typical SDK and
9016 only contains a minimal set of components with which to build simple
9017 test applications, as well as run the debugger: $ bitbake
9018 meta-toolchain
9019
9020 A final method is to build Gdb itself within the build system: $
9021 bitbake gdb-cross-architecture Doing so produces a temporary copy of
9022 ``cross-gdb`` you can use for debugging during development. While
9023 this is the quickest approach, the two previous methods in this step
9024 are better when considering long-term maintenance strategies.
9025
9026 .. note::
9027
9028 If you run
9029 bitbake gdb-cross
9030 , the OpenEmbedded build system suggests the actual image (e.g.
9031 gdb-cross-i586
9032 ). The suggestion is usually the actual name you want to use.
9033
90344. *Set up the* ``debugfs``
9035
9036 Run the following commands to set up the ``debugfs``: $ mkdir debugfs
9037 $ cd debugfs $ tar xvfj
9038 build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2 $ tar
9039 xvfj
9040 build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2
9041
90425. *Set up GDB*
9043
9044 Install the SDK (if you built one) and then source the correct
9045 environment file. Sourcing the environment file puts the SDK in your
9046 ``PATH`` environment variable.
9047
9048 If you are using the build system, Gdb is located in
9049 build-dir/tmp/sysroots/host/usr/bin/architecture/architecture-gdb
9050
90516. *Boot the target:*
9052
9053 For information on how to run QEMU, see the `QEMU
9054 Documentation <http://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__.
9055
9056 .. note::
9057
9058 Be sure to verify that your host can access the target via TCP.
9059
90607. *Debug a program:*
9061
9062 Debugging a program involves running gdbserver on the target and then
9063 running Gdb on the host. The example in this step debugs ``gzip``:
9064 root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help For
9065 additional gdbserver options, see the `GDB Server
9066 Documentation <https://www.gnu.org/software/gdb/documentation/>`__.
9067
9068 After running gdbserver on the target, you need to run Gdb on the
9069 host and configure it and connect to the target. Use these commands:
9070 $ cd directory-holding-the-debugfs-directory $ arch-gdb (gdb) set
9071 sysroot debugfs (gdb) set substitute-path /usr/src/debug
9072 debugfs/usr/src/debug (gdb) target remote IP-of-target:1234 At this
9073 point, everything should automatically load (i.e. matching binaries,
9074 symbols and headers).
9075
9076 .. note::
9077
9078 The Gdb
9079 set
9080 commands in the previous example can be placed into the users
9081 ~/.gdbinit
9082 file. Upon starting, Gdb automatically runs whatever commands are
9083 in that file.
9084
90858. *Deploying without a full image rebuild:*
9086
9087 In many cases, during development you want a quick method to deploy a
9088 new binary to the target and debug it, without waiting for a full
9089 image build.
9090
9091 One approach to solving this situation is to just build the component
9092 you want to debug. Once you have built the component, copy the
9093 executable directly to both the target and the host ``debugfs``.
9094
9095 If the binary is processed through the debug splitting in
9096 OpenEmbedded, you should also copy the debug items (i.e. ``.debug``
9097 contents and corresponding ``/usr/src/debug`` files) from the work
9098 directory. Here is an example: $ bitbake bash $ bitbake -c devshell
9099 bash $ cd .. $ scp packages-split/bash/bin/bash target:/bin/bash $ cp
9100 -a packages-split/bash-dbg/\* path/debugfs
9101
9102Debugging with the GNU Project Debugger (GDB) on the Target
9103-----------------------------------------------------------
9104
9105The previous section addressed using GDB remotely for debugging
9106purposes, which is the most usual case due to the inherent hardware
9107limitations on many embedded devices. However, debugging in the target
9108hardware itself is also possible with more powerful devices. This
9109section describes what you need to do in order to support using GDB to
9110debug on the target hardware.
9111
9112To support this kind of debugging, you need do the following:
9113
9114- Ensure that GDB is on the target. You can do this by adding "gdb" to
9115 ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__:
9116 IMAGE_INSTALL_append = " gdb" Alternatively, you can add
9117 "tools-debug" to
9118 ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__:
9119 IMAGE_FEATURES_append = " tools-debug"
9120
9121- Ensure that debug symbols are present. You can make sure these
9122 symbols are present by installing ``-dbg``: IMAGE_INSTALL_append = "
9123 packagename-dbg" Alternatively, you can do the following to include
9124 all the debug symbols: IMAGE_FEATURES_append = " dbg-pkgs"
9125
9126.. note::
9127
9128 To improve the debug information accuracy, you can reduce the level
9129 of optimization used by the compiler. For example, when adding the
9130 following line to your
9131 local.conf
9132 file, you will reduce optimization from
9133 FULL_OPTIMIZATION
9134 of "-O2" to
9135 DEBUG_OPTIMIZATION
9136 of "-O -fno-omit-frame-pointer":
9137 ::
9138
9139 DEBUG_BUILD = "1"
9140
9141
9142 Consider that this will reduce the application's performance and is
9143 recommended only for debugging purposes.
9144
9145.. _dev-other-debugging-others:
9146
9147Other Debugging Tips
9148--------------------
9149
9150Here are some other tips that you might find useful:
9151
9152- When adding new packages, it is worth watching for undesirable items
9153 making their way into compiler command lines. For example, you do not
9154 want references to local system files like ``/usr/lib/`` or
9155 ``/usr/include/``.
9156
9157- If you want to remove the ``psplash`` boot splashscreen, add
9158 ``psplash=false`` to the kernel command line. Doing so prevents
9159 ``psplash`` from loading and thus allows you to see the console. It
9160 is also possible to switch out of the splashscreen by switching the
9161 virtual console (e.g. Fn+Left or Fn+Right on a Zaurus).
9162
9163- Removing ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ (usually
9164 ``tmp/``, within the `Build
9165 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__) can often fix
9166 temporary build issues. Removing ``TMPDIR`` is usually a relatively
9167 cheap operation, because task output will be cached in
9168 ```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ (usually
9169 ``sstate-cache/``, which is also in the Build Directory).
9170
9171 .. note::
9172
9173 Removing
9174 TMPDIR
9175 might be a workaround rather than a fix. Consequently, trying to
9176 determine the underlying cause of an issue before removing the
9177 directory is a good idea.
9178
9179- Understanding how a feature is used in practice within existing
9180 recipes can be very helpful. It is recommended that you configure
9181 some method that allows you to quickly search through files.
9182
9183 Using GNU Grep, you can use the following shell function to
9184 recursively search through common recipe-related files, skipping
9185 binary files, ``.git`` directories, and the Build Directory (assuming
9186 its name starts with "build"): g() { grep -Ir \\ --exclude-dir=.git
9187 \\ --exclude-dir='build*' \\ --include='*.bb*' \\ --include='*.inc*'
9188 \\ --include='*.conf*' \\ --include='*.py*' \\ "$@" } Following are
9189 some usage examples: $ g FOO # Search recursively for "FOO" $ g -i
9190 foo # Search recursively for "foo", ignoring case $ g -w FOO # Search
9191 recursively for "FOO" as a word, ignoring e.g. "FOOBAR" If figuring
9192 out how some feature works requires a lot of searching, it might
9193 indicate that the documentation should be extended or improved. In
9194 such cases, consider filing a documentation bug using the Yocto
9195 Project implementation of
9196 `Bugzilla <https://bugzilla.yoctoproject.org/>`__. For information on
9197 how to submit a bug against the Yocto Project, see the Yocto Project
9198 Bugzilla `wiki
9199 page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__
9200 and the "`Submitting a Defect Against the Yocto
9201 Project <#submitting-a-defect-against-the-yocto-project>`__" section.
9202
9203 .. note::
9204
9205 The manuals might not be the right place to document variables
9206 that are purely internal and have a limited scope (e.g. internal
9207 variables used to implement a single
9208 .bbclass
9209 file).
9210
9211Making Changes to the Yocto Project
9212===================================
9213
9214Because the Yocto Project is an open-source, community-based project,
9215you can effect changes to the project. This section presents procedures
9216that show you how to submit a defect against the project and how to
9217submit a change.
9218
9219Submitting a Defect Against the Yocto Project
9220---------------------------------------------
9221
9222Use the Yocto Project implementation of
9223`Bugzilla <http://www.bugzilla.org/about/>`__ to submit a defect (bug)
9224against the Yocto Project. For additional information on this
9225implementation of Bugzilla see the "`Yocto Project
9226Bugzilla <&YOCTO_DOCS_REF_URL;#resources-bugtracker>`__" section in the
9227Yocto Project Reference Manual. For more detail on any of the following
9228steps, see the Yocto Project `Bugzilla wiki
9229page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__.
9230
9231Use the following general steps to submit a bug"
9232
92331. Open the Yocto Project implementation of
9234 `Bugzilla <&YOCTO_BUGZILLA_URL;>`__.
9235
92362. Click "File a Bug" to enter a new bug.
9237
92383. Choose the appropriate "Classification", "Product", and "Component"
9239 for which the bug was found. Bugs for the Yocto Project fall into
9240 one of several classifications, which in turn break down into
9241 several products and components. For example, for a bug against the
9242 ``meta-intel`` layer, you would choose "Build System, Metadata &
9243 Runtime", "BSPs", and "bsps-meta-intel", respectively.
9244
92454. Choose the "Version" of the Yocto Project for which you found the
9246 bug (e.g. DISTRO).
9247
92485. Determine and select the "Severity" of the bug. The severity
9249 indicates how the bug impacted your work.
9250
92516. Choose the "Hardware" that the bug impacts.
9252
92537. Choose the "Architecture" that the bug impacts.
9254
92558. Choose a "Documentation change" item for the bug. Fixing a bug might
9256 or might not affect the Yocto Project documentation. If you are
9257 unsure of the impact to the documentation, select "Don't Know".
9258
92599. Provide a brief "Summary" of the bug. Try to limit your summary to
9260 just a line or two and be sure to capture the essence of the bug.
9261
926210. Provide a detailed "Description" of the bug. You should provide as
9263 much detail as you can about the context, behavior, output, and so
9264 forth that surrounds the bug. You can even attach supporting files
9265 for output from logs by using the "Add an attachment" button.
9266
926711. Click the "Submit Bug" button submit the bug. A new Bugzilla number
9268 is assigned to the bug and the defect is logged in the bug tracking
9269 system.
9270
9271Once you file a bug, the bug is processed by the Yocto Project Bug
9272Triage Team and further details concerning the bug are assigned (e.g.
9273priority and owner). You are the "Submitter" of the bug and any further
9274categorization, progress, or comments on the bug result in Bugzilla
9275sending you an automated email concerning the particular change or
9276progress to the bug.
9277
9278.. _how-to-submit-a-change:
9279
9280Submitting a Change to the Yocto Project
9281----------------------------------------
9282
9283Contributions to the Yocto Project and OpenEmbedded are very welcome.
9284Because the system is extremely configurable and flexible, we recognize
9285that developers will want to extend, configure or optimize it for their
9286specific uses.
9287
9288The Yocto Project uses a mailing list and a patch-based workflow that is
9289similar to the Linux kernel but contains important differences. In
9290general, a mailing list exists through which you can submit patches. You
9291should send patches to the appropriate mailing list so that they can be
9292reviewed and merged by the appropriate maintainer. The specific mailing
9293list you need to use depends on the location of the code you are
9294changing. Each component (e.g. layer) should have a ``README`` file that
9295indicates where to send the changes and which process to follow.
9296
9297You can send the patch to the mailing list using whichever approach you
9298feel comfortable with to generate the patch. Once sent, the patch is
9299usually reviewed by the community at large. If somebody has concerns
9300with the patch, they will usually voice their concern over the mailing
9301list. If a patch does not receive any negative reviews, the maintainer
9302of the affected layer typically takes the patch, tests it, and then
9303based on successful testing, merges the patch.
9304
9305The "poky" repository, which is the Yocto Project's reference build
9306environment, is a hybrid repository that contains several individual
9307pieces (e.g. BitBake, Metadata, documentation, and so forth) built using
9308the combo-layer tool. The upstream location used for submitting changes
9309varies by component:
9310
9311- *Core Metadata:* Send your patch to the
9312 `openembedded-core <http://lists.openembedded.org/mailman/listinfo/openembedded-core>`__
9313 mailing list. For example, a change to anything under the ``meta`` or
9314 ``scripts`` directories should be sent to this mailing list.
9315
9316- *BitBake:* For changes to BitBake (i.e. anything under the
9317 ``bitbake`` directory), send your patch to the
9318 `bitbake-devel <http://lists.openembedded.org/mailman/listinfo/bitbake-devel>`__
9319 mailing list.
9320
9321- *"meta-*" trees:* These trees contain Metadata. Use the
9322 `poky <https://lists.yoctoproject.org/listinfo/poky>`__ mailing list.
9323
9324For changes to other layers hosted in the Yocto Project source
9325repositories (i.e. ``yoctoproject.org``), tools, and the Yocto Project
9326documentation, use the `Yocto
9327Project <https://lists.yoctoproject.org/listinfo/yocto>`__ general
9328mailing list.
9329
9330.. note::
9331
9332 Sometimes a layer's documentation specifies to use a particular
9333 mailing list. If so, use that list.
9334
9335For additional recipes that do not fit into the core Metadata, you
9336should determine which layer the recipe should go into and submit the
9337change in the manner recommended by the documentation (e.g. the
9338``README`` file) supplied with the layer. If in doubt, please ask on the
9339Yocto general mailing list or on the openembedded-devel mailing list.
9340
9341You can also push a change upstream and request a maintainer to pull the
9342change into the component's upstream repository. You do this by pushing
9343to a contribution repository that is upstream. See the "`Git Workflows
9344and the Yocto
9345Project <&YOCTO_DOCS_OM_URL;#gs-git-workflows-and-the-yocto-project>`__"
9346section in the Yocto Project Overview and Concepts Manual for additional
9347concepts on working in the Yocto Project development environment.
9348
9349Two commonly used testing repositories exist for OpenEmbedded-Core:
9350
9351- *"ross/mut" branch:* The "mut" (master-under-test) tree exists in the
9352 ``poky-contrib`` repository in the `Yocto Project source
9353 repositories <&YOCTO_GIT_URL;>`__.
9354
9355- *"master-next" branch:* This branch is part of the main "poky"
9356 repository in the Yocto Project source repositories.
9357
9358Maintainers use these branches to test submissions prior to merging
9359patches. Thus, you can get an idea of the status of a patch based on
9360whether the patch has been merged into one of these branches.
9361
9362.. note::
9363
9364 This system is imperfect and changes can sometimes get lost in the
9365 flow. Asking about the status of a patch or change is reasonable if
9366 the change has been idle for a while with no feedback. The Yocto
9367 Project does have plans to use
9368 Patchwork
9369 to track the status of patches and also to automatically preview
9370 patches.
9371
9372The following sections provide procedures for submitting a change.
9373
9374.. _pushing-a-change-upstream:
9375
9376Using Scripts to Push a Change Upstream and Request a Pull
9377~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9378
9379Follow this procedure to push a change to an upstream "contrib" Git
9380repository:
9381
9382.. note::
9383
9384 You can find general Git information on how to push a change upstream
9385 in the
9386 Git Community Book
9387 .
9388
93891. *Make Your Changes Locally:* Make your changes in your local Git
9390 repository. You should make small, controlled, isolated changes.
9391 Keeping changes small and isolated aids review, makes
9392 merging/rebasing easier and keeps the change history clean should
9393 anyone need to refer to it in future.
9394
93952. *Stage Your Changes:* Stage your changes by using the ``git add``
9396 command on each file you changed.
9397
93983. *Commit Your Changes:* Commit the change by using the ``git commit``
9399 command. Make sure your commit information follows standards by
9400 following these accepted conventions:
9401
9402 - Be sure to include a "Signed-off-by:" line in the same style as
9403 required by the Linux kernel. Adding this line signifies that you,
9404 the submitter, have agreed to the Developer's Certificate of
9405 Origin 1.1 as follows: Developer's Certificate of Origin 1.1 By
9406 making a contribution to this project, I certify that: (a) The
9407 contribution was created in whole or in part by me and I have the
9408 right to submit it under the open source license indicated in the
9409 file; or (b) The contribution is based upon previous work that, to
9410 the best of my knowledge, is covered under an appropriate open
9411 source license and I have the right under that license to submit
9412 that work with modifications, whether created in whole or in part
9413 by me, under the same open source license (unless I am permitted
9414 to submit under a different license), as indicated in the file; or
9415 (c) The contribution was provided directly to me by some other
9416 person who certified (a), (b) or (c) and I have not modified it.
9417 (d) I understand and agree that this project and the contribution
9418 are public and that a record of the contribution (including all
9419 personal information I submit with it, including my sign-off) is
9420 maintained indefinitely and may be redistributed consistent with
9421 this project or the open source license(s) involved.
9422
9423 - Provide a single-line summary of the change. and, if more
9424 explanation is needed, provide more detail in the body of the
9425 commit. This summary is typically viewable in the "shortlist" of
9426 changes. Thus, providing something short and descriptive that
9427 gives the reader a summary of the change is useful when viewing a
9428 list of many commits. You should prefix this short description
9429 with the recipe name (if changing a recipe), or else with the
9430 short form path to the file being changed.
9431
9432 - For the body of the commit message, provide detailed information
9433 that describes what you changed, why you made the change, and the
9434 approach you used. It might also be helpful if you mention how you
9435 tested the change. Provide as much detail as you can in the body
9436 of the commit message.
9437
9438 .. note::
9439
9440 You do not need to provide a more detailed explanation of a
9441 change if the change is minor to the point of the single line
9442 summary providing all the information.
9443
9444 - If the change addresses a specific bug or issue that is associated
9445 with a bug-tracking ID, include a reference to that ID in your
9446 detailed description. For example, the Yocto Project uses a
9447 specific convention for bug references - any commit that addresses
9448 a specific bug should use the following form for the detailed
9449 description. Be sure to use the actual bug-tracking ID from
9450 Bugzilla for bug-id: Fixes [YOCTO #bug-id] detailed description of
9451 change
9452
94534. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for
9454 permissions to push to an upstream contrib repository, push the
9455 change to that repository: $ git push upstream_remote_repo
9456 local_branch_name For example, suppose you have permissions to push
9457 into the upstream ``meta-intel-contrib`` repository and you are
9458 working in a local branch named your_name\ ``/README``. The following
9459 command pushes your local commits to the ``meta-intel-contrib``
9460 upstream repository and puts the commit in a branch named
9461 your_name\ ``/README``: $ git push meta-intel-contrib
9462 your_name/README
9463
94645. *Determine Who to Notify:* Determine the maintainer or the mailing
9465 list that you need to notify for the change.
9466
9467 Before submitting any change, you need to be sure who the maintainer
9468 is or what mailing list that you need to notify. Use either these
9469 methods to find out:
9470
9471 - *Maintenance File:* Examine the ``maintainers.inc`` file, which is
9472 located in the `Source
9473 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ at
9474 ``meta/conf/distro/include``, to see who is responsible for code.
9475
9476 - *Search by File:* Using `Git <&YOCTO_DOCS_OM_URL;#git>`__, you can
9477 enter the following command to bring up a short list of all
9478 commits against a specific file: git shortlog -- filename Just
9479 provide the name of the file for which you are interested. The
9480 information returned is not ordered by history but does include a
9481 list of everyone who has committed grouped by name. From the list,
9482 you can see who is responsible for the bulk of the changes against
9483 the file.
9484
9485 - *Examine the List of Mailing Lists:* For a list of the Yocto
9486 Project and related mailing lists, see the "`Mailing
9487 lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__" section in
9488 the Yocto Project Reference Manual.
9489
94906. *Make a Pull Request:* Notify the maintainer or the mailing list that
9491 you have pushed a change by making a pull request.
9492
9493 The Yocto Project provides two scripts that conveniently let you
9494 generate and send pull requests to the Yocto Project. These scripts
9495 are ``create-pull-request`` and ``send-pull-request``. You can find
9496 these scripts in the ``scripts`` directory within the `Source
9497 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
9498 ``~/poky/scripts``).
9499
9500 Using these scripts correctly formats the requests without
9501 introducing any whitespace or HTML formatting. The maintainer that
9502 receives your patches either directly or through the mailing list
9503 needs to be able to save and apply them directly from your emails.
9504 Using these scripts is the preferred method for sending patches.
9505
9506 First, create the pull request. For example, the following command
9507 runs the script, specifies the upstream repository in the contrib
9508 directory into which you pushed the change, and provides a subject
9509 line in the created patch files: $ ~/poky/scripts/create-pull-request
9510 -u meta-intel-contrib -s "Updated Manual Section Reference in README"
9511 Running this script forms ``*.patch`` files in a folder named
9512 ``pull-``\ PID in the current directory. One of the patch files is a
9513 cover letter.
9514
9515 Before running the ``send-pull-request`` script, you must edit the
9516 cover letter patch to insert information about your change. After
9517 editing the cover letter, send the pull request. For example, the
9518 following command runs the script and specifies the patch directory
9519 and email address. In this example, the email address is a mailing
9520 list: $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565
9521 -t meta-intel@yoctoproject.org You need to follow the prompts as the
9522 script is interactive.
9523
9524 .. note::
9525
9526 For help on using these scripts, simply provide the
9527 -h
9528 argument as follows:
9529 ::
9530
9531 $ poky/scripts/create-pull-request -h
9532 $ poky/scripts/send-pull-request -h
9533
9534
9535.. _submitting-a-patch:
9536
9537Using Email to Submit a Patch
9538~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9539
9540You can submit patches without using the ``create-pull-request`` and
9541``send-pull-request`` scripts described in the previous section.
9542However, keep in mind, the preferred method is to use the scripts.
9543
9544Depending on the components changed, you need to submit the email to a
9545specific mailing list. For some guidance on which mailing list to use,
9546see the `list <#figuring-out-the-mailing-list-to-use>`__ at the
9547beginning of this section. For a description of all the available
9548mailing lists, see the "`Mailing
9549Lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__" section in the
9550Yocto Project Reference Manual.
9551
9552Here is the general procedure on how to submit a patch through email
9553without using the scripts:
9554
95551. *Make Your Changes Locally:* Make your changes in your local Git
9556 repository. You should make small, controlled, isolated changes.
9557 Keeping changes small and isolated aids review, makes
9558 merging/rebasing easier and keeps the change history clean should
9559 anyone need to refer to it in future.
9560
95612. *Stage Your Changes:* Stage your changes by using the ``git add``
9562 command on each file you changed.
9563
95643. *Commit Your Changes:* Commit the change by using the
9565 ``git commit --signoff`` command. Using the ``--signoff`` option
9566 identifies you as the person making the change and also satisfies the
9567 Developer's Certificate of Origin (DCO) shown earlier.
9568
9569 When you form a commit, you must follow certain standards established
9570 by the Yocto Project development team. See `Step
9571 3 <#making-sure-you-have-correct-commit-information>`__ in the
9572 previous section for information on how to provide commit information
9573 that meets Yocto Project commit message standards.
9574
95754. *Format the Commit:* Format the commit into an email message. To
9576 format commits, use the ``git format-patch`` command. When you
9577 provide the command, you must include a revision list or a number of
9578 patches as part of the command. For example, either of these two
9579 commands takes your most recent single commit and formats it as an
9580 email message in the current directory: $ git format-patch -1 or $
9581 git format-patch HEAD~
9582
9583 After the command is run, the current directory contains a numbered
9584 ``.patch`` file for the commit.
9585
9586 If you provide several commits as part of the command, the
9587 ``git format-patch`` command produces a series of numbered files in
9588 the current directory – one for each commit. If you have more than
9589 one patch, you should also use the ``--cover`` option with the
9590 command, which generates a cover letter as the first "patch" in the
9591 series. You can then edit the cover letter to provide a description
9592 for the series of patches. For information on the
9593 ``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed
9594 using the ``man git-format-patch`` command.
9595
9596 .. note::
9597
9598 If you are or will be a frequent contributor to the Yocto Project
9599 or to OpenEmbedded, you might consider requesting a contrib area
9600 and the necessary associated rights.
9601
96025. *Import the Files Into Your Mail Client:* Import the files into your
9603 mail client by using the ``git send-email`` command.
9604
9605 .. note::
9606
9607 In order to use
9608 git send-email
9609 , you must have the proper Git packages installed on your host.
9610 For Ubuntu, Debian, and Fedora the package is
9611 git-email
9612 .
9613
9614 The ``git send-email`` command sends email by using a local or remote
9615 Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or
9616 through a direct ``smtp`` configuration in your Git ``~/.gitconfig``
9617 file. If you are submitting patches through email only, it is very
9618 important that you submit them without any whitespace or HTML
9619 formatting that either you or your mailer introduces. The maintainer
9620 that receives your patches needs to be able to save and apply them
9621 directly from your emails. A good way to verify that what you are
9622 sending will be applicable by the maintainer is to do a dry run and
9623 send them to yourself and then save and apply them as the maintainer
9624 would.
9625
9626 The ``git send-email`` command is the preferred method for sending
9627 your patches using email since there is no risk of compromising
9628 whitespace in the body of the message, which can occur when you use
9629 your own mail client. The command also has several options that let
9630 you specify recipients and perform further editing of the email
9631 message. For information on how to use the ``git send-email``
9632 command, see ``GIT-SEND-EMAIL(1)`` displayed using the
9633 ``man git-send-email`` command.
9634
9635Working With Licenses
9636=====================
9637
9638As mentioned in the "`Licensing <&YOCTO_DOCS_OM_URL;#licensing>`__"
9639section in the Yocto Project Overview and Concepts Manual, open source
9640projects are open to the public and they consequently have different
9641licensing structures in place. This section describes the mechanism by
9642which the `OpenEmbedded build
9643system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ tracks changes to
9644licensing text and covers how to maintain open source license compliance
9645during your project's lifecycle. The section also describes how to
9646enable commercially licensed recipes, which by default are disabled.
9647
9648.. _usingpoky-configuring-LIC_FILES_CHKSUM:
9649
9650Tracking License Changes
9651------------------------
9652
9653The license of an upstream project might change in the future. In order
9654to prevent these changes going unnoticed, the
9655```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
9656variable tracks changes to the license text. The checksums are validated
9657at the end of the configure step, and if the checksums do not match, the
9658build will fail.
9659
9660.. _usingpoky-specifying-LIC_FILES_CHKSUM:
9661
9662Specifying the ``LIC_FILES_CHKSUM`` Variable
9663~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9664
9665The ``LIC_FILES_CHKSUM`` variable contains checksums of the license text
9666in the source code for the recipe. Following is an example of how to
9667specify ``LIC_FILES_CHKSUM``: LIC_FILES_CHKSUM =
9668"file://COPYING;md5=xxxx \\
9669file://licfile1.txt;beginline=5;endline=29;md5=yyyy \\
9670file://licfile2.txt;endline=50;md5=zzzz \\ ..."
9671
9672.. note::
9673
9674 - When using "beginline" and "endline", realize that line numbering
9675 begins with one and not zero. Also, the included lines are
9676 inclusive (i.e. lines five through and including 29 in the
9677 previous example for ``licfile1.txt``).
9678
9679 - When a license check fails, the selected license text is included
9680 as part of the QA message. Using this output, you can determine
9681 the exact start and finish for the needed license text.
9682
9683The build system uses the ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__
9684variable as the default directory when searching files listed in
9685``LIC_FILES_CHKSUM``. The previous example employs the default
9686directory.
9687
9688Consider this next example: LIC_FILES_CHKSUM =
9689"file://src/ls.c;beginline=5;endline=16;\\
9690md5=bb14ed3c4cda583abc85401304b5cd4e" LIC_FILES_CHKSUM =
9691"file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
9692
9693The first line locates a file in ``${S}/src/ls.c`` and isolates lines
9694five through 16 as license text. The second line refers to a file in
9695```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__.
9696
9697Note that ``LIC_FILES_CHKSUM`` variable is mandatory for all recipes,
9698unless the ``LICENSE`` variable is set to "CLOSED".
9699
9700.. _usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax:
9701
9702Explanation of Syntax
9703~~~~~~~~~~~~~~~~~~~~~
9704
9705As mentioned in the previous section, the ``LIC_FILES_CHKSUM`` variable
9706lists all the important files that contain the license text for the
9707source code. It is possible to specify a checksum for an entire file, or
9708a specific section of a file (specified by beginning and ending line
9709numbers with the "beginline" and "endline" parameters, respectively).
9710The latter is useful for source files with a license notice header,
9711README documents, and so forth. If you do not use the "beginline"
9712parameter, then it is assumed that the text begins on the first line of
9713the file. Similarly, if you do not use the "endline" parameter, it is
9714assumed that the license text ends with the last line of the file.
9715
9716The "md5" parameter stores the md5 checksum of the license text. If the
9717license text changes in any way as compared to this parameter then a
9718mismatch occurs. This mismatch triggers a build failure and notifies the
9719developer. Notification allows the developer to review and address the
9720license text changes. Also note that if a mismatch occurs during the
9721build, the correct md5 checksum is placed in the build log and can be
9722easily copied to the recipe.
9723
9724There is no limit to how many files you can specify using the
9725``LIC_FILES_CHKSUM`` variable. Generally, however, every project
9726requires a few specifications for license tracking. Many projects have a
9727"COPYING" file that stores the license information for all the source
9728code files. This practice allows you to just track the "COPYING" file as
9729long as it is kept up to date.
9730
9731.. note::
9732
9733 - If you specify an empty or invalid "md5" parameter,
9734 `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ returns an md5
9735 mis-match error and displays the correct "md5" parameter value
9736 during the build. The correct parameter is also captured in the
9737 build log.
9738
9739 - If the whole file contains only license text, you do not need to
9740 use the "beginline" and "endline" parameters.
9741
9742Enabling Commercially Licensed Recipes
9743--------------------------------------
9744
9745By default, the OpenEmbedded build system disables components that have
9746commercial or other special licensing requirements. Such requirements
9747are defined on a recipe-by-recipe basis through the
9748```LICENSE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS>`__ variable
9749definition in the affected recipe. For instance, the
9750``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
9751contains the following statement: LICENSE_FLAGS = "commercial" Here is a
9752slightly more complicated example that contains both an explicit recipe
9753name and version (after variable expansion): LICENSE_FLAGS =
9754"license_${PN}_${PV}" In order for a component restricted by a
9755``LICENSE_FLAGS`` definition to be enabled and included in an image, it
9756needs to have a matching entry in the global
9757```LICENSE_FLAGS_WHITELIST`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST>`__
9758variable, which is a variable typically defined in your ``local.conf``
9759file. For example, to enable the
9760``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
9761could add either the string "commercial_gst-plugins-ugly" or the more
9762general string "commercial" to ``LICENSE_FLAGS_WHITELIST``. See the
9763"`License Flag Matching <#license-flag-matching>`__" section for a full
9764explanation of how ``LICENSE_FLAGS`` matching works. Here is the
9765example: LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
9766Likewise, to additionally enable the package built from the recipe
9767containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
9768the actual recipe name was ``emgd_1.10.bb``, the following string would
9769enable that package as well as the original ``gst-plugins-ugly``
9770package: LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly
9771license_emgd_1.10" As a convenience, you do not need to specify the
9772complete license string in the whitelist for every package. You can use
9773an abbreviated form, which consists of just the first portion or
9774portions of the license string before the initial underscore character
9775or characters. A partial string will match any license that contains the
9776given string as the first portion of its license. For example, the
9777following whitelist string will also match both of the packages
9778previously mentioned as well as any other packages that have licenses
9779starting with "commercial" or "license". LICENSE_FLAGS_WHITELIST =
9780"commercial license"
9781
9782License Flag Matching
9783~~~~~~~~~~~~~~~~~~~~~
9784
9785License flag matching allows you to control what recipes the
9786OpenEmbedded build system includes in the build. Fundamentally, the
9787build system attempts to match ``LICENSE_FLAGS`` strings found in
9788recipes against ``LICENSE_FLAGS_WHITELIST`` strings found in the
9789whitelist. A match causes the build system to include a recipe in the
9790build, while failure to find a match causes the build system to exclude
9791a recipe.
9792
9793In general, license flag matching is simple. However, understanding some
9794concepts will help you correctly and effectively use matching.
9795
9796Before a flag defined by a particular recipe is tested against the
9797contents of the whitelist, the expanded string ``_${PN}`` is appended to
9798the flag. This expansion makes each ``LICENSE_FLAGS`` value
9799recipe-specific. After expansion, the string is then matched against the
9800whitelist. Thus, specifying ``LICENSE_FLAGS = "commercial"`` in recipe
9801"foo", for example, results in the string ``"commercial_foo"``. And, to
9802create a match, that string must appear in the whitelist.
9803
9804Judicious use of the ``LICENSE_FLAGS`` strings and the contents of the
9805``LICENSE_FLAGS_WHITELIST`` variable allows you a lot of flexibility for
9806including or excluding recipes based on licensing. For example, you can
9807broaden the matching capabilities by using license flags string subsets
9808in the whitelist.
9809
9810.. note::
9811
9812 When using a string subset, be sure to use the part of the expanded
9813 string that precedes the appended underscore character (e.g.
9814 usethispart_1.3
9815 ,
9816 usethispart_1.4
9817 , and so forth).
9818
9819For example, simply specifying the string "commercial" in the whitelist
9820matches any expanded ``LICENSE_FLAGS`` definition that starts with the
9821string "commercial" such as "commercial_foo" and "commercial_bar", which
9822are the strings the build system automatically generates for
9823hypothetical recipes named "foo" and "bar" assuming those recipes simply
9824specify the following: LICENSE_FLAGS = "commercial" Thus, you can choose
9825to exhaustively enumerate each license flag in the whitelist and allow
9826only specific recipes into the image, or you can use a string subset
9827that causes a broader range of matches to allow a range of recipes into
9828the image.
9829
9830This scheme works even if the ``LICENSE_FLAGS`` string already has
9831``_${PN}`` appended. For example, the build system turns the license
9832flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
9833both the general "commercial" and the specific "commercial_1.2_foo"
9834strings found in the whitelist, as expected.
9835
9836Here are some other scenarios:
9837
9838- You can specify a versioned string in the recipe such as
9839 "commercial_foo_1.2" in a "foo" recipe. The build system expands this
9840 string to "commercial_foo_1.2_foo". Combine this license flag with a
9841 whitelist that has the string "commercial" and you match the flag
9842 along with any other flag that starts with the string "commercial".
9843
9844- Under the same circumstances, you can use "commercial_foo" in the
9845 whitelist and the build system not only matches "commercial_foo_1.2"
9846 but also matches any license flag with the string "commercial_foo",
9847 regardless of the version.
9848
9849- You can be very specific and use both the package and version parts
9850 in the whitelist (e.g. "commercial_foo_1.2") to specifically match a
9851 versioned recipe.
9852
9853Other Variables Related to Commercial Licenses
9854~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
9855
9856Other helpful variables related to commercial license handling exist and
9857are defined in the
9858``poky/meta/conf/distro/include/default-distrovars.inc`` file:
9859COMMERCIAL_AUDIO_PLUGINS ?= "" COMMERCIAL_VIDEO_PLUGINS ?= "" If you
9860want to enable these components, you can do so by making sure you have
9861statements similar to the following in your ``local.conf`` configuration
9862file: COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \\
9863gst-plugins-ugly-mpegaudioparse" COMMERCIAL_VIDEO_PLUGINS =
9864"gst-plugins-ugly-mpeg2dec \\ gst-plugins-ugly-mpegstream
9865gst-plugins-bad-mpegvideoparse" LICENSE_FLAGS_WHITELIST =
9866"commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
9867Of course, you could also create a matching whitelist for those
9868components using the more general "commercial" in the whitelist, but
9869that would also enable all the other packages with ``LICENSE_FLAGS``
9870containing "commercial", which you may or may not want:
9871LICENSE_FLAGS_WHITELIST = "commercial"
9872
9873Specifying audio and video plugins as part of the
9874``COMMERCIAL_AUDIO_PLUGINS`` and ``COMMERCIAL_VIDEO_PLUGINS`` statements
9875(along with the enabling ``LICENSE_FLAGS_WHITELIST``) includes the
9876plugins or components into built images, thus adding support for media
9877formats or components.
9878
9879Maintaining Open Source License Compliance During Your Product's Lifecycle
9880--------------------------------------------------------------------------
9881
9882One of the concerns for a development organization using open source
9883software is how to maintain compliance with various open source
9884licensing during the lifecycle of the product. While this section does
9885not provide legal advice or comprehensively cover all scenarios, it does
9886present methods that you can use to assist you in meeting the compliance
9887requirements during a software release.
9888
9889With hundreds of different open source licenses that the Yocto Project
9890tracks, it is difficult to know the requirements of each and every
9891license. However, the requirements of the major FLOSS licenses can begin
9892to be covered by assuming that three main areas of concern exist:
9893
9894- Source code must be provided.
9895
9896- License text for the software must be provided.
9897
9898- Compilation scripts and modifications to the source code must be
9899 provided.
9900
9901There are other requirements beyond the scope of these three and the
9902methods described in this section (e.g. the mechanism through which
9903source code is distributed).
9904
9905As different organizations have different methods of complying with open
9906source licensing, this section is not meant to imply that there is only
9907one single way to meet your compliance obligations, but rather to
9908describe one method of achieving compliance. The remainder of this
9909section describes methods supported to meet the previously mentioned
9910three requirements. Once you take steps to meet these requirements, and
9911prior to releasing images, sources, and the build system, you should
9912audit all artifacts to ensure completeness.
9913
9914.. note::
9915
9916 The Yocto Project generates a license manifest during image creation
9917 that is located in
9918 ${DEPLOY_DIR}/licenses/
9919 image_name-datestamp
9920 to assist with any audits.
9921
9922Providing the Source Code
9923~~~~~~~~~~~~~~~~~~~~~~~~~
9924
9925Compliance activities should begin before you generate the final image.
9926The first thing you should look at is the requirement that tops the list
9927for most compliance groups - providing the source. The Yocto Project has
9928a few ways of meeting this requirement.
9929
9930One of the easiest ways to meet this requirement is to provide the
9931entire ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ used by the
9932build. This method, however, has a few issues. The most obvious is the
9933size of the directory since it includes all sources used in the build
9934and not just the source used in the released image. It will include
9935toolchain source, and other artifacts, which you would not generally
9936release. However, the more serious issue for most companies is
9937accidental release of proprietary software. The Yocto Project provides
9938an ```archiver`` <&YOCTO_DOCS_REF_URL;#ref-classes-archiver>`__ class to
9939help avoid some of these concerns.
9940
9941Before you employ ``DL_DIR`` or the ``archiver`` class, you need to
9942decide how you choose to provide source. The source ``archiver`` class
9943can generate tarballs and SRPMs and can create them with various levels
9944of compliance in mind.
9945
9946One way of doing this (but certainly not the only way) is to release
9947just the source as a tarball. You can do this by adding the following to
9948the ``local.conf`` file found in the `Build
9949Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__: INHERIT +=
9950"archiver" ARCHIVER_MODE[src] = "original" During the creation of your
9951image, the source from all recipes that deploy packages to the image is
9952placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
9953```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ for each recipe.
9954Releasing the entire directory enables you to comply with requirements
9955concerning providing the unmodified source. It is important to note that
9956the size of the directory can get large.
9957
9958A way to help mitigate the size issue is to only release tarballs for
9959licenses that require the release of source. Let us assume you are only
9960concerned with GPL code as identified by running the following script: #
9961Script to archive a subset of packages matching specific license(s) #
9962Source and license files are copied into sub folders of package folder #
9963Must be run from build folder #!/bin/bash
9964src_release_dir="source-release" mkdir -p $src_release_dir for a in
9965tmp/deploy/sources/*; do for d in $a/*; do # Get package name from path
9966p=`basename $d\` p=${p%-*} p=${p%-*} # Only archive GPL packages (update
9967\*GPL\* regex for your license check) numfiles=`ls
9968tmp/deploy/licenses/$p/*GPL\* 2> /dev/null \| wc -l\` if [ $numfiles -gt
99691 ]; then echo Archiving $p mkdir -p $src_release_dir/$p/source cp $d/\*
9970$src_release_dir/$p/source 2> /dev/null mkdir -p
9971$src_release_dir/$p/license cp tmp/deploy/licenses/$p/\*
9972$src_release_dir/$p/license 2> /dev/null fi done done At this point, you
9973could create a tarball from the ``gpl_source_release`` directory and
9974provide that to the end user. This method would be a step toward
9975achieving compliance with section 3a of GPLv2 and with section 6 of
9976GPLv3.
9977
9978Providing License Text
9979~~~~~~~~~~~~~~~~~~~~~~
9980
9981One requirement that is often overlooked is inclusion of license text.
9982This requirement also needs to be dealt with prior to generating the
9983final image. Some licenses require the license text to accompany the
9984binary. You can achieve this by adding the following to your
9985``local.conf`` file: COPY_LIC_MANIFEST = "1" COPY_LIC_DIRS = "1"
9986LICENSE_CREATE_PACKAGE = "1" Adding these statements to the
9987configuration file ensures that the licenses collected during package
9988generation are included on your image.
9989
9990.. note::
9991
9992 Setting all three variables to "1" results in the image having two
9993 copies of the same license file. One copy resides in
9994 ``/usr/share/common-licenses`` and the other resides in
9995 ``/usr/share/license``.
9996
9997 The reason for this behavior is because
9998 ```COPY_LIC_DIRS`` <&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS>`__ and
9999 ```COPY_LIC_MANIFEST`` <&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST>`__
10000 add a copy of the license when the image is built but do not offer a
10001 path for adding licenses for newly installed packages to an image.
10002 ```LICENSE_CREATE_PACKAGE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE>`__
10003 adds a separate package and an upgrade path for adding licenses to an
10004 image.
10005
10006As the source ``archiver`` class has already archived the original
10007unmodified source that contains the license files, you would have
10008already met the requirements for inclusion of the license information
10009with source as defined by the GPL and other open source licenses.
10010
10011Providing Compilation Scripts and Source Code Modifications
10012~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10013
10014At this point, we have addressed all we need to prior to generating the
10015image. The next two requirements are addressed during the final
10016packaging of the release.
10017
10018By releasing the version of the OpenEmbedded build system and the layers
10019used during the build, you will be providing both compilation scripts
10020and the source code modifications in one step.
10021
10022If the deployment team has a `BSP
10023layer <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__ and a distro layer, and those
10024those layers are used to patch, compile, package, or modify (in any way)
10025any open source software included in your released images, you might be
10026required to release those layers under section 3 of GPLv2 or section 1
10027of GPLv3. One way of doing that is with a clean checkout of the version
10028of the Yocto Project and layers used during your build. Here is an
10029example: # We built using the DISTRO_NAME_NO_CAP branch of the poky repo
10030$ git clone -b DISTRO_NAME_NO_CAP git://git.yoctoproject.org/poky $ cd
10031poky # We built using the release_branch for our layers $ git clone -b
10032release_branch git://git.mycompany.com/meta-my-bsp-layer $ git clone -b
10033release_branch git://git.mycompany.com/meta-my-software-layer # clean up
10034the .git repos $ find . -name ".git" -type d -exec rm -rf {} \\; One
10035thing a development organization might want to consider for end-user
10036convenience is to modify ``meta-poky/conf/bblayers.conf.sample`` to
10037ensure that when the end user utilizes the released build system to
10038build an image, the development organization's layers are included in
10039the ``bblayers.conf`` file automatically: # POKY_BBLAYERS_CONF_VERSION
10040is increased each time build/conf/bblayers.conf # changes incompatibly
10041POKY_BBLAYERS_CONF_VERSION = "2" BBPATH = "${TOPDIR}" BBFILES ?= ""
10042BBLAYERS ?= " \\ ##OEROOT##/meta \\ ##OEROOT##/meta-poky \\
10043##OEROOT##/meta-yocto-bsp \\ ##OEROOT##/meta-mylayer \\ " Creating and
10044providing an archive of the `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__
10045layers (recipes, configuration files, and so forth) enables you to meet
10046your requirements to include the scripts to control compilation as well
10047as any modifications to the original source.
10048
10049Copying Licenses that Do Not Exist
10050----------------------------------
10051
10052Some packages, such as the linux-firmware package, have many licenses
10053that are not in any way common. You can avoid adding a lot of these
10054types of common license files, which are only applicable to a specific
10055package, by using the
10056```NO_GENERIC_LICENSE`` <&YOCTO_DOCS_REF_URL;#var-NO_GENERIC_LICENSE>`__
10057variable. Using this variable also avoids QA errors when you use a
10058non-common, non-CLOSED license in a recipe.
10059
10060The following is an example that uses the ``LICENSE.Abilis.txt`` file as
10061the license from the fetched source: NO_GENERIC_LICENSE[Firmware-Abilis]
10062= "LICENSE.Abilis.txt"
10063
10064Using the Error Reporting Tool
10065==============================
10066
10067The error reporting tool allows you to submit errors encountered during
10068builds to a central database. Outside of the build environment, you can
10069use a web interface to browse errors, view statistics, and query for
10070errors. The tool works using a client-server system where the client
10071portion is integrated with the installed Yocto Project `Source
10072Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``).
10073The server receives the information collected and saves it in a
10074database.
10075
10076A live instance of the error reporting server exists at
10077` <http://errors.yoctoproject.org>`__. This server exists so that when
10078you want to get help with build failures, you can submit all of the
10079information on the failure easily and then point to the URL in your bug
10080report or send an email to the mailing list.
10081
10082.. note::
10083
10084 If you send error reports to this server, the reports become publicly
10085 visible.
10086
10087Enabling and Using the Tool
10088---------------------------
10089
10090By default, the error reporting tool is disabled. You can enable it by
10091inheriting the
10092```report-error`` <&YOCTO_DOCS_REF_URL;#ref-classes-report-error>`__
10093class by adding the following statement to the end of your
10094``local.conf`` file in your `Build
10095Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. INHERIT +=
10096"report-error"
10097
10098By default, the error reporting feature stores information in
10099``${``\ ```LOG_DIR`` <&YOCTO_DOCS_REF_URL;#var-LOG_DIR>`__\ ``}/error-report``.
10100However, you can specify a directory to use by adding the following to
10101your ``local.conf`` file: ERR_REPORT_DIR = "path" Enabling error
10102reporting causes the build process to collect the errors and store them
10103in a file as previously described. When the build system encounters an
10104error, it includes a command as part of the console output. You can run
10105the command to send the error file to the server. For example, the
10106following command sends the errors to an upstream server: $
10107send-error-report
10108/home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
10109In the previous example, the errors are sent to a public database
10110available at ` <http://errors.yoctoproject.org>`__, which is used by the
10111entire community. If you specify a particular server, you can send the
10112errors to a different database. Use the following command for more
10113information on available options: $ send-error-report --help
10114
10115When sending the error file, you are prompted to review the data being
10116sent as well as to provide a name and optional email address. Once you
10117satisfy these prompts, the command returns a link from the server that
10118corresponds to your entry in the database. For example, here is a
10119typical link: http://errors.yoctoproject.org/Errors/Details/9522/
10120Following the link takes you to a web interface where you can browse,
10121query the errors, and view statistics.
10122
10123Disabling the Tool
10124------------------
10125
10126To disable the error reporting feature, simply remove or comment out the
10127following statement from the end of your ``local.conf`` file in your
10128`Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. INHERIT +=
10129"report-error"
10130
10131Setting Up Your Own Error Reporting Server
10132------------------------------------------
10133
10134If you want to set up your own error reporting server, you can obtain
10135the code from the Git repository at
10136` <http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/>`__.
10137Instructions on how to set it up are in the README document.
10138
10139.. _dev-using-wayland-and-weston:
10140
10141Using Wayland and Weston
10142========================
10143
10144`Wayland <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)>`__
10145is a computer display server protocol that provides a method for
10146compositing window managers to communicate directly with applications
10147and video hardware and expects them to communicate with input hardware
10148using other libraries. Using Wayland with supporting targets can result
10149in better control over graphics frame rendering than an application
10150might otherwise achieve.
10151
10152The Yocto Project provides the Wayland protocol libraries and the
10153reference
10154`Weston <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston>`__
10155compositor as part of its release. You can find the integrated packages
10156in the ``meta`` layer of the `Source
10157Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. Specifically, you
10158can find the recipes that build both Wayland and Weston at
10159``meta/recipes-graphics/wayland``.
10160
10161You can build both the Wayland and Weston packages for use only with
10162targets that accept the `Mesa 3D and Direct Rendering
10163Infrastructure <https://en.wikipedia.org/wiki/Mesa_(computer_graphics)>`__,
10164which is also known as Mesa DRI. This implies that you cannot build and
10165use the packages if your target uses, for example, the Intel Embedded
10166Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI.
10167
10168.. note::
10169
10170 Due to lack of EGL support, Weston 1.0.3 will not run directly on the
10171 emulated QEMU hardware. However, this version of Weston will run
10172 under X emulation without issues.
10173
10174This section describes what you need to do to implement Wayland and use
10175the Weston compositor when building an image for a supporting target.
10176
10177Enabling Wayland in an Image
10178----------------------------
10179
10180To enable Wayland, you need to enable it to be built and enable it to be
10181included (installed) in the image.
10182
10183.. _enable-building:
10184
10185Building
10186~~~~~~~~
10187
10188To cause Mesa to build the ``wayland-egl`` platform and Weston to build
10189Wayland with Kernel Mode Setting
10190(`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__)
10191support, include the "wayland" flag in the
10192```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
10193statement in your ``local.conf`` file: DISTRO_FEATURES_append = "
10194wayland"
10195
10196.. note::
10197
10198 If X11 has been enabled elsewhere, Weston will build Wayland with X11
10199 support
10200
10201.. _enable-installation-in-an-image:
10202
10203Installing
10204~~~~~~~~~~
10205
10206To install the Wayland feature into an image, you must include the
10207following
10208```CORE_IMAGE_EXTRA_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL>`__
10209statement in your ``local.conf`` file: CORE_IMAGE_EXTRA_INSTALL +=
10210"wayland weston"
10211
10212Running Weston
10213--------------
10214
10215To run Weston inside X11, enabling it as described earlier and building
10216a Sato image is sufficient. If you are running your image under Sato, a
10217Weston Launcher appears in the "Utility" category.
10218
10219Alternatively, you can run Weston through the command-line interpretor
10220(CLI), which is better suited for development work. To run Weston under
10221the CLI, you need to do the following after your image is built:
10222
102231. Run these commands to export ``XDG_RUNTIME_DIR``: mkdir -p
10224 /tmp/$USER-weston chmod 0700 /tmp/$USER-weston export
10225 XDG_RUNTIME_DIR=/tmp/$USER-weston
10226
102272. Launch Weston in the shell: weston
diff --git a/documentation/dev-manual/dev-manual-intro.rst b/documentation/dev-manual/dev-manual-intro.rst
new file mode 100644
index 0000000000..5b26d9eb18
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-intro.rst
@@ -0,0 +1,62 @@
1******************************************
2The Yocto Project Development Tasks Manual
3******************************************
4
5.. _dev-welcome:
6
7Welcome
8=======
9
10Welcome to the Yocto Project Development Tasks Manual! This manual
11provides relevant procedures necessary for developing in the Yocto
12Project environment (i.e. developing embedded Linux images and
13user-space applications that run on targeted devices). The manual groups
14related procedures into higher-level sections. Procedures can consist of
15high-level steps or low-level steps depending on the topic.
16
17This manual provides the following:
18
19- Procedures that help you get going with the Yocto Project. For
20 example, procedures that show you how to set up a build host and work
21 with the Yocto Project source repositories.
22
23- Procedures that show you how to submit changes to the Yocto Project.
24 Changes can be improvements, new features, or bug fixes.
25
26- Procedures related to "everyday" tasks you perform while developing
27 images and applications using the Yocto Project. For example,
28 procedures to create a layer, customize an image, write a new recipe,
29 and so forth.
30
31This manual does not provide the following:
32
33- Redundant Step-by-step Instructions: For example, the `Yocto Project
34 Application Development and the Extensible Software Development Kit
35 (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual contains detailed
36 instructions on how to install an SDK, which is used to develop
37 applications for target hardware.
38
39- Reference or Conceptual Material: This type of material resides in an
40 appropriate reference manual. For example, system variables are
41 documented in the `Yocto Project Reference
42 Manual <&YOCTO_DOCS_REF_URL;>`__.
43
44- Detailed Public Information Not Specific to the Yocto Project: For
45 example, exhaustive information on how to use the Source Control
46 Manager Git is better covered with Internet searches and official Git
47 Documentation than through the Yocto Project documentation.
48
49Other Information
50=================
51
52Because this manual presents information for many different topics,
53supplemental information is recommended for full comprehension. For
54introductory information on the Yocto Project, see the `Yocto Project
55Website <&YOCTO_HOME_URL;>`__. If you want to build an image with no
56knowledge of Yocto Project as a way of quickly testing it out, see the
57`Yocto Project Quick Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
58
59For a comprehensive list of links and other documentation, see the
60"`Links and Related
61Documentation <&YOCTO_DOCS_REF_URL;#resources-links-and-related-documentation>`__"
62section in the Yocto Project Reference Manual.
diff --git a/documentation/dev-manual/dev-manual-qemu.rst b/documentation/dev-manual/dev-manual-qemu.rst
new file mode 100644
index 0000000000..b3cd69d805
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-qemu.rst
@@ -0,0 +1,429 @@
1*******************************
2Using the Quick EMUlator (QEMU)
3*******************************
4
5The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
6Open Source project as part of the Yocto Project development "tool set".
7This chapter provides both procedures that show you how to use the Quick
8EMUlator (QEMU) and other QEMU information helpful for development
9purposes.
10
11.. _qemu-dev-overview:
12
13Overview
14========
15
16Within the context of the Yocto Project, QEMU is an emulator and
17virtualization machine that allows you to run a complete image you have
18built using the Yocto Project as just another task on your build system.
19QEMU is useful for running and testing images and applications on
20supported Yocto Project architectures without having actual hardware.
21Among other things, the Yocto Project uses QEMU to run automated Quality
22Assurance (QA) tests on final images shipped with each release.
23
24.. note::
25
26 This implementation is not the same as QEMU in general.
27
28This section provides a brief reference for the Yocto Project
29implementation of QEMU.
30
31For official information and documentation on QEMU in general, see the
32following references:
33
34- `QEMU Website <http://wiki.qemu.org/Main_Page>`__\ *:* The official
35 website for the QEMU Open Source project.
36
37- `Documentation <http://wiki.qemu.org/Manual>`__\ *:* The QEMU user
38 manual.
39
40.. _qemu-running-qemu:
41
42Running QEMU
43============
44
45To use QEMU, you need to have QEMU installed and initialized as well as
46have the proper artifacts (i.e. image files and root filesystems)
47available. Follow these general steps to run QEMU:
48
491. *Install QEMU:* QEMU is made available with the Yocto Project a
50 number of ways. One method is to install a Software Development Kit
51 (SDK). See "`The QEMU
52 Emulator <&YOCTO_DOCS_SDK_URL;#the-qemu-emulator>`__" section in the
53 Yocto Project Application Development and the Extensible Software
54 Development Kit (eSDK) manual for information on how to install QEMU.
55
562. *Setting Up the Environment:* How you set up the QEMU environment
57 depends on how you installed QEMU:
58
59 - If you cloned the ``poky`` repository or you downloaded and
60 unpacked a Yocto Project release tarball, you can source the build
61 environment script (i.e.
62 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__): $ cd
63 ~/poky $ source oe-init-build-env
64
65 - If you installed a cross-toolchain, you can run the script that
66 initializes the toolchain. For example, the following commands run
67 the initialization script from the default ``poky_sdk`` directory:
68 . ~/poky_sdk/environment-setup-core2-64-poky-linux
69
703. *Ensure the Artifacts are in Place:* You need to be sure you have a
71 pre-built kernel that will boot in QEMU. You also need the target
72 root filesystem for your target machine’s architecture:
73
74 - If you have previously built an image for QEMU (e.g. ``qemux86``,
75 ``qemuarm``, and so forth), then the artifacts are in place in
76 your `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
77
78 - If you have not built an image, you can go to the
79 `machines/qemu <&YOCTO_MACHINES_DL_URL;>`__ area and download a
80 pre-built image that matches your architecture and can be run on
81 QEMU.
82
83 See the "`Extracting the Root
84 Filesystem <&YOCTO_DOCS_SDK_URL;#sdk-extracting-the-root-filesystem>`__"
85 section in the Yocto Project Application Development and the
86 Extensible Software Development Kit (eSDK) manual for information on
87 how to extract a root filesystem.
88
894. *Run QEMU:* The basic ``runqemu`` command syntax is as follows: $
90 runqemu [option ] [...] Based on what you provide on the command
91 line, ``runqemu`` does a good job of figuring out what you are trying
92 to do. For example, by default, QEMU looks for the most recently
93 built image according to the timestamp when it needs to look for an
94 image. Minimally, through the use of options, you must provide either
95 a machine name, a virtual machine image (``*wic.vmdk``), or a kernel
96 image (``*.bin``).
97
98 Here are some additional examples to help illustrate further QEMU:
99
100 - This example starts QEMU with MACHINE set to "qemux86-64".
101 Assuming a standard `Build
102 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, ``runqemu``
103 automatically finds the ``bzImage-qemux86-64.bin`` image file and
104 the ``core-image-minimal-qemux86-64-20200218002850.rootfs.ext4``
105 (assuming the current build created a ``core-image-minimal``
106 image).
107
108 .. note::
109
110 When more than one image with the same name exists, QEMU finds
111 and uses the most recently built image according to the
112 timestamp.
113
114 $ runqemu qemux86-64
115
116 - This example produces the exact same results as the previous
117 example. This command, however, specifically provides the image
118 and root filesystem type. $ runqemu qemux86-64 core-image-minimal
119 ext4
120
121 - This example specifies to boot an initial RAM disk image and to
122 enable audio in QEMU. For this case, ``runqemu`` set the internal
123 variable ``FSTYPE`` to "cpio.gz". Also, for audio to be enabled,
124 an appropriate driver must be installed (see the previous
125 description for the ``audio`` option for more information). $
126 runqemu qemux86-64 ramfs audio
127
128 - This example does not provide enough information for QEMU to
129 launch. While the command does provide a root filesystem type, it
130 must also minimally provide a MACHINE, KERNEL, or VM option. $
131 runqemu ext4
132
133 - This example specifies to boot a virtual machine image
134 (``.wic.vmdk`` file). From the ``.wic.vmdk``, ``runqemu``
135 determines the QEMU architecture (MACHINE) to be "qemux86-64" and
136 the root filesystem type to be "vmdk". $ runqemu
137 /home/scott-lenovo/vm/core-image-minimal-qemux86-64.wic.vmdk
138
139Switching Between Consoles
140==========================
141
142When booting or running QEMU, you can switch between supported consoles
143by using Ctrl+Alt+number. For example, Ctrl+Alt+3 switches you to the
144serial console as long as that console is enabled. Being able to switch
145consoles is helpful, for example, if the main QEMU console breaks for
146some reason.
147
148.. note::
149
150 Usually, "2" gets you to the main console and "3" gets you to the
151 serial console.
152
153Removing the Splash Screen
154==========================
155
156You can remove the splash screen when QEMU is booting by using Alt+left.
157Removing the splash screen allows you to see what is happening in the
158background.
159
160Disabling the Cursor Grab
161=========================
162
163The default QEMU integration captures the cursor within the main window.
164It does this since standard mouse devices only provide relative input
165and not absolute coordinates. You then have to break out of the grab
166using the "Ctrl+Alt" key combination. However, the Yocto Project's
167integration of QEMU enables the wacom USB touch pad driver by default to
168allow input of absolute coordinates. This default means that the mouse
169can enter and leave the main window without the grab taking effect
170leading to a better user experience.
171
172.. _qemu-running-under-a-network-file-system-nfs-server:
173
174Running Under a Network File System (NFS) Server
175================================================
176
177One method for running QEMU is to run it on an NFS server. This is
178useful when you need to access the same file system from both the build
179and the emulated system at the same time. It is also worth noting that
180the system does not need root privileges to run. It uses a user space
181NFS server to avoid that. Follow these steps to set up for running QEMU
182using an NFS server.
183
1841. *Extract a Root Filesystem:* Once you are able to run QEMU in your
185 environment, you can use the ``runqemu-extract-sdk`` script, which is
186 located in the ``scripts`` directory along with the ``runqemu``
187 script.
188
189 The ``runqemu-extract-sdk`` takes a root filesystem tarball and
190 extracts it into a location that you specify. Here is an example that
191 takes a file system and extracts it to a directory named
192 ``test-nfs``: runqemu-extract-sdk
193 ./tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.tar.bz2
194 test-nfs
195
1962. *Start QEMU:* Once you have extracted the file system, you can run
197 ``runqemu`` normally with the additional location of the file system.
198 You can then also make changes to the files within ``./test-nfs`` and
199 see those changes appear in the image in real time. Here is an
200 example using the ``qemux86`` image: runqemu qemux86-64 ./test-nfs
201
202.. note::
203
204 Should you need to start, stop, or restart the NFS share, you can use
205 the following commands:
206
207 - The following command starts the NFS share: runqemu-export-rootfs
208 start file-system-location
209
210 - The following command stops the NFS share: runqemu-export-rootfs
211 stop file-system-location
212
213 - The following command restarts the NFS share:
214 runqemu-export-rootfs restart file-system-location
215
216.. _qemu-kvm-cpu-compatibility:
217
218QEMU CPU Compatibility Under KVM
219================================
220
221By default, the QEMU build compiles for and targets 64-bit and x86 Intel
222Core2 Duo processors and 32-bit x86 Intel Pentium II processors. QEMU
223builds for and targets these CPU types because they display a broad
224range of CPU feature compatibility with many commonly used CPUs.
225
226Despite this broad range of compatibility, the CPUs could support a
227feature that your host CPU does not support. Although this situation is
228not a problem when QEMU uses software emulation of the feature, it can
229be a problem when QEMU is running with KVM enabled. Specifically,
230software compiled with a certain CPU feature crashes when run on a CPU
231under KVM that does not support that feature. To work around this
232problem, you can override QEMU's runtime CPU setting by changing the
233``QB_CPU_KVM`` variable in ``qemuboot.conf`` in the `Build
234Directory's <&YOCTO_DOCS_REF_URL;#build-directory>`__ ``deploy/image``
235directory. This setting specifies a ``-cpu`` option passed into QEMU in
236the ``runqemu`` script. Running ``qemu -cpu help`` returns a list of
237available supported CPU types.
238
239.. _qemu-dev-performance:
240
241QEMU Performance
242================
243
244Using QEMU to emulate your hardware can result in speed issues depending
245on the target and host architecture mix. For example, using the
246``qemux86`` image in the emulator on an Intel-based 32-bit (x86) host
247machine is fast because the target and host architectures match. On the
248other hand, using the ``qemuarm`` image on the same Intel-based host can
249be slower. But, you still achieve faithful emulation of ARM-specific
250issues.
251
252To speed things up, the QEMU images support using ``distcc`` to call a
253cross-compiler outside the emulated system. If you used ``runqemu`` to
254start QEMU, and the ``distccd`` application is present on the host
255system, any BitBake cross-compiling toolchain available from the build
256system is automatically used from within QEMU simply by calling
257``distcc``. You can accomplish this by defining the cross-compiler
258variable (e.g. ``export CC="distcc"``). Alternatively, if you are using
259a suitable SDK image or the appropriate stand-alone toolchain is
260present, the toolchain is also automatically used.
261
262.. note::
263
264 Several mechanisms exist that let you connect to the system running
265 on the QEMU emulator:
266
267 - QEMU provides a framebuffer interface that makes standard consoles
268 available.
269
270 - Generally, headless embedded devices have a serial port. If so,
271 you can configure the operating system of the running image to use
272 that port to run a console. The connection uses standard IP
273 networking.
274
275 - SSH servers exist in some QEMU images. The ``core-image-sato``
276 QEMU image has a Dropbear secure shell (SSH) server that runs with
277 the root password disabled. The ``core-image-full-cmdline`` and
278 ``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear.
279 Including these SSH servers allow you to use standard ``ssh`` and
280 ``scp`` commands. The ``core-image-minimal`` QEMU image, however,
281 contains no SSH server.
282
283 - You can use a provided, user-space NFS server to boot the QEMU
284 session using a local copy of the root filesystem on the host. In
285 order to make this connection, you must extract a root filesystem
286 tarball by using the ``runqemu-extract-sdk`` command. After
287 running the command, you must then point the ``runqemu`` script to
288 the extracted directory instead of a root filesystem image file.
289 See the "`Running Under a Network File System (NFS)
290 Server <#qemu-running-under-a-network-file-system-nfs-server>`__"
291 section for more information.
292
293.. _qemu-dev-command-line-syntax:
294
295QEMU Command-Line Syntax
296========================
297
298The basic ``runqemu`` command syntax is as follows: $ runqemu [option ]
299[...] Based on what you provide on the command line, ``runqemu`` does a
300good job of figuring out what you are trying to do. For example, by
301default, QEMU looks for the most recently built image according to the
302timestamp when it needs to look for an image. Minimally, through the use
303of options, you must provide either a machine name, a virtual machine
304image (``*wic.vmdk``), or a kernel image (``*.bin``).
305
306Following is the command-line help output for the ``runqemu`` command: $
307runqemu --help Usage: you can run this script with any valid combination
308of the following environment variables (in any order): KERNEL - the
309kernel image file to use ROOTFS - the rootfs image file or nfsroot
310directory to use MACHINE - the machine name (optional, autodetected from
311KERNEL filename if unspecified) Simplified QEMU command-line options can
312be passed with: nographic - disable video console serial - enable a
313serial console on /dev/ttyS0 slirp - enable user networking, no root
314privileges is required kvm - enable KVM when running x86/x86_64
315(VT-capable CPU required) kvm-vhost - enable KVM with vhost when running
316x86/x86_64 (VT-capable CPU required) publicvnc - enable a VNC server
317open to all hosts audio - enable audio [*/]ovmf\* - OVMF firmware file
318or base name for booting with UEFI tcpserial=<port> - specify tcp serial
319port number biosdir=<dir> - specify custom bios dir
320biosfilename=<filename> - specify bios filename qemuparams=<xyz> -
321specify custom parameters to QEMU bootparams=<xyz> - specify custom
322kernel parameters during boot help, -h, --help: print this text
323Examples: runqemu runqemu qemuarm runqemu tmp/deploy/images/qemuarm
324runqemu tmp/deploy/images/qemux86/<qemuboot.conf> runqemu qemux86-64
325core-image-sato ext4 runqemu qemux86-64 wic-image-minimal wic runqemu
326path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial runqemu qemux86
327iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz... runqemu qemux86
328qemuparams="-m 256" runqemu qemux86 bootparams="psplash=false" runqemu
329path/to/<image>-<machine>.wic runqemu path/to/<image>-<machine>.wic.vmdk
330
331.. _qemu-dev-runqemu-command-line-options:
332
333``runqemu`` Command-Line Options
334================================
335
336Following is a description of ``runqemu`` options you can provide on the
337command line:
338
339.. note::
340
341 If you do provide some "illegal" option combination or perhaps you do
342 not provide enough in the way of options,
343 runqemu
344 provides appropriate error messaging to help you correct the problem.
345
346- QEMUARCH: The QEMU machine architecture, which must be "qemuarm",
347 "qemuarm64", "qemumips", "qemumips64", "qemuppc", "qemux86", or
348 "qemux86-64".
349
350- ``VM``: The virtual machine image, which must be a ``.wic.vmdk``
351 file. Use this option when you want to boot a ``.wic.vmdk`` image.
352 The image filename you provide must contain one of the following
353 strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64",
354 "qemumips", "qemuppc", or "qemush4".
355
356- ROOTFS: A root filesystem that has one of the following filetype
357 extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If
358 the filename you provide for this option uses “nfs”, it must provide
359 an explicit root filesystem path.
360
361- KERNEL: A kernel image, which is a ``.bin`` file. When you provide a
362 ``.bin`` file, ``runqemu`` detects it and assumes the file is a
363 kernel image.
364
365- MACHINE: The architecture of the QEMU machine, which must be one of
366 the following: "qemux86", "qemux86-64", "qemuarm", "qemuarm64",
367 "qemumips", “qemumips64", or "qemuppc". The MACHINE and QEMUARCH
368 options are basically identical. If you do not provide a MACHINE
369 option, ``runqemu`` tries to determine it based on other options.
370
371- ``ramfs``: Indicates you are booting an initial RAM disk (initramfs)
372 image, which means the ``FSTYPE`` is ``cpio.gz``.
373
374- ``iso``: Indicates you are booting an ISO image, which means the
375 ``FSTYPE`` is ``.iso``.
376
377- ``nographic``: Disables the video console, which sets the console to
378 "ttys0". This option is useful when you have logged into a server and
379 you do not want to disable forwarding from the X Window System (X11)
380 to your workstation or laptop.
381
382- ``serial``: Enables a serial console on ``/dev/ttyS0``.
383
384- ``biosdir``: Establishes a custom directory for BIOS, VGA BIOS and
385 keymaps.
386
387- ``biosfilename``: Establishes a custom BIOS name.
388
389- ``qemuparams=\"xyz\"``: Specifies custom QEMU parameters. Use this
390 option to pass options other than the simple "kvm" and "serial"
391 options.
392
393- ``bootparams=\"xyz\"``: Specifies custom boot parameters for the
394 kernel.
395
396- ``audio``: Enables audio in QEMU. The MACHINE option must be either
397 "qemux86" or "qemux86-64" in order for audio to be enabled.
398 Additionally, the ``snd_intel8x0`` or ``snd_ens1370`` driver must be
399 installed in linux guest.
400
401- ``slirp``: Enables "slirp" networking, which is a different way of
402 networking that does not need root access but also is not as easy to
403 use or comprehensive as the default.
404
405- ``kvm``: Enables KVM when running "qemux86" or "qemux86-64" QEMU
406 architectures. For KVM to work, all the following conditions must be
407 met:
408
409 - Your MACHINE must be either qemux86" or "qemux86-64".
410
411 - Your build host has to have the KVM modules installed, which are
412 ``/dev/kvm``.
413
414 - The build host ``/dev/kvm`` directory has to be both writable and
415 readable.
416
417- ``kvm-vhost``: Enables KVM with VHOST support when running "qemux86"
418 or "qemux86-64" QEMU architectures. For KVM with VHOST to work, the
419 following conditions must be met:
420
421 - `kvm <#kvm-cond>`__ option conditions must be met.
422
423 - Your build host has to have virtio net device, which are
424 ``/dev/vhost-net``.
425
426 - The build host ``/dev/vhost-net`` directory has to be either
427 readable or writable and “slirp-enabled”.
428
429- ``publicvnc``: Enables a VNC server open to all hosts.
diff --git a/documentation/dev-manual/dev-manual-start.rst b/documentation/dev-manual/dev-manual-start.rst
new file mode 100644
index 0000000000..9ddd29c664
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-start.rst
@@ -0,0 +1,873 @@
1***********************************
2Setting Up to Use the Yocto Project
3***********************************
4
5This chapter provides guidance on how to prepare to use the Yocto
6Project. You can learn about creating a team environment that develops
7using the Yocto Project, how to set up a `build
8host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__, how to locate
9Yocto Project source repositories, and how to create local Git
10repositories.
11
12.. _usingpoky-changes-collaborate:
13
14Creating a Team Development Environment
15=======================================
16
17It might not be immediately clear how you can use the Yocto Project in a
18team development environment, or how to scale it for a large team of
19developers. You can adapt the Yocto Project to many different use cases
20and scenarios; however, this flexibility could cause difficulties if you
21are trying to create a working setup that scales effectively.
22
23To help you understand how to set up this type of environment, this
24section presents a procedure that gives you information that can help
25you get the results you want. The procedure is high-level and presents
26some of the project's most successful experiences, practices, solutions,
27and available technologies that have proved to work well in the past;
28however, keep in mind, the procedure here is simply a starting point.
29You can build off these steps and customize the procedure to fit any
30particular working environment and set of practices.
31
321. *Determine Who is Going to be Developing:* You first need to
33 understand who is going to be doing anything related to the Yocto
34 Project and determine their roles. Making this determination is
35 essential to completing subsequent steps, which are to get your
36 equipment together and set up your development environment's
37 hardware topology.
38
39 The following roles exist:
40
41 - *Application Developer:* This type of developer does application
42 level work on top of an existing software stack.
43
44 - *Core System Developer:* This type of developer works on the
45 contents of the operating system image itself.
46
47 - *Build Engineer:* This type of developer manages Autobuilders and
48 releases. Depending on the specifics of the environment, not all
49 situations might need a Build Engineer.
50
51 - *Test Engineer:* This type of developer creates and manages
52 automated tests that are used to ensure all application and core
53 system development meets desired quality standards.
54
552. *Gather the Hardware:* Based on the size and make-up of the team,
56 get the hardware together. Ideally, any development, build, or test
57 engineer uses a system that runs a supported Linux distribution.
58 These systems, in general, should be high performance (e.g. dual,
59 six-core Xeons with 24 Gbytes of RAM and plenty of disk space). You
60 can help ensure efficiency by having any machines used for testing
61 or that run Autobuilders be as high performance as possible.
62
63 .. note::
64
65 Given sufficient processing power, you might also consider
66 building Yocto Project development containers to be run under
67 Docker, which is described later.
68
693. *Understand the Hardware Topology of the Environment:* Once you
70 understand the hardware involved and the make-up of the team, you
71 can understand the hardware topology of the development environment.
72 You can get a visual idea of the machines and their roles across the
73 development environment.
74
754. *Use Git as Your Source Control Manager (SCM):* Keeping your
76 `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ (i.e. recipes,
77 configuration files, classes, and so forth) and any software you are
78 developing under the control of an SCM system that is compatible
79 with the OpenEmbedded build system is advisable. Of all of the SCMs
80 supported by BitBake, the Yocto Project team strongly recommends
81 using `Git <&YOCTO_DOCS_OM_URL;#git>`__. Git is a distributed system
82 that is easy to back up, allows you to work remotely, and then
83 connects back to the infrastructure.
84
85 .. note::
86
87 For information about BitBake, see the
88 BitBake User Manual
89 .
90
91 It is relatively easy to set up Git services and create
92 infrastructure like
93 `http://git.yoctoproject.org <&YOCTO_GIT_URL;>`__, which is based on
94 server software called ``gitolite`` with ``cgit`` being used to
95 generate the web interface that lets you view the repositories. The
96 ``gitolite`` software identifies users using SSH keys and allows
97 branch-based access controls to repositories that you can control as
98 little or as much as necessary.
99
100 .. note::
101
102 The setup of these services is beyond the scope of this manual.
103 However, sites such as the following exist that describe how to
104 perform setup:
105
106 - `Git documentation <http://git-scm.com/book/ch4-8.html>`__:
107 Describes how to install ``gitolite`` on the server.
108
109 - `Gitolite <http://gitolite.com>`__: Information for
110 ``gitolite``.
111
112 - `Interfaces, frontends, and
113 tools <https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools>`__:
114 Documentation on how to create interfaces and frontends for
115 Git.
116
1175. *Set up the Application Development Machines:* As mentioned earlier,
118 application developers are creating applications on top of existing
119 software stacks. Following are some best practices for setting up
120 machines used for application development:
121
122 - Use a pre-built toolchain that contains the software stack
123 itself. Then, develop the application code on top of the stack.
124 This method works well for small numbers of relatively isolated
125 applications.
126
127 - Keep your cross-development toolchains updated. You can do this
128 through provisioning either as new toolchain downloads or as
129 updates through a package update mechanism using ``opkg`` to
130 provide updates to an existing toolchain. The exact mechanics of
131 how and when to do this depend on local policy.
132
133 - Use multiple toolchains installed locally into different
134 locations to allow development across versions.
135
1366. *Set up the Core Development Machines:* As mentioned earlier, core
137 developers work on the contents of the operating system itself.
138 Following are some best practices for setting up machines used for
139 developing images:
140
141 - Have the `OpenEmbedded build
142 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ available on
143 the developer workstations so developers can run their own builds
144 and directly rebuild the software stack.
145
146 - Keep the core system unchanged as much as possible and do your
147 work in layers on top of the core system. Doing so gives you a
148 greater level of portability when upgrading to new versions of
149 the core system or Board Support Packages (BSPs).
150
151 - Share layers amongst the developers of a particular project and
152 contain the policy configuration that defines the project.
153
1547. *Set up an Autobuilder:* Autobuilders are often the core of the
155 development environment. It is here that changes from individual
156 developers are brought together and centrally tested. Based on this
157 automated build and test environment, subsequent decisions about
158 releases can be made. Autobuilders also allow for "continuous
159 integration" style testing of software components and regression
160 identification and tracking.
161
162 See "`Yocto Project
163 Autobuilder <http://autobuilder.yoctoproject.org>`__" for more
164 information and links to buildbot. The Yocto Project team has found
165 this implementation works well in this role. A public example of
166 this is the Yocto Project Autobuilders, which the Yocto Project team
167 uses to test the overall health of the project.
168
169 The features of this system are:
170
171 - Highlights when commits break the build.
172
173 - Populates an `sstate
174 cache <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__ from which
175 developers can pull rather than requiring local builds.
176
177 - Allows commit hook triggers, which trigger builds when commits
178 are made.
179
180 - Allows triggering of automated image booting and testing under
181 the QuickEMUlator (QEMU).
182
183 - Supports incremental build testing and from-scratch builds.
184
185 - Shares output that allows developer testing and historical
186 regression investigation.
187
188 - Creates output that can be used for releases.
189
190 - Allows scheduling of builds so that resources can be used
191 efficiently.
192
1938. *Set up Test Machines:* Use a small number of shared, high
194 performance systems for testing purposes. Developers can use these
195 systems for wider, more extensive testing while they continue to
196 develop locally using their primary development system.
197
1989. *Document Policies and Change Flow:* The Yocto Project uses a
199 hierarchical structure and a pull model. Scripts exist to create and
200 send pull requests (i.e. ``create-pull-request`` and
201 ``send-pull-request``). This model is in line with other open source
202 projects where maintainers are responsible for specific areas of the
203 project and a single maintainer handles the final "top-of-tree"
204 merges.
205
206 .. note::
207
208 You can also use a more collective push model. The
209 gitolite
210 software supports both the push and pull models quite easily.
211
212 As with any development environment, it is important to document the
213 policy used as well as any main project guidelines so they are
214 understood by everyone. It is also a good idea to have
215 well-structured commit messages, which are usually a part of a
216 project's guidelines. Good commit messages are essential when
217 looking back in time and trying to understand why changes were made.
218
219 If you discover that changes are needed to the core layer of the
220 project, it is worth sharing those with the community as soon as
221 possible. Chances are if you have discovered the need for changes,
222 someone else in the community needs them also.
223
22410. *Development Environment Summary:* Aside from the previous steps,
225 some best practices exist within the Yocto Project development
226 environment. Consider the following:
227
228 - Use `Git <&YOCTO_DOCS_OM_URL;#git>`__ as the source control
229 system.
230
231 - Maintain your Metadata in layers that make sense for your
232 situation. See the "`The Yocto Project Layer
233 Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__"
234 section in the Yocto Project Overview and Concepts Manual and the
235 "`Understanding and Creating
236 Layers <#understanding-and-creating-layers>`__" section for more
237 information on layers.
238
239 - Separate the project's Metadata and code by using separate Git
240 repositories. See the "`Yocto Project Source
241 Repositories <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__"
242 section in the Yocto Project Overview and Concepts Manual for
243 information on these repositories. See the "`Locating Yocto
244 Project Source Files <#locating-yocto-project-source-files>`__"
245 section for information on how to set up local Git repositories
246 for related upstream Yocto Project Git repositories.
247
248 - Set up the directory for the shared state cache
249 (```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__) where
250 it makes sense. For example, set up the sstate cache on a system
251 used by developers in the same organization and share the same
252 source directories on their machines.
253
254 - Set up an Autobuilder and have it populate the sstate cache and
255 source directories.
256
257 - The Yocto Project community encourages you to send patches to the
258 project to fix bugs or add features. If you do submit patches,
259 follow the project commit guidelines for writing good commit
260 messages. See the "`Submitting a Change to the Yocto
261 Project <#how-to-submit-a-change>`__" section.
262
263 - Send changes to the core sooner than later as others are likely
264 to run into the same issues. For some guidance on mailing lists
265 to use, see the list in the "`Submitting a Change to the Yocto
266 Project <#how-to-submit-a-change>`__" section. For a description
267 of the available mailing lists, see the "`Mailing
268 Lists <&YOCTO_DOCS_REF_URL;#resources-mailinglist>`__" section in
269 the Yocto Project Reference Manual.
270
271.. _dev-preparing-the-build-host:
272
273Preparing the Build Host
274========================
275
276This section provides procedures to set up a system to be used as your
277`build host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ for
278development using the Yocto Project. Your build host can be a native
279Linux machine (recommended), it can be a machine (Linux, Mac, or
280Windows) that uses `CROPS <https://github.com/crops/poky-container>`__,
281which leverages `Docker Containers <https://www.docker.com/>`__ or it
282can be a Windows machine capable of running Windows Subsystem For Linux
283v2 (WSL).
284
285.. note::
286
287 The Yocto Project is not compatible with
288 Windows Subsystem for Linux v1
289 . It is compatible but not officially supported nor validated with
290 WSLv2. If you still decide to use WSL please upgrade to
291 WSLv2
292 .
293
294Once your build host is set up to use the Yocto Project, further steps
295are necessary depending on what you want to accomplish. See the
296following references for information on how to prepare for Board Support
297Package (BSP) development and kernel development:
298
299- *BSP Development:* See the "`Preparing Your Build Host to Work With
300 BSP
301 Layers <&YOCTO_DOCS_BSP_URL;#preparing-your-build-host-to-work-with-bsp-layers>`__"
302 section in the Yocto Project Board Support Package (BSP) Developer's
303 Guide.
304
305- *Kernel Development:* See the "`Preparing the Build Host to Work on
306 the
307 Kernel <&YOCTO_DOCS_KERNEL_DEV_URL;#preparing-the-build-host-to-work-on-the-kernel>`__"
308 section in the Yocto Project Linux Kernel Development Manual.
309
310Setting Up a Native Linux Host
311------------------------------
312
313Follow these steps to prepare a native Linux machine as your Yocto
314Project Build Host:
315
3161. *Use a Supported Linux Distribution:* You should have a reasonably
317 current Linux-based host system. You will have the best results with
318 a recent release of Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS
319 as these releases are frequently tested against the Yocto Project and
320 officially supported. For a list of the distributions under
321 validation and their status, see the "`Supported Linux
322 Distributions <&YOCTO_DOCS_REF_URL;#detailed-supported-distros>`__"
323 section in the Yocto Project Reference Manual and the wiki page at
324 `Distribution
325 Support <&YOCTO_WIKI_URL;/wiki/Distribution_Support>`__.
326
3272. *Have Enough Free Memory:* Your system should have at least 50 Gbytes
328 of free disk space for building images.
329
3303. *Meet Minimal Version Requirements:* The OpenEmbedded build system
331 should be able to run on any modern distribution that has the
332 following versions for Git, tar, Python and gcc.
333
334 - Git 1.8.3.1 or greater
335
336 - tar 1.28 or greater
337
338 - Python 3.5.0 or greater.
339
340 - gcc 5.0 or greater.
341
342 If your build host does not meet any of these three listed version
343 requirements, you can take steps to prepare the system so that you
344 can still use the Yocto Project. See the "`Required Git, tar, Python
345 and gcc
346 Versions <&YOCTO_DOCS_REF_URL;#required-git-tar-python-and-gcc-versions>`__"
347 section in the Yocto Project Reference Manual for information.
348
3494. *Install Development Host Packages:* Required development host
350 packages vary depending on your build host and what you want to do
351 with the Yocto Project. Collectively, the number of required packages
352 is large if you want to be able to cover all cases.
353
354 For lists of required packages for all scenarios, see the "`Required
355 Packages for the Build
356 Host <&YOCTO_DOCS_REF_URL;#required-packages-for-the-build-host>`__"
357 section in the Yocto Project Reference Manual.
358
359Once you have completed the previous steps, you are ready to continue
360using a given development path on your native Linux machine. If you are
361going to use BitBake, see the "`Cloning the ``poky``
362Repository <#cloning-the-poky-repository>`__" section. If you are going
363to use the Extensible SDK, see the "`Using the Extensible
364SDK <&YOCTO_DOCS_SDK_URL;#sdk-extensible>`__" Chapter in the Yocto
365Project Application Development and the Extensible Software Development
366Kit (eSDK) manual. If you want to work on the kernel, see the `Yocto
367Project Linux Kernel Development
368Manual <&YOCTO_DOCS_KERNEL_DEV_URL;>`__. If you are going to use
369Toaster, see the "`Setting Up and Using
370Toaster <&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use>`__"
371section in the Toaster User Manual.
372
373.. _setting-up-to-use-crops:
374
375Setting Up to Use CROss PlatformS (CROPS)
376-----------------------------------------
377
378With `CROPS <https://github.com/crops/poky-container>`__, which
379leverages `Docker Containers <https://www.docker.com/>`__, you can
380create a Yocto Project development environment that is operating system
381agnostic. You can set up a container in which you can develop using the
382Yocto Project on a Windows, Mac, or Linux machine.
383
384Follow these general steps to prepare a Windows, Mac, or Linux machine
385as your Yocto Project build host:
386
3871. *Determine What Your Build Host Needs:*
388 `Docker <https://www.docker.com/what-docker>`__ is a software
389 container platform that you need to install on the build host.
390 Depending on your build host, you might have to install different
391 software to support Docker containers. Go to the Docker installation
392 page and read about the platform requirements in "`Supported
393 Platforms <https://docs.docker.com/install/#supported-platforms>`__"
394 your build host needs to run containers.
395
3962. *Choose What To Install:* Depending on whether or not your build host
397 meets system requirements, you need to install "Docker CE Stable" or
398 the "Docker Toolbox". Most situations call for Docker CE. However, if
399 you have a build host that does not meet requirements (e.g.
400 Pre-Windows 10 or Windows 10 "Home" version), you must install Docker
401 Toolbox instead.
402
4033. *Go to the Install Site for Your Platform:* Click the link for the
404 Docker edition associated with your build host's native software. For
405 example, if your build host is running Microsoft Windows Version 10
406 and you want the Docker CE Stable edition, click that link under
407 "Supported Platforms".
408
4094. *Install the Software:* Once you have understood all the
410 pre-requisites, you can download and install the appropriate
411 software. Follow the instructions for your specific machine and the
412 type of the software you need to install:
413
414 - Install `Docker CE for
415 Windows <https://docs.docker.com/docker-for-windows/install/#install-docker-for-windows-desktop-app>`__
416 for Windows build hosts that meet requirements.
417
418 - Install `Docker CE for
419 Macs <https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-for-mac>`__
420 for Mac build hosts that meet requirements.
421
422 - Install `Docker Toolbox for
423 Windows <https://docs.docker.com/toolbox/toolbox_install_windows/>`__
424 for Windows build hosts that do not meet Docker requirements.
425
426 - Install `Docker Toolbox for
427 MacOS <https://docs.docker.com/toolbox/toolbox_install_mac/>`__
428 for Mac build hosts that do not meet Docker requirements.
429
430 - Install `Docker CE for
431 CentOS <https://docs.docker.com/install/linux/docker-ce/centos/>`__
432 for Linux build hosts running the CentOS distribution.
433
434 - Install `Docker CE for
435 Debian <https://docs.docker.com/install/linux/docker-ce/debian/>`__
436 for Linux build hosts running the Debian distribution.
437
438 - Install `Docker CE for
439 Fedora <https://docs.docker.com/install/linux/docker-ce/fedora/>`__
440 for Linux build hosts running the Fedora distribution.
441
442 - Install `Docker CE for
443 Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/>`__
444 for Linux build hosts running the Ubuntu distribution.
445
4465. *Optionally Orient Yourself With Docker:* If you are unfamiliar with
447 Docker and the container concept, you can learn more here -
448 ` <https://docs.docker.com/get-started/>`__.
449
4506. *Launch Docker or Docker Toolbox:* You should be able to launch
451 Docker or the Docker Toolbox and have a terminal shell on your
452 development host.
453
4547. *Set Up the Containers to Use the Yocto Project:* Go to
455 ` <https://github.com/crops/docker-win-mac-docs/wiki>`__ and follow
456 the directions for your particular build host (i.e. Linux, Mac, or
457 Windows).
458
459 Once you complete the setup instructions for your machine, you have
460 the Poky, Extensible SDK, and Toaster containers available. You can
461 click those links from the page and learn more about using each of
462 those containers.
463
464Once you have a container set up, everything is in place to develop just
465as if you were running on a native Linux machine. If you are going to
466use the Poky container, see the "`Cloning the ``poky``
467Repository <#cloning-the-poky-repository>`__" section. If you are going
468to use the Extensible SDK container, see the "`Using the Extensible
469SDK <&YOCTO_DOCS_SDK_URL;#sdk-extensible>`__" Chapter in the Yocto
470Project Application Development and the Extensible Software Development
471Kit (eSDK) manual. If you are going to use the Toaster container, see
472the "`Setting Up and Using
473Toaster <&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use>`__"
474section in the Toaster User Manual.
475
476.. _setting-up-to-use-wsl:
477
478Setting Up to Use Windows Subsystem For Linux (WSLv2)
479-----------------------------------------------------
480
481With `Windows Subsystem for Linux
482(WSLv2) <https://docs.microsoft.com/en-us/windows/wsl/wsl2-about>`__,
483you can create a Yocto Project development environment that allows you
484to build on Windows. You can set up a Linux distribution inside Windows
485in which you can develop using the Yocto Project.
486
487Follow these general steps to prepare a Windows machine using WSLv2 as
488your Yocto Project build host:
489
4901. *Make sure your Windows 10 machine is capable of running WSLv2:*
491 WSLv2 is only available for Windows 10 builds > 18917. To check which
492 build version you are running, you may open a command prompt on
493 Windows and execute the command "ver". C:\Users\myuser> ver Microsoft
494 Windows [Version 10.0.19041.153] If your build is capable of running
495 WSLv2 you may continue, for more information on this subject or
496 instructions on how to upgrade to WSLv2 visit `Windows 10
497 WSLv2 <https://docs.microsoft.com/en-us/windows/wsl/wsl2-install>`__
498
4992. *Install the Linux distribution of your choice inside Windows 10:*
500 Once you know your version of Windows 10 supports WSLv2, you can
501 install the distribution of your choice from the Microsoft Store.
502 Open the Microsoft Store and search for Linux. While there are
503 several Linux distributions available, the assumption is that your
504 pick will be one of the distributions supported by the Yocto Project
505 as stated on the instructions for using a native Linux host. After
506 making your selection, simply click "Get" to download and install the
507 distribution.
508
5093. *Check your Linux distribution is using WSLv2:* Open a Windows
510 PowerShell and run: C:\WINDOWS\system32> wsl -l -v NAME STATE VERSION
511 \*Ubuntu Running 2 Note the version column which says the WSL version
512 being used by your distribution, on compatible systems, this can be
513 changed back at any point in time.
514
5154. *Optionally Orient Yourself on WSL:* If you are unfamiliar with WSL,
516 you can learn more here -
517 ` <https://docs.microsoft.com/en-us/windows/wsl/wsl2-about>`__.
518
5195. *Launch your WSL Distibution:* From the Windows start menu simply
520 launch your WSL distribution just like any other application.
521
5226. *Optimize your WSLv2 storage often:* Due to the way storage is
523 handled on WSLv2, the storage space used by the undelying Linux
524 distribution is not reflected immedately, and since bitbake heavily
525 uses storage, after several builds, you may be unaware you are
526 running out of space. WSLv2 uses a VHDX file for storage, this issue
527 can be easily avoided by manually optimizing this file often, this
528 can be done in the following way:
529
530 1. *Find the location of your VHDX file:* First you need to find the
531 distro app package directory, to achieve this open a Windows
532 Powershell as Administrator and run: C:\WINDOWS\system32>
533 Get-AppxPackage -Name "*Ubuntu*" \| Select PackageFamilyName
534 PackageFamilyName -----------------
535 CanonicalGroupLimited.UbuntuonWindows_79abcdefgh You should now
536 replace the PackageFamilyName and your user on the following path
537 to find your VHDX file:
538 ``C:\Users\user\AppData\Local\Packages\PackageFamilyName\LocalState\``
539 For example: ls
540 C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\\
541 Mode LastWriteTime Length Name -a---- 3/14/2020 9:52 PM
542 57418973184 ext4.vhdx Your VHDX file path is:
543 ``C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx``
544
545 2. *Optimize your VHDX file:* Open a Windows Powershell as
546 Administrator to optimize your VHDX file, shutting down WSL first:
547 C:\WINDOWS\system32> wsl --shutdown C:\WINDOWS\system32>
548 optimize-vhd -Path
549 C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx
550 -Mode full A progress bar should be shown while optimizing the
551 VHDX file, and storage should now be reflected correctly on the
552 Windows Explorer.
553
554.. note::
555
556 The current implementation of WSLv2 does not have out-of-the-box
557 access to external devices such as those connected through a USB
558 port, but it automatically mounts your
559 C:
560 drive on
561 /mnt/c/
562 (and others), which you can use to share deploy artifacts to be later
563 flashed on hardware through Windows, but your build directory should
564 not reside inside this mountpoint.
565
566Once you have WSLv2 set up, everything is in place to develop just as if
567you were running on a native Linux machine. If you are going to use the
568Extensible SDK container, see the "`Using the Extensible
569SDK <&YOCTO_DOCS_SDK_URL;#sdk-extensible>`__" Chapter in the Yocto
570Project Application Development and the Extensible Software Development
571Kit (eSDK) manual. If you are going to use the Toaster container, see
572the "`Setting Up and Using
573Toaster <&YOCTO_DOCS_TOAST_URL;#toaster-manual-setup-and-use>`__"
574section in the Toaster User Manual.
575
576Locating Yocto Project Source Files
577===================================
578
579This section shows you how to locate, fetch and configure the source
580files you'll need to work with the Yocto Project.
581
582.. note::
583
584 - For concepts and introductory information about Git as it is used
585 in the Yocto Project, see the "`Git <&YOCTO_DOCS_OM_URL;#git>`__"
586 section in the Yocto Project Overview and Concepts Manual.
587
588 - For concepts on Yocto Project source repositories, see the "`Yocto
589 Project Source
590 Repositories <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__"
591 section in the Yocto Project Overview and Concepts Manual."
592
593Accessing Source Repositories
594-----------------------------
595
596Working from a copy of the upstream Yocto Project `Source
597Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ is the
598preferred method for obtaining and using a Yocto Project release. You
599can view the Yocto Project Source Repositories at
600` <&YOCTO_GIT_URL;>`__. In particular, you can find the ``poky``
601repository at ` <http://git.yoctoproject.org/cgit/cgit.cgi/poky/>`__.
602
603Use the following procedure to locate the latest upstream copy of the
604``poky`` Git repository:
605
6061. *Access Repositories:* Open a browser and go to
607 ` <&YOCTO_GIT_URL;>`__ to access the GUI-based interface into the
608 Yocto Project source repositories.
609
6102. *Select the Repository:* Click on the repository in which you are
611 interested (e.g. ``poky``).
612
6133. *Find the URL Used to Clone the Repository:* At the bottom of the
614 page, note the URL used to
615 `clone <&YOCTO_DOCS_OM_URL;#git-commands-clone>`__ that repository
616 (e.g. ``YOCTO_GIT_URL/poky``).
617
618 .. note::
619
620 For information on cloning a repository, see the "
621 Cloning the
622 poky
623 Repository
624 " section.
625
626Accessing Index of Releases
627---------------------------
628
629Yocto Project maintains an Index of Releases area that contains related
630files that contribute to the Yocto Project. Rather than Git
631repositories, these files are tarballs that represent snapshots in time
632of a given component.
633
634.. note::
635
636 The recommended method for accessing Yocto Project components is to
637 use Git to clone the upstream repository and work from within that
638 locally cloned repository. The procedure in this section exists
639 should you desire a tarball snapshot of any given component.
640
641Follow these steps to locate and download a particular tarball:
642
6431. *Access the Index of Releases:* Open a browser and go to
644 ` <&YOCTO_DL_URL;/releases>`__ to access the Index of Releases. The
645 list represents released components (e.g. ``bitbake``, ``sato``, and
646 so on).
647
648 .. note::
649
650 The
651 yocto
652 directory contains the full array of released Poky tarballs. The
653 poky
654 directory in the Index of Releases was historically used for very
655 early releases and exists now only for retroactive completeness.
656
6572. *Select a Component:* Click on any released component in which you
658 are interested (e.g. ``yocto``).
659
6603. *Find the Tarball:* Drill down to find the associated tarball. For
661 example, click on ``yocto-DISTRO`` to view files associated with the
662 Yocto Project DISTRO release (e.g.
663 ``poky-DISTRO_NAME_NO_CAP-POKYVERSION.tar.bz2``, which is the
664 released Poky tarball).
665
6664. *Download the Tarball:* Click the tarball to download and save a
667 snapshot of the given component.
668
669Using the Downloads Page
670------------------------
671
672The `Yocto Project Website <&YOCTO_HOME_URL;>`__ uses a "DOWNLOADS" page
673from which you can locate and download tarballs of any Yocto Project
674release. Rather than Git repositories, these files represent snapshot
675tarballs similar to the tarballs located in the Index of Releases
676described in the "`Accessing Index of
677Releases <#accessing-index-of-releases>`__" section.
678
679.. note::
680
681 The recommended method for accessing Yocto Project components is to
682 use Git to clone a repository and work from within that local
683 repository. The procedure in this section exists should you desire a
684 tarball snapshot of any given component.
685
6861. *Go to the Yocto Project Website:* Open The `Yocto Project
687 Website <&YOCTO_HOME_URL;>`__ in your browser.
688
6892. *Get to the Downloads Area:* Select the "DOWNLOADS" item from the
690 pull-down "SOFTWARE" tab menu near the top of the page.
691
6923. *Select a Yocto Project Release:* Use the menu next to "RELEASE" to
693 display and choose a recent or past supported Yocto Project release
694 (e.g. DISTRO_NAME_NO_CAP, DISTRO_NAME_NO_CAP_MINUS_ONE, and so
695 forth).
696
697 .. note::
698
699 For a "map" of Yocto Project releases to version numbers, see the
700 Releases
701 wiki page.
702
703 You can use the "RELEASE ARCHIVE" link to reveal a menu of all Yocto
704 Project releases.
705
7064. *Download Tools or Board Support Packages (BSPs):* From the
707 "DOWNLOADS" page, you can download tools or BSPs as well. Just scroll
708 down the page and look for what you need.
709
710Accessing Nightly Builds
711------------------------
712
713Yocto Project maintains an area for nightly builds that contains tarball
714releases at ` <&YOCTO_AB_NIGHTLY_URL;>`__. These builds include Yocto
715Project releases ("poky"), toolchains, and builds for supported
716machines.
717
718Should you ever want to access a nightly build of a particular Yocto
719Project component, use the following procedure:
720
7211. *Locate the Index of Nightly Builds:* Open a browser and go to
722 ` <&YOCTO_AB_NIGHTLY_URL;>`__ to access the Nightly Builds.
723
7242. *Select a Date:* Click on the date in which you are interested. If
725 you want the latest builds, use "CURRENT".
726
7273. *Select a Build:* Choose the area in which you are interested. For
728 example, if you are looking for the most recent toolchains, select
729 the "toolchain" link.
730
7314. *Find the Tarball:* Drill down to find the associated tarball.
732
7335. *Download the Tarball:* Click the tarball to download and save a
734 snapshot of the given component.
735
736Cloning and Checking Out Branches
737=================================
738
739To use the Yocto Project for development, you need a release locally
740installed on your development system. This locally installed set of
741files is referred to as the `Source
742Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ in the Yocto
743Project documentation.
744
745The preferred method of creating your Source Directory is by using
746`Git <&YOCTO_DOCS_OM_URL;#git>`__ to clone a local copy of the upstream
747``poky`` repository. Working from a cloned copy of the upstream
748repository allows you to contribute back into the Yocto Project or to
749simply work with the latest software on a development branch. Because
750Git maintains and creates an upstream repository with a complete history
751of changes and you are working with a local clone of that repository,
752you have access to all the Yocto Project development branches and tag
753names used in the upstream repository.
754
755Cloning the ``poky`` Repository
756-------------------------------
757
758Follow these steps to create a local version of the upstream
759```poky`` <&YOCTO_DOCS_REF_URL;#poky>`__ Git repository.
760
7611. *Set Your Directory:* Change your working directory to where you want
762 to create your local copy of ``poky``.
763
7642. *Clone the Repository:* The following example command clones the
765 ``poky`` repository and uses the default name "poky" for your local
766 repository: $ git clone git://git.yoctoproject.org/poky Cloning into
767 'poky'... remote: Counting objects: 432160, done. remote: Compressing
768 objects: 100% (102056/102056), done. remote: Total 432160 (delta
769 323116), reused 432037 (delta 323000) Receiving objects: 100%
770 (432160/432160), 153.81 MiB \| 8.54 MiB/s, done. Resolving deltas:
771 100% (323116/323116), done. Checking connectivity... done. Unless you
772 specify a specific development branch or tag name, Git clones the
773 "master" branch, which results in a snapshot of the latest
774 development changes for "master". For information on how to check out
775 a specific development branch or on how to check out a local branch
776 based on a tag name, see the "`Checking Out By Branch in
777 Poky <#checking-out-by-branch-in-poky>`__" and `Checking Out By Tag
778 in Poky <#checkout-out-by-tag-in-poky>`__" sections, respectively.
779
780 Once the local repository is created, you can change to that
781 directory and check its status. Here, the single "master" branch
782 exists on your system and by default, it is checked out: $ cd ~/poky
783 $ git status On branch master Your branch is up-to-date with
784 'origin/master'. nothing to commit, working directory clean $ git
785 branch \* master Your local repository of poky is identical to the
786 upstream poky repository at the time from which it was cloned. As you
787 work with the local branch, you can periodically use the
788 ``git pull DASHDASHrebase`` command to be sure you are up-to-date
789 with the upstream branch.
790
791Checking Out by Branch in Poky
792------------------------------
793
794When you clone the upstream poky repository, you have access to all its
795development branches. Each development branch in a repository is unique
796as it forks off the "master" branch. To see and use the files of a
797particular development branch locally, you need to know the branch name
798and then specifically check out that development branch.
799
800.. note::
801
802 Checking out an active development branch by branch name gives you a
803 snapshot of that particular branch at the time you check it out.
804 Further development on top of the branch that occurs after check it
805 out can occur.
806
8071. *Switch to the Poky Directory:* If you have a local poky Git
808 repository, switch to that directory. If you do not have the local
809 copy of poky, see the "`Cloning the ``poky``
810 Repository <#cloning-the-poky-repository>`__" section.
811
8122. *Determine Existing Branch Names:* $ git branch -a \* master
813 remotes/origin/1.1_M1 remotes/origin/1.1_M2 remotes/origin/1.1_M3
814 remotes/origin/1.1_M4 remotes/origin/1.2_M1 remotes/origin/1.2_M2
815 remotes/origin/1.2_M3 . . . remotes/origin/thud
816 remotes/origin/thud-next remotes/origin/warrior
817 remotes/origin/warrior-next remotes/origin/zeus
818 remotes/origin/zeus-next ... and so on ...
819
8203. *Check out the Branch:* Check out the development branch in which you
821 want to work. For example, to access the files for the Yocto Project
822 DISTRO Release (DISTRO_NAME), use the following command: $ git
823 checkout -b DISTRO_NAME_NO_CAP origin/DISTRO_NAME_NO_CAP Branch
824 DISTRO_NAME_NO_CAP set up to track remote branch DISTRO_NAME_NO_CAP
825 from origin. Switched to a new branch 'DISTRO_NAME_NO_CAP' The
826 previous command checks out the "DISTRO_NAME_NO_CAP" development
827 branch and reports that the branch is tracking the upstream
828 "origin/DISTRO_NAME_NO_CAP" branch.
829
830 The following command displays the branches that are now part of your
831 local poky repository. The asterisk character indicates the branch
832 that is currently checked out for work: $ git branch master \*
833 DISTRO_NAME_NO_CAP
834
835.. _checkout-out-by-tag-in-poky:
836
837Checking Out by Tag in Poky
838---------------------------
839
840Similar to branches, the upstream repository uses tags to mark specific
841commits associated with significant points in a development branch (i.e.
842a release point or stage of a release). You might want to set up a local
843branch based on one of those points in the repository. The process is
844similar to checking out by branch name except you use tag names.
845
846.. note::
847
848 Checking out a branch based on a tag gives you a stable set of files
849 not affected by development on the branch above the tag.
850
8511. *Switch to the Poky Directory:* If you have a local poky Git
852 repository, switch to that directory. If you do not have the local
853 copy of poky, see the "`Cloning the ``poky``
854 Repository <#cloning-the-poky-repository>`__" section.
855
8562. *Fetch the Tag Names:* To checkout the branch based on a tag name,
857 you need to fetch the upstream tags into your local repository: $ git
858 fetch --tags $
859
8603. *List the Tag Names:* You can list the tag names now: $ git tag
861 1.1_M1.final 1.1_M1.rc1 1.1_M1.rc2 1.1_M2.final 1.1_M2.rc1 . . .
862 yocto-2.5 yocto-2.5.1 yocto-2.5.2 yocto-2.5.3 yocto-2.6 yocto-2.6.1
863 yocto-2.6.2 yocto-2.7 yocto_1.5_M5.rc8
864
8654. *Check out the Branch:* $ git checkout tags/DISTRO_REL_TAG -b
866 my_yocto_DISTRO Switched to a new branch 'my_yocto_DISTRO' $ git
867 branch master \* my_yocto_DISTRO The previous command creates and
868 checks out a local branch named "my_yocto_DISTRO", which is based on
869 the commit in the upstream poky repository that has the same tag. In
870 this example, the files you have available locally as a result of the
871 ``checkout`` command are a snapshot of the "DISTRO_NAME_NO_CAP"
872 development branch at the point where Yocto Project DISTRO was
873 released.
diff --git a/documentation/dev-manual/dev-manual.rst b/documentation/dev-manual/dev-manual.rst
new file mode 100644
index 0000000000..f447dd205e
--- /dev/null
+++ b/documentation/dev-manual/dev-manual.rst
@@ -0,0 +1,12 @@
1======================================
2Yocto Project Development Tasks Manual
3======================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 dev-manual-intro
10 dev-manual-start
11 dev-manual-common-tasks
12 dev-manual-qemu
diff --git a/documentation/index.rst b/documentation/index.rst
index 1470d2d5de..42686dadb4 100644
--- a/documentation/index.rst
+++ b/documentation/index.rst
@@ -9,3 +9,14 @@ Welcome to The Yocto Project's documentation!
9.. toctree:: 9.. toctree::
10 :maxdepth: 1 10 :maxdepth: 1
11 11
12 brief-yoctoprojectqs/brief-yoctoprojectqs
13 overview-manual/overview-manual
14 bsp-guide/bsp-guide
15 ref-manual/ref-manual
16 dev-manual/dev-manual
17 adt-manual/adt-manual
18 kernel-dev/kernel-dev
19 profile-manual/profile-manual
20 sdk-manual/sdk-manual
21 toaster-manual/toaster-manual
22 test-manual/test-manual
diff --git a/documentation/kernel-dev/kernel-dev-advanced.rst b/documentation/kernel-dev/kernel-dev-advanced.rst
new file mode 100644
index 0000000000..be76bd7b52
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-advanced.rst
@@ -0,0 +1,762 @@
1*******************************************************
2Working with Advanced Metadata (``yocto-kernel-cache``)
3*******************************************************
4
5.. _kernel-dev-advanced-overview:
6
7Overview
8========
9
10In addition to supporting configuration fragments and patches, the Yocto
11Project kernel tools also support rich
12`Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ that you can use to define
13complex policies and Board Support Package (BSP) support. The purpose of
14the Metadata and the tools that manage it is to help you manage the
15complexity of the configuration and sources used to support multiple
16BSPs and Linux kernel types.
17
18Kernel Metadata exists in many places. One area in the Yocto Project
19`Source Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ is the
20``yocto-kernel-cache`` Git repository. You can find this repository
21grouped under the "Yocto Linux Kernel" heading in the `Yocto Project
22Source Repositories <&YOCTO_GIT_URL;>`__.
23
24Kernel development tools ("kern-tools") exist also in the Yocto Project
25Source Repositories under the "Yocto Linux Kernel" heading in the
26``yocto-kernel-tools`` Git repository. The recipe that builds these
27tools is ``meta/recipes-kernel/kern-tools/kern-tools-native_git.bb`` in
28the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
29``poky``).
30
31Using Kernel Metadata in a Recipe
32=================================
33
34As mentioned in the introduction, the Yocto Project contains kernel
35Metadata, which is located in the ``yocto-kernel-cache`` Git repository.
36This Metadata defines Board Support Packages (BSPs) that correspond to
37definitions in linux-yocto recipes for corresponding BSPs. A BSP
38consists of an aggregation of kernel policy and enabled
39hardware-specific features. The BSP can be influenced from within the
40linux-yocto recipe.
41
42.. note::
43
44 A Linux kernel recipe that contains kernel Metadata (e.g. inherits
45 from the
46 linux-yocto.inc
47 file) is said to be a "linux-yocto style" recipe.
48
49Every linux-yocto style recipe must define the
50```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__ variable. This
51variable is typically set to the same value as the ``MACHINE`` variable,
52which is used by `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__.
53However, in some cases, the variable might instead refer to the
54underlying platform of the ``MACHINE``.
55
56Multiple BSPs can reuse the same ``KMACHINE`` name if they are built
57using the same BSP description. Multiple Corei7-based BSPs could share
58the same "intel-corei7-64" value for ``KMACHINE``. It is important to
59realize that ``KMACHINE`` is just for kernel mapping, while ``MACHINE``
60is the machine type within a BSP Layer. Even with this distinction,
61however, these two variables can hold the same value. See the `BSP
62Descriptions <#bsp-descriptions>`__ section for more information.
63
64Every linux-yocto style recipe must also indicate the Linux kernel
65source repository branch used to build the Linux kernel. The
66```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable must be set
67to indicate the branch.
68
69.. note::
70
71 You can use the
72 KBRANCH
73 value to define an alternate branch typically with a machine override
74 as shown here from the
75 meta-yocto-bsp
76 layer:
77 ::
78
79 KBRANCH_edgerouter = "standard/edgerouter"
80
81
82The linux-yocto style recipes can optionally define the following
83variables: KERNEL_FEATURES LINUX_KERNEL_TYPE
84
85```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
86defines the kernel type to be used in assembling the configuration. If
87you do not specify a ``LINUX_KERNEL_TYPE``, it defaults to "standard".
88Together with ``KMACHINE``, ``LINUX_KERNEL_TYPE`` defines the search
89arguments used by the kernel tools to find the appropriate description
90within the kernel Metadata with which to build out the sources and
91configuration. The linux-yocto recipes define "standard", "tiny", and
92"preempt-rt" kernel types. See the "`Kernel Types <#kernel-types>`__"
93section for more information on kernel types.
94
95During the build, the kern-tools search for the BSP description file
96that most closely matches the ``KMACHINE`` and ``LINUX_KERNEL_TYPE``
97variables passed in from the recipe. The tools use the first BSP
98description it finds that match both variables. If the tools cannot find
99a match, they issue a warning.
100
101The tools first search for the ``KMACHINE`` and then for the
102``LINUX_KERNEL_TYPE``. If the tools cannot find a partial match, they
103will use the sources from the ``KBRANCH`` and any configuration
104specified in the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__.
105
106You can use the
107```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
108variable to include features (configuration fragments, patches, or both)
109that are not already included by the ``KMACHINE`` and
110``LINUX_KERNEL_TYPE`` variable combination. For example, to include a
111feature specified as "features/netfilter/netfilter.scc", specify:
112KERNEL_FEATURES += "features/netfilter/netfilter.scc" To include a
113feature called "cfg/sound.scc" just for the ``qemux86`` machine,
114specify: KERNEL_FEATURES_append_qemux86 = " cfg/sound.scc" The value of
115the entries in ``KERNEL_FEATURES`` are dependent on their location
116within the kernel Metadata itself. The examples here are taken from the
117``yocto-kernel-cache`` repository. Each branch of this repository
118contains "features" and "cfg" subdirectories at the top-level. For more
119information, see the "`Kernel Metadata
120Syntax <#kernel-metadata-syntax>`__" section.
121
122Kernel Metadata Syntax
123======================
124
125The kernel Metadata consists of three primary types of files: ``scc``
126 [1]_ description files, configuration fragments, and patches. The
127``scc`` files define variables and include or otherwise reference any of
128the three file types. The description files are used to aggregate all
129types of kernel Metadata into what ultimately describes the sources and
130the configuration required to build a Linux kernel tailored to a
131specific machine.
132
133The ``scc`` description files are used to define two fundamental types
134of kernel Metadata:
135
136- Features
137
138- Board Support Packages (BSPs)
139
140Features aggregate sources in the form of patches and configuration
141fragments into a modular reusable unit. You can use features to
142implement conceptually separate kernel Metadata descriptions such as
143pure configuration fragments, simple patches, complex features, and
144kernel types. `Kernel types <#kernel-types>`__ define general kernel
145features and policy to be reused in the BSPs.
146
147BSPs define hardware-specific features and aggregate them with kernel
148types to form the final description of what will be assembled and built.
149
150While the kernel Metadata syntax does not enforce any logical separation
151of configuration fragments, patches, features or kernel types, best
152practices dictate a logical separation of these types of Metadata. The
153following Metadata file hierarchy is recommended: base/ bsp/ cfg/
154features/ ktypes/ patches/
155
156The ``bsp`` directory contains the `BSP
157descriptions <#bsp-descriptions>`__. The remaining directories all
158contain "features". Separating ``bsp`` from the rest of the structure
159aids conceptualizing intended usage.
160
161Use these guidelines to help place your ``scc`` description files within
162the structure:
163
164- If your file contains only configuration fragments, place the file in
165 the ``cfg`` directory.
166
167- If your file contains only source-code fixes, place the file in the
168 ``patches`` directory.
169
170- If your file encapsulates a major feature, often combining sources
171 and configurations, place the file in ``features`` directory.
172
173- If your file aggregates non-hardware configuration and patches in
174 order to define a base kernel policy or major kernel type to be
175 reused across multiple BSPs, place the file in ``ktypes`` directory.
176
177These distinctions can easily become blurred - especially as out-of-tree
178features slowly merge upstream over time. Also, remember that how the
179description files are placed is a purely logical organization and has no
180impact on the functionality of the kernel Metadata. There is no impact
181because all of ``cfg``, ``features``, ``patches``, and ``ktypes``,
182contain "features" as far as the kernel tools are concerned.
183
184Paths used in kernel Metadata files are relative to base, which is
185either
186```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__ if
187you are creating Metadata in `recipe-space <#recipe-space-metadata>`__,
188or the top level of
189```yocto-kernel-cache`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/>`__
190if you are creating `Metadata outside of the
191recipe-space <#metadata-outside-the-recipe-space>`__.
192
193Configuration
194-------------
195
196The simplest unit of kernel Metadata is the configuration-only feature.
197This feature consists of one or more Linux kernel configuration
198parameters in a configuration fragment file (``.cfg``) and a ``.scc``
199file that describes the fragment.
200
201As an example, consider the Symmetric Multi-Processing (SMP) fragment
202used with the ``linux-yocto-4.12`` kernel as defined outside of the
203recipe space (i.e. ``yocto-kernel-cache``). This Metadata consists of
204two files: ``smp.scc`` and ``smp.cfg``. You can find these files in the
205``cfg`` directory of the ``yocto-4.12`` branch in the
206``yocto-kernel-cache`` Git repository: cfg/smp.scc: define
207KFEATURE_DESCRIPTION "Enable SMP for 32 bit builds" define
208KFEATURE_COMPATIBILITY all kconf hardware smp.cfg cfg/smp.cfg:
209CONFIG_SMP=y CONFIG_SCHED_SMT=y # Increase default NR_CPUS from 8 to 64
210so that platform with # more than 8 processors can be all activated at
211boot time CONFIG_NR_CPUS=64 # The following is needed when setting
212NR_CPUS to something # greater than 8 on x86 architectures, it should be
213automatically # disregarded by Kconfig when using a different arch
214CONFIG_X86_BIGSMP=y You can find general information on configuration
215fragment files in the "`Creating Configuration
216Fragments <#creating-config-fragments>`__" section.
217
218Within the ``smp.scc`` file, the
219```KFEATURE_DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-KFEATURE_DESCRIPTION>`__
220statement provides a short description of the fragment. Higher level
221kernel tools use this description.
222
223Also within the ``smp.scc`` file, the ``kconf`` command includes the
224actual configuration fragment in an ``.scc`` file, and the "hardware"
225keyword identifies the fragment as being hardware enabling, as opposed
226to general policy, which would use the "non-hardware" keyword. The
227distinction is made for the benefit of the configuration validation
228tools, which warn you if a hardware fragment overrides a policy set by a
229non-hardware fragment.
230
231.. note::
232
233 The description file can include multiple
234 kconf
235 statements, one per fragment.
236
237As described in the "`Validating
238Configuration <#validating-configuration>`__" section, you can use the
239following BitBake command to audit your configuration: $ bitbake
240linux-yocto -c kernel_configcheck -f
241
242Patches
243-------
244
245Patch descriptions are very similar to configuration fragment
246descriptions, which are described in the previous section. However,
247instead of a ``.cfg`` file, these descriptions work with source patches
248(i.e. ``.patch`` files).
249
250A typical patch includes a description file and the patch itself. As an
251example, consider the build patches used with the ``linux-yocto-4.12``
252kernel as defined outside of the recipe space (i.e.
253``yocto-kernel-cache``). This Metadata consists of several files:
254``build.scc`` and a set of ``*.patch`` files. You can find these files
255in the ``patches/build`` directory of the ``yocto-4.12`` branch in the
256``yocto-kernel-cache`` Git repository.
257
258The following listings show the ``build.scc`` file and part of the
259``modpost-mask-trivial-warnings.patch`` file: patches/build/build.scc:
260patch arm-serialize-build-targets.patch patch
261powerpc-serialize-image-targets.patch patch
262kbuild-exclude-meta-directory-from-distclean-processi.patch # applied by
263kgit # patch kbuild-add-meta-files-to-the-ignore-li.patch patch
264modpost-mask-trivial-warnings.patch patch
265menuconfig-check-lxdiaglog.sh-Allow-specification-of.patch
266patches/build/modpost-mask-trivial-warnings.patch: From
267bd48931bc142bdd104668f3a062a1f22600aae61 Mon Sep 17 00:00:00 2001 From:
268Paul Gortmaker <paul.gortmaker@windriver.com> Date: Sun, 25 Jan 2009
26917:58:09 -0500 Subject: [PATCH] modpost: mask trivial warnings Newer
270HOSTCC will complain about various stdio fcns because . . . char
271\*dump_write = NULL, \*files_source = NULL; int opt; -- 2.10.1 generated
272by cgit v0.10.2 at 2017-09-28 15:23:23 (GMT) The description file can
273include multiple patch statements where each statement handles a single
274patch. In the example ``build.scc`` file, five patch statements exist
275for the five patches in the directory.
276
277You can create a typical ``.patch`` file using ``diff -Nurp`` or
278``git format-patch`` commands. For information on how to create patches,
279see the "`Using ``devtool`` to Patch the
280Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
281Kernel Development to Patch the
282Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
283sections.
284
285Features
286--------
287
288Features are complex kernel Metadata types that consist of configuration
289fragments, patches, and possibly other feature description files. As an
290example, consider the following generic listing: features/myfeature.scc
291define KFEATURE_DESCRIPTION "Enable myfeature" patch
2920001-myfeature-core.patch patch 0002-myfeature-interface.patch include
293cfg/myfeature_dependency.scc kconf non-hardware myfeature.cfg This
294example shows how the ``patch`` and ``kconf`` commands are used as well
295as how an additional feature description file is included with the
296``include`` command.
297
298Typically, features are less granular than configuration fragments and
299are more likely than configuration fragments and patches to be the types
300of things you want to specify in the ``KERNEL_FEATURES`` variable of the
301Linux kernel recipe. See the "`Using Kernel Metadata in a
302Recipe <#using-kernel-metadata-in-a-recipe>`__" section earlier in the
303manual.
304
305Kernel Types
306------------
307
308A kernel type defines a high-level kernel policy by aggregating
309non-hardware configuration fragments with patches you want to use when
310building a Linux kernel of a specific type (e.g. a real-time kernel).
311Syntactically, kernel types are no different than features as described
312in the "`Features <#features>`__" section. The
313```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
314variable in the kernel recipe selects the kernel type. For example, in
315the ``linux-yocto_4.12.bb`` kernel recipe found in
316``poky/meta/recipes-kernel/linux``, a
317```require`` <&YOCTO_DOCS_BB_URL;#require-inclusion>`__ directive
318includes the ``poky/meta/recipes-kernel/linux/linux-yocto.inc`` file,
319which has the following statement that defines the default kernel type:
320LINUX_KERNEL_TYPE ??= "standard"
321
322Another example would be the real-time kernel (i.e.
323``linux-yocto-rt_4.12.bb``). This kernel recipe directly sets the kernel
324type as follows: LINUX_KERNEL_TYPE = "preempt-rt"
325
326.. note::
327
328 You can find kernel recipes in the
329 meta/recipes-kernel/linux
330 directory of the
331 Source Directory
332 (e.g.
333 poky/meta/recipes-kernel/linux/linux-yocto_4.12.bb
334 ). See the "
335 Using Kernel Metadata in a Recipe
336 " section for more information.
337
338Three kernel types ("standard", "tiny", and "preempt-rt") are supported
339for Linux Yocto kernels:
340
341- "standard": Includes the generic Linux kernel policy of the Yocto
342 Project linux-yocto kernel recipes. This policy includes, among other
343 things, which file systems, networking options, core kernel features,
344 and debugging and tracing options are supported.
345
346- "preempt-rt": Applies the ``PREEMPT_RT`` patches and the
347 configuration options required to build a real-time Linux kernel.
348 This kernel type inherits from the "standard" kernel type.
349
350- "tiny": Defines a bare minimum configuration meant to serve as a base
351 for very small Linux kernels. The "tiny" kernel type is independent
352 from the "standard" configuration. Although the "tiny" kernel type
353 does not currently include any source changes, it might in the
354 future.
355
356For any given kernel type, the Metadata is defined by the ``.scc`` (e.g.
357``standard.scc``). Here is a partial listing for the ``standard.scc``
358file, which is found in the ``ktypes/standard`` directory of the
359``yocto-kernel-cache`` Git repository: # Include this kernel type
360fragment to get the standard features and # configuration values. #
361Note: if only the features are desired, but not the configuration # then
362this should be included as: # include ktypes/standard/standard.scc nocfg
363# if no chained configuration is desired, include it as: # include
364ktypes/standard/standard.scc nocfg inherit include ktypes/base/base.scc
365branch standard kconf non-hardware standard.cfg include
366features/kgdb/kgdb.scc . . . include cfg/net/ip6_nf.scc include
367cfg/net/bridge.scc include cfg/systemd.scc include
368features/rfkill/rfkill.scc
369
370As with any ``.scc`` file, a kernel type definition can aggregate other
371``.scc`` files with ``include`` commands. These definitions can also
372directly pull in configuration fragments and patches with the ``kconf``
373and ``patch`` commands, respectively.
374
375.. note::
376
377 It is not strictly necessary to create a kernel type
378 .scc
379 file. The Board Support Package (BSP) file can implicitly define the
380 kernel type using a
381 define
382 KTYPE
383 myktype
384 line. See the "
385 BSP Descriptions
386 " section for more information.
387
388BSP Descriptions
389----------------
390
391BSP descriptions (i.e. ``*.scc`` files) combine kernel types with
392hardware-specific features. The hardware-specific Metadata is typically
393defined independently in the BSP layer, and then aggregated with each
394supported kernel type.
395
396.. note::
397
398 For BSPs supported by the Yocto Project, the BSP description files
399 are located in the
400 bsp
401 directory of the
402 yocto-kernel-cache
403 repository organized under the "Yocto Linux Kernel" heading in the
404 Yocto Project Source Repositories
405 .
406
407This section overviews the BSP description structure, the aggregation
408concepts, and presents a detailed example using a BSP supported by the
409Yocto Project (i.e. BeagleBone Board). For complete information on BSP
410layer file hierarchy, see the `Yocto Project Board Support Package (BSP)
411Developer's Guide <&YOCTO_DOCS_BSP_URL;>`__.
412
413.. _bsp-description-file-overview:
414
415Overview
416~~~~~~~~
417
418For simplicity, consider the following root BSP layer description files
419for the BeagleBone board. These files employ both a structure and naming
420convention for consistency. The naming convention for the file is as
421follows: bsp_root_name-kernel_type.scc Here are some example root layer
422BSP filenames for the BeagleBone Board BSP, which is supported by the
423Yocto Project: beaglebone-standard.scc beaglebone-preempt-rt.scc Each
424file uses the root name (i.e "beaglebone") BSP name followed by the
425kernel type.
426
427Examine the ``beaglebone-standard.scc`` file: define KMACHINE beaglebone
428define KTYPE standard define KARCH arm include
429ktypes/standard/standard.scc branch beaglebone include beaglebone.scc #
430default policy for standard kernels include
431features/latencytop/latencytop.scc include
432features/profiling/profiling.scc Every top-level BSP description file
433should define the ```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
434```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__, and
435```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__ variables. These
436variables allow the OpenEmbedded build system to identify the
437description as meeting the criteria set by the recipe being built. This
438example supports the "beaglebone" machine for the "standard" kernel and
439the "arm" architecture.
440
441Be aware that a hard link between the ``KTYPE`` variable and a kernel
442type description file does not exist. Thus, if you do not have the
443kernel type defined in your kernel Metadata as it is here, you only need
444to ensure that the
445```LINUX_KERNEL_TYPE`` <&YOCTO_DOCS_REF_URL;#var-LINUX_KERNEL_TYPE>`__
446variable in the kernel recipe and the ``KTYPE`` variable in the BSP
447description file match.
448
449To separate your kernel policy from your hardware configuration, you
450include a kernel type (``ktype``), such as "standard". In the previous
451example, this is done using the following: include
452ktypes/standard/standard.scc This file aggregates all the configuration
453fragments, patches, and features that make up your standard kernel
454policy. See the "`Kernel Types <#kernel-types>`__" section for more
455information.
456
457To aggregate common configurations and features specific to the kernel
458for mybsp, use the following: include mybsp.scc You can see that in the
459BeagleBone example with the following: include beaglebone.scc For
460information on how to break a complete ``.config`` file into the various
461configuration fragments, see the "`Creating Configuration
462Fragments <#creating-config-fragments>`__" section.
463
464Finally, if you have any configurations specific to the hardware that
465are not in a ``*.scc`` file, you can include them as follows: kconf
466hardware mybsp-extra.cfg The BeagleBone example does not include these
467types of configurations. However, the Malta 32-bit board does
468("mti-malta32"). Here is the ``mti-malta32-le-standard.scc`` file:
469define KMACHINE mti-malta32-le define KMACHINE qemumipsel define KTYPE
470standard define KARCH mips include ktypes/standard/standard.scc branch
471mti-malta32 include mti-malta32.scc kconf hardware mti-malta32-le.cfg
472
473.. _bsp-description-file-example-minnow:
474
475Example
476~~~~~~~
477
478Many real-world examples are more complex. Like any other ``.scc`` file,
479BSP descriptions can aggregate features. Consider the Minnow BSP
480definition given the ``linux-yocto-4.4`` branch of the
481``yocto-kernel-cache`` (i.e.
482``yocto-kernel-cache/bsp/minnow/minnow.scc``):
483
484.. note::
485
486 Although the Minnow Board BSP is unused, the Metadata remains and is
487 being used here just as an example.
488
489include cfg/x86.scc include features/eg20t/eg20t.scc include
490cfg/dmaengine.scc include features/power/intel.scc include cfg/efi.scc
491include features/usb/ehci-hcd.scc include features/usb/ohci-hcd.scc
492include features/usb/usb-gadgets.scc include
493features/usb/touchscreen-composite.scc include cfg/timer/hpet.scc
494include features/leds/leds.scc include features/spi/spidev.scc include
495features/i2c/i2cdev.scc include features/mei/mei-txe.scc # Earlyprintk
496and port debug requires 8250 kconf hardware cfg/8250.cfg kconf hardware
497minnow.cfg kconf hardware minnow-dev.cfg
498
499The ``minnow.scc`` description file includes a hardware configuration
500fragment (``minnow.cfg``) specific to the Minnow BSP as well as several
501more general configuration fragments and features enabling hardware
502found on the machine. This ``minnow.scc`` description file is then
503included in each of the three "minnow" description files for the
504supported kernel types (i.e. "standard", "preempt-rt", and "tiny").
505Consider the "minnow" description for the "standard" kernel type (i.e.
506``minnow-standard.scc``: define KMACHINE minnow define KTYPE standard
507define KARCH i386 include ktypes/standard include minnow.scc # Extra
508minnow configs above the minimal defined in minnow.scc include
509cfg/efi-ext.scc include features/media/media-all.scc include
510features/sound/snd_hda_intel.scc # The following should really be in
511standard.scc # USB live-image support include cfg/usb-mass-storage.scc
512include cfg/boot-live.scc # Basic profiling include
513features/latencytop/latencytop.scc include
514features/profiling/profiling.scc # Requested drivers that don't have an
515existing scc kconf hardware minnow-drivers-extra.cfg The ``include``
516command midway through the file includes the ``minnow.scc`` description
517that defines all enabled hardware for the BSP that is common to all
518kernel types. Using this command significantly reduces duplication.
519
520Now consider the "minnow" description for the "tiny" kernel type (i.e.
521``minnow-tiny.scc``): define KMACHINE minnow define KTYPE tiny define
522KARCH i386 include ktypes/tiny include minnow.scc As you might expect,
523the "tiny" description includes quite a bit less. In fact, it includes
524only the minimal policy defined by the "tiny" kernel type and the
525hardware-specific configuration required for booting the machine along
526with the most basic functionality of the system as defined in the base
527"minnow" description file.
528
529Notice again the three critical variables:
530```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
531```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__, and
532```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__. Of these variables, only
533``KTYPE`` has changed to specify the "tiny" kernel type.
534
535Kernel Metadata Location
536========================
537
538Kernel Metadata always exists outside of the kernel tree either defined
539in a kernel recipe (recipe-space) or outside of the recipe. Where you
540choose to define the Metadata depends on what you want to do and how you
541intend to work. Regardless of where you define the kernel Metadata, the
542syntax used applies equally.
543
544If you are unfamiliar with the Linux kernel and only wish to apply a
545configuration and possibly a couple of patches provided to you by
546others, the recipe-space method is recommended. This method is also a
547good approach if you are working with Linux kernel sources you do not
548control or if you just do not want to maintain a Linux kernel Git
549repository on your own. For partial information on how you can define
550kernel Metadata in the recipe-space, see the "`Modifying an Existing
551Recipe <#modifying-an-existing-recipe>`__" section.
552
553Conversely, if you are actively developing a kernel and are already
554maintaining a Linux kernel Git repository of your own, you might find it
555more convenient to work with kernel Metadata kept outside the
556recipe-space. Working with Metadata in this area can make iterative
557development of the Linux kernel more efficient outside of the BitBake
558environment.
559
560Recipe-Space Metadata
561---------------------
562
563When stored in recipe-space, the kernel Metadata files reside in a
564directory hierarchy below
565```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__. For
566a linux-yocto recipe or for a Linux kernel recipe derived by copying and
567modifying
568``oe-core/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb`` to
569a recipe in your layer, ``FILESEXTRAPATHS`` is typically set to
570``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``.
571See the "`Modifying an Existing
572Recipe <#modifying-an-existing-recipe>`__" section for more information.
573
574Here is an example that shows a trivial tree of kernel Metadata stored
575in recipe-space within a BSP layer: meta-my_bsp_layer/ \`--
576recipes-kernel \`-- linux \`-- linux-yocto \|-- bsp-standard.scc \|--
577bsp.cfg \`-- standard.cfg
578
579When the Metadata is stored in recipe-space, you must take steps to
580ensure BitBake has the necessary information to decide what files to
581fetch and when they need to be fetched again. It is only necessary to
582specify the ``.scc`` files on the
583```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__. BitBake parses them
584and fetches any files referenced in the ``.scc`` files by the
585``include``, ``patch``, or ``kconf`` commands. Because of this, it is
586necessary to bump the recipe ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__
587value when changing the content of files not explicitly listed in the
588``SRC_URI``.
589
590If the BSP description is in recipe space, you cannot simply list the
591``*.scc`` in the ``SRC_URI`` statement. You need to use the following
592form from your kernel append file: SRC_URI_append_myplatform = " \\
593file://myplatform;type=kmeta;destsuffix=myplatform \\ "
594
595Metadata Outside the Recipe-Space
596---------------------------------
597
598When stored outside of the recipe-space, the kernel Metadata files
599reside in a separate repository. The OpenEmbedded build system adds the
600Metadata to the build as a "type=kmeta" repository through the
601```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable. As an
602example, consider the following ``SRC_URI`` statement from the
603``linux-yocto_4.12.bb`` kernel recipe: SRC_URI =
604"git://git.yoctoproject.org/linux-yocto-4.12.git;name=machine;branch=${KBRANCH};
605\\
606git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
607``${KMETA}``, in this context, is simply used to name the directory into
608which the Git fetcher places the Metadata. This behavior is no different
609than any multi-repository ``SRC_URI`` statement used in a recipe (e.g.
610see the previous section).
611
612You can keep kernel Metadata in a "kernel-cache", which is a directory
613containing configuration fragments. As with any Metadata kept outside
614the recipe-space, you simply need to use the ``SRC_URI`` statement with
615the "type=kmeta" attribute. Doing so makes the kernel Metadata available
616during the configuration phase.
617
618If you modify the Metadata, you must not forget to update the ``SRCREV``
619statements in the kernel's recipe. In particular, you need to update the
620``SRCREV_meta`` variable to match the commit in the ``KMETA`` branch you
621wish to use. Changing the data in these branches and not updating the
622``SRCREV`` statements to match will cause the build to fetch an older
623commit.
624
625Organizing Your Source
626======================
627
628Many recipes based on the ``linux-yocto-custom.bb`` recipe use Linux
629kernel sources that have only a single branch - "master". This type of
630repository structure is fine for linear development supporting a single
631machine and architecture. However, if you work with multiple boards and
632architectures, a kernel source repository with multiple branches is more
633efficient. For example, suppose you need a series of patches for one
634board to boot. Sometimes, these patches are works-in-progress or
635fundamentally wrong, yet they are still necessary for specific boards.
636In these situations, you most likely do not want to include these
637patches in every kernel you build (i.e. have the patches as part of the
638lone "master" branch). It is situations like these that give rise to
639multiple branches used within a Linux kernel sources Git repository.
640
641Repository organization strategies exist that maximize source reuse,
642remove redundancy, and logically order your changes. This section
643presents strategies for the following cases:
644
645- Encapsulating patches in a feature description and only including the
646 patches in the BSP descriptions of the applicable boards.
647
648- Creating a machine branch in your kernel source repository and
649 applying the patches on that branch only.
650
651- Creating a feature branch in your kernel source repository and
652 merging that branch into your BSP when needed.
653
654The approach you take is entirely up to you and depends on what works
655best for your development model.
656
657Encapsulating Patches
658---------------------
659
660if you are reusing patches from an external tree and are not working on
661the patches, you might find the encapsulated feature to be appropriate.
662Given this scenario, you do not need to create any branches in the
663source repository. Rather, you just take the static patches you need and
664encapsulate them within a feature description. Once you have the feature
665description, you simply include that into the BSP description as
666described in the "`BSP Descriptions <#bsp-descriptions>`__" section.
667
668You can find information on how to create patches and BSP descriptions
669in the "`Patches <#patches>`__" and "`BSP
670Descriptions <#bsp-descriptions>`__" sections.
671
672Machine Branches
673----------------
674
675When you have multiple machines and architectures to support, or you are
676actively working on board support, it is more efficient to create
677branches in the repository based on individual machines. Having machine
678branches allows common source to remain in the "master" branch with any
679features specific to a machine stored in the appropriate machine branch.
680This organization method frees you from continually reintegrating your
681patches into a feature.
682
683Once you have a new branch, you can set up your kernel Metadata to use
684the branch a couple different ways. In the recipe, you can specify the
685new branch as the ``KBRANCH`` to use for the board as follows: KBRANCH =
686"mynewbranch" Another method is to use the ``branch`` command in the BSP
687description: mybsp.scc: define KMACHINE mybsp define KTYPE standard
688define KARCH i386 include standard.scc branch mynewbranch include
689mybsp-hw.scc
690
691If you find yourself with numerous branches, you might consider using a
692hierarchical branching system similar to what the Yocto Linux Kernel Git
693repositories use: common/kernel_type/machine
694
695If you had two kernel types, "standard" and "small" for instance, three
696machines, and common as ``mydir``, the branches in your Git repository
697might look like this: mydir/base mydir/standard/base
698mydir/standard/machine_a mydir/standard/machine_b
699mydir/standard/machine_c mydir/small/base mydir/small/machine_a
700
701This organization can help clarify the branch relationships. In this
702case, ``mydir/standard/machine_a`` includes everything in ``mydir/base``
703and ``mydir/standard/base``. The "standard" and "small" branches add
704sources specific to those kernel types that for whatever reason are not
705appropriate for the other branches.
706
707.. note::
708
709 The "base" branches are an artifact of the way Git manages its data
710 internally on the filesystem: Git will not allow you to use
711 mydir/standard
712 and
713 mydir/standard/machine_a
714 because it would have to create a file and a directory named
715 "standard".
716
717Feature Branches
718----------------
719
720When you are actively developing new features, it can be more efficient
721to work with that feature as a branch, rather than as a set of patches
722that have to be regularly updated. The Yocto Project Linux kernel tools
723provide for this with the ``git merge`` command.
724
725To merge a feature branch into a BSP, insert the ``git merge`` command
726after any ``branch`` commands: mybsp.scc: define KMACHINE mybsp define
727KTYPE standard define KARCH i386 include standard.scc branch mynewbranch
728git merge myfeature include mybsp-hw.scc
729
730.. _scc-reference:
731
732SCC Description File Reference
733==============================
734
735This section provides a brief reference for the commands you can use
736within an SCC description file (``.scc``):
737
738- ``branch [ref]``: Creates a new branch relative to the current branch
739 (typically ``${KTYPE}``) using the currently checked-out branch, or
740 "ref" if specified.
741
742- ``define``: Defines variables, such as
743 ```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__,
744 ```KTYPE`` <&YOCTO_DOCS_REF_URL;#var-KTYPE>`__,
745 ```KARCH`` <&YOCTO_DOCS_REF_URL;#var-KARCH>`__, and
746 ```KFEATURE_DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-KFEATURE_DESCRIPTION>`__.
747
748- ``include SCC_FILE``: Includes an SCC file in the current file. The
749 file is parsed as if you had inserted it inline.
750
751- ``kconf [hardware|non-hardware] CFG_FILE``: Queues a configuration
752 fragment for merging into the final Linux ``.config`` file.
753
754- ``git merge GIT_BRANCH``: Merges the feature branch into the current
755 branch.
756
757- ``patch PATCH_FILE``: Applies the patch to the current Git branch.
758
759.. [1]
760 ``scc`` stands for Series Configuration Control, but the naming has
761 less significance in the current implementation of the tooling than
762 it had in the past. Consider ``scc`` files to be description files.
diff --git a/documentation/kernel-dev/kernel-dev-common.rst b/documentation/kernel-dev/kernel-dev-common.rst
new file mode 100644
index 0000000000..bf87edbdc4
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-common.rst
@@ -0,0 +1,1728 @@
1************
2Common Tasks
3************
4
5This chapter presents several common tasks you perform when you work
6with the Yocto Project Linux kernel. These tasks include preparing your
7host development system for kernel development, preparing a layer,
8modifying an existing recipe, patching the kernel, configuring the
9kernel, iterative development, working with your own sources, and
10incorporating out-of-tree modules.
11
12.. note::
13
14 The examples presented in this chapter work with the Yocto Project
15 2.4 Release and forward.
16
17Preparing the Build Host to Work on the Kernel
18==============================================
19
20Before you can do any kernel development, you need to be sure your build
21host is set up to use the Yocto Project. For information on how to get
22set up, see the "`Preparing the Build
23Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section in
24the Yocto Project Development Tasks Manual. Part of preparing the system
25is creating a local Git repository of the `Source
26Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (``poky``) on your
27system. Follow the steps in the "`Cloning the ``poky``
28Repository <&YOCTO_DOCS_DEV_URL;#cloning-the-poky-repository>`__"
29section in the Yocto Project Development Tasks Manual to set up your
30Source Directory.
31
32.. note::
33
34 Be sure you check out the appropriate development branch or you
35 create your local branch by checking out a specific tag to get the
36 desired version of Yocto Project. See the "
37 Checking Out by Branch in Poky
38 " and "
39 Checking Out by Tag in Poky
40 " sections in the Yocto Project Development Tasks Manual for more
41 information.
42
43Kernel development is best accomplished using
44```devtool`` <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__
45and not through traditional kernel workflow methods. The remainder of
46this section provides information for both scenarios.
47
48Getting Ready to Develop Using ``devtool``
49------------------------------------------
50
51Follow these steps to prepare to update the kernel image using
52``devtool``. Completing this procedure leaves you with a clean kernel
53image and ready to make modifications as described in the "`Using
54``devtool`` to Patch the Kernel <#using-devtool-to-patch-the-kernel>`__"
55section:
56
571. *Initialize the BitBake Environment:* Before building an extensible
58 SDK, you need to initialize the BitBake build environment by sourcing
59 the build environment script (i.e.
60 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__):
61 $ cd ~/poky $ source oe-init-build-env
62
63 .. note::
64
65 The previous commands assume the
66 Source Repositories
67 (i.e.
68 poky
69 ) have been cloned using Git and the local repository is named
70 "poky".
71
722. *Prepare Your ``local.conf`` File:* By default, the
73 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable is set to
74 "qemux86-64", which is fine if you are building for the QEMU emulator
75 in 64-bit mode. However, if you are not, you need to set the
76 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
77 found in the `Build
78 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (i.e.
79 ``~/poky/build`` in this example).
80
81 Also, since you are preparing to work on the kernel image, you need
82 to set the
83 ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
84 variable to include kernel modules.
85
86 In this example we wish to build for qemux86 so we must set the
87 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
88 As described we do this by appending to ``conf/local.conf``: MACHINE
89 = "qemux86" MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
90
913. *Create a Layer for Patches:* You need to create a layer to hold
92 patches created for the kernel image. You can use the
93 ``bitbake-layers create-layer`` command as follows: $ cd ~/poky/build
94 $ bitbake-layers create-layer ../../meta-mylayer NOTE: Starting
95 bitbake server... Add your new layer with 'bitbake-layers add-layer
96 ../../meta-mylayer' $
97
98 .. note::
99
100 For background information on working with common and BSP layers,
101 see the "
102 Understanding and Creating Layers
103 " section in the Yocto Project Development Tasks Manual and the "
104 BSP Layers
105 " section in the Yocto Project Board Support (BSP) Developer's
106 Guide, respectively. For information on how to use the
107 bitbake-layers create-layer
108 command to quickly set up a layer, see the "
109 Creating a General Layer Using the
110 bitbake-layers
111 Script
112 " section in the Yocto Project Development Tasks Manual.
113
1144. *Inform the BitBake Build Environment About Your Layer:* As directed
115 when you created your layer, you need to add the layer to the
116 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable in the
117 ``bblayers.conf`` file as follows: $ cd ~/poky/build $ bitbake-layers
118 add-layer ../../meta-mylayer NOTE: Starting bitbake server... $
119
1205. *Build the Extensible SDK:* Use BitBake to build the extensible SDK
121 specifically for use with images to be run using QEMU: $ cd
122 ~/poky/build $ bitbake core-image-minimal -c populate_sdk_ext Once
123 the build finishes, you can find the SDK installer file (i.e.
124 ``*.sh`` file) in the following directory:
125 ~/poky/build/tmp/deploy/sdk For this example, the installer file is
126 named
127 ``poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-DISTRO.sh``
128
1296. *Install the Extensible SDK:* Use the following command to install
130 the SDK. For this example, install the SDK in the default
131 ``~/poky_sdk`` directory: $ cd ~/poky/build/tmp/deploy/sdk $
132 ./poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-DISTRO.sh
133 Poky (Yocto Project Reference Distro) Extensible SDK installer
134 version DISTRO
135 ============================================================================
136 Enter target directory for SDK (default: ~/poky_sdk): You are about
137 to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
138 Extracting SDK......................................done Setting it
139 up... Extracting buildtools... Preparing build system... Parsing
140 recipes: 100%
141 \|#################################################################\|
142 Time: 0:00:52 Initializing tasks: 100% \|##############
143 ###############################################\| Time: 0:00:04
144 Checking sstate mirror object availability: 100%
145 \|######################################\| Time: 0:00:00 Parsing
146 recipes: 100%
147 \|#################################################################\|
148 Time: 0:00:33 Initializing tasks: 100%
149 \|##############################################################\|
150 Time: 0:00:00 done SDK has been successfully set up and is ready to
151 be used. Each time you wish to use the SDK in a new shell session,
152 you need to source the environment setup script e.g. $ .
153 /home/scottrif/poky_sdk/environment-setup-i586-poky-linux
154
1557. *Set Up a New Terminal to Work With the Extensible SDK:* You must set
156 up a new terminal to work with the SDK. You cannot use the same
157 BitBake shell used to build the installer.
158
159 After opening a new shell, run the SDK environment setup script as
160 directed by the output from installing the SDK: $ source
161 ~/poky_sdk/environment-setup-i586-poky-linux "SDK environment now set
162 up; additionally you may now run devtool to perform development
163 tasks. Run devtool --help for further details.
164
165 .. note::
166
167 If you get a warning about attempting to use the extensible SDK in
168 an environment set up to run BitBake, you did not use a new shell.
169
1708. *Build the Clean Image:* The final step in preparing to work on the
171 kernel is to build an initial image using ``devtool`` in the new
172 terminal you just set up and initialized for SDK work: $ devtool
173 build-image Parsing recipes: 100%
174 \|##########################################\| Time: 0:00:05 Parsing
175 of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47
176 skipped, 0 masked, 0 errors. WARNING: No packages to add, building
177 image core-image-minimal unmodified Loading cache: 100%
178 \|############################################\| Time: 0:00:00 Loaded
179 1299 entries from dependency cache. NOTE: Resolving any missing task
180 queue dependencies Initializing tasks: 100%
181 \|#######################################\| Time: 0:00:07 Checking
182 sstate mirror object availability: 100% \|###############\| Time:
183 0:00:00 NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks
184 NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need
185 to be rerun and all succeeded. NOTE: Successfully built
186 core-image-minimal. You can find output files in
187 /home/scottrif/poky_sdk/tmp/deploy/images/qemux86 If you were
188 building for actual hardware and not for emulation, you could flash
189 the image to a USB stick on ``/dev/sdd`` and boot your device. For an
190 example that uses a Minnowboard, see the
191 `TipsAndTricks/KernelDevelopmentWithEsdk <https://wiki.yoctoproject.org/wiki/TipsAndTricks/KernelDevelopmentWithEsdk>`__
192 Wiki page.
193
194At this point you have set up to start making modifications to the
195kernel by using the extensible SDK. For a continued example, see the
196"`Using ``devtool`` to Patch the
197Kernel <#using-devtool-to-patch-the-kernel>`__" section.
198
199Getting Ready for Traditional Kernel Development
200------------------------------------------------
201
202Getting ready for traditional kernel development using the Yocto Project
203involves many of the same steps as described in the previous section.
204However, you need to establish a local copy of the kernel source since
205you will be editing these files.
206
207Follow these steps to prepare to update the kernel image using
208traditional kernel development flow with the Yocto Project. Completing
209this procedure leaves you ready to make modifications to the kernel
210source as described in the "`Using Traditional Kernel Development to
211Patch the
212Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
213section:
214
2151. *Initialize the BitBake Environment:* Before you can do anything
216 using BitBake, you need to initialize the BitBake build environment
217 by sourcing the build environment script (i.e.
218 ```oe-init-build-env`` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__).
219 Also, for this example, be sure that the local branch you have
220 checked out for ``poky`` is the Yocto Project DISTRO_NAME branch. If
221 you need to checkout out the DISTRO_NAME branch, see the "`Checking
222 out by Branch in
223 Poky <&YOCTO_DOCS_DEV_URL;#checking-out-by-branch-in-poky>`__"
224 section in the Yocto Project Development Tasks Manual. $ cd ~/poky $
225 git branch master \* DISTRO_NAME $ source oe-init-build-env
226
227 .. note::
228
229 The previous commands assume the
230 Source Repositories
231 (i.e.
232 poky
233 ) have been cloned using Git and the local repository is named
234 "poky".
235
2362. *Prepare Your ``local.conf`` File:* By default, the
237 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable is set to
238 "qemux86-64", which is fine if you are building for the QEMU emulator
239 in 64-bit mode. However, if you are not, you need to set the
240 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
241 found in the `Build
242 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ (i.e.
243 ``~/poky/build`` in this example).
244
245 Also, since you are preparing to work on the kernel image, you need
246 to set the
247 ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
248 variable to include kernel modules.
249
250 In this example we wish to build for qemux86 so we must set the
251 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
252 As described we do this by appending to ``conf/local.conf``: MACHINE
253 = "qemux86" MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
254
2553. *Create a Layer for Patches:* You need to create a layer to hold
256 patches created for the kernel image. You can use the
257 ``bitbake-layers create-layer`` command as follows: $ cd ~/poky/build
258 $ bitbake-layers create-layer ../../meta-mylayer NOTE: Starting
259 bitbake server... Add your new layer with 'bitbake-layers add-layer
260 ../../meta-mylayer'
261
262 .. note::
263
264 For background information on working with common and BSP layers,
265 see the "
266 Understanding and Creating Layers
267 " section in the Yocto Project Development Tasks Manual and the "
268 BSP Layers
269 " section in the Yocto Project Board Support (BSP) Developer's
270 Guide, respectively. For information on how to use the
271 bitbake-layers create-layer
272 command to quickly set up a layer, see the "
273 Creating a General Layer Using the
274 bitbake-layers
275 Script
276 " section in the Yocto Project Development Tasks Manual.
277
2784. *Inform the BitBake Build Environment About Your Layer:* As directed
279 when you created your layer, you need to add the layer to the
280 ```BBLAYERS`` <&YOCTO_DOCS_REF_URL;#var-BBLAYERS>`__ variable in the
281 ``bblayers.conf`` file as follows: $ cd ~/poky/build $ bitbake-layers
282 add-layer ../../meta-mylayer NOTE: Starting bitbake server ... $
283
2845. *Create a Local Copy of the Kernel Git Repository:* You can find Git
285 repositories of supported Yocto Project kernels organized under
286 "Yocto Linux Kernel" in the Yocto Project Source Repositories at
287 ` <&YOCTO_GIT_URL;>`__.
288
289 For simplicity, it is recommended that you create your copy of the
290 kernel Git repository outside of the `Source
291 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__, which is
292 usually named ``poky``. Also, be sure you are in the
293 ``standard/base`` branch.
294
295 The following commands show how to create a local copy of the
296 ``linux-yocto-4.12`` kernel and be in the ``standard/base`` branch.
297
298 .. note::
299
300 The
301 linux-yocto-4.12
302 kernel can be used with the Yocto Project 2.4 release and forward.
303 You cannot use the
304 linux-yocto-4.12
305 kernel with releases prior to Yocto Project 2.4:
306
307 $ cd ~ $ git clone git://git.yoctoproject.org/linux-yocto-4.12
308 --branch standard/base Cloning into 'linux-yocto-4.12'... remote:
309 Counting objects: 6097195, done. remote: Compressing objects: 100%
310 (901026/901026), done. remote: Total 6097195 (delta 5152604), reused
311 6096847 (delta 5152256) Receiving objects: 100% (6097195/6097195),
312 1.24 GiB \| 7.81 MiB/s, done. Resolving deltas: 100%
313 (5152604/5152604), done. Checking connectivity... done. Checking out
314 files: 100% (59846/59846), done.
315
3166. *Create a Local Copy of the Kernel Cache Git Repository:* For
317 simplicity, it is recommended that you create your copy of the kernel
318 cache Git repository outside of the `Source
319 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__, which is
320 usually named ``poky``. Also, for this example, be sure you are in
321 the ``yocto-4.12`` branch.
322
323 The following commands show how to create a local copy of the
324 ``yocto-kernel-cache`` and be in the ``yocto-4.12`` branch: $ cd ~ $
325 git clone git://git.yoctoproject.org/yocto-kernel-cache --branch
326 yocto-4.12 Cloning into 'yocto-kernel-cache'... remote: Counting
327 objects: 22639, done. remote: Compressing objects: 100% (9761/9761),
328 done. remote: Total 22639 (delta 12400), reused 22586 (delta 12347)
329 Receiving objects: 100% (22639/22639), 22.34 MiB \| 6.27 MiB/s, done.
330 Resolving deltas: 100% (12400/12400), done. Checking connectivity...
331 done.
332
333At this point, you are ready to start making modifications to the kernel
334using traditional kernel development steps. For a continued example, see
335the "`Using Traditional Kernel Development to Patch the
336Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
337section.
338
339Creating and Preparing a Layer
340==============================
341
342If you are going to be modifying kernel recipes, it is recommended that
343you create and prepare your own layer in which to do your work. Your
344layer contains its own `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__
345append files (``.bbappend``) and provides a convenient mechanism to
346create your own recipe files (``.bb``) as well as store and use kernel
347patch files. For background information on working with layers, see the
348"`Understanding and Creating
349Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
350section in the Yocto Project Development Tasks Manual.
351
352.. note::
353
354 The Yocto Project comes with many tools that simplify tasks you need
355 to perform. One such tool is the
356 bitbake-layers create-layer
357 command, which simplifies creating a new layer. See the "
358 Creating a General Layer Using the
359 bitbake-layers
360 Script
361 " section in the Yocto Project Development Tasks Manual for
362 information on how to use this script to quick set up a new layer.
363
364To better understand the layer you create for kernel development, the
365following section describes how to create a layer without the aid of
366tools. These steps assume creation of a layer named ``mylayer`` in your
367home directory:
368
3691. *Create Structure*: Create the layer's structure: $ cd $HOME $ mkdir
370 meta-mylayer $ mkdir meta-mylayer/conf $ mkdir
371 meta-mylayer/recipes-kernel $ mkdir meta-mylayer/recipes-kernel/linux
372 $ mkdir meta-mylayer/recipes-kernel/linux/linux-yocto The ``conf``
373 directory holds your configuration files, while the
374 ``recipes-kernel`` directory holds your append file and eventual
375 patch files.
376
3772. *Create the Layer Configuration File*: Move to the
378 ``meta-mylayer/conf`` directory and create the ``layer.conf`` file as
379 follows: # We have a conf and classes directory, add to BBPATH BBPATH
380 .= ":${LAYERDIR}" # We have recipes-\* directories, add to BBFILES
381 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \\
382 ${LAYERDIR}/recipes-*/*/*.bbappend" BBFILE_COLLECTIONS += "mylayer"
383 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/" BBFILE_PRIORITY_mylayer =
384 "5" Notice ``mylayer`` as part of the last three statements.
385
3863. *Create the Kernel Recipe Append File*: Move to the
387 ``meta-mylayer/recipes-kernel/linux`` directory and create the
388 kernel's append file. This example uses the ``linux-yocto-4.12``
389 kernel. Thus, the name of the append file is
390 ``linux-yocto_4.12.bbappend``: FILESEXTRAPATHS_prepend :=
391 "${THISDIR}/${PN}:" SRC_URI_append = " file://patch-file-one"
392 SRC_URI_append = " file://patch-file-two" SRC_URI_append = "
393 file://patch-file-three" The
394 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
395 and ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
396 enable the OpenEmbedded build system to find patch files. For more
397 information on using append files, see the "`Using .bbappend Files in
398 Your Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in
399 the Yocto Project Development Tasks Manual.
400
401Modifying an Existing Recipe
402============================
403
404In many cases, you can customize an existing linux-yocto recipe to meet
405the needs of your project. Each release of the Yocto Project provides a
406few Linux kernel recipes from which you can choose. These are located in
407the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ in
408``meta/recipes-kernel/linux``.
409
410Modifying an existing recipe can consist of the following:
411
412- Creating the append file
413
414- Applying patches
415
416- Changing the configuration
417
418Before modifying an existing recipe, be sure that you have created a
419minimal, custom layer from which you can work. See the "`Creating and
420Preparing a Layer <#creating-and-preparing-a-layer>`__" section for
421information.
422
423Creating the Append File
424------------------------
425
426You create this file in your custom layer. You also name it accordingly
427based on the linux-yocto recipe you are using. For example, if you are
428modifying the ``meta/recipes-kernel/linux/linux-yocto_4.12.bb`` recipe,
429the append file will typically be located as follows within your custom
430layer: your-layer/recipes-kernel/linux/linux-yocto_4.12.bbappend The
431append file should initially extend the
432```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ search path by
433prepending the directory that contains your files to the
434```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
435variable as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" The
436path
437``${``\ ```THISDIR`` <&YOCTO_DOCS_REF_URL;#var-THISDIR>`__\ ``}/${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}``
438expands to "linux-yocto" in the current directory for this example. If
439you add any new files that modify the kernel recipe and you have
440extended ``FILESPATH`` as described above, you must place the files in
441your layer in the following area:
442your-layer/recipes-kernel/linux/linux-yocto/
443
444.. note::
445
446 If you are working on a new machine Board Support Package (BSP), be
447 sure to refer to the
448 Yocto Project Board Support Package (BSP) Developer's Guide
449 .
450
451As an example, consider the following append file used by the BSPs in
452``meta-yocto-bsp``:
453meta-yocto-bsp/recipes-kernel/linux/linux-yocto_4.12.bbappend The
454following listing shows the file. Be aware that the actual commit ID
455strings in this example listing might be different than the actual
456strings in the file from the ``meta-yocto-bsp`` layer upstream.
457KBRANCH_genericx86 = "standard/base" KBRANCH_genericx86-64 =
458"standard/base" KMACHINE_genericx86 ?= "common-pc"
459KMACHINE_genericx86-64 ?= "common-pc-64" KBRANCH_edgerouter =
460"standard/edgerouter" KBRANCH_beaglebone = "standard/beaglebone"
461SRCREV_machine_genericx86 ?= "d09f2ce584d60ecb7890550c22a80c48b83c2e19"
462SRCREV_machine_genericx86-64 ?=
463"d09f2ce584d60ecb7890550c22a80c48b83c2e19" SRCREV_machine_edgerouter ?=
464"b5c8cfda2dfe296410d51e131289fb09c69e1e7d" SRCREV_machine_beaglebone ?=
465"b5c8cfda2dfe296410d51e131289fb09c69e1e7d" COMPATIBLE_MACHINE_genericx86
466= "genericx86" COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
467COMPATIBLE_MACHINE_edgerouter = "edgerouter"
468COMPATIBLE_MACHINE_beaglebone = "beaglebone" LINUX_VERSION_genericx86 =
469"4.12.7" LINUX_VERSION_genericx86-64 = "4.12.7" LINUX_VERSION_edgerouter
470= "4.12.10" LINUX_VERSION_beaglebone = "4.12.10" This append file
471contains statements used to support several BSPs that ship with the
472Yocto Project. The file defines machines using the
473```COMPATIBLE_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE>`__
474variable and uses the
475```KMACHINE`` <&YOCTO_DOCS_REF_URL;#var-KMACHINE>`__ variable to ensure
476the machine name used by the OpenEmbedded build system maps to the
477machine name used by the Linux Yocto kernel. The file also uses the
478optional ```KBRANCH`` <&YOCTO_DOCS_REF_URL;#var-KBRANCH>`__ variable to
479ensure the build process uses the appropriate kernel branch.
480
481Although this particular example does not use it, the
482```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
483variable could be used to enable features specific to the kernel. The
484append file points to specific commits in the `Source
485Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ Git repository and
486the ``meta`` Git repository branches to identify the exact kernel needed
487to build the BSP.
488
489One thing missing in this particular BSP, which you will typically need
490when developing a BSP, is the kernel configuration file (``.config``)
491for your BSP. When developing a BSP, you probably have a kernel
492configuration file or a set of kernel configuration files that, when
493taken together, define the kernel configuration for your BSP. You can
494accomplish this definition by putting the configurations in a file or a
495set of files inside a directory located at the same level as your
496kernel's append file and having the same name as the kernel's main
497recipe file. With all these conditions met, simply reference those files
498in the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement in
499the append file.
500
501For example, suppose you had some configuration options in a file called
502``network_configs.cfg``. You can place that file inside a directory
503named ``linux-yocto`` and then add a ``SRC_URI`` statement such as the
504following to the append file. When the OpenEmbedded build system builds
505the kernel, the configuration options are picked up and applied. SRC_URI
506+= "file://network_configs.cfg"
507
508To group related configurations into multiple files, you perform a
509similar procedure. Here is an example that groups separate
510configurations specifically for Ethernet and graphics into their own
511files and adds the configurations by using a ``SRC_URI`` statement like
512the following in your append file: SRC_URI += "file://myconfig.cfg \\
513file://eth.cfg \\ file://gfx.cfg"
514
515Another variable you can use in your kernel recipe append file is the
516```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
517variable. When you use this statement, you are extending the locations
518used by the OpenEmbedded system to look for files and patches as the
519recipe is processed.
520
521.. note::
522
523 Other methods exist to accomplish grouping and defining configuration
524 options. For example, if you are working with a local clone of the
525 kernel repository, you could checkout the kernel's ``meta`` branch,
526 make your changes, and then push the changes to the local bare clone
527 of the kernel. The result is that you directly add configuration
528 options to the ``meta`` branch for your BSP. The configuration
529 options will likely end up in that location anyway if the BSP gets
530 added to the Yocto Project.
531
532 In general, however, the Yocto Project maintainers take care of
533 moving the ``SRC_URI``-specified configuration options to the
534 kernel's ``meta`` branch. Not only is it easier for BSP developers to
535 not have to worry about putting those configurations in the branch,
536 but having the maintainers do it allows them to apply 'global'
537 knowledge about the kinds of common configuration options multiple
538 BSPs in the tree are typically using. This allows for promotion of
539 common configurations into common features.
540
541Applying Patches
542----------------
543
544If you have a single patch or a small series of patches that you want to
545apply to the Linux kernel source, you can do so just as you would with
546any other recipe. You first copy the patches to the path added to
547```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__ in
548your ``.bbappend`` file as described in the previous section, and then
549reference them in ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
550statements.
551
552For example, you can apply a three-patch series by adding the following
553lines to your linux-yocto ``.bbappend`` file in your layer: SRC_URI +=
554"file://0001-first-change.patch" SRC_URI +=
555"file://0002-second-change.patch" SRC_URI +=
556"file://0003-third-change.patch" The next time you run BitBake to build
557the Linux kernel, BitBake detects the change in the recipe and fetches
558and applies the patches before building the kernel.
559
560For a detailed example showing how to patch the kernel using
561``devtool``, see the "`Using ``devtool`` to Patch the
562Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
563Kernel Development to Patch the
564Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
565sections.
566
567Changing the Configuration
568--------------------------
569
570You can make wholesale or incremental changes to the final ``.config``
571file used for the eventual Linux kernel configuration by including a
572``defconfig`` file and by specifying configuration fragments in the
573```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ to be applied to that
574file.
575
576If you have a complete, working Linux kernel ``.config`` file you want
577to use for the configuration, as before, copy that file to the
578appropriate ``${PN}`` directory in your layer's ``recipes-kernel/linux``
579directory, and rename the copied file to "defconfig". Then, add the
580following lines to the linux-yocto ``.bbappend`` file in your layer:
581FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
582"file://defconfig" The ``SRC_URI`` tells the build system how to search
583for the file, while the
584```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
585extends the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
586variable (search directories) to include the ``${PN}`` directory you
587created to hold the configuration changes.
588
589.. note::
590
591 The build system applies the configurations from the
592 defconfig
593 file before applying any subsequent configuration fragments. The
594 final kernel configuration is a combination of the configurations in
595 the
596 defconfig
597 file and any configuration fragments you provide. You need to realize
598 that if you have any configuration fragments, the build system
599 applies these on top of and after applying the existing
600 defconfig
601 file configurations.
602
603Generally speaking, the preferred approach is to determine the
604incremental change you want to make and add that as a configuration
605fragment. For example, if you want to add support for a basic serial
606console, create a file named ``8250.cfg`` in the ``${PN}`` directory
607with the following content (without indentation): CONFIG_SERIAL_8250=y
608CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_PCI=y
609CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_8250_RUNTIME_UARTS=4
610CONFIG_SERIAL_CORE=y CONFIG_SERIAL_CORE_CONSOLE=y Next, include this
611configuration fragment and extend the ``FILESPATH`` variable in your
612``.bbappend`` file: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
613SRC_URI += "file://8250.cfg" The next time you run BitBake to build the
614Linux kernel, BitBake detects the change in the recipe and fetches and
615applies the new configuration before building the kernel.
616
617For a detailed example showing how to configure the kernel, see the
618"`Configuring the Kernel <#configuring-the-kernel>`__" section.
619
620Using an "In-Tree"  ``defconfig`` File
621--------------------------------------
622
623It might be desirable to have kernel configuration fragment support
624through a ``defconfig`` file that is pulled from the kernel source tree
625for the configured machine. By default, the OpenEmbedded build system
626looks for ``defconfig`` files in the layer used for Metadata, which is
627"out-of-tree", and then configures them using the following: SRC_URI +=
628"file://defconfig" If you do not want to maintain copies of
629``defconfig`` files in your layer but would rather allow users to use
630the default configuration from the kernel tree and still be able to add
631configuration fragments to the
632```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ through, for example,
633append files, you can direct the OpenEmbedded build system to use a
634``defconfig`` file that is "in-tree".
635
636To specify an "in-tree" ``defconfig`` file, use the following statement
637form: KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file Here is an example
638that assigns the ``KBUILD_DEFCONFIG`` variable based on "raspberrypi2"
639and provides the path to the "in-tree" ``defconfig`` file to be used for
640a Raspberry Pi 2, which is based on the Broadcom 2708/2709 chipset:
641KBUILD_DEFCONFIG_raspberrypi2 ?= "bcm2709_defconfig"
642
643Aside from modifying your kernel recipe and providing your own
644``defconfig`` file, you need to be sure no files or statements set
645``SRC_URI`` to use a ``defconfig`` other than your "in-tree" file (e.g.
646a kernel's ``linux-``\ machine\ ``.inc`` file). In other words, if the
647build system detects a statement that identifies an "out-of-tree"
648``defconfig`` file, that statement will override your
649``KBUILD_DEFCONFIG`` variable.
650
651See the
652```KBUILD_DEFCONFIG`` <&YOCTO_DOCS_REF_URL;#var-KBUILD_DEFCONFIG>`__
653variable description for more information.
654
655Using ``devtool`` to Patch the Kernel
656=====================================
657
658The steps in this procedure show you how you can patch the kernel using
659the extensible SDK and ``devtool``.
660
661.. note::
662
663 Before attempting this procedure, be sure you have performed the
664 steps to get ready for updating the kernel as described in the "
665 Getting Ready to Develop Using
666 devtool
667 " section.
668
669Patching the kernel involves changing or adding configurations to an
670existing kernel, changing or adding recipes to the kernel that are
671needed to support specific hardware features, or even altering the
672source code itself.
673
674This example creates a simple patch by adding some QEMU emulator console
675output at boot time through ``printk`` statements in the kernel's
676``calibrate.c`` source code file. Applying the patch and booting the
677modified image causes the added messages to appear on the emulator's
678console. The example is a continuation of the setup procedure found in
679the "`Getting Ready to Develop Using
680``devtool`` <#getting-ready-to-develop-using-devtool>`__" Section.
681
6821. *Check Out the Kernel Source Files:* First you must use ``devtool``
683 to checkout the kernel source code in its workspace. Be sure you are
684 in the terminal set up to do work with the extensible SDK.
685
686 .. note::
687
688 See this
689 step
690 in the "
691 Getting Ready to Develop Using
692 devtool
693 " section for more information.
694
695 Use the following ``devtool`` command to check out the code: $
696 devtool modify linux-yocto
697
698 .. note::
699
700 During the checkout operation, a bug exists that could cause
701 errors such as the following to appear:
702 ::
703
704 ERROR: Taskhash mismatch 2c793438c2d9f8c3681fd5f7bc819efa versus
705 be3a89ce7c47178880ba7bf6293d7404 for
706 /path/to/esdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_4.10.bb.do_unpack
707
708
709 You can safely ignore these messages. The source code is correctly
710 checked out.
711
7122. *Edit the Source Files* Follow these steps to make some simple
713 changes to the source files:
714
715 1. *Change the working directory*: In the previous step, the output
716 noted where you can find the source files (e.g.
717 ``~/poky_sdk/workspace/sources/linux-yocto``). Change to where the
718 kernel source code is before making your edits to the
719 ``calibrate.c`` file: $ cd
720 ~/poky_sdk/workspace/sources/linux-yocto
721
722 2. *Edit the source file*: Edit the ``init/calibrate.c`` file to have
723 the following changes: void calibrate_delay(void) { unsigned long
724 lpj; static bool printed; int this_cpu = smp_processor_id();
725 printk("*************************************\n"); printk("\*
726 \*\n"); printk("\* HELLO YOCTO KERNEL \*\n"); printk("\* \*\n");
727 printk("*************************************\n"); if
728 (per_cpu(cpu_loops_per_jiffy, this_cpu)) { . . .
729
7303. *Build the Updated Kernel Source:* To build the updated kernel
731 source, use ``devtool``: $ devtool build linux-yocto
732
7334. *Create the Image With the New Kernel:* Use the
734 ``devtool build-image`` command to create a new image that has the
735 new kernel.
736
737 .. note::
738
739 If the image you originally created resulted in a Wic file, you
740 can use an alternate method to create the new image with the
741 updated kernel. For an example, see the steps in the
742 TipsAndTricks/KernelDevelopmentWithEsdk
743 Wiki Page.
744
745 $ cd ~ $ devtool build-image core-image-minimal
746
7475. *Test the New Image:* For this example, you can run the new image
748 using QEMU to verify your changes:
749
750 1. *Boot the image*: Boot the modified image in the QEMU emulator
751 using this command: $ runqemu qemux86
752
753 2. *Verify the changes*: Log into the machine using ``root`` with no
754 password and then use the following shell command to scroll
755 through the console's boot output. # dmesg \| less You should see
756 the results of your ``printk`` statements as part of the output
757 when you scroll down the console window.
758
7596. *Stage and commit your changes*: Within your eSDK terminal, change
760 your working directory to where you modified the ``calibrate.c`` file
761 and use these Git commands to stage and commit your changes: $ cd
762 ~/poky_sdk/workspace/sources/linux-yocto $ git status $ git add
763 init/calibrate.c $ git commit -m "calibrate: Add printk example"
764
7657. *Export the Patches and Create an Append File:* To export your
766 commits as patches and create a ``.bbappend`` file, use the following
767 command in the terminal used to work with the extensible SDK. This
768 example uses the previously established layer named ``meta-mylayer``.
769
770 .. note::
771
772 See Step 3 of the "
773 Getting Ready to Develop Using devtool
774 " section for information on setting up this layer.
775
776 $ devtool finish linux-yocto ~/meta-mylayer Once the command
777 finishes, the patches and the ``.bbappend`` file are located in the
778 ``~/meta-mylayer/recipes-kernel/linux`` directory.
779
7808. *Build the Image With Your Modified Kernel:* You can now build an
781 image that includes your kernel patches. Execute the following
782 command from your `Build
783 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in the terminal
784 set up to run BitBake: $ cd ~/poky/build $ bitbake core-image-minimal
785
786Using Traditional Kernel Development to Patch the Kernel
787========================================================
788
789The steps in this procedure show you how you can patch the kernel using
790traditional kernel development (i.e. not using ``devtool`` and the
791extensible SDK as described in the "`Using ``devtool`` to Patch the
792Kernel <#using-devtool-to-patch-the-kernel>`__" section).
793
794.. note::
795
796 Before attempting this procedure, be sure you have performed the
797 steps to get ready for updating the kernel as described in the "
798 Getting Ready for Traditional Kernel Development
799 " section.
800
801Patching the kernel involves changing or adding configurations to an
802existing kernel, changing or adding recipes to the kernel that are
803needed to support specific hardware features, or even altering the
804source code itself.
805
806The example in this section creates a simple patch by adding some QEMU
807emulator console output at boot time through ``printk`` statements in
808the kernel's ``calibrate.c`` source code file. Applying the patch and
809booting the modified image causes the added messages to appear on the
810emulator's console. The example is a continuation of the setup procedure
811found in the "`Getting Ready for Traditional Kernel
812Development <#getting-ready-for-traditional-kernel-development>`__"
813Section.
814
8151. *Edit the Source Files* Prior to this step, you should have used Git
816 to create a local copy of the repository for your kernel. Assuming
817 you created the repository as directed in the "`Getting Ready for
818 Traditional Kernel
819 Development <#getting-ready-for-traditional-kernel-development>`__"
820 section, use the following commands to edit the ``calibrate.c`` file:
821
822 1. *Change the working directory*: You need to locate the source
823 files in the local copy of the kernel Git repository: Change to
824 where the kernel source code is before making your edits to the
825 ``calibrate.c`` file: $ cd ~/linux-yocto-4.12/init
826
827 2. *Edit the source file*: Edit the ``calibrate.c`` file to have the
828 following changes: void calibrate_delay(void) { unsigned long lpj;
829 static bool printed; int this_cpu = smp_processor_id();
830 printk("*************************************\n"); printk("\*
831 \*\n"); printk("\* HELLO YOCTO KERNEL \*\n"); printk("\* \*\n");
832 printk("*************************************\n"); if
833 (per_cpu(cpu_loops_per_jiffy, this_cpu)) { . . .
834
8352. *Stage and Commit Your Changes:* Use standard Git commands to stage
836 and commit the changes you just made: $ git add calibrate.c $ git
837 commit -m "calibrate.c - Added some printk statements" If you do not
838 stage and commit your changes, the OpenEmbedded Build System will not
839 pick up the changes.
840
8413. *Update Your ``local.conf`` File to Point to Your Source Files:* In
842 addition to your ``local.conf`` file specifying to use
843 "kernel-modules" and the "qemux86" machine, it must also point to the
844 updated kernel source files. Add
845 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ and
846 ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ statements similar
847 to the following to your ``local.conf``: $ cd ~/poky/build/conf Add
848 the following to the ``local.conf``: SRC_URI_pn-linux-yocto =
849 "git:///path-to/linux-yocto-4.12;protocol=file;name=machine;branch=standard/base;
850 \\
851 git:///path-to/yocto-kernel-cache;protocol=file;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
852 SRCREV_meta_qemux86 = "${AUTOREV}" SRCREV_machine_qemux86 =
853 "${AUTOREV}"
854
855 .. note::
856
857 Be sure to replace
858 path-to
859 with the pathname to your local Git repositories. Also, you must
860 be sure to specify the correct branch and machine types. For this
861 example, the branch is
862 standard/base
863 and the machine is "qemux86".
864
8654. *Build the Image:* With the source modified, your changes staged and
866 committed, and the ``local.conf`` file pointing to the kernel files,
867 you can now use BitBake to build the image: $ cd ~/poky/build $
868 bitbake core-image-minimal
869
8705. *Boot the image*: Boot the modified image in the QEMU emulator using
871 this command. When prompted to login to the QEMU console, use "root"
872 with no password: $ cd ~/poky/build $ runqemu qemux86
873
8746. *Look for Your Changes:* As QEMU booted, you might have seen your
875 changes rapidly scroll by. If not, use these commands to see your
876 changes: # dmesg \| less You should see the results of your
877 ``printk`` statements as part of the output when you scroll down the
878 console window.
879
8807. *Generate the Patch File:* Once you are sure that your patch works
881 correctly, you can generate a ``*.patch`` file in the kernel source
882 repository: $ cd ~/linux-yocto-4.12/init $ git format-patch -1
883 0001-calibrate.c-Added-some-printk-statements.patch
884
8858. *Move the Patch File to Your Layer:* In order for subsequent builds
886 to pick up patches, you need to move the patch file you created in
887 the previous step to your layer ``meta-mylayer``. For this example,
888 the layer created earlier is located in your home directory as
889 ``meta-mylayer``. When the layer was created using the
890 ``yocto-create`` script, no additional hierarchy was created to
891 support patches. Before moving the patch file, you need to add
892 additional structure to your layer using the following commands: $ cd
893 ~/meta-mylayer $ mkdir recipes-kernel $ mkdir recipes-kernel/linux $
894 mkdir recipes-kernel/linux/linux-yocto Once you have created this
895 hierarchy in your layer, you can move the patch file using the
896 following command: $ mv
897 ~/linux-yocto-4.12/init/0001-calibrate.c-Added-some-printk-statements.patch
898 ~/meta-mylayer/recipes-kernel/linux/linux-yocto
899
9009. *Create the Append File:* Finally, you need to create the
901 ``linux-yocto_4.12.bbappend`` file and insert statements that allow
902 the OpenEmbedded build system to find the patch. The append file
903 needs to be in your layer's ``recipes-kernel/linux`` directory and it
904 must be named ``linux-yocto_4.12.bbappend`` and have the following
905 contents: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
906 SRC_URI_append = "
907 file://0001-calibrate.c-Added-some-printk-statements.patch" The
908 ```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
909 and ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
910 enable the OpenEmbedded build system to find the patch file.
911
912 For more information on append files and patches, see the "`Creating
913 the Append File <#creating-the-append-file>`__" and "`Applying
914 Patches <#applying-patches>`__" sections. You can also see the
915 "`Using .bbappend Files in Your
916 Layer" <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
917 Yocto Project Development Tasks Manual.
918
919 .. note::
920
921 To build
922 core-image-minimal
923 again and see the effects of your patch, you can essentially
924 eliminate the temporary source files saved in
925 poky/build/tmp/work/...
926 and residual effects of the build by entering the following
927 sequence of commands:
928 ::
929
930 $ cd ~/poky/build
931 $ bitbake -c cleanall yocto-linux
932 $ bitbake core-image-minimal -c cleanall
933 $ bitbake core-image-minimal
934 $ runqemu qemux86
935
936
937Configuring the Kernel
938======================
939
940Configuring the Yocto Project kernel consists of making sure the
941``.config`` file has all the right information in it for the image you
942are building. You can use the ``menuconfig`` tool and configuration
943fragments to make sure your ``.config`` file is just how you need it.
944You can also save known configurations in a ``defconfig`` file that the
945build system can use for kernel configuration.
946
947This section describes how to use ``menuconfig``, create and use
948configuration fragments, and how to interactively modify your
949``.config`` file to create the leanest kernel configuration file
950possible.
951
952For more information on kernel configuration, see the "`Changing the
953Configuration <#changing-the-configuration>`__" section.
954
955Using  ``menuconfig``
956---------------------
957
958The easiest way to define kernel configurations is to set them through
959the ``menuconfig`` tool. This tool provides an interactive method with
960which to set kernel configurations. For general information on
961``menuconfig``, see ` <http://en.wikipedia.org/wiki/Menuconfig>`__.
962
963To use the ``menuconfig`` tool in the Yocto Project development
964environment, you must do the following:
965
966- Because you launch ``menuconfig`` using BitBake, you must be sure to
967 set up your environment by running the
968 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ script found in
969 the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
970
971- You must be sure of the state of your build's configuration in the
972 `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
973
974- Your build host must have the following two packages installed:
975 libncurses5-dev libtinfo-dev
976
977The following commands initialize the BitBake environment, run the
978```do_kernel_configme`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configme>`__
979task, and launch ``menuconfig``. These commands assume the Source
980Directory's top-level folder is ``~/poky``: $ cd poky $ source
981oe-init-build-env $ bitbake linux-yocto -c kernel_configme -f $ bitbake
982linux-yocto -c menuconfig Once ``menuconfig`` comes up, its standard
983interface allows you to interactively examine and configure all the
984kernel configuration parameters. After making your changes, simply exit
985the tool and save your changes to create an updated version of the
986``.config`` configuration file.
987
988.. note::
989
990 You can use the entire
991 .config
992 file as the
993 defconfig
994 file. For information on
995 defconfig
996 files, see the "
997 Changing the Configuration
998 ", "
999 Using an In-Tree
1000 defconfig
1001 File
1002 , and "
1003 Creating a
1004 defconfig
1005 File
1006 " sections.
1007
1008Consider an example that configures the "CONFIG_SMP" setting for the
1009``linux-yocto-4.12`` kernel.
1010
1011.. note::
1012
1013 The OpenEmbedded build system recognizes this kernel as
1014 linux-yocto
1015 through Metadata (e.g.
1016 PREFERRED_VERSION
1017 \_linux-yocto ?= "12.4%"
1018 ).
1019
1020Once ``menuconfig`` launches, use the interface to navigate through the
1021selections to find the configuration settings in which you are
1022interested. For this example, you deselect "CONFIG_SMP" by clearing the
1023"Symmetric Multi-Processing Support" option. Using the interface, you
1024can find the option under "Processor Type and Features". To deselect
1025"CONFIG_SMP", use the arrow keys to highlight "Symmetric
1026Multi-Processing Support" and enter "N" to clear the asterisk. When you
1027are finished, exit out and save the change.
1028
1029Saving the selections updates the ``.config`` configuration file. This
1030is the file that the OpenEmbedded build system uses to configure the
1031kernel during the build. You can find and examine this file in the Build
1032Directory in ``tmp/work/``. The actual ``.config`` is located in the
1033area where the specific kernel is built. For example, if you were
1034building a Linux Yocto kernel based on the ``linux-yocto-4.12`` kernel
1035and you were building a QEMU image targeted for ``x86`` architecture,
1036the ``.config`` file would be:
1037poky/build/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+gitAUTOINC+eda4d18...
1038...967-r0/linux-qemux86-standard-build/.config
1039
1040.. note::
1041
1042 The previous example directory is artificially split and many of the
1043 characters in the actual filename are omitted in order to make it
1044 more readable. Also, depending on the kernel you are using, the exact
1045 pathname might differ.
1046
1047Within the ``.config`` file, you can see the kernel settings. For
1048example, the following entry shows that symmetric multi-processor
1049support is not set: # CONFIG_SMP is not set
1050
1051A good method to isolate changed configurations is to use a combination
1052of the ``menuconfig`` tool and simple shell commands. Before changing
1053configurations with ``menuconfig``, copy the existing ``.config`` and
1054rename it to something else, use ``menuconfig`` to make as many changes
1055as you want and save them, then compare the renamed configuration file
1056against the newly created file. You can use the resulting differences as
1057your base to create configuration fragments to permanently save in your
1058kernel layer.
1059
1060.. note::
1061
1062 Be sure to make a copy of the
1063 .config
1064 file and do not just rename it. The build system needs an existing
1065 .config
1066 file from which to work.
1067
1068Creating a  ``defconfig`` File
1069------------------------------
1070
1071A ``defconfig`` file in the context of the Yocto Project is often a
1072``.config`` file that is copied from a build or a ``defconfig`` taken
1073from the kernel tree and moved into recipe space. You can use a
1074``defconfig`` file to retain a known set of kernel configurations from
1075which the OpenEmbedded build system can draw to create the final
1076``.config`` file.
1077
1078.. note::
1079
1080 Out-of-the-box, the Yocto Project never ships a
1081 defconfig
1082 or
1083 .config
1084 file. The OpenEmbedded build system creates the final
1085 .config
1086 file used to configure the kernel.
1087
1088To create a ``defconfig``, start with a complete, working Linux kernel
1089``.config`` file. Copy that file to the appropriate
1090``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` directory in
1091your layer's ``recipes-kernel/linux`` directory, and rename the copied
1092file to "defconfig" (e.g.
1093``~/meta-mylayer/recipes-kernel/linux/linux-yocto/defconfig``). Then,
1094add the following lines to the linux-yocto ``.bbappend`` file in your
1095layer: FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
1096"file://defconfig" The
1097```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ tells the build
1098system how to search for the file, while the
1099```FILESEXTRAPATHS`` <&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS>`__
1100extends the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__
1101variable (search directories) to include the ``${PN}`` directory you
1102created to hold the configuration changes.
1103
1104.. note::
1105
1106 The build system applies the configurations from the
1107 defconfig
1108 file before applying any subsequent configuration fragments. The
1109 final kernel configuration is a combination of the configurations in
1110 the
1111 defconfig
1112 file and any configuration fragments you provide. You need to realize
1113 that if you have any configuration fragments, the build system
1114 applies these on top of and after applying the existing defconfig
1115 file configurations.
1116
1117For more information on configuring the kernel, see the "`Changing the
1118Configuration <#changing-the-configuration>`__" section.
1119
1120.. _creating-config-fragments:
1121
1122Creating Configuration Fragments
1123--------------------------------
1124
1125Configuration fragments are simply kernel options that appear in a file
1126placed where the OpenEmbedded build system can find and apply them. The
1127build system applies configuration fragments after applying
1128configurations from a ``defconfig`` file. Thus, the final kernel
1129configuration is a combination of the configurations in the
1130``defconfig`` file and then any configuration fragments you provide. The
1131build system applies fragments on top of and after applying the existing
1132defconfig file configurations.
1133
1134Syntactically, the configuration statement is identical to what would
1135appear in the ``.config`` file, which is in the `Build
1136Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
1137
1138.. note::
1139
1140 For more information about where the
1141 .config
1142 file is located, see the example in the "
1143 Using
1144 menuconfig
1145 " section.
1146
1147It is simple to create a configuration fragment. One method is to use
1148shell commands. For example, issuing the following from the shell
1149creates a configuration fragment file named ``my_smp.cfg`` that enables
1150multi-processor support within the kernel: $ echo "CONFIG_SMP=y" >>
1151my_smp.cfg
1152
1153.. note::
1154
1155 All configuration fragment files must use the
1156 .cfg
1157 extension in order for the OpenEmbedded build system to recognize
1158 them as a configuration fragment.
1159
1160Another method is to create a configuration fragment using the
1161differences between two configuration files: one previously created and
1162saved, and one freshly created using the ``menuconfig`` tool.
1163
1164To create a configuration fragment using this method, follow these
1165steps:
1166
11671. *Complete a Build Through Kernel Configuration:* Complete a build at
1168 least through the kernel configuration task as follows: $ bitbake
1169 linux-yocto -c kernel_configme -f This step ensures that you create a
1170 ``.config`` file from a known state. Because situations exist where
1171 your build state might become unknown, it is best to run this task
1172 prior to starting ``menuconfig``.
1173
11742. *Launch ``menuconfig``:* Run the ``menuconfig`` command: $ bitbake
1175 linux-yocto -c menuconfig
1176
11773. *Create the Configuration Fragment:* Run the ``diffconfig`` command
1178 to prepare a configuration fragment. The resulting file
1179 ``fragment.cfg`` is placed in the
1180 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``
1181 directory: $ bitbake linux-yocto -c diffconfig
1182
1183The ``diffconfig`` command creates a file that is a list of Linux kernel
1184``CONFIG_`` assignments. See the "`Changing the
1185Configuration <#changing-the-configuration>`__" section for additional
1186information on how to use the output as a configuration fragment.
1187
1188.. note::
1189
1190 You can also use this method to create configuration fragments for a
1191 BSP. See the "
1192 BSP Descriptions
1193 " section for more information.
1194
1195Where do you put your configuration fragment files? You can place these
1196files in an area pointed to by
1197```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ as directed by your
1198``bblayers.conf`` file, which is located in your layer. The OpenEmbedded
1199build system picks up the configuration and adds it to the kernel's
1200configuration. For example, suppose you had a set of configuration
1201options in a file called ``myconfig.cfg``. If you put that file inside a
1202directory named ``linux-yocto`` that resides in the same directory as
1203the kernel's append file within your layer and then add the following
1204statements to the kernel's append file, those configuration options will
1205be picked up and applied when the kernel is built:
1206FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" SRC_URI +=
1207"file://myconfig.cfg"
1208
1209As mentioned earlier, you can group related configurations into multiple
1210files and name them all in the ``SRC_URI`` statement as well. For
1211example, you could group separate configurations specifically for
1212Ethernet and graphics into their own files and add those by using a
1213``SRC_URI`` statement like the following in your append file: SRC_URI +=
1214"file://myconfig.cfg \\ file://eth.cfg \\ file://gfx.cfg"
1215
1216Validating Configuration
1217------------------------
1218
1219You can use the
1220```do_kernel_configcheck`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configcheck>`__
1221task to provide configuration validation: $ bitbake linux-yocto -c
1222kernel_configcheck -f Running this task produces warnings for when a
1223requested configuration does not appear in the final ``.config`` file or
1224when you override a policy configuration in a hardware configuration
1225fragment.
1226
1227In order to run this task, you must have an existing ``.config`` file.
1228See the "`Using ``menuconfig`` <#using-menuconfig>`__" section for
1229information on how to create a configuration file.
1230
1231Following is sample output from the ``do_kernel_configcheck`` task:
1232Loading cache: 100%
1233\|########################################################\| Time:
12340:00:00 Loaded 1275 entries from dependency cache. NOTE: Resolving any
1235missing task queue dependencies Build Configuration: . . . NOTE:
1236Executing SetScene Tasks NOTE: Executing RunQueue Tasks WARNING:
1237linux-yocto-4.12.12+gitAUTOINC+eda4d18ce4_16de014967-r0
1238do_kernel_configcheck: [kernel config]: specified values did not make it
1239into the kernel's final configuration: ---------- CONFIG_X86_TSC
1240----------------- Config: CONFIG_X86_TSC From:
1241/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc-cpu.cfg
1242Requested value: CONFIG_X86_TSC=y Actual value: ----------
1243CONFIG_X86_BIGSMP ----------------- Config: CONFIG_X86_BIGSMP From:
1244/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1245/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1246Requested value: # CONFIG_X86_BIGSMP is not set Actual value: ----------
1247CONFIG_NR_CPUS ----------------- Config: CONFIG_NR_CPUS From:
1248/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1249/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc.cfg
1250/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1251Requested value: CONFIG_NR_CPUS=8 Actual value: CONFIG_NR_CPUS=1
1252---------- CONFIG_SCHED_SMT ----------------- Config: CONFIG_SCHED_SMT
1253From:
1254/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1255/home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1256Requested value: CONFIG_SCHED_SMT=y Actual value: NOTE: Tasks Summary:
1257Attempted 288 tasks of which 285 didn't need to be rerun and all
1258succeeded. Summary: There were 3 WARNING messages shown.
1259
1260.. note::
1261
1262 The previous output example has artificial line breaks to make it
1263 more readable.
1264
1265The output describes the various problems that you can encounter along
1266with where to find the offending configuration items. You can use the
1267information in the logs to adjust your configuration files and then
1268repeat the
1269```do_kernel_configme`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configme>`__
1270and
1271```do_kernel_configcheck`` <&YOCTO_DOCS_REF_URL;#ref-tasks-kernel_configcheck>`__
1272tasks until they produce no warnings.
1273
1274For more information on how to use the ``menuconfig`` tool, see the
1275"`Using ``menuconfig`` <#using-menuconfig>`__" section.
1276
1277Fine-Tuning the Kernel Configuration File
1278-----------------------------------------
1279
1280You can make sure the ``.config`` file is as lean or efficient as
1281possible by reading the output of the kernel configuration fragment
1282audit, noting any issues, making changes to correct the issues, and then
1283repeating.
1284
1285As part of the kernel build process, the ``do_kernel_configcheck`` task
1286runs. This task validates the kernel configuration by checking the final
1287``.config`` file against the input files. During the check, the task
1288produces warning messages for the following issues:
1289
1290- Requested options that did not make the final ``.config`` file.
1291
1292- Configuration items that appear twice in the same configuration
1293 fragment.
1294
1295- Configuration items tagged as "required" that were overridden.
1296
1297- A board overrides a non-board specific option.
1298
1299- Listed options not valid for the kernel being processed. In other
1300 words, the option does not appear anywhere.
1301
1302.. note::
1303
1304 The
1305 do_kernel_configcheck
1306 task can also optionally report if an option is overridden during
1307 processing.
1308
1309For each output warning, a message points to the file that contains a
1310list of the options and a pointer to the configuration fragment that
1311defines them. Collectively, the files are the key to streamlining the
1312configuration.
1313
1314To streamline the configuration, do the following:
1315
13161. *Use a Working Configuration:* Start with a full configuration that
1317 you know works. Be sure the configuration builds and boots
1318 successfully. Use this configuration file as your baseline.
1319
13202. *Run Configure and Check Tasks:* Separately run the
1321 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks: $ bitbake
1322 linux-yocto -c kernel_configme -f $ bitbake linux-yocto -c
1323 kernel_configcheck -f
1324
13253. *Process the Results:* Take the resulting list of files from the
1326 ``do_kernel_configcheck`` task warnings and do the following:
1327
1328 - Drop values that are redefined in the fragment but do not change
1329 the final ``.config`` file.
1330
1331 - Analyze and potentially drop values from the ``.config`` file that
1332 override required configurations.
1333
1334 - Analyze and potentially remove non-board specific options.
1335
1336 - Remove repeated and invalid options.
1337
13384. *Re-Run Configure and Check Tasks:* After you have worked through the
1339 output of the kernel configuration audit, you can re-run the
1340 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks to see the
1341 results of your changes. If you have more issues, you can deal with
1342 them as described in the previous step.
1343
1344Iteratively working through steps two through four eventually yields a
1345minimal, streamlined configuration file. Once you have the best
1346``.config``, you can build the Linux Yocto kernel.
1347
1348Expanding Variables
1349===================
1350
1351Sometimes it is helpful to determine what a variable expands to during a
1352build. You can do examine the values of variables by examining the
1353output of the ``bitbake -e`` command. The output is long and is more
1354easily managed in a text file, which allows for easy searches: $ bitbake
1355-e virtual/kernel > some_text_file Within the text file, you can see
1356exactly how each variable is expanded and used by the OpenEmbedded build
1357system.
1358
1359Working with a "Dirty" Kernel Version String
1360============================================
1361
1362If you build a kernel image and the version string has a "+" or a
1363"-dirty" at the end, uncommitted modifications exist in the kernel's
1364source directory. Follow these steps to clean up the version string:
1365
13661. *Discover the Uncommitted Changes:* Go to the kernel's locally cloned
1367 Git repository (source directory) and use the following Git command
1368 to list the files that have been changed, added, or removed: $ git
1369 status
1370
13712. *Commit the Changes:* You should commit those changes to the kernel
1372 source tree regardless of whether or not you will save, export, or
1373 use the changes: $ git add $ git commit -s -a -m "getting rid of
1374 -dirty"
1375
13763. *Rebuild the Kernel Image:* Once you commit the changes, rebuild the
1377 kernel.
1378
1379 Depending on your particular kernel development workflow, the
1380 commands you use to rebuild the kernel might differ. For information
1381 on building the kernel image when using ``devtool``, see the "`Using
1382 ``devtool`` to Patch the
1383 Kernel <#using-devtool-to-patch-the-kernel>`__" section. For
1384 information on building the kernel image when using Bitbake, see the
1385 "`Using Traditional Kernel Development to Patch the
1386 Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
1387 section.
1388
1389Working With Your Own Sources
1390=============================
1391
1392If you cannot work with one of the Linux kernel versions supported by
1393existing linux-yocto recipes, you can still make use of the Yocto
1394Project Linux kernel tooling by working with your own sources. When you
1395use your own sources, you will not be able to leverage the existing
1396kernel `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ and stabilization
1397work of the linux-yocto sources. However, you will be able to manage
1398your own Metadata in the same format as the linux-yocto sources.
1399Maintaining format compatibility facilitates converging with linux-yocto
1400on a future, mutually-supported kernel version.
1401
1402To help you use your own sources, the Yocto Project provides a
1403linux-yocto custom recipe (``linux-yocto-custom.bb``) that uses
1404``kernel.org`` sources and the Yocto Project Linux kernel tools for
1405managing kernel Metadata. You can find this recipe in the ``poky`` Git
1406repository of the Yocto Project `Source Repository <&YOCTO_GIT_URL;>`__
1407at: poky/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
1408
1409Here are some basic steps you can use to work with your own sources:
1410
14111. *Create a Copy of the Kernel Recipe:* Copy the
1412 ``linux-yocto-custom.bb`` recipe to your layer and give it a
1413 meaningful name. The name should include the version of the Yocto
1414 Linux kernel you are using (e.g. ``linux-yocto-myproject_4.12.bb``,
1415 where "4.12" is the base version of the Linux kernel with which you
1416 would be working).
1417
14182. *Create a Directory for Your Patches:* In the same directory inside
1419 your layer, create a matching directory to store your patches and
1420 configuration files (e.g. ``linux-yocto-myproject``).
1421
14223. *Ensure You Have Configurations:* Make sure you have either a
1423 ``defconfig`` file or configuration fragment files in your layer.
1424 When you use the ``linux-yocto-custom.bb`` recipe, you must specify a
1425 configuration. If you do not have a ``defconfig`` file, you can run
1426 the following: $ make defconfig After running the command, copy the
1427 resulting ``.config`` file to the ``files`` directory in your layer
1428 as "defconfig" and then add it to the
1429 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable in the
1430 recipe.
1431
1432 Running the ``make defconfig`` command results in the default
1433 configuration for your architecture as defined by your kernel.
1434 However, no guarantee exists that this configuration is valid for
1435 your use case, or that your board will even boot. This is
1436 particularly true for non-x86 architectures.
1437
1438 To use non-x86 ``defconfig`` files, you need to be more specific and
1439 find one that matches your board (i.e. for arm, you look in
1440 ``arch/arm/configs`` and use the one that is the best starting point
1441 for your board).
1442
14434. *Edit the Recipe:* Edit the following variables in your recipe as
1444 appropriate for your project:
1445
1446 - ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__: The
1447 ``SRC_URI`` should specify a Git repository that uses one of the
1448 supported Git fetcher protocols (i.e. ``file``, ``git``, ``http``,
1449 and so forth). The ``SRC_URI`` variable should also specify either
1450 a ``defconfig`` file or some configuration fragment files. The
1451 skeleton recipe provides an example ``SRC_URI`` as a syntax
1452 reference.
1453
1454 - ```LINUX_VERSION`` <&YOCTO_DOCS_REF_URL;#var-LINUX_VERSION>`__:
1455 The Linux kernel version you are using (e.g. "4.12").
1456
1457 - ```LINUX_VERSION_EXTENSION`` <&YOCTO_DOCS_REF_URL;#var-LINUX_VERSION_EXTENSION>`__:
1458 The Linux kernel ``CONFIG_LOCALVERSION`` that is compiled into the
1459 resulting kernel and visible through the ``uname`` command.
1460
1461 - ```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__: The commit ID
1462 from which you want to build.
1463
1464 - ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__: Treat this variable the
1465 same as you would in any other recipe. Increment the variable to
1466 indicate to the OpenEmbedded build system that the recipe has
1467 changed.
1468
1469 - ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The default ``PV``
1470 assignment is typically adequate. It combines the
1471 ``LINUX_VERSION`` with the Source Control Manager (SCM) revision
1472 as derived from the ```SRCPV`` <&YOCTO_DOCS_REF_URL;#var-SRCPV>`__
1473 variable. The combined results are a string with the following
1474 form:
1475 3.19.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c60d32f0d770931e3cf343e2
1476 While lengthy, the extra verbosity in ``PV`` helps ensure you are
1477 using the exact sources from which you intend to build.
1478
1479 - ```COMPATIBLE_MACHINE`` <&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE>`__:
1480 A list of the machines supported by your new recipe. This variable
1481 in the example recipe is set by default to a regular expression
1482 that matches only the empty string, "(^$)". This default setting
1483 triggers an explicit build failure. You must change it to match a
1484 list of the machines that your new recipe supports. For example,
1485 to support the ``qemux86`` and ``qemux86-64`` machines, use the
1486 following form: COMPATIBLE_MACHINE = "qemux86|qemux86-64"
1487
14885. *Customize Your Recipe as Needed:* Provide further customizations to
1489 your recipe as needed just as you would customize an existing
1490 linux-yocto recipe. See the "`Modifying an Existing
1491 Recipe <#modifying-an-existing-recipe>`__" section for information.
1492
1493Working with Out-of-Tree Modules
1494================================
1495
1496This section describes steps to build out-of-tree modules on your target
1497and describes how to incorporate out-of-tree modules in the build.
1498
1499Building Out-of-Tree Modules on the Target
1500------------------------------------------
1501
1502While the traditional Yocto Project development model would be to
1503include kernel modules as part of the normal build process, you might
1504find it useful to build modules on the target. This could be the case if
1505your target system is capable and powerful enough to handle the
1506necessary compilation. Before deciding to build on your target, however,
1507you should consider the benefits of using a proper cross-development
1508environment from your build host.
1509
1510If you want to be able to build out-of-tree modules on the target, there
1511are some steps you need to take on the target that is running your SDK
1512image. Briefly, the ``kernel-dev`` package is installed by default on
1513all ``*.sdk`` images and the ``kernel-devsrc`` package is installed on
1514many of the ``*.sdk`` images. However, you need to create some scripts
1515prior to attempting to build the out-of-tree modules on the target that
1516is running that image.
1517
1518Prior to attempting to build the out-of-tree modules, you need to be on
1519the target as root and you need to change to the ``/usr/src/kernel``
1520directory. Next, ``make`` the scripts: # cd /usr/src/kernel # make
1521scripts Because all SDK image recipes include ``dev-pkgs``, the
1522``kernel-dev`` packages will be installed as part of the SDK image and
1523the ``kernel-devsrc`` packages will be installed as part of applicable
1524SDK images. The SDK uses the scripts when building out-of-tree modules.
1525Once you have switched to that directory and created the scripts, you
1526should be able to build your out-of-tree modules on the target.
1527
1528Incorporating Out-of-Tree Modules
1529---------------------------------
1530
1531While it is always preferable to work with sources integrated into the
1532Linux kernel sources, if you need an external kernel module, the
1533``hello-mod.bb`` recipe is available as a template from which you can
1534create your own out-of-tree Linux kernel module recipe.
1535
1536This template recipe is located in the ``poky`` Git repository of the
1537Yocto Project `Source Repository <&YOCTO_GIT_URL;>`__ at:
1538poky/meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb
1539
1540To get started, copy this recipe to your layer and give it a meaningful
1541name (e.g. ``mymodule_1.0.bb``). In the same directory, create a new
1542directory named ``files`` where you can store any source files, patches,
1543or other files necessary for building the module that do not come with
1544the sources. Finally, update the recipe as needed for the module.
1545Typically, you will need to set the following variables:
1546
1547- ```DESCRIPTION`` <&YOCTO_DOCS_REF_URL;#var-DESCRIPTION>`__
1548
1549- ```LICENSE*`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__
1550
1551- ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
1552
1553- ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__
1554
1555Depending on the build system used by the module sources, you might need
1556to make some adjustments. For example, a typical module ``Makefile``
1557looks much like the one provided with the ``hello-mod`` template: obj-m
1558:= hello.o SRC := $(shell pwd) all: $(MAKE) -C $(KERNEL_SRC) M=$(SRC)
1559modules_install: $(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install ...
1560
1561The important point to note here is the
1562```KERNEL_SRC`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_SRC>`__ variable. The
1563```module`` <&YOCTO_DOCS_REF_URL;#ref-classes-module>`__ class sets this
1564variable and the
1565```KERNEL_PATH`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_PATH>`__ variable to
1566``${STAGING_KERNEL_DIR}`` with the necessary Linux kernel build
1567information to build modules. If your module ``Makefile`` uses a
1568different variable, you might want to override the
1569```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ step, or
1570create a patch to the ``Makefile`` to work with the more typical
1571``KERNEL_SRC`` or ``KERNEL_PATH`` variables.
1572
1573After you have prepared your recipe, you will likely want to include the
1574module in your images. To do this, see the documentation for the
1575following variables in the Yocto Project Reference Manual and set one of
1576them appropriately for your machine configuration file:
1577
1578- ```MACHINE_ESSENTIAL_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RDEPENDS>`__
1579
1580- ```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
1581
1582- ```MACHINE_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RDEPENDS>`__
1583
1584- ```MACHINE_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RRECOMMENDS>`__
1585
1586Modules are often not required for boot and can be excluded from certain
1587build configurations. The following allows for the most flexibility:
1588MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule" The value is
1589derived by appending the module filename without the ``.ko`` extension
1590to the string "kernel-module-".
1591
1592Because the variable is
1593```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ and not a
1594```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__ variable, the build
1595will not fail if this module is not available to include in the image.
1596
1597Inspecting Changes and Commits
1598==============================
1599
1600A common question when working with a kernel is: "What changes have been
1601applied to this tree?" Rather than using "grep" across directories to
1602see what has changed, you can use Git to inspect or search the kernel
1603tree. Using Git is an efficient way to see what has changed in the tree.
1604
1605What Changed in a Kernel?
1606-------------------------
1607
1608Following are a few examples that show how to use Git commands to
1609examine changes. These examples are by no means the only way to see
1610changes.
1611
1612.. note::
1613
1614 In the following examples, unless you provide a commit range,
1615 kernel.org
1616 history is blended with Yocto Project kernel changes. You can form
1617 ranges by using branch names from the kernel tree as the upper and
1618 lower commit markers with the Git commands. You can see the branch
1619 names through the web interface to the Yocto Project source
1620 repositories at
1621 .
1622
1623To see a full range of the changes, use the ``git whatchanged`` command
1624and specify a commit range for the branch (commit\ ``..``\ commit).
1625
1626Here is an example that looks at what has changed in the ``emenlow``
1627branch of the ``linux-yocto-3.19`` kernel. The lower commit range is the
1628commit associated with the ``standard/base`` branch, while the upper
1629commit range is the commit associated with the ``standard/emenlow``
1630branch. $ git whatchanged origin/standard/base..origin/standard/emenlow
1631
1632To see short, one line summaries of changes use the ``git log`` command:
1633$ git log --oneline origin/standard/base..origin/standard/emenlow
1634
1635Use this command to see code differences for the changes: $ git diff
1636origin/standard/base..origin/standard/emenlow
1637
1638Use this command to see the commit log messages and the text
1639differences: $ git show origin/standard/base..origin/standard/emenlow
1640
1641Use this command to create individual patches for each change. Here is
1642an example that that creates patch files for each commit and places them
1643in your ``Documents`` directory: $ git format-patch -o $HOME/Documents
1644origin/standard/base..origin/standard/emenlow
1645
1646Showing a Particular Feature or Branch Change
1647---------------------------------------------
1648
1649Tags in the Yocto Project kernel tree divide changes for significant
1650features or branches. The ``git show`` tag command shows changes based
1651on a tag. Here is an example that shows ``systemtap`` changes: $ git
1652show systemtap You can use the ``git branch --contains`` tag command to
1653show the branches that contain a particular feature. This command shows
1654the branches that contain the ``systemtap`` feature: $ git branch
1655--contains systemtap
1656
1657Adding Recipe-Space Kernel Features
1658===================================
1659
1660You can add kernel features in the
1661`recipe-space <#recipe-space-metadata>`__ by using the
1662```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
1663variable and by specifying the feature's ``.scc`` file path in the
1664```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statement. When you
1665add features using this method, the OpenEmbedded build system checks to
1666be sure the features are present. If the features are not present, the
1667build stops. Kernel features are the last elements processed for
1668configuring and patching the kernel. Therefore, adding features in this
1669manner is a way to enforce specific features are present and enabled
1670without needing to do a full audit of any other layer's additions to the
1671``SRC_URI`` statement.
1672
1673You add a kernel feature by providing the feature as part of the
1674``KERNEL_FEATURES`` variable and by providing the path to the feature's
1675``.scc`` file, which is relative to the root of the kernel Metadata. The
1676OpenEmbedded build system searches all forms of kernel Metadata on the
1677``SRC_URI`` statement regardless of whether the Metadata is in the
1678"kernel-cache", system kernel Metadata, or a recipe-space Metadata (i.e.
1679part of the kernel recipe). See the "`Kernel Metadata
1680Location <#kernel-metadata-location>`__" section for additional
1681information.
1682
1683When you specify the feature's ``.scc`` file on the ``SRC_URI``
1684statement, the OpenEmbedded build system adds the directory of that
1685``.scc`` file along with all its subdirectories to the kernel feature
1686search path. Because subdirectories are searched, you can reference a
1687single ``.scc`` file in the ``SRC_URI`` statement to reference multiple
1688kernel features.
1689
1690Consider the following example that adds the "test.scc" feature to the
1691build.
1692
16931. *Create the Feature File:* Create a ``.scc`` file and locate it just
1694 as you would any other patch file, ``.cfg`` file, or fetcher item you
1695 specify in the ``SRC_URI`` statement.
1696
1697 .. note::
1698
1699 - You must add the directory of the ``.scc`` file to the
1700 fetcher's search path in the same manner as you would add a
1701 ``.patch`` file.
1702
1703 - You can create additional ``.scc`` files beneath the directory
1704 that contains the file you are adding. All subdirectories are
1705 searched during the build as potential feature directories.
1706
1707 Continuing with the example, suppose the "test.scc" feature you are
1708 adding has a ``test.scc`` file in the following directory: my_recipe
1709 \| +-linux-yocto \| +-test.cfg +-test.scc In this example, the
1710 ``linux-yocto`` directory has both the feature ``test.scc`` file and
1711 a similarly named configuration fragment file ``test.cfg``.
1712
17132. *Add the Feature File to ``SRC_URI``:* Add the ``.scc`` file to the
1714 recipe's ``SRC_URI`` statement: SRC_URI_append = " file://test.scc"
1715 The leading space before the path is important as the path is
1716 appended to the existing path.
1717
17183. *Specify the Feature as a Kernel Feature:* Use the
1719 ``KERNEL_FEATURES`` statement to specify the feature as a kernel
1720 feature: KERNEL_FEATURES_append = " test.scc" The OpenEmbedded build
1721 system processes the kernel feature when it builds the kernel.
1722
1723 .. note::
1724
1725 If other features are contained below "test.scc", then their
1726 directories are relative to the directory containing the
1727 test.scc
1728 file.
diff --git a/documentation/kernel-dev/kernel-dev-concepts-appx.rst b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
new file mode 100644
index 0000000000..e8addbd307
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
@@ -0,0 +1,409 @@
1************************
2Advanced Kernel Concepts
3************************
4
5.. _kernel-big-picture:
6
7Yocto Project Kernel Development and Maintenance
8================================================
9
10Kernels available through the Yocto Project (Yocto Linux kernels), like
11other kernels, are based off the Linux kernel releases from
12` <http://www.kernel.org>`__. At the beginning of a major Linux kernel
13development cycle, the Yocto Project team chooses a Linux kernel based
14on factors such as release timing, the anticipated release timing of
15final upstream ``kernel.org`` versions, and Yocto Project feature
16requirements. Typically, the Linux kernel chosen is in the final stages
17of development by the Linux community. In other words, the Linux kernel
18is in the release candidate or "rc" phase and has yet to reach final
19release. But, by being in the final stages of external development, the
20team knows that the ``kernel.org`` final release will clearly be within
21the early stages of the Yocto Project development window.
22
23This balance allows the Yocto Project team to deliver the most
24up-to-date Yocto Linux kernel possible, while still ensuring that the
25team has a stable official release for the baseline Linux kernel
26version.
27
28As implied earlier, the ultimate source for Yocto Linux kernels are
29released kernels from ``kernel.org``. In addition to a foundational
30kernel from ``kernel.org``, the available Yocto Linux kernels contain a
31mix of important new mainline developments, non-mainline developments
32(when no alternative exists), Board Support Package (BSP) developments,
33and custom features. These additions result in a commercially released
34Yocto Project Linux kernel that caters to specific embedded designer
35needs for targeted hardware.
36
37You can find a web interface to the Yocto Linux kernels in the `Source
38Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ at
39` <&YOCTO_GIT_URL;>`__. If you look at the interface, you will see to
40the left a grouping of Git repositories titled "Yocto Linux Kernel".
41Within this group, you will find several Linux Yocto kernels developed
42and included with Yocto Project releases:
43
44- *``linux-yocto-4.1``:* The stable Yocto Project kernel to use with
45 the Yocto Project Release 2.0. This kernel is based on the Linux 4.1
46 released kernel.
47
48- *``linux-yocto-4.4``:* The stable Yocto Project kernel to use with
49 the Yocto Project Release 2.1. This kernel is based on the Linux 4.4
50 released kernel.
51
52- *``linux-yocto-4.6``:* A temporary kernel that is not tied to any
53 Yocto Project release.
54
55- *``linux-yocto-4.8``:* The stable yocto Project kernel to use with
56 the Yocto Project Release 2.2.
57
58- *``linux-yocto-4.9``:* The stable Yocto Project kernel to use with
59 the Yocto Project Release 2.3. This kernel is based on the Linux 4.9
60 released kernel.
61
62- *``linux-yocto-4.10``:* The default stable Yocto Project kernel to
63 use with the Yocto Project Release 2.3. This kernel is based on the
64 Linux 4.10 released kernel.
65
66- *``linux-yocto-4.12``:* The default stable Yocto Project kernel to
67 use with the Yocto Project Release 2.4. This kernel is based on the
68 Linux 4.12 released kernel.
69
70- *``yocto-kernel-cache``:* The ``linux-yocto-cache`` contains patches
71 and configurations for the linux-yocto kernel tree. This repository
72 is useful when working on the linux-yocto kernel. For more
73 information on this "Advanced Kernel Metadata", see the "`Working
74 With Advanced Metadata
75 (``yocto-kernel-cache``) <#kernel-dev-advanced>`__" Chapter.
76
77- *``linux-yocto-dev``:* A development kernel based on the latest
78 upstream release candidate available.
79
80.. note::
81
82 Long Term Support Initiative (LTSI) for Yocto Linux kernels is as
83 follows:
84
85 - For Yocto Project releases 1.7, 1.8, and 2.0, the LTSI kernel is
86 ``linux-yocto-3.14``.
87
88 - For Yocto Project releases 2.1, 2.2, and 2.3, the LTSI kernel is
89 ``linux-yocto-4.1``.
90
91 - For Yocto Project release 2.4, the LTSI kernel is
92 ``linux-yocto-4.9``
93
94 - ``linux-yocto-4.4`` is an LTS kernel.
95
96Once a Yocto Linux kernel is officially released, the Yocto Project team
97goes into their next development cycle, or upward revision (uprev)
98cycle, while still continuing maintenance on the released kernel. It is
99important to note that the most sustainable and stable way to include
100feature development upstream is through a kernel uprev process.
101Back-porting hundreds of individual fixes and minor features from
102various kernel versions is not sustainable and can easily compromise
103quality.
104
105During the uprev cycle, the Yocto Project team uses an ongoing analysis
106of Linux kernel development, BSP support, and release timing to select
107the best possible ``kernel.org`` Linux kernel version on which to base
108subsequent Yocto Linux kernel development. The team continually monitors
109Linux community kernel development to look for significant features of
110interest. The team does consider back-porting large features if they
111have a significant advantage. User or community demand can also trigger
112a back-port or creation of new functionality in the Yocto Project
113baseline kernel during the uprev cycle.
114
115Generally speaking, every new Linux kernel both adds features and
116introduces new bugs. These consequences are the basic properties of
117upstream Linux kernel development and are managed by the Yocto Project
118team's Yocto Linux kernel development strategy. It is the Yocto Project
119team's policy to not back-port minor features to the released Yocto
120Linux kernel. They only consider back-porting significant technological
121jumps DASH and, that is done after a complete gap analysis. The reason
122for this policy is that back-porting any small to medium sized change
123from an evolving Linux kernel can easily create mismatches,
124incompatibilities and very subtle errors.
125
126The policies described in this section result in both a stable and a
127cutting edge Yocto Linux kernel that mixes forward ports of existing
128Linux kernel features and significant and critical new functionality.
129Forward porting Linux kernel functionality into the Yocto Linux kernels
130available through the Yocto Project can be thought of as a "micro
131uprev." The many “micro uprevs” produce a Yocto Linux kernel version
132with a mix of important new mainline, non-mainline, BSP developments and
133feature integrations. This Yocto Linux kernel gives insight into new
134features and allows focused amounts of testing to be done on the kernel,
135which prevents surprises when selecting the next major uprev. The
136quality of these cutting edge Yocto Linux kernels is evolving and the
137kernels are used in leading edge feature and BSP development.
138
139Yocto Linux Kernel Architecture and Branching Strategies
140========================================================
141
142As mentioned earlier, a key goal of the Yocto Project is to present the
143developer with a kernel that has a clear and continuous history that is
144visible to the user. The architecture and mechanisms, in particular the
145branching strategies, used achieve that goal in a manner similar to
146upstream Linux kernel development in ``kernel.org``.
147
148You can think of a Yocto Linux kernel as consisting of a baseline Linux
149kernel with added features logically structured on top of the baseline.
150The features are tagged and organized by way of a branching strategy
151implemented by the Yocto Project team using the Source Code Manager
152(SCM) Git.
153
154.. note::
155
156 - Git is the obvious SCM for meeting the Yocto Linux kernel
157 organizational and structural goals described in this section. Not
158 only is Git the SCM for Linux kernel development in ``kernel.org``
159 but, Git continues to grow in popularity and supports many
160 different work flows, front-ends and management techniques.
161
162 - You can find documentation on Git at
163 ` <http://git-scm.com/documentation>`__. You can also get an
164 introduction to Git as it applies to the Yocto Project in the
165 "`Git <&YOCTO_DOCS_OM_URL;#git>`__" section in the Yocto Project
166 Overview and Concepts Manual. The latter reference provides an
167 overview of Git and presents a minimal set of Git commands that
168 allows you to be functional using Git. You can use as much, or as
169 little, of what Git has to offer to accomplish what you need for
170 your project. You do not have to be a "Git Expert" in order to use
171 it with the Yocto Project.
172
173Using Git's tagging and branching features, the Yocto Project team
174creates kernel branches at points where functionality is no longer
175shared and thus, needs to be isolated. For example, board-specific
176incompatibilities would require different functionality and would
177require a branch to separate the features. Likewise, for specific kernel
178features, the same branching strategy is used.
179
180This "tree-like" architecture results in a structure that has features
181organized to be specific for particular functionality, single kernel
182types, or a subset of kernel types. Thus, the user has the ability to
183see the added features and the commits that make up those features. In
184addition to being able to see added features, the user can also view the
185history of what made up the baseline Linux kernel.
186
187Another consequence of this strategy results in not having to store the
188same feature twice internally in the tree. Rather, the kernel team
189stores the unique differences required to apply the feature onto the
190kernel type in question.
191
192.. note::
193
194 The Yocto Project team strives to place features in the tree such
195 that features can be shared by all boards and kernel types where
196 possible. However, during development cycles or when large features
197 are merged, the team cannot always follow this practice. In those
198 cases, the team uses isolated branches to merge features.
199
200BSP-specific code additions are handled in a similar manner to
201kernel-specific additions. Some BSPs only make sense given certain
202kernel types. So, for these types, the team creates branches off the end
203of that kernel type for all of the BSPs that are supported on that
204kernel type. From the perspective of the tools that create the BSP
205branch, the BSP is really no different than a feature. Consequently, the
206same branching strategy applies to BSPs as it does to kernel features.
207So again, rather than store the BSP twice, the team only stores the
208unique differences for the BSP across the supported multiple kernels.
209
210While this strategy can result in a tree with a significant number of
211branches, it is important to realize that from the developer's point of
212view, there is a linear path that travels from the baseline
213``kernel.org``, through a select group of features and ends with their
214BSP-specific commits. In other words, the divisions of the kernel are
215transparent and are not relevant to the developer on a day-to-day basis.
216From the developer's perspective, this path is the "master" branch in
217Git terms. The developer does not need to be aware of the existence of
218any other branches at all. Of course, value exists in the having these
219branches in the tree, should a person decide to explore them. For
220example, a comparison between two BSPs at either the commit level or at
221the line-by-line code ``diff`` level is now a trivial operation.
222
223The following illustration shows the conceptual Yocto Linux kernel.
224
225In the illustration, the "Kernel.org Branch Point" marks the specific
226spot (or Linux kernel release) from which the Yocto Linux kernel is
227created. From this point forward in the tree, features and differences
228are organized and tagged.
229
230The "Yocto Project Baseline Kernel" contains functionality that is
231common to every kernel type and BSP that is organized further along in
232the tree. Placing these common features in the tree this way means
233features do not have to be duplicated along individual branches of the
234tree structure.
235
236From the "Yocto Project Baseline Kernel", branch points represent
237specific functionality for individual Board Support Packages (BSPs) as
238well as real-time kernels. The illustration represents this through
239three BSP-specific branches and a real-time kernel branch. Each branch
240represents some unique functionality for the BSP or for a real-time
241Yocto Linux kernel.
242
243In this example structure, the "Real-time (rt) Kernel" branch has common
244features for all real-time Yocto Linux kernels and contains more
245branches for individual BSP-specific real-time kernels. The illustration
246shows three branches as an example. Each branch points the way to
247specific, unique features for a respective real-time kernel as they
248apply to a given BSP.
249
250The resulting tree structure presents a clear path of markers (or
251branches) to the developer that, for all practical purposes, is the
252Yocto Linux kernel needed for any given set of requirements.
253
254.. note::
255
256 Keep in mind the figure does not take into account all the supported
257 Yocto Linux kernels, but rather shows a single generic kernel just
258 for conceptual purposes. Also keep in mind that this structure
259 represents the Yocto Project
260 Source Repositories
261 that are either pulled from during the build or established on the
262 host development system prior to the build by either cloning a
263 particular kernel's Git repository or by downloading and unpacking a
264 tarball.
265
266Working with the kernel as a structured tree follows recognized
267community best practices. In particular, the kernel as shipped with the
268product, should be considered an "upstream source" and viewed as a
269series of historical and documented modifications (commits). These
270modifications represent the development and stabilization done by the
271Yocto Project kernel development team.
272
273Because commits only change at significant release points in the product
274life cycle, developers can work on a branch created from the last
275relevant commit in the shipped Yocto Project Linux kernel. As mentioned
276previously, the structure is transparent to the developer because the
277kernel tree is left in this state after cloning and building the kernel.
278
279Kernel Build File Hierarchy
280===========================
281
282Upstream storage of all the available kernel source code is one thing,
283while representing and using the code on your host development system is
284another. Conceptually, you can think of the kernel source repositories
285as all the source files necessary for all the supported Yocto Linux
286kernels. As a developer, you are just interested in the source files for
287the kernel on which you are working. And, furthermore, you need them
288available on your host system.
289
290Kernel source code is available on your host system several different
291ways:
292
293- *Files Accessed While using ``devtool``:* ``devtool``, which is
294 available with the Yocto Project, is the preferred method by which to
295 modify the kernel. See the "`Kernel Modification
296 Workflow <#kernel-modification-workflow>`__" section.
297
298- *Cloned Repository:* If you are working in the kernel all the time,
299 you probably would want to set up your own local Git repository of
300 the Yocto Linux kernel tree. For information on how to clone a Yocto
301 Linux kernel Git repository, see the "`Preparing the Build Host to
302 Work on the
303 Kernel <#preparing-the-build-host-to-work-on-the-kernel>`__" section.
304
305- *Temporary Source Files from a Build:* If you just need to make some
306 patches to the kernel using a traditional BitBake workflow (i.e. not
307 using the ``devtool``), you can access temporary kernel source files
308 that were extracted and used during a kernel build.
309
310The temporary kernel source files resulting from a build using BitBake
311have a particular hierarchy. When you build the kernel on your
312development system, all files needed for the build are taken from the
313source repositories pointed to by the
314```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ variable and gathered
315in a temporary work area where they are subsequently used to create the
316unique kernel. Thus, in a sense, the process constructs a local source
317tree specific to your kernel from which to generate the new kernel
318image.
319
320The following figure shows the temporary file structure created on your
321host system when you build the kernel using Bitbake. This `Build
322Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ contains all the
323source files used during the build.
324
325Again, for additional information on the Yocto Project kernel's
326architecture and its branching strategy, see the "`Yocto Linux Kernel
327Architecture and Branching
328Strategies <#yocto-linux-kernel-architecture-and-branching-strategies>`__"
329section. You can also reference the "`Using ``devtool`` to Patch the
330Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
331Kernel Development to Patch the
332Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
333sections for detailed example that modifies the kernel.
334
335Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase
336=======================================================================================
337
338This section describes part of the kernel configuration audit phase that
339most developers can ignore. For general information on kernel
340configuration including ``menuconfig``, ``defconfig`` files, and
341configuration fragments, see the "`Configuring the
342Kernel <#configuring-the-kernel>`__" section.
343
344During this part of the audit phase, the contents of the final
345``.config`` file are compared against the fragments specified by the
346system. These fragments can be system fragments, distro fragments, or
347user-specified configuration elements. Regardless of their origin, the
348OpenEmbedded build system warns the user if a specific option is not
349included in the final kernel configuration.
350
351By default, in order to not overwhelm the user with configuration
352warnings, the system only reports missing "hardware" options as they
353could result in a boot failure or indicate that important hardware is
354not available.
355
356To determine whether or not a given option is "hardware" or
357"non-hardware", the kernel Metadata in ``yocto-kernel-cache`` contains
358files that classify individual or groups of options as either hardware
359or non-hardware. To better show this, consider a situation where the
360``yocto-kernel-cache`` contains the following files:
361yocto-kernel-cache/features/drm-psb/hardware.cfg
362yocto-kernel-cache/features/kgdb/hardware.cfg
363yocto-kernel-cache/ktypes/base/hardware.cfg
364yocto-kernel-cache/bsp/mti-malta32/hardware.cfg
365yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg
366yocto-kernel-cache/bsp/qemuarma9/hardware.cfg
367yocto-kernel-cache/bsp/mti-malta64/hardware.cfg
368yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
369yocto-kernel-cache/bsp/common-pc/hardware.cfg
370yocto-kernel-cache/bsp/common-pc-64/hardware.cfg
371yocto-kernel-cache/features/rfkill/non-hardware.cfg
372yocto-kernel-cache/ktypes/base/non-hardware.cfg
373yocto-kernel-cache/features/aufs/non-hardware.kcf
374yocto-kernel-cache/features/ocf/non-hardware.kcf
375yocto-kernel-cache/ktypes/base/non-hardware.kcf
376yocto-kernel-cache/ktypes/base/hardware.kcf
377yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf The following list
378provides explanations for the various files:
379
380- ``hardware.kcf``: Specifies a list of kernel Kconfig files that
381 contain hardware options only.
382
383- ``non-hardware.kcf``: Specifies a list of kernel Kconfig files that
384 contain non-hardware options only.
385
386- ``hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options that
387 are hardware, regardless of whether or not they are within a Kconfig
388 file specified by a hardware or non-hardware Kconfig file (i.e.
389 ``hardware.kcf`` or ``non-hardware.kcf``).
390
391- ``non-hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options
392 that are not hardware, regardless of whether or not they are within a
393 Kconfig file specified by a hardware or non-hardware Kconfig file
394 (i.e. ``hardware.kcf`` or ``non-hardware.kcf``).
395
396Here is a specific example using the
397``kernel-cache/bsp/mti-malta32/hardware.cfg``: CONFIG_SERIAL_8250
398CONFIG_SERIAL_8250_CONSOLE CONFIG_SERIAL_8250_NR_UARTS
399CONFIG_SERIAL_8250_PCI CONFIG_SERIAL_CORE CONFIG_SERIAL_CORE_CONSOLE
400CONFIG_VGA_ARB The kernel configuration audit automatically detects
401these files (hence the names must be exactly the ones discussed here),
402and uses them as inputs when generating warnings about the final
403``.config`` file.
404
405A user-specified kernel Metadata repository, or recipe space feature,
406can use these same files to classify options that are found within its
407``.cfg`` files as hardware or non-hardware, to prevent the OpenEmbedded
408build system from producing an error or warning when an option is not in
409the final ``.config`` file.
diff --git a/documentation/kernel-dev/kernel-dev-faq.rst b/documentation/kernel-dev/kernel-dev-faq.rst
new file mode 100644
index 0000000000..b852769683
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-faq.rst
@@ -0,0 +1,43 @@
1**********************
2Kernel Development FAQ
3**********************
4
5.. _kernel-dev-faq-section:
6
7Common Questions and Solutions
8==============================
9
10The following lists some solutions for common questions. How do I use my
11own Linux kernel ``.config`` file? Refer to the "`Changing the
12Configuration <#changing-the-configuration>`__" section for information.
13How do I create configuration fragments? Refer to the "`Creating
14Configuration Fragments <#creating-config-fragments>`__" section for
15information. How do I use my own Linux kernel sources? Refer to the
16"`Working With Your Own Sources <#working-with-your-own-sources>`__"
17section for information. How do I install/not-install the kernel image
18on the rootfs? The kernel image (e.g. ``vmlinuz``) is provided by the
19``kernel-image`` package. Image recipes depend on ``kernel-base``. To
20specify whether or not the kernel image is installed in the generated
21root filesystem, override ``RDEPENDS_kernel-base`` to include or not
22include "kernel-image". See the "`Using .bbappend Files in Your
23Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
24Yocto Project Development Tasks Manual for information on how to use an
25append file to override metadata. How do I install a specific kernel
26module? Linux kernel modules are packaged individually. To ensure a
27specific kernel module is included in an image, include it in the
28appropriate machine
29```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__ variable.
30These other variables are useful for installing specific modules:
31```MACHINE_ESSENTIAL_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RDEPENDS>`__
32```MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS>`__
33```MACHINE_EXTRA_RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RDEPENDS>`__
34```MACHINE_EXTRA_RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-MACHINE_EXTRA_RRECOMMENDS>`__
35For example, set the following in the ``qemux86.conf`` file to include
36the ``ab123`` kernel modules with images built for the ``qemux86``
37machine: MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123" For more
38information, see the "`Incorporating Out-of-Tree
39Modules <#incorporating-out-of-tree-modules>`__" section. How do I
40change the Linux kernel command line? The Linux kernel command line is
41typically specified in the machine config using the ``APPEND`` variable.
42For example, you can add some helpful debug information doing the
43following: APPEND += "printk.time=y initcall_debug debug"
diff --git a/documentation/kernel-dev/kernel-dev-intro.rst b/documentation/kernel-dev/kernel-dev-intro.rst
new file mode 100644
index 0000000000..0f87f8cf1d
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-intro.rst
@@ -0,0 +1,178 @@
1************
2Introduction
3************
4
5.. _kernel-dev-overview:
6
7Overview
8========
9
10Regardless of how you intend to make use of the Yocto Project, chances
11are you will work with the Linux kernel. This manual describes how to
12set up your build host to support kernel development, introduces the
13kernel development process, provides background information on the Yocto
14Linux kernel `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__, describes
15common tasks you can perform using the kernel tools, shows you how to
16use the kernel Metadata needed to work with the kernel inside the Yocto
17Project, and provides insight into how the Yocto Project team develops
18and maintains Yocto Linux kernel Git repositories and Metadata.
19
20Each Yocto Project release has a set of Yocto Linux kernel recipes,
21whose Git repositories you can view in the Yocto `Source
22Repositories <&YOCTO_GIT_URL;>`__ under the "Yocto Linux Kernel"
23heading. New recipes for the release track the latest Linux kernel
24upstream developments from ` <http://www.kernel.org>`__ and introduce
25newly-supported platforms. Previous recipes in the release are refreshed
26and supported for at least one additional Yocto Project release. As they
27align, these previous releases are updated to include the latest from
28the Long Term Support Initiative (LTSI) project. You can learn more
29about Yocto Linux kernels and LTSI in the "`Yocto Project Kernel
30Development and Maintenance <#kernel-big-picture>`__" section.
31
32Also included is a Yocto Linux kernel development recipe
33(``linux-yocto-dev.bb``) should you want to work with the very latest in
34upstream Yocto Linux kernel development and kernel Metadata development.
35
36.. note::
37
38 For more on Yocto Linux kernels, see the "
39 Yocto Project Kernel Development and Maintenance
40 section.
41
42The Yocto Project also provides a powerful set of kernel tools for
43managing Yocto Linux kernel sources and configuration data. You can use
44these tools to make a single configuration change, apply multiple
45patches, or work with your own kernel sources.
46
47In particular, the kernel tools allow you to generate configuration
48fragments that specify only what you must, and nothing more.
49Configuration fragments only need to contain the highest level visible
50``CONFIG`` options as presented by the Yocto Linux kernel ``menuconfig``
51system. Contrast this against a complete Yocto Linux kernel ``.config``
52file, which includes all the automatically selected ``CONFIG`` options.
53This efficiency reduces your maintenance effort and allows you to
54further separate your configuration in ways that make sense for your
55project. A common split separates policy and hardware. For example, all
56your kernels might support the ``proc`` and ``sys`` filesystems, but
57only specific boards require sound, USB, or specific drivers. Specifying
58these configurations individually allows you to aggregate them together
59as needed, but maintains them in only one place. Similar logic applies
60to separating source changes.
61
62If you do not maintain your own kernel sources and need to make only
63minimal changes to the sources, the released recipes provide a vetted
64base upon which to layer your changes. Doing so allows you to benefit
65from the continual kernel integration and testing performed during
66development of the Yocto Project.
67
68If, instead, you have a very specific Linux kernel source tree and are
69unable to align with one of the official Yocto Linux kernel recipes, an
70alternative exists by which you can use the Yocto Project Linux kernel
71tools with your own kernel sources.
72
73The remainder of this manual provides instructions for completing
74specific Linux kernel development tasks. These instructions assume you
75are comfortable working with
76`BitBake <http://openembedded.org/wiki/Bitbake>`__ recipes and basic
77open-source development tools. Understanding these concepts will
78facilitate the process of working with the kernel recipes. If you find
79you need some additional background, please be sure to review and
80understand the following documentation:
81
82- `Yocto Project Quick Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
83
84- `Yocto Project Overview and Concepts Manual <&YOCTO_DOCS_OM_URL;>`__.
85
86- ```devtool``
87 workflow <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__
88 as described in the Yocto Project Application Development and the
89 Extensible Software Development Kit (eSDK) manual.
90
91- The "`Understanding and Creating
92 Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
93 section in the Yocto Project Development Tasks Manual.
94
95- The "`Kernel Modification
96 Workflow <#kernel-modification-workflow>`__" section.
97
98Kernel Modification Workflow
99============================
100
101Kernel modification involves changing the Yocto Project kernel, which
102could involve changing configuration options as well as adding new
103kernel recipes. Configuration changes can be added in the form of
104configuration fragments, while recipe modification comes through the
105kernel's ``recipes-kernel`` area in a kernel layer you create.
106
107This section presents a high-level overview of the Yocto Project kernel
108modification workflow. The illustration and accompanying list provide
109general information and references for further information.
110
1111. *Set up Your Host Development System to Support Development Using the
112 Yocto Project*: See the "`Setting Up the Development Host to Use the
113 Yocto Project <&YOCTO_DOCS_DEV_URL;#dev-manual-start>`__" section in
114 the Yocto Project Development Tasks Manual for options on how to get
115 a build host ready to use the Yocto Project.
116
1172. *Set Up Your Host Development System for Kernel Development:* It is
118 recommended that you use ``devtool`` and an extensible SDK for kernel
119 development. Alternatively, you can use traditional kernel
120 development methods with the Yocto Project. Either way, there are
121 steps you need to take to get the development environment ready.
122
123 Using ``devtool`` and the eSDK requires that you have a clean build
124 of the image and that you are set up with the appropriate eSDK. For
125 more information, see the "`Getting Ready to Develop Using
126 ``devtool`` <#getting-ready-to-develop-using-devtool>`__" section.
127
128 Using traditional kernel development requires that you have the
129 kernel source available in an isolated local Git repository. For more
130 information, see the "`Getting Ready for Traditional Kernel
131 Development <#getting-ready-for-traditional-kernel-development>`__"
132 section.
133
1343. *Make Changes to the Kernel Source Code if applicable:* Modifying the
135 kernel does not always mean directly changing source files. However,
136 if you have to do this, you make the changes to the files in the
137 eSDK's Build Directory if you are using ``devtool``. For more
138 information, see the "`Using ``devtool`` to Patch the
139 Kernel <#using-devtool-to-patch-the-kernel>`__" section.
140
141 If you are using traditional kernel development, you edit the source
142 files in the kernel's local Git repository. For more information, see
143 the "`Using Traditional Kernel Development to Patch the
144 Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
145 section.
146
1474. *Make Kernel Configuration Changes if Applicable:* If your situation
148 calls for changing the kernel's configuration, you can use
149 ```menuconfig`` <#using-menuconfig>`__, which allows you to
150 interactively develop and test the configuration changes you are
151 making to the kernel. Saving changes you make with ``menuconfig``
152 updates the kernel's ``.config`` file.
153
154 .. note::
155
156 Try to resist the temptation to directly edit an existing
157 .config
158 file, which is found in the Build Directory among the source code
159 used for the build. Doing so, can produce unexpected results when
160 the OpenEmbedded build system regenerates the configuration file.
161
162 Once you are satisfied with the configuration changes made using
163 ``menuconfig`` and you have saved them, you can directly compare the
164 resulting ``.config`` file against an existing original and gather
165 those changes into a `configuration fragment
166 file <#creating-config-fragments>`__ to be referenced from within the
167 kernel's ``.bbappend`` file.
168
169 Additionally, if you are working in a BSP layer and need to modify
170 the BSP's kernel's configuration, you can use ``menuconfig``.
171
1725. *Rebuild the Kernel Image With Your Changes:* Rebuilding the kernel
173 image applies your changes. Depending on your target hardware, you
174 can verify your changes on actual hardware or perhaps QEMU.
175
176The remainder of this developer's guide covers common tasks typically
177used during kernel development, advanced Metadata usage, and Yocto Linux
178kernel maintenance concepts.
diff --git a/documentation/kernel-dev/kernel-dev-maint-appx.rst b/documentation/kernel-dev/kernel-dev-maint-appx.rst
new file mode 100644
index 0000000000..4653e51eba
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-maint-appx.rst
@@ -0,0 +1,226 @@
1******************
2Kernel Maintenance
3******************
4
5Tree Construction
6=================
7
8This section describes construction of the Yocto Project kernel source
9repositories as accomplished by the Yocto Project team to create Yocto
10Linux kernel repositories. These kernel repositories are found under the
11heading "Yocto Linux Kernel" at `YOCTO_GIT_URL <&YOCTO_GIT_URL;>`__ and
12are shipped as part of a Yocto Project release. The team creates these
13repositories by compiling and executing the set of feature descriptions
14for every BSP and feature in the product. Those feature descriptions
15list all necessary patches, configurations, branches, tags, and feature
16divisions found in a Yocto Linux kernel. Thus, the Yocto Project Linux
17kernel repository (or tree) and accompanying Metadata in the
18``yocto-kernel-cache`` are built.
19
20The existence of these repositories allow you to access and clone a
21particular Yocto Project Linux kernel repository and use it to build
22images based on their configurations and features.
23
24You can find the files used to describe all the valid features and BSPs
25in the Yocto Project Linux kernel in any clone of the Yocto Project
26Linux kernel source repository and ``yocto-kernel-cache`` Git trees. For
27example, the following commands clone the Yocto Project baseline Linux
28kernel that branches off ``linux.org`` version 4.12 and the
29``yocto-kernel-cache``, which contains stores of kernel Metadata: $ git
30clone git://git.yoctoproject.org/linux-yocto-4.12 $ git clone
31git://git.yoctoproject.org/linux-kernel-cache For more information on
32how to set up a local Git repository of the Yocto Project Linux kernel
33files, see the "`Preparing the Build Host to Work on the
34Kernel <#preparing-the-build-host-to-work-on-the-kernel>`__" section.
35
36Once you have cloned the kernel Git repository and the cache of Metadata
37on your local machine, you can discover the branches that are available
38in the repository using the following Git command: $ git branch -a
39Checking out a branch allows you to work with a particular Yocto Linux
40kernel. For example, the following commands check out the
41"standard/beagleboard" branch of the Yocto Linux kernel repository and
42the "yocto-4.12" branch of the ``yocto-kernel-cache`` repository: $ cd
43~/linux-yocto-4.12 $ git checkout -b my-kernel-4.12
44remotes/origin/standard/beagleboard $ cd ~/linux-kernel-cache $ git
45checkout -b my-4.12-metadata remotes/origin/yocto-4.12
46
47.. note::
48
49 Branches in the
50 yocto-kernel-cache
51 repository correspond to Yocto Linux kernel versions (e.g.
52 "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
53
54Once you have checked out and switched to appropriate branches, you can
55see a snapshot of all the kernel source files used to used to build that
56particular Yocto Linux kernel for a particular board.
57
58To see the features and configurations for a particular Yocto Linux
59kernel, you need to examine the ``yocto-kernel-cache`` Git repository.
60As mentioned, branches in the ``yocto-kernel-cache`` repository
61correspond to Yocto Linux kernel versions (e.g. ``yocto-4.12``).
62Branches contain descriptions in the form of ``.scc`` and ``.cfg``
63files.
64
65You should realize, however, that browsing your local
66``yocto-kernel-cache`` repository for feature descriptions and patches
67is not an effective way to determine what is in a particular kernel
68branch. Instead, you should use Git directly to discover the changes in
69a branch. Using Git is an efficient and flexible way to inspect changes
70to the kernel.
71
72.. note::
73
74 Ground up reconstruction of the complete kernel tree is an action
75 only taken by the Yocto Project team during an active development
76 cycle. When you create a clone of the kernel Git repository, you are
77 simply making it efficiently available for building and development.
78
79The following steps describe what happens when the Yocto Project Team
80constructs the Yocto Project kernel source Git repository (or tree)
81found at ` <&YOCTO_GIT_URL;>`__ given the introduction of a new
82top-level kernel feature or BSP. The following actions effectively
83provide the Metadata and create the tree that includes the new feature,
84patch, or BSP:
85
861. *Pass Feature to the OpenEmbedded Build System:* A top-level kernel
87 feature is passed to the kernel build subsystem. Normally, this
88 feature is a BSP for a particular kernel type.
89
902. *Locate Feature:* The file that describes the top-level feature is
91 located by searching these system directories:
92
93 - The in-tree kernel-cache directories, which are located in the
94 ```yocto-kernel-cache`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache/tree/bsp>`__
95 repository organized under the "Yocto Linux Kernel" heading in the
96 `Yocto Project Source
97 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
98
99 - Areas pointed to by ``SRC_URI`` statements found in kernel recipes
100
101 For a typical build, the target of the search is a feature
102 description in an ``.scc`` file whose name follows this format (e.g.
103 ``beaglebone-standard.scc`` and ``beaglebone-preempt-rt.scc``):
104 bsp_root_name-kernel_type.scc
105
1063. *Expand Feature:* Once located, the feature description is either
107 expanded into a simple script of actions, or into an existing
108 equivalent script that is already part of the shipped kernel.
109
1104. *Append Extra Features:* Extra features are appended to the top-level
111 feature description. These features can come from the
112 ```KERNEL_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_FEATURES>`__
113 variable in recipes.
114
1155. *Locate, Expand, and Append Each Feature:* Each extra feature is
116 located, expanded and appended to the script as described in step
117 three.
118
1196. *Execute the Script:* The script is executed to produce files
120 ``.scc`` and ``.cfg`` files in appropriate directories of the
121 ``yocto-kernel-cache`` repository. These files are descriptions of
122 all the branches, tags, patches and configurations that need to be
123 applied to the base Git repository to completely create the source
124 (build) branch for the new BSP or feature.
125
1267. *Clone Base Repository:* The base repository is cloned, and the
127 actions listed in the ``yocto-kernel-cache`` directories are applied
128 to the tree.
129
1308. *Perform Cleanup:* The Git repositories are left with the desired
131 branches checked out and any required branching, patching and tagging
132 has been performed.
133
134The kernel tree and cache are ready for developer consumption to be
135locally cloned, configured, and built into a Yocto Project kernel
136specific to some target hardware.
137
138.. note::
139
140 - The generated ``yocto-kernel-cache`` repository adds to the kernel
141 as shipped with the Yocto Project release. Any add-ons and
142 configuration data are applied to the end of an existing branch.
143 The full repository generation that is found in the official Yocto
144 Project kernel repositories at
145 `http://git.yoctoproject.org <&YOCTO_GIT_URL;>`__ is the
146 combination of all supported boards and configurations.
147
148 - The technique the Yocto Project team uses is flexible and allows
149 for seamless blending of an immutable history with additional
150 patches specific to a deployment. Any additions to the kernel
151 become an integrated part of the branches.
152
153 - The full kernel tree that you see on ` <&YOCTO_GIT_URL;>`__ is
154 generated through repeating the above steps for all valid BSPs.
155 The end result is a branched, clean history tree that makes up the
156 kernel for a given release. You can see the script (``kgit-scc``)
157 responsible for this in the
158 ```yocto-kernel-tools`` <&YOCTO_GIT_URL;/cgit.cgi/yocto-kernel-tools/tree/tools>`__
159 repository.
160
161 - The steps used to construct the full kernel tree are the same
162 steps that BitBake uses when it builds a kernel image.
163
164Build Strategy
165==============
166
167Once you have cloned a Yocto Linux kernel repository and the cache
168repository (``yocto-kernel-cache``) onto your development system, you
169can consider the compilation phase of kernel development, which is
170building a kernel image. Some prerequisites exist that are validated by
171the build process before compilation starts:
172
173- The ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ points to the
174 kernel Git repository.
175
176- A BSP build branch with Metadata exists in the ``yocto-kernel-cache``
177 repository. The branch is based on the Yocto Linux kernel version and
178 has configurations and features grouped under the
179 ``yocto-kernel-cache/bsp`` directory. For example, features and
180 configurations for the BeagleBone Board assuming a
181 ``linux-yocto_4.12`` kernel reside in the following area of the
182 ``yocto-kernel-cache`` repository: yocto-kernel-cache/bsp/beaglebone
183
184 .. note::
185
186 In the previous example, the "yocto-4.12" branch is checked out in
187 the
188 yocto-kernel-cache
189 repository.
190
191The OpenEmbedded build system makes sure these conditions exist before
192attempting compilation. Other means, however, do exist, such as as
193bootstrapping a BSP.
194
195Before building a kernel, the build process verifies the tree and
196configures the kernel by processing all of the configuration "fragments"
197specified by feature descriptions in the ``.scc`` files. As the features
198are compiled, associated kernel configuration fragments are noted and
199recorded in the series of directories in their compilation order. The
200fragments are migrated, pre-processed and passed to the Linux Kernel
201Configuration subsystem (``lkc``) as raw input in the form of a
202``.config`` file. The ``lkc`` uses its own internal dependency
203constraints to do the final processing of that information and generates
204the final ``.config`` file that is used during compilation.
205
206Using the board's architecture and other relevant values from the
207board's template, kernel compilation is started and a kernel image is
208produced.
209
210The other thing that you notice once you configure a kernel is that the
211build process generates a build tree that is separate from your kernel's
212local Git source repository tree. This build tree has a name that uses
213the following form, where ``${MACHINE}`` is the metadata name of the
214machine (BSP) and "kernel_type" is one of the Yocto Project supported
215kernel types (e.g. "standard"): linux-${MACHINE}-kernel_type-build
216
217The existing support in the ``kernel.org`` tree achieves this default
218functionality.
219
220This behavior means that all the generated files for a particular
221machine or BSP are now in the build tree directory. The files include
222the final ``.config`` file, all the ``.o`` files, the ``.a`` files, and
223so forth. Since each machine or BSP has its own separate `Build
224Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ in its own separate
225branch of the Git repository, you can easily switch between different
226builds.
diff --git a/documentation/kernel-dev/kernel-dev.rst b/documentation/kernel-dev/kernel-dev.rst
new file mode 100644
index 0000000000..734ed1881c
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev.rst
@@ -0,0 +1,14 @@
1=============================================
2Yocto Project Linux Kernel Development Manual
3=============================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 kernel-dev-intro
10 kernel-dev-common
11 kernel-dev-advanced
12 kernel-dev-concepts-appx
13 kernel-dev-maint-appx
14 kernel-dev-faq
diff --git a/documentation/overview-manual/overview-manual-concepts.rst b/documentation/overview-manual/overview-manual-concepts.rst
new file mode 100644
index 0000000000..b37adbbba6
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-concepts.rst
@@ -0,0 +1,2133 @@
1**********************
2Yocto Project Concepts
3**********************
4
5This chapter provides explanations for Yocto Project concepts that go
6beyond the surface of "how-to" information and reference (or look-up)
7material. Concepts such as components, the `OpenEmbedded build
8system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ workflow,
9cross-development toolchains, shared state cache, and so forth are
10explained.
11
12Yocto Project Components
13========================
14
15The `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ task executor
16together with various types of configuration files form the
17`OpenEmbedded-Core <&YOCTO_DOCS_REF_URL;#oe-core>`__. This section
18overviews these components by describing their use and how they
19interact.
20
21BitBake handles the parsing and execution of the data files. The data
22itself is of various types:
23
24- *Recipes:* Provides details about particular pieces of software.
25
26- *Class Data:* Abstracts common build information (e.g. how to build a
27 Linux kernel).
28
29- *Configuration Data:* Defines machine-specific settings, policy
30 decisions, and so forth. Configuration data acts as the glue to bind
31 everything together.
32
33BitBake knows how to combine multiple data sources together and refers
34to each data source as a layer. For information on layers, see the
35"`Understanding and Creating
36Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
37section of the Yocto Project Development Tasks Manual.
38
39Following are some brief details on these core components. For
40additional information on how these components interact during a build,
41see the "`OpenEmbedded Build System
42Concepts <#openembedded-build-system-build-concepts>`__" section.
43
44.. _usingpoky-components-bitbake:
45
46BitBake
47-------
48
49BitBake is the tool at the heart of the `OpenEmbedded build
50system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ and is responsible
51for parsing the `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__, generating
52a list of tasks from it, and then executing those tasks.
53
54This section briefly introduces BitBake. If you want more information on
55BitBake, see the `BitBake User Manual <&YOCTO_DOCS_BB_URL;>`__.
56
57To see a list of the options BitBake supports, use either of the
58following commands: $ bitbake -h $ bitbake --help
59
60The most common usage for BitBake is ``bitbake packagename``, where
61``packagename`` is the name of the package you want to build (referred
62to as the "target"). The target often equates to the first part of a
63recipe's filename (e.g. "foo" for a recipe named ``foo_1.3.0-r0.bb``).
64So, to process the ``matchbox-desktop_1.2.3.bb`` recipe file, you might
65type the following: $ bitbake matchbox-desktop Several different
66versions of ``matchbox-desktop`` might exist. BitBake chooses the one
67selected by the distribution configuration. You can get more details
68about how BitBake chooses between different target versions and
69providers in the
70"`Preferences <&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences>`__" section
71of the BitBake User Manual.
72
73BitBake also tries to execute any dependent tasks first. So for example,
74before building ``matchbox-desktop``, BitBake would build a cross
75compiler and ``glibc`` if they had not already been built.
76
77A useful BitBake option to consider is the ``-k`` or ``--continue``
78option. This option instructs BitBake to try and continue processing the
79job as long as possible even after encountering an error. When an error
80occurs, the target that failed and those that depend on it cannot be
81remade. However, when you use this option other dependencies can still
82be processed.
83
84.. _overview-components-recipes:
85
86Recipes
87-------
88
89Files that have the ``.bb`` suffix are "recipes" files. In general, a
90recipe contains information about a single piece of software. This
91information includes the location from which to download the unaltered
92source, any source patches to be applied to that source (if needed),
93which special configuration options to apply, how to compile the source
94files, and how to package the compiled output.
95
96The term "package" is sometimes used to refer to recipes. However, since
97the word "package" is used for the packaged output from the OpenEmbedded
98build system (i.e. ``.ipk`` or ``.deb`` files), this document avoids
99using the term "package" when referring to recipes.
100
101.. _overview-components-classes:
102
103Classes
104-------
105
106Class files (``.bbclass``) contain information that is useful to share
107between recipes files. An example is the
108```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class,
109which contains common settings for any application that Autotools uses.
110The "`Classes <&YOCTO_DOCS_REF_URL;#ref-classes>`__" chapter in the
111Yocto Project Reference Manual provides details about classes and how to
112use them.
113
114.. _overview-components-configurations:
115
116Configurations
117--------------
118
119The configuration files (``.conf``) define various configuration
120variables that govern the OpenEmbedded build process. These files fall
121into several areas that define machine configuration options,
122distribution configuration options, compiler tuning options, general
123common configuration options, and user configuration options in
124``conf/local.conf``, which is found in the `Build
125Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
126
127.. _overview-layers:
128
129Layers
130======
131
132Layers are repositories that contain related metadata (i.e. sets of
133instructions) that tell the OpenEmbedded build system how to build a
134target. Yocto Project's `layer model <#the-yocto-project-layer-model>`__
135facilitates collaboration, sharing, customization, and reuse within the
136Yocto Project development environment. Layers logically separate
137information for your project. For example, you can use a layer to hold
138all the configurations for a particular piece of hardware. Isolating
139hardware-specific configurations allows you to share other metadata by
140using a different layer where that metadata might be common across
141several pieces of hardware.
142
143Many layers exist that work in the Yocto Project development
144environment. The `Yocto Project Curated Layer
145Index <https://caffelli-staging.yoctoproject.org/software-overview/layers/>`__
146and `OpenEmbedded Layer
147Index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__
148both contain layers from which you can use or leverage.
149
150By convention, layers in the Yocto Project follow a specific form.
151Conforming to a known structure allows BitBake to make assumptions
152during builds on where to find types of metadata. You can find
153procedures and learn about tools (i.e. ``bitbake-layers``) for creating
154layers suitable for the Yocto Project in the "`Understanding and
155Creating
156Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
157section of the Yocto Project Development Tasks Manual.
158
159.. _openembedded-build-system-build-concepts:
160
161OpenEmbedded Build System Concepts
162==================================
163
164This section takes a more detailed look inside the build process used by
165the `OpenEmbedded build
166system <&YOCTO_DOCS_REF_URL;#build-system-term>`__, which is the build
167system specific to the Yocto Project. At the heart of the build system
168is BitBake, the task executor.
169
170The following diagram represents the high-level workflow of a build. The
171remainder of this section expands on the fundamental input, output,
172process, and metadata logical blocks that make up the workflow.
173
174In general, the build's workflow consists of several functional areas:
175
176- *User Configuration:* metadata you can use to control the build
177 process.
178
179- *Metadata Layers:* Various layers that provide software, machine, and
180 distro metadata.
181
182- *Source Files:* Upstream releases, local projects, and SCMs.
183
184- *Build System:* Processes under the control of
185 `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__. This block expands
186 on how BitBake fetches source, applies patches, completes
187 compilation, analyzes output for package generation, creates and
188 tests packages, generates images, and generates cross-development
189 tools.
190
191- *Package Feeds:* Directories containing output packages (RPM, DEB or
192 IPK), which are subsequently used in the construction of an image or
193 Software Development Kit (SDK), produced by the build system. These
194 feeds can also be copied and shared using a web server or other means
195 to facilitate extending or updating existing images on devices at
196 runtime if runtime package management is enabled.
197
198- *Images:* Images produced by the workflow.
199
200- *Application Development SDK:* Cross-development tools that are
201 produced along with an image or separately with BitBake.
202
203User Configuration
204------------------
205
206User configuration helps define the build. Through user configuration,
207you can tell BitBake the target architecture for which you are building
208the image, where to store downloaded source, and other build properties.
209
210The following figure shows an expanded representation of the "User
211Configuration" box of the `general workflow
212figure <#general-workflow-figure>`__:
213
214BitBake needs some basic configuration files in order to complete a
215build. These files are ``*.conf`` files. The minimally necessary ones
216reside as example files in the ``build/conf`` directory of the `Source
217Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. For simplicity,
218this section refers to the Source Directory as the "Poky Directory."
219
220When you clone the `Poky <&YOCTO_DOCS_REF_URL;#poky>`__ Git repository
221or you download and unpack a Yocto Project release, you can set up the
222Source Directory to be named anything you want. For this discussion, the
223cloned repository uses the default name ``poky``.
224
225.. note::
226
227 The Poky repository is primarily an aggregation of existing
228 repositories. It is not a canonical upstream source.
229
230The ``meta-poky`` layer inside Poky contains a ``conf`` directory that
231has example configuration files. These example files are used as a basis
232for creating actual configuration files when you source
233````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__, which is the
234build environment script.
235
236Sourcing the build environment script creates a `Build
237Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ if one does not
238already exist. BitBake uses the Build Directory for all its work during
239builds. The Build Directory has a ``conf`` directory that contains
240default versions of your ``local.conf`` and ``bblayers.conf``
241configuration files. These default configuration files are created only
242if versions do not already exist in the Build Directory at the time you
243source the build environment setup script.
244
245Because the Poky repository is fundamentally an aggregation of existing
246repositories, some users might be familiar with running the ```` script
247in the context of separate
248`OpenEmbedded-Core <&YOCTO_DOCS_REF_URL;#oe-core>`__ and BitBake
249repositories rather than a single Poky repository. This discussion
250assumes the script is executed from within a cloned or unpacked version
251of Poky.
252
253Depending on where the script is sourced, different sub-scripts are
254called to set up the Build Directory (Yocto or OpenEmbedded).
255Specifically, the script ``scripts/oe-setup-builddir`` inside the poky
256directory sets up the Build Directory and seeds the directory (if
257necessary) with configuration files appropriate for the Yocto Project
258development environment.
259
260.. note::
261
262 The
263 scripts/oe-setup-builddir
264 script uses the
265 $TEMPLATECONF
266 variable to determine which sample configuration files to locate.
267
268The ``local.conf`` file provides many basic variables that define a
269build environment. Here is a list of a few. To see the default
270configurations in a ``local.conf`` file created by the build environment
271script, see the
272```local.conf.sample`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample>`__
273in the ``meta-poky`` layer:
274
275- *Target Machine Selection:* Controlled by the
276 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable.
277
278- *Download Directory:* Controlled by the
279 ```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ variable.
280
281- *Shared State Directory:* Controlled by the
282 ```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ variable.
283
284- *Build Output:* Controlled by the
285 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ variable.
286
287- *Distribution Policy:* Controlled by the
288 ```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__ variable.
289
290- *Packaging Format:* Controlled by the
291 ```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
292 variable.
293
294- *SDK Target Architecture:* Controlled by the
295 ```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__ variable.
296
297- *Extra Image Packages:* Controlled by the
298 ```EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES>`__
299 variable.
300
301.. note::
302
303 Configurations set in the
304 conf/local.conf
305 file can also be set in the
306 conf/site.conf
307 and
308 conf/auto.conf
309 configuration files.
310
311The ``bblayers.conf`` file tells BitBake what layers you want considered
312during the build. By default, the layers listed in this file include
313layers minimally needed by the build system. However, you must manually
314add any custom layers you have created. You can find more information on
315working with the ``bblayers.conf`` file in the "`Enabling Your
316Layer <&YOCTO_DOCS_DEV_URL;#enabling-your-layer>`__" section in the
317Yocto Project Development Tasks Manual.
318
319The files ``site.conf`` and ``auto.conf`` are not created by the
320environment initialization script. If you want the ``site.conf`` file,
321you need to create that yourself. The ``auto.conf`` file is typically
322created by an autobuilder:
323
324- *``site.conf``:* You can use the ``conf/site.conf`` configuration
325 file to configure multiple build directories. For example, suppose
326 you had several build environments and they shared some common
327 features. You can set these default build properties here. A good
328 example is perhaps the packaging format to use through the
329 ```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
330 variable.
331
332 One useful scenario for using the ``conf/site.conf`` file is to
333 extend your ```BBPATH`` <&YOCTO_DOCS_REF_URL;#var-BBPATH>`__ variable
334 to include the path to a ``conf/site.conf``. Then, when BitBake looks
335 for Metadata using ``BBPATH``, it finds the ``conf/site.conf`` file
336 and applies your common configurations found in the file. To override
337 configurations in a particular build directory, alter the similar
338 configurations within that build directory's ``conf/local.conf``
339 file.
340
341- *``auto.conf``:* The file is usually created and written to by an
342 autobuilder. The settings put into the file are typically the same as
343 you would find in the ``conf/local.conf`` or the ``conf/site.conf``
344 files.
345
346You can edit all configuration files to further define any particular
347build environment. This process is represented by the "User
348Configuration Edits" box in the figure.
349
350When you launch your build with the ``bitbake target`` command, BitBake
351sorts out the configurations to ultimately define your build
352environment. It is important to understand that the `OpenEmbedded build
353system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ reads the
354configuration files in a specific order: ``site.conf``, ``auto.conf``,
355and ``local.conf``. And, the build system applies the normal assignment
356statement rules as described in the "`Syntax and
357Operators <&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata>`__" chapter
358of the BitBake User Manual. Because the files are parsed in a specific
359order, variable assignments for the same variable could be affected. For
360example, if the ``auto.conf`` file and the ``local.conf`` set variable1
361to different values, because the build system parses ``local.conf``
362after ``auto.conf``, variable1 is assigned the value from the
363``local.conf`` file.
364
365Metadata, Machine Configuration, and Policy Configuration
366---------------------------------------------------------
367
368The previous section described the user configurations that define
369BitBake's global behavior. This section takes a closer look at the
370layers the build system uses to further control the build. These layers
371provide Metadata for the software, machine, and policies.
372
373In general, three types of layer input exists. You can see them below
374the "User Configuration" box in the `general workflow
375figure <#general-workflow-figure>`__:
376
377- *Metadata (``.bb`` + Patches):* Software layers containing
378 user-supplied recipe files, patches, and append files. A good example
379 of a software layer might be the
380 ```meta-qt5`` <https://github.com/meta-qt5/meta-qt5>`__ layer from
381 the `OpenEmbedded Layer
382 Index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
383 This layer is for version 5.0 of the popular
384 `Qt <https://wiki.qt.io/About_Qt>`__ cross-platform application
385 development framework for desktop, embedded and mobile.
386
387- *Machine BSP Configuration:* Board Support Package (BSP) layers (i.e.
388 "BSP Layer" in the following figure) providing machine-specific
389 configurations. This type of information is specific to a particular
390 target architecture. A good example of a BSP layer from the `Poky
391 Reference Distribution <#gs-reference-distribution-poky>`__ is the
392 ```meta-yocto-bsp`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp>`__
393 layer.
394
395- *Policy Configuration:* Distribution Layers (i.e. "Distro Layer" in
396 the following figure) providing top-level or general policies for the
397 images or SDKs being built for a particular distribution. For
398 example, in the Poky Reference Distribution the distro layer is the
399 ```meta-poky`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky>`__
400 layer. Within the distro layer is a ``conf/distro`` directory that
401 contains distro configuration files (e.g.
402 ```poky.conf`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky/conf/distro/poky.conf>`__
403 that contain many policy configurations for the Poky distribution.
404
405The following figure shows an expanded representation of these three
406layers from the `general workflow figure <#general-workflow-figure>`__:
407
408In general, all layers have a similar structure. They all contain a
409licensing file (e.g. ``COPYING.MIT``) if the layer is to be distributed,
410a ``README`` file as good practice and especially if the layer is to be
411distributed, a configuration directory, and recipe directories. You can
412learn about the general structure for layers used with the Yocto Project
413in the "`Creating Your Own
414Layer <&YOCTO_DOCS_DEV_URL;#creating-your-own-layer>`__" section in the
415Yocto Project Development Tasks Manual. For a general discussion on
416layers and the many layers from which you can draw, see the
417"`Layers <#overview-layers>`__" and "`The Yocto Project Layer
418Model <#the-yocto-project-layer-model>`__" sections both earlier in this
419manual.
420
421If you explored the previous links, you discovered some areas where many
422layers that work with the Yocto Project exist. The `Source
423Repositories <http://git.yoctoproject.org/>`__ also shows layers
424categorized under "Yocto Metadata Layers."
425
426.. note::
427
428 Layers exist in the Yocto Project Source Repositories that cannot be
429 found in the OpenEmbedded Layer Index. These layers are either
430 deprecated or experimental in nature.
431
432BitBake uses the ``conf/bblayers.conf`` file, which is part of the user
433configuration, to find what layers it should be using as part of the
434build.
435
436Distro Layer
437~~~~~~~~~~~~
438
439The distribution layer provides policy configurations for your
440distribution. Best practices dictate that you isolate these types of
441configurations into their own layer. Settings you provide in
442``conf/distro/distro.conf`` override similar settings that BitBake finds
443in your ``conf/local.conf`` file in the Build Directory.
444
445The following list provides some explanation and references for what you
446typically find in the distribution layer:
447
448- *classes:* Class files (``.bbclass``) hold common functionality that
449 can be shared among recipes in the distribution. When your recipes
450 inherit a class, they take on the settings and functions for that
451 class. You can read more about class files in the
452 "`Classes <&YOCTO_DOCS_REF_URL;#ref-classes>`__" chapter of the Yocto
453 Reference Manual.
454
455- *conf:* This area holds configuration files for the layer
456 (``conf/layer.conf``), the distribution
457 (``conf/distro/distro.conf``), and any distribution-wide include
458 files.
459
460- *recipes-*:* Recipes and append files that affect common
461 functionality across the distribution. This area could include
462 recipes and append files to add distribution-specific configuration,
463 initialization scripts, custom image recipes, and so forth. Examples
464 of ``recipes-*`` directories are ``recipes-core`` and
465 ``recipes-extra``. Hierarchy and contents within a ``recipes-*``
466 directory can vary. Generally, these directories contain recipe files
467 (``*.bb``), recipe append files (``*.bbappend``), directories that
468 are distro-specific for configuration files, and so forth.
469
470BSP Layer
471~~~~~~~~~
472
473The BSP Layer provides machine configurations that target specific
474hardware. Everything in this layer is specific to the machine for which
475you are building the image or the SDK. A common structure or form is
476defined for BSP layers. You can learn more about this structure in the
477`Yocto Project Board Support Package (BSP) Developer's
478Guide <&YOCTO_DOCS_BSP_URL;>`__.
479
480.. note::
481
482 In order for a BSP layer to be considered compliant with the Yocto
483 Project, it must meet some structural requirements.
484
485The BSP Layer's configuration directory contains configuration files for
486the machine (``conf/machine/machine.conf``) and, of course, the layer
487(``conf/layer.conf``).
488
489The remainder of the layer is dedicated to specific recipes by function:
490``recipes-bsp``, ``recipes-core``, ``recipes-graphics``,
491``recipes-kernel``, and so forth. Metadata can exist for multiple
492formfactors, graphics support systems, and so forth.
493
494.. note::
495
496 While the figure shows several
497 recipes-\*
498 directories, not all these directories appear in all BSP layers.
499
500Software Layer
501~~~~~~~~~~~~~~
502
503The software layer provides the Metadata for additional software
504packages used during the build. This layer does not include Metadata
505that is specific to the distribution or the machine, which are found in
506their respective layers.
507
508This layer contains any recipes, append files, and patches, that your
509project needs.
510
511.. _sources-dev-environment:
512
513Sources
514-------
515
516In order for the OpenEmbedded build system to create an image or any
517target, it must be able to access source files. The `general workflow
518figure <#general-workflow-figure>`__ represents source files using the
519"Upstream Project Releases", "Local Projects", and "SCMs (optional)"
520boxes. The figure represents mirrors, which also play a role in locating
521source files, with the "Source Materials" box.
522
523The method by which source files are ultimately organized is a function
524of the project. For example, for released software, projects tend to use
525tarballs or other archived files that can capture the state of a release
526guaranteeing that it is statically represented. On the other hand, for a
527project that is more dynamic or experimental in nature, a project might
528keep source files in a repository controlled by a Source Control Manager
529(SCM) such as Git. Pulling source from a repository allows you to
530control the point in the repository (the revision) from which you want
531to build software. Finally, a combination of the two might exist, which
532would give the consumer a choice when deciding where to get source
533files.
534
535BitBake uses the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
536variable to point to source files regardless of their location. Each
537recipe must have a ``SRC_URI`` variable that points to the source.
538
539Another area that plays a significant role in where source files come
540from is pointed to by the
541```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ variable. This area is
542a cache that can hold previously downloaded source. You can also
543instruct the OpenEmbedded build system to create tarballs from Git
544repositories, which is not the default behavior, and store them in the
545``DL_DIR`` by using the
546```BB_GENERATE_MIRROR_TARBALLS`` <&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS>`__
547variable.
548
549Judicious use of a ``DL_DIR`` directory can save the build system a trip
550across the Internet when looking for files. A good method for using a
551download directory is to have ``DL_DIR`` point to an area outside of
552your Build Directory. Doing so allows you to safely delete the Build
553Directory if needed without fear of removing any downloaded source file.
554
555The remainder of this section provides a deeper look into the source
556files and the mirrors. Here is a more detailed look at the source file
557area of the `general workflow figure <#general-workflow-figure>`__:
558
559Upstream Project Releases
560~~~~~~~~~~~~~~~~~~~~~~~~~
561
562Upstream project releases exist anywhere in the form of an archived file
563(e.g. tarball or zip file). These files correspond to individual
564recipes. For example, the figure uses specific releases each for
565BusyBox, Qt, and Dbus. An archive file can be for any released product
566that can be built using a recipe.
567
568Local Projects
569~~~~~~~~~~~~~~
570
571Local projects are custom bits of software the user provides. These bits
572reside somewhere local to a project - perhaps a directory into which the
573user checks in items (e.g. a local directory containing a development
574source tree used by the group).
575
576The canonical method through which to include a local project is to use
577the ```externalsrc`` <&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc>`__
578class to include that local project. You use either the ``local.conf``
579or a recipe's append file to override or set the recipe to point to the
580local directory on your disk to pull in the whole source tree.
581
582.. _scms:
583
584Source Control Managers (Optional)
585~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
586
587Another place from which the build system can get source files is with
588`fetchers <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__ employing various Source
589Control Managers (SCMs) such as Git or Subversion. In such cases, a
590repository is cloned or checked out. The
591```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ task inside
592BitBake uses the ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
593variable and the argument's prefix to determine the correct fetcher
594module.
595
596.. note::
597
598 For information on how to have the OpenEmbedded build system generate
599 tarballs for Git repositories and place them in the
600 DL_DIR
601 directory, see the
602 BB_GENERATE_MIRROR_TARBALLS
603 variable in the Yocto Project Reference Manual.
604
605When fetching a repository, BitBake uses the
606```SRCREV`` <&YOCTO_DOCS_REF_URL;#var-SRCREV>`__ variable to determine
607the specific revision from which to build.
608
609Source Mirror(s)
610~~~~~~~~~~~~~~~~
611
612Two kinds of mirrors exist: pre-mirrors and regular mirrors. The
613```PREMIRRORS`` <&YOCTO_DOCS_REF_URL;#var-PREMIRRORS>`__ and
614```MIRRORS`` <&YOCTO_DOCS_REF_URL;#var-MIRRORS>`__ variables point to
615these, respectively. BitBake checks pre-mirrors before looking upstream
616for any source files. Pre-mirrors are appropriate when you have a shared
617directory that is not a directory defined by the
618```DL_DIR`` <&YOCTO_DOCS_REF_URL;#var-DL_DIR>`__ variable. A Pre-mirror
619typically points to a shared directory that is local to your
620organization.
621
622Regular mirrors can be any site across the Internet that is used as an
623alternative location for source code should the primary site not be
624functioning for some reason or another.
625
626.. _package-feeds-dev-environment:
627
628Package Feeds
629-------------
630
631When the OpenEmbedded build system generates an image or an SDK, it gets
632the packages from a package feed area located in the `Build
633Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. The `general
634workflow figure <#general-workflow-figure>`__ shows this package feeds
635area in the upper-right corner.
636
637This section looks a little closer into the package feeds area used by
638the build system. Here is a more detailed look at the area:
639
640Package feeds are an intermediary step in the build process. The
641OpenEmbedded build system provides classes to generate different package
642types, and you specify which classes to enable through the
643```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__
644variable. Before placing the packages into package feeds, the build
645process validates them with generated output quality assurance checks
646through the ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__
647class.
648
649The package feed area resides in the Build Directory. The directory the
650build system uses to temporarily store packages is determined by a
651combination of variables and the particular package manager in use. See
652the "Package Feeds" box in the illustration and note the information to
653the right of that area. In particular, the following defines where
654package files are kept:
655
656- ```DEPLOY_DIR`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR>`__: Defined as
657 ``tmp/deploy`` in the Build Directory.
658
659- ``DEPLOY_DIR_*``: Depending on the package manager used, the package
660 type sub-folder. Given RPM, IPK, or DEB packaging and tarball
661 creation, the
662 ```DEPLOY_DIR_RPM`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_RPM>`__,
663 ```DEPLOY_DIR_IPK`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IPK>`__,
664 ```DEPLOY_DIR_DEB`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_DEB>`__, or
665 ```DEPLOY_DIR_TAR`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_TAR>`__,
666 variables are used, respectively.
667
668- ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__: Defines
669 architecture-specific sub-folders. For example, packages could exist
670 for the i586 or qemux86 architectures.
671
672BitBake uses the
673```do_package_write_*`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb>`__
674tasks to generate packages and place them into the package holding area
675(e.g. ``do_package_write_ipk`` for IPK packages). See the
676"```do_package_write_deb`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb>`__",
677"```do_package_write_ipk`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_ipk>`__",
678"```do_package_write_rpm`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_rpm>`__",
679and
680"```do_package_write_tar`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_tar>`__"
681sections in the Yocto Project Reference Manual for additional
682information. As an example, consider a scenario where an IPK packaging
683manager is being used and package architecture support for both i586 and
684qemux86 exist. Packages for the i586 architecture are placed in
685``build/tmp/deploy/ipk/i586``, while packages for the qemux86
686architecture are placed in ``build/tmp/deploy/ipk/qemux86``.
687
688.. _bitbake-dev-environment:
689
690BitBake
691-------
692
693The OpenEmbedded build system uses
694`BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ to produce images and
695Software Development Kits (SDKs). You can see from the `general workflow
696figure <#general-workflow-figure>`__, the BitBake area consists of
697several functional areas. This section takes a closer look at each of
698those areas.
699
700.. note::
701
702 Separate documentation exists for the BitBake tool. See the
703 BitBake User Manual
704 for reference material on BitBake.
705
706.. _source-fetching-dev-environment:
707
708Source Fetching
709~~~~~~~~~~~~~~~
710
711The first stages of building a recipe are to fetch and unpack the source
712code:
713
714The ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__ and
715```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ tasks fetch
716the source files and unpack them into the `Build
717Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__.
718
719.. note::
720
721 For every local file (e.g.
722 file://
723 ) that is part of a recipe's
724 SRC_URI
725 statement, the OpenEmbedded build system takes a checksum of the file
726 for the recipe and inserts the checksum into the signature for the
727 do_fetch
728 task. If any local file has been modified, the
729 do_fetch
730 task and all tasks that depend on it are re-executed.
731
732By default, everything is accomplished in the Build Directory, which has
733a defined structure. For additional general information on the Build
734Directory, see the
735"```build/`` <&YOCTO_DOCS_REF_URL;#structure-core-build>`__" section in
736the Yocto Project Reference Manual.
737
738Each recipe has an area in the Build Directory where the unpacked source
739code resides. The ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ variable points
740to this area for a recipe's unpacked source code. The name of that
741directory for any given recipe is defined from several different
742variables. The preceding figure and the following list describe the
743Build Directory's hierarchy:
744
745- ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__: The base directory
746 where the OpenEmbedded build system performs all its work during the
747 build. The default base directory is the ``tmp`` directory.
748
749- ```PACKAGE_ARCH`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH>`__: The
750 architecture of the built package or packages. Depending on the
751 eventual destination of the package or packages (i.e. machine
752 architecture, `build
753 host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__, SDK, or
754 specific machine), ``PACKAGE_ARCH`` varies. See the variable's
755 description for details.
756
757- ```TARGET_OS`` <&YOCTO_DOCS_REF_URL;#var-TARGET_OS>`__: The operating
758 system of the target device. A typical value would be "linux" (e.g.
759 "qemux86-poky-linux").
760
761- ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__: The name of the recipe used
762 to build the package. This variable can have multiple meanings.
763 However, when used in the context of input files, ``PN`` represents
764 the the name of the recipe.
765
766- ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__: The location
767 where the OpenEmbedded build system builds a recipe (i.e. does the
768 work to create the package).
769
770 - ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The version of the
771 recipe used to build the package.
772
773 - ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__: The revision of the
774 recipe used to build the package.
775
776- ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__: Contains the unpacked source
777 files for a given recipe.
778
779 - ```BPN`` <&YOCTO_DOCS_REF_URL;#var-BPN>`__: The name of the recipe
780 used to build the package. The ``BPN`` variable is a version of
781 the ``PN`` variable but with common prefixes and suffixes removed.
782
783 - ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__: The version of the
784 recipe used to build the package.
785
786.. note::
787
788 In the previous figure, notice that two sample hierarchies exist: one
789 based on package architecture (i.e.
790 PACKAGE_ARCH
791 ) and one based on a machine (i.e.
792 MACHINE
793 ). The underlying structures are identical. The differentiator being
794 what the OpenEmbedded build system is using as a build target (e.g.
795 general architecture, a build host, an SDK, or a specific machine).
796
797.. _patching-dev-environment:
798
799Patching
800~~~~~~~~
801
802Once source code is fetched and unpacked, BitBake locates patch files
803and applies them to the source files:
804
805The ```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task uses a
806recipe's ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
807and the ```FILESPATH`` <&YOCTO_DOCS_REF_URL;#var-FILESPATH>`__ variable
808to locate applicable patch files.
809
810Default processing for patch files assumes the files have either
811``*.patch`` or ``*.diff`` file types. You can use ``SRC_URI`` parameters
812to change the way the build system recognizes patch files. See the
813```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task for more
814information.
815
816BitBake finds and applies multiple patches for a single recipe in the
817order in which it locates the patches. The ``FILESPATH`` variable
818defines the default set of directories that the build system uses to
819search for patch files. Once found, patches are applied to the recipe's
820source files, which are located in the
821```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ directory.
822
823For more information on how the source directories are created, see the
824"`Source Fetching <#source-fetching-dev-environment>`__" section. For
825more information on how to create patches and how the build system
826processes patches, see the "`Patching
827Code <&YOCTO_DOCS_DEV_URL;#new-recipe-patching-code>`__" section in the
828Yocto Project Development Tasks Manual. You can also see the "`Use
829``devtool modify`` to Modify the Source of an Existing
830Component <&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component>`__"
831section in the Yocto Project Application Development and the Extensible
832Software Development Kit (SDK) manual and the "`Using Traditional Kernel
833Development to Patch the
834Kernel <&YOCTO_DOCS_KERNEL_DEV_URL;#using-traditional-kernel-development-to-patch-the-kernel>`__"
835section in the Yocto Project Linux Kernel Development Manual.
836
837.. _configuration-compilation-and-staging-dev-environment:
838
839Configuration, Compilation, and Staging
840~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
841
842After source code is patched, BitBake executes tasks that configure and
843compile the source code. Once compilation occurs, the files are copied
844to a holding area (staged) in preparation for packaging:
845
846This step in the build process consists of the following tasks:
847
848- ```do_prepare_recipe_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-prepare_recipe_sysroot>`__:
849 This task sets up the two sysroots in
850 ``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}``
851 (i.e. ``recipe-sysroot`` and ``recipe-sysroot-native``) so that
852 during the packaging phase the sysroots can contain the contents of
853 the
854 ```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
855 tasks of the recipes on which the recipe containing the tasks
856 depends. A sysroot exists for both the target and for the native
857 binaries, which run on the host system.
858
859- *``do_configure``*: This task configures the source by enabling and
860 disabling any build-time and configuration options for the software
861 being built. Configurations can come from the recipe itself as well
862 as from an inherited class. Additionally, the software itself might
863 configure itself depending on the target for which it is being built.
864
865 The configurations handled by the
866 ```do_configure`` <&YOCTO_DOCS_REF_URL;#ref-tasks-configure>`__ task
867 are specific to configurations for the source code being built by the
868 recipe.
869
870 If you are using the
871 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class,
872 you can add additional configuration options by using the
873 ```EXTRA_OECONF`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF>`__ or
874 ```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
875 variables. For information on how this variable works within that
876 class, see the
877 ```autotools`` <&YOCTO_DOCS_REF_URL;#ref-classes-autotools>`__ class
878 `here <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/autotools.bbclass>`__.
879
880- *``do_compile``*: Once a configuration task has been satisfied,
881 BitBake compiles the source using the
882 ```do_compile`` <&YOCTO_DOCS_REF_URL;#ref-tasks-compile>`__ task.
883 Compilation occurs in the directory pointed to by the
884 ```B`` <&YOCTO_DOCS_REF_URL;#var-B>`__ variable. Realize that the
885 ``B`` directory is, by default, the same as the
886 ```S`` <&YOCTO_DOCS_REF_URL;#var-S>`__ directory.
887
888- *``do_install``*: After compilation completes, BitBake executes the
889 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task.
890 This task copies files from the ``B`` directory and places them in a
891 holding area pointed to by the ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__
892 variable. Packaging occurs later using files from this holding
893 directory.
894
895.. _package-splitting-dev-environment:
896
897Package Splitting
898~~~~~~~~~~~~~~~~~
899
900After source code is configured, compiled, and staged, the build system
901analyzes the results and splits the output into packages:
902
903The ```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ and
904```do_packagedata`` <&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata>`__
905tasks combine to analyze the files found in the
906```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__ directory and split them into
907subsets based on available packages and files. Analysis involves the
908following as well as other items: splitting out debugging symbols,
909looking at shared library dependencies between packages, and looking at
910package relationships.
911
912The ``do_packagedata`` task creates package metadata based on the
913analysis such that the build system can generate the final packages. The
914```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__
915task stages (copies) a subset of the files installed by the
916```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task into
917the appropriate sysroot. Working, staged, and intermediate results of
918the analysis and package splitting process use several areas:
919
920- ```PKGD`` <&YOCTO_DOCS_REF_URL;#var-PKGD>`__: The destination
921 directory (i.e. ``package``) for packages before they are split into
922 individual packages.
923
924- ```PKGDESTWORK`` <&YOCTO_DOCS_REF_URL;#var-PKGDESTWORK>`__: A
925 temporary work area (i.e. ``pkgdata``) used by the ``do_package``
926 task to save package metadata.
927
928- ```PKGDEST`` <&YOCTO_DOCS_REF_URL;#var-PKGDEST>`__: The parent
929 directory (i.e. ``packages-split``) for packages after they have been
930 split.
931
932- ```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__: A shared,
933 global-state directory that holds packaging metadata generated during
934 the packaging process. The packaging process copies metadata from
935 ``PKGDESTWORK`` to the ``PKGDATA_DIR`` area where it becomes globally
936 available.
937
938- ```STAGING_DIR_HOST`` <&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_HOST>`__:
939 The path for the sysroot for the system on which a component is built
940 to run (i.e. ``recipe-sysroot``).
941
942- ```STAGING_DIR_NATIVE`` <&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_NATIVE>`__:
943 The path for the sysroot used when building components for the build
944 host (i.e. ``recipe-sysroot-native``).
945
946- ```STAGING_DIR_TARGET`` <&YOCTO_DOCS_REF_URL;#var-STAGING_DIR_TARGET>`__:
947 The path for the sysroot used when a component that is built to
948 execute on a system and it generates code for yet another machine
949 (e.g. cross-canadian recipes).
950
951The ```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ variable defines the
952files that go into each package in
953```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__. If you want
954details on how this is accomplished, you can look at
955```package.bbclass`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/package.bbclass>`__.
956
957Depending on the type of packages being created (RPM, DEB, or IPK), the
958```do_package_write_*`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb>`__
959task creates the actual packages and places them in the Package Feed
960area, which is ``${TMPDIR}/deploy``. You can see the "`Package
961Feeds <#package-feeds-dev-environment>`__" section for more detail on
962that part of the build process.
963
964.. note::
965
966 Support for creating feeds directly from the
967 deploy/\*
968 directories does not exist. Creating such feeds usually requires some
969 kind of feed maintenance mechanism that would upload the new packages
970 into an official package feed (e.g. the Ångström distribution). This
971 functionality is highly distribution-specific and thus is not
972 provided out of the box.
973
974.. _image-generation-dev-environment:
975
976Image Generation
977~~~~~~~~~~~~~~~~
978
979Once packages are split and stored in the Package Feeds area, the build
980system uses BitBake to generate the root filesystem image:
981
982The image generation process consists of several stages and depends on
983several tasks and variables. The
984```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__ task creates
985the root filesystem (file and directory structure) for an image. This
986task uses several key variables to help create the list of packages to
987actually install:
988
989- ```IMAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL>`__: Lists
990 out the base set of packages from which to install from the Package
991 Feeds area.
992
993- ```PACKAGE_EXCLUDE`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE>`__:
994 Specifies packages that should not be installed into the image.
995
996- ```IMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES>`__:
997 Specifies features to include in the image. Most of these features
998 map to additional packages for installation.
999
1000- ```PACKAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES>`__:
1001 Specifies the package backend (e.g. RPM, DEB, or IPK) to use and
1002 consequently helps determine where to locate packages within the
1003 Package Feeds area.
1004
1005- ```IMAGE_LINGUAS`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_LINGUAS>`__:
1006 Determines the language(s) for which additional language support
1007 packages are installed.
1008
1009- ```PACKAGE_INSTALL`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL>`__:
1010 The final list of packages passed to the package manager for
1011 installation into the image.
1012
1013With ```IMAGE_ROOTFS`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_ROOTFS>`__
1014pointing to the location of the filesystem under construction and the
1015``PACKAGE_INSTALL`` variable providing the final list of packages to
1016install, the root file system is created.
1017
1018Package installation is under control of the package manager (e.g.
1019dnf/rpm, opkg, or apt/dpkg) regardless of whether or not package
1020management is enabled for the target. At the end of the process, if
1021package management is not enabled for the target, the package manager's
1022data files are deleted from the root filesystem. As part of the final
1023stage of package installation, post installation scripts that are part
1024of the packages are run. Any scripts that fail to run on the build host
1025are run on the target when the target system is first booted. If you are
1026using a `read-only root
1027filesystem <&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem>`__,
1028all the post installation scripts must succeed on the build host during
1029the package installation phase since the root filesystem on the target
1030is read-only.
1031
1032The final stages of the ``do_rootfs`` task handle post processing. Post
1033processing includes creation of a manifest file and optimizations.
1034
1035The manifest file (``.manifest``) resides in the same directory as the
1036root filesystem image. This file lists out, line-by-line, the installed
1037packages. The manifest file is useful for the
1038```testimage`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__ class,
1039for example, to determine whether or not to run specific tests. See the
1040```IMAGE_MANIFEST`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_MANIFEST>`__
1041variable for additional information.
1042
1043Optimizing processes that are run across the image include ``mklibs``,
1044``prelink``, and any other post-processing commands as defined by the
1045```ROOTFS_POSTPROCESS_COMMAND`` <&YOCTO_DOCS_REF_URL;#var-ROOTFS_POSTPROCESS_COMMAND>`__
1046variable. The ``mklibs`` process optimizes the size of the libraries,
1047while the ``prelink`` process optimizes the dynamic linking of shared
1048libraries to reduce start up time of executables.
1049
1050After the root filesystem is built, processing begins on the image
1051through the ```do_image`` <&YOCTO_DOCS_REF_URL;#ref-tasks-image>`__
1052task. The build system runs any pre-processing commands as defined by
1053the
1054```IMAGE_PREPROCESS_COMMAND`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_PREPROCESS_COMMAND>`__
1055variable. This variable specifies a list of functions to call before the
1056build system creates the final image output files.
1057
1058The build system dynamically creates ``do_image_*`` tasks as needed,
1059based on the image types specified in the
1060```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__ variable.
1061The process turns everything into an image file or a set of image files
1062and can compress the root filesystem image to reduce the overall size of
1063the image. The formats used for the root filesystem depend on the
1064``IMAGE_FSTYPES`` variable. Compression depends on whether the formats
1065support compression.
1066
1067As an example, a dynamically created task when creating a particular
1068image type would take the following form: do_image_type So, if the type
1069as specified by the ``IMAGE_FSTYPES`` were ``ext4``, the dynamically
1070generated task would be as follows: do_image_ext4
1071
1072The final task involved in image creation is the
1073```do_image_complete`` <&YOCTO_DOCS_REF_URL;#ref-tasks-image-complete>`__
1074task. This task completes the image by applying any image post
1075processing as defined through the
1076```IMAGE_POSTPROCESS_COMMAND`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_POSTPROCESS_COMMAND>`__
1077variable. The variable specifies a list of functions to call once the
1078build system has created the final image output files.
1079
1080.. note::
1081
1082 The entire image generation process is run under
1083 Pseudo
1084 . Running under Pseudo ensures that the files in the root filesystem
1085 have correct ownership.
1086
1087.. _sdk-generation-dev-environment:
1088
1089SDK Generation
1090~~~~~~~~~~~~~~
1091
1092The OpenEmbedded build system uses BitBake to generate the Software
1093Development Kit (SDK) installer scripts for both the standard SDK and
1094the extensible SDK (eSDK):
1095
1096.. note::
1097
1098 For more information on the cross-development toolchain generation,
1099 see the "
1100 Cross-Development Toolchain Generation
1101 " section. For information on advantages gained when building a
1102 cross-development toolchain using the
1103 do_populate_sdk
1104 task, see the "
1105 Building an SDK Installer
1106 " section in the Yocto Project Application Development and the
1107 Extensible Software Development Kit (eSDK) manual.
1108
1109Like image generation, the SDK script process consists of several stages
1110and depends on many variables. The
1111```do_populate_sdk`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk>`__
1112and
1113```do_populate_sdk_ext`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk_ext>`__
1114tasks use these key variables to help create the list of packages to
1115actually install. For information on the variables listed in the figure,
1116see the "`Application Development SDK <#sdk-dev-environment>`__"
1117section.
1118
1119The ``do_populate_sdk`` task helps create the standard SDK and handles
1120two parts: a target part and a host part. The target part is the part
1121built for the target hardware and includes libraries and headers. The
1122host part is the part of the SDK that runs on the
1123```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__.
1124
1125The ``do_populate_sdk_ext`` task helps create the extensible SDK and
1126handles host and target parts differently than its counter part does for
1127the standard SDK. For the extensible SDK, the task encapsulates the
1128build system, which includes everything needed (host and target) for the
1129SDK.
1130
1131Regardless of the type of SDK being constructed, the tasks perform some
1132cleanup after which a cross-development environment setup script and any
1133needed configuration files are created. The final output is the
1134Cross-development toolchain installation script (``.sh`` file), which
1135includes the environment setup script.
1136
1137Stamp Files and the Rerunning of Tasks
1138~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1139
1140For each task that completes successfully, BitBake writes a stamp file
1141into the ```STAMPS_DIR`` <&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR>`__
1142directory. The beginning of the stamp file's filename is determined by
1143the ```STAMP`` <&YOCTO_DOCS_REF_URL;#var-STAMP>`__ variable, and the end
1144of the name consists of the task's name and current `input
1145checksum <#overview-checksums>`__.
1146
1147.. note::
1148
1149 This naming scheme assumes that
1150 BB_SIGNATURE_HANDLER
1151 is "OEBasicHash", which is almost always the case in current
1152 OpenEmbedded.
1153
1154To determine if a task needs to be rerun, BitBake checks if a stamp file
1155with a matching input checksum exists for the task. If such a stamp file
1156exists, the task's output is assumed to exist and still be valid. If the
1157file does not exist, the task is rerun.
1158
1159.. note::
1160
1161 The stamp mechanism is more general than the shared state (sstate)
1162 cache mechanism described in the "`Setscene Tasks and Shared
1163 State <#setscene-tasks-and-shared-state>`__" section. BitBake avoids
1164 rerunning any task that has a valid stamp file, not just tasks that
1165 can be accelerated through the sstate cache.
1166
1167 However, you should realize that stamp files only serve as a marker
1168 that some work has been done and that these files do not record task
1169 output. The actual task output would usually be somewhere in
1170 ```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__ (e.g. in some
1171 recipe's ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__.) What
1172 the sstate cache mechanism adds is a way to cache task output that
1173 can then be shared between build machines.
1174
1175Since ``STAMPS_DIR`` is usually a subdirectory of ``TMPDIR``, removing
1176``TMPDIR`` will also remove ``STAMPS_DIR``, which means tasks will
1177properly be rerun to repopulate ``TMPDIR``.
1178
1179If you want some task to always be considered "out of date", you can
1180mark it with the ```nostamp`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__
1181varflag. If some other task depends on such a task, then that task will
1182also always be considered out of date, which might not be what you want.
1183
1184For details on how to view information about a task's signature, see the
1185"`Viewing Task Variable
1186Dependencies <&YOCTO_DOCS_DEV_URL;#dev-viewing-task-variable-dependencies>`__"
1187section in the Yocto Project Development Tasks Manual.
1188
1189Setscene Tasks and Shared State
1190~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1191
1192The description of tasks so far assumes that BitBake needs to build
1193everything and no available prebuilt objects exist. BitBake does support
1194skipping tasks if prebuilt objects are available. These objects are
1195usually made available in the form of a shared state (sstate) cache.
1196
1197.. note::
1198
1199 For information on variables affecting sstate, see the
1200 SSTATE_DIR
1201 and
1202 SSTATE_MIRRORS
1203 variables.
1204
1205The idea of a setscene task (i.e ``do_``\ taskname\ ``_setscene``) is a
1206version of the task where instead of building something, BitBake can
1207skip to the end result and simply place a set of files into specific
1208locations as needed. In some cases, it makes sense to have a setscene
1209task variant (e.g. generating package files in the
1210```do_package_write_*`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb>`__
1211task). In other cases, it does not make sense (e.g. a
1212```do_patch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-patch>`__ task or a
1213```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__ task) since
1214the work involved would be equal to or greater than the underlying task.
1215
1216In the build system, the common tasks that have setscene variants are
1217```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__,
1218``do_package_write_*``,
1219```do_deploy`` <&YOCTO_DOCS_REF_URL;#ref-tasks-deploy>`__,
1220```do_packagedata`` <&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata>`__, and
1221```do_populate_sysroot`` <&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot>`__.
1222Notice that these tasks represent most of the tasks whose output is an
1223end result.
1224
1225The build system has knowledge of the relationship between these tasks
1226and other preceding tasks. For example, if BitBake runs
1227``do_populate_sysroot_setscene`` for something, it does not make sense
1228to run any of the ``do_fetch``, ``do_unpack``, ``do_patch``,
1229``do_configure``, ``do_compile``, and ``do_install`` tasks. However, if
1230``do_package`` needs to be run, BitBake needs to run those other tasks.
1231
1232It becomes more complicated if everything can come from an sstate cache
1233because some objects are simply not required at all. For example, you do
1234not need a compiler or native tools, such as quilt, if nothing exists to
1235compile or patch. If the ``do_package_write_*`` packages are available
1236from sstate, BitBake does not need the ``do_package`` task data.
1237
1238To handle all these complexities, BitBake runs in two phases. The first
1239is the "setscene" stage. During this stage, BitBake first checks the
1240sstate cache for any targets it is planning to build. BitBake does a
1241fast check to see if the object exists rather than a complete download.
1242If nothing exists, the second phase, which is the setscene stage,
1243completes and the main build proceeds.
1244
1245If objects are found in the sstate cache, the build system works
1246backwards from the end targets specified by the user. For example, if an
1247image is being built, the build system first looks for the packages
1248needed for that image and the tools needed to construct an image. If
1249those are available, the compiler is not needed. Thus, the compiler is
1250not even downloaded. If something was found to be unavailable, or the
1251download or setscene task fails, the build system then tries to install
1252dependencies, such as the compiler, from the cache.
1253
1254The availability of objects in the sstate cache is handled by the
1255function specified by the
1256```BB_HASHCHECK_FUNCTION`` <&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION>`__
1257variable and returns a list of available objects. The function specified
1258by the
1259```BB_SETSCENE_DEPVALID`` <&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID>`__
1260variable is the function that determines whether a given dependency
1261needs to be followed, and whether for any given relationship the
1262function needs to be passed. The function returns a True or False value.
1263
1264.. _images-dev-environment:
1265
1266Images
1267------
1268
1269The images produced by the build system are compressed forms of the root
1270filesystem and are ready to boot on a target device. You can see from
1271the `general workflow figure <#general-workflow-figure>`__ that BitBake
1272output, in part, consists of images. This section takes a closer look at
1273this output:
1274
1275.. note::
1276
1277 For a list of example images that the Yocto Project provides, see the
1278 "
1279 Images
1280 " chapter in the Yocto Project Reference Manual.
1281
1282The build process writes images out to the `Build
1283Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ inside the
1284``tmp/deploy/images/machine/`` folder as shown in the figure. This
1285folder contains any files expected to be loaded on the target device.
1286The ```DEPLOY_DIR`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR>`__ variable
1287points to the ``deploy`` directory, while the
1288```DEPLOY_DIR_IMAGE`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IMAGE>`__
1289variable points to the appropriate directory containing images for the
1290current configuration.
1291
1292- kernel-image: A kernel binary file. The
1293 ```KERNEL_IMAGETYPE`` <&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE>`__
1294 variable determines the naming scheme for the kernel image file.
1295 Depending on this variable, the file could begin with a variety of
1296 naming strings. The ``deploy/images/``\ machine directory can contain
1297 multiple image files for the machine.
1298
1299- root-filesystem-image: Root filesystems for the target device (e.g.
1300 ``*.ext3`` or ``*.bz2`` files). The
1301 ```IMAGE_FSTYPES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES>`__
1302 variable determines the root filesystem image type. The
1303 ``deploy/images/``\ machine directory can contain multiple root
1304 filesystems for the machine.
1305
1306- kernel-modules: Tarballs that contain all the modules built for the
1307 kernel. Kernel module tarballs exist for legacy purposes and can be
1308 suppressed by setting the
1309 ```MODULE_TARBALL_DEPLOY`` <&YOCTO_DOCS_REF_URL;#var-MODULE_TARBALL_DEPLOY>`__
1310 variable to "0". The ``deploy/images/``\ machine directory can
1311 contain multiple kernel module tarballs for the machine.
1312
1313- bootloaders: If applicable to the target machine, bootloaders
1314 supporting the image. The ``deploy/images/``\ machine directory can
1315 contain multiple bootloaders for the machine.
1316
1317- symlinks: The ``deploy/images/``\ machine folder contains a symbolic
1318 link that points to the most recently built file for each machine.
1319 These links might be useful for external scripts that need to obtain
1320 the latest version of each file.
1321
1322.. _sdk-dev-environment:
1323
1324Application Development SDK
1325---------------------------
1326
1327In the `general workflow figure <#general-workflow-figure>`__, the
1328output labeled "Application Development SDK" represents an SDK. The SDK
1329generation process differs depending on whether you build an extensible
1330SDK (e.g. ``bitbake -c populate_sdk_ext`` imagename) or a standard SDK
1331(e.g. ``bitbake -c populate_sdk`` imagename). This section takes a
1332closer look at this output:
1333
1334The specific form of this output is a set of files that includes a
1335self-extracting SDK installer (``*.sh``), host and target manifest
1336files, and files used for SDK testing. When the SDK installer file is
1337run, it installs the SDK. The SDK consists of a cross-development
1338toolchain, a set of libraries and headers, and an SDK environment setup
1339script. Running this installer essentially sets up your
1340cross-development environment. You can think of the cross-toolchain as
1341the "host" part because it runs on the SDK machine. You can think of the
1342libraries and headers as the "target" part because they are built for
1343the target hardware. The environment setup script is added so that you
1344can initialize the environment before using the tools.
1345
1346.. note::
1347
1348 - The Yocto Project supports several methods by which you can set up
1349 this cross-development environment. These methods include
1350 downloading pre-built SDK installers or building and installing
1351 your own SDK installer.
1352
1353 - For background information on cross-development toolchains in the
1354 Yocto Project development environment, see the "`Cross-Development
1355 Toolchain Generation <#cross-development-toolchain-generation>`__"
1356 section.
1357
1358 - For information on setting up a cross-development environment, see
1359 the `Yocto Project Application Development and the Extensible
1360 Software Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
1361
1362All the output files for an SDK are written to the ``deploy/sdk`` folder
1363inside the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ as
1364shown in the previous figure. Depending on the type of SDK, several
1365variables exist that help configure these files. The following list
1366shows the variables associated with an extensible SDK:
1367
1368- ```DEPLOY_DIR`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR>`__: Points to
1369 the ``deploy`` directory.
1370
1371- ```SDK_EXT_TYPE`` <&YOCTO_DOCS_REF_URL;#var-SDK_EXT_TYPE>`__:
1372 Controls whether or not shared state artifacts are copied into the
1373 extensible SDK. By default, all required shared state artifacts are
1374 copied into the SDK.
1375
1376- ```SDK_INCLUDE_PKGDATA`` <&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_PKGDATA>`__:
1377 Specifies whether or not packagedata is included in the extensible
1378 SDK for all recipes in the "world" target.
1379
1380- ```SDK_INCLUDE_TOOLCHAIN`` <&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_TOOLCHAIN>`__:
1381 Specifies whether or not the toolchain is included when building the
1382 extensible SDK.
1383
1384- ```SDK_LOCAL_CONF_WHITELIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_WHITELIST>`__:
1385 A list of variables allowed through from the build system
1386 configuration into the extensible SDK configuration.
1387
1388- ```SDK_LOCAL_CONF_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_BLACKLIST>`__:
1389 A list of variables not allowed through from the build system
1390 configuration into the extensible SDK configuration.
1391
1392- ```SDK_INHERIT_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_INHERIT_BLACKLIST>`__:
1393 A list of classes to remove from the
1394 ```INHERIT`` <&YOCTO_DOCS_REF_URL;#var-INHERIT>`__ value globally
1395 within the extensible SDK configuration.
1396
1397This next list, shows the variables associated with a standard SDK:
1398
1399- ```DEPLOY_DIR`` <&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR>`__: Points to
1400 the ``deploy`` directory.
1401
1402- ```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__: Specifies
1403 the architecture of the machine on which the cross-development tools
1404 are run to create packages for the target hardware.
1405
1406- ```SDKIMAGE_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-SDKIMAGE_FEATURES>`__:
1407 Lists the features to include in the "target" part of the SDK.
1408
1409- ```TOOLCHAIN_HOST_TASK`` <&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_HOST_TASK>`__:
1410 Lists packages that make up the host part of the SDK (i.e. the part
1411 that runs on the ``SDKMACHINE``). When you use
1412 ``bitbake -c populate_sdk imagename`` to create the SDK, a set of
1413 default packages apply. This variable allows you to add more
1414 packages.
1415
1416- ```TOOLCHAIN_TARGET_TASK`` <&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_TARGET_TASK>`__:
1417 Lists packages that make up the target part of the SDK (i.e. the part
1418 built for the target hardware).
1419
1420- ```SDKPATH`` <&YOCTO_DOCS_REF_URL;#var-SDKPATH>`__: Defines the
1421 default SDK installation path offered by the installation script.
1422
1423- ```SDK_HOST_MANIFEST`` <&YOCTO_DOCS_REF_URL;#var-SDK_HOST_MANIFEST>`__:
1424 Lists all the installed packages that make up the host part of the
1425 SDK. This variable also plays a minor role for extensible SDK
1426 development as well. However, it is mainly used for the standard SDK.
1427
1428- ```SDK_TARGET_MANIFEST`` <&YOCTO_DOCS_REF_URL;#var-SDK_TARGET_MANIFEST>`__:
1429 Lists all the installed packages that make up the target part of the
1430 SDK. This variable also plays a minor role for extensible SDK
1431 development as well. However, it is mainly used for the standard SDK.
1432
1433Cross-Development Toolchain Generation
1434======================================
1435
1436The Yocto Project does most of the work for you when it comes to
1437creating `cross-development
1438toolchains <&YOCTO_DOCS_REF_URL;#cross-development-toolchain>`__. This
1439section provides some technical background on how cross-development
1440toolchains are created and used. For more information on toolchains, you
1441can also see the `Yocto Project Application Development and the
1442Extensible Software Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__
1443manual.
1444
1445In the Yocto Project development environment, cross-development
1446toolchains are used to build images and applications that run on the
1447target hardware. With just a few commands, the OpenEmbedded build system
1448creates these necessary toolchains for you.
1449
1450The following figure shows a high-level build environment regarding
1451toolchain construction and use.
1452
1453Most of the work occurs on the Build Host. This is the machine used to
1454build images and generally work within the the Yocto Project
1455environment. When you run
1456`BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ to create an image, the
1457OpenEmbedded build system uses the host ``gcc`` compiler to bootstrap a
1458cross-compiler named ``gcc-cross``. The ``gcc-cross`` compiler is what
1459BitBake uses to compile source files when creating the target image. You
1460can think of ``gcc-cross`` simply as an automatically generated
1461cross-compiler that is used internally within BitBake only.
1462
1463.. note::
1464
1465 The extensible SDK does not use
1466 gcc-cross-canadian
1467 since this SDK ships a copy of the OpenEmbedded build system and the
1468 sysroot within it contains
1469 gcc-cross
1470 .
1471
1472The chain of events that occurs when ``gcc-cross`` is bootstrapped is as
1473follows: gcc -> binutils-cross -> gcc-cross-initial ->
1474linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime
1475
1476- ``gcc``: The build host's GNU Compiler Collection (GCC).
1477
1478- ``binutils-cross``: The bare minimum binary utilities needed in order
1479 to run the ``gcc-cross-initial`` phase of the bootstrap operation.
1480
1481- ``gcc-cross-initial``: An early stage of the bootstrap process for
1482 creating the cross-compiler. This stage builds enough of the
1483 ``gcc-cross``, the C library, and other pieces needed to finish
1484 building the final cross-compiler in later stages. This tool is a
1485 "native" package (i.e. it is designed to run on the build host).
1486
1487- ``linux-libc-headers``: Headers needed for the cross-compiler.
1488
1489- ``glibc-initial``: An initial version of the Embedded GNU C Library
1490 (GLIBC) needed to bootstrap ``glibc``.
1491
1492- ``glibc``: The GNU C Library.
1493
1494- ``gcc-cross``: The final stage of the bootstrap process for the
1495 cross-compiler. This stage results in the actual cross-compiler that
1496 BitBake uses when it builds an image for a targeted device.
1497
1498 .. note::
1499
1500 If you are replacing this cross compiler toolchain with a custom
1501 version, you must replace
1502 gcc-cross
1503 .
1504
1505 This tool is also a "native" package (i.e. it is designed to run on
1506 the build host).
1507
1508- ``gcc-runtime``: Runtime libraries resulting from the toolchain
1509 bootstrapping process. This tool produces a binary that consists of
1510 the runtime libraries need for the targeted device.
1511
1512You can use the OpenEmbedded build system to build an installer for the
1513relocatable SDK used to develop applications. When you run the
1514installer, it installs the toolchain, which contains the development
1515tools (e.g., ``gcc-cross-canadian``, ``binutils-cross-canadian``, and
1516other ``nativesdk-*`` tools), which are tools native to the SDK (i.e.
1517native to ```SDK_ARCH`` <&YOCTO_DOCS_REF_URL;#var-SDK_ARCH>`__), you
1518need to cross-compile and test your software. The figure shows the
1519commands you use to easily build out this toolchain. This
1520cross-development toolchain is built to execute on the
1521```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__, which might or
1522might not be the same machine as the Build Host.
1523
1524.. note::
1525
1526 If your target architecture is supported by the Yocto Project, you
1527 can take advantage of pre-built images that ship with the Yocto
1528 Project and already contain cross-development toolchain installers.
1529
1530Here is the bootstrap process for the relocatable toolchain: gcc ->
1531binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers ->
1532glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian
1533
1534- ``gcc``: The build host's GNU Compiler Collection (GCC).
1535
1536- ``binutils-crosssdk``: The bare minimum binary utilities needed in
1537 order to run the ``gcc-crosssdk-initial`` phase of the bootstrap
1538 operation.
1539
1540- ``gcc-crosssdk-initial``: An early stage of the bootstrap process for
1541 creating the cross-compiler. This stage builds enough of the
1542 ``gcc-crosssdk`` and supporting pieces so that the final stage of the
1543 bootstrap process can produce the finished cross-compiler. This tool
1544 is a "native" binary that runs on the build host.
1545
1546- ``linux-libc-headers``: Headers needed for the cross-compiler.
1547
1548- ``glibc-initial``: An initial version of the Embedded GLIBC needed to
1549 bootstrap ``nativesdk-glibc``.
1550
1551- ``nativesdk-glibc``: The Embedded GLIBC needed to bootstrap the
1552 ``gcc-crosssdk``.
1553
1554- ``gcc-crosssdk``: The final stage of the bootstrap process for the
1555 relocatable cross-compiler. The ``gcc-crosssdk`` is a transitory
1556 compiler and never leaves the build host. Its purpose is to help in
1557 the bootstrap process to create the eventual ``gcc-cross-canadian``
1558 compiler, which is relocatable. This tool is also a "native" package
1559 (i.e. it is designed to run on the build host).
1560
1561- ``gcc-cross-canadian``: The final relocatable cross-compiler. When
1562 run on the ```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__,
1563 this tool produces executable code that runs on the target device.
1564 Only one cross-canadian compiler is produced per architecture since
1565 they can be targeted at different processor optimizations using
1566 configurations passed to the compiler through the compile commands.
1567 This circumvents the need for multiple compilers and thus reduces the
1568 size of the toolchains.
1569
1570.. note::
1571
1572 For information on advantages gained when building a
1573 cross-development toolchain installer, see the "
1574 Building an SDK Installer
1575 " appendix in the Yocto Project Application Development and the
1576 Extensible Software Development Kit (eSDK) manual.
1577
1578Shared State Cache
1579==================
1580
1581By design, the OpenEmbedded build system builds everything from scratch
1582unless `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ can determine
1583that parts do not need to be rebuilt. Fundamentally, building from
1584scratch is attractive as it means all parts are built fresh and no
1585possibility of stale data exists that can cause problems. When
1586developers hit problems, they typically default back to building from
1587scratch so they have a know state from the start.
1588
1589Building an image from scratch is both an advantage and a disadvantage
1590to the process. As mentioned in the previous paragraph, building from
1591scratch ensures that everything is current and starts from a known
1592state. However, building from scratch also takes much longer as it
1593generally means rebuilding things that do not necessarily need to be
1594rebuilt.
1595
1596The Yocto Project implements shared state code that supports incremental
1597builds. The implementation of the shared state code answers the
1598following questions that were fundamental roadblocks within the
1599OpenEmbedded incremental build support system:
1600
1601- What pieces of the system have changed and what pieces have not
1602 changed?
1603
1604- How are changed pieces of software removed and replaced?
1605
1606- How are pre-built components that do not need to be rebuilt from
1607 scratch used when they are available?
1608
1609For the first question, the build system detects changes in the "inputs"
1610to a given task by creating a checksum (or signature) of the task's
1611inputs. If the checksum changes, the system assumes the inputs have
1612changed and the task needs to be rerun. For the second question, the
1613shared state (sstate) code tracks which tasks add which output to the
1614build process. This means the output from a given task can be removed,
1615upgraded or otherwise manipulated. The third question is partly
1616addressed by the solution for the second question assuming the build
1617system can fetch the sstate objects from remote locations and install
1618them if they are deemed to be valid.
1619
1620.. note::
1621
1622 - The build system does not maintain
1623 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ information as part of
1624 the shared state packages. Consequently, considerations exist that
1625 affect maintaining shared state feeds. For information on how the
1626 build system works with packages and can track incrementing ``PR``
1627 information, see the "`Automatically Incrementing a Binary Package
1628 Revision
1629 Number <&YOCTO_DOCS_DEV_URL;#automatically-incrementing-a-binary-package-revision-number>`__"
1630 section in the Yocto Project Development Tasks Manual.
1631
1632 - The code in the build system that supports incremental builds is
1633 not simple code. For techniques that help you work around issues
1634 related to shared state code, see the "`Viewing Metadata Used to
1635 Create the Input Signature of a Shared State
1636 Task <&YOCTO_DOCS_DEV_URL;#dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task>`__"
1637 and "`Invalidating Shared State to Force a Task to
1638 Run <&YOCTO_DOCS_DEV_URL;#dev-invalidating-shared-state-to-force-a-task-to-run>`__"
1639 sections both in the Yocto Project Development Tasks Manual.
1640
1641The rest of this section goes into detail about the overall incremental
1642build architecture, the checksums (signatures), and shared state.
1643
1644.. _concepts-overall-architecture:
1645
1646Overall Architecture
1647--------------------
1648
1649When determining what parts of the system need to be built, BitBake
1650works on a per-task basis rather than a per-recipe basis. You might
1651wonder why using a per-task basis is preferred over a per-recipe basis.
1652To help explain, consider having the IPK packaging backend enabled and
1653then switching to DEB. In this case, the
1654```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ and
1655```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ task outputs
1656are still valid. However, with a per-recipe approach, the build would
1657not include the ``.deb`` files. Consequently, you would have to
1658invalidate the whole build and rerun it. Rerunning everything is not the
1659best solution. Also, in this case, the core must be "taught" much about
1660specific tasks. This methodology does not scale well and does not allow
1661users to easily add new tasks in layers or as external recipes without
1662touching the packaged-staging core.
1663
1664.. _overview-checksums:
1665
1666Checksums (Signatures)
1667----------------------
1668
1669The shared state code uses a checksum, which is a unique signature of a
1670task's inputs, to determine if a task needs to be run again. Because it
1671is a change in a task's inputs that triggers a rerun, the process needs
1672to detect all the inputs to a given task. For shell tasks, this turns
1673out to be fairly easy because the build process generates a "run" shell
1674script for each task and it is possible to create a checksum that gives
1675you a good idea of when the task's data changes.
1676
1677To complicate the problem, there are things that should not be included
1678in the checksum. First, there is the actual specific build path of a
1679given task - the ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__. It
1680does not matter if the work directory changes because it should not
1681affect the output for target packages. Also, the build process has the
1682objective of making native or cross packages relocatable.
1683
1684.. note::
1685
1686 Both native and cross packages run on the
1687 build host
1688 . However, cross packages generate output for the target
1689 architecture.
1690
1691The checksum therefore needs to exclude ``WORKDIR``. The simplistic
1692approach for excluding the work directory is to set ``WORKDIR`` to some
1693fixed value and create the checksum for the "run" script.
1694
1695Another problem results from the "run" scripts containing functions that
1696might or might not get called. The incremental build solution contains
1697code that figures out dependencies between shell functions. This code is
1698used to prune the "run" scripts down to the minimum set, thereby
1699alleviating this problem and making the "run" scripts much more readable
1700as a bonus.
1701
1702So far, solutions for shell scripts exist. What about Python tasks? The
1703same approach applies even though these tasks are more difficult. The
1704process needs to figure out what variables a Python function accesses
1705and what functions it calls. Again, the incremental build solution
1706contains code that first figures out the variable and function
1707dependencies, and then creates a checksum for the data used as the input
1708to the task.
1709
1710Like the ``WORKDIR`` case, situations exist where dependencies should be
1711ignored. For these situations, you can instruct the build process to
1712ignore a dependency by using a line like the following:
1713PACKAGE_ARCHS[vardepsexclude] = "MACHINE" This example ensures that the
1714```PACKAGE_ARCHS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCHS>`__ variable
1715does not depend on the value of
1716```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__, even if it does
1717reference it.
1718
1719Equally, there are cases where you need to add dependencies BitBake is
1720not able to find. You can accomplish this by using a line like the
1721following: PACKAGE_ARCHS[vardeps] = "MACHINE" This example explicitly
1722adds the ``MACHINE`` variable as a dependency for ``PACKAGE_ARCHS``.
1723
1724As an example, consider a case with in-line Python where BitBake is not
1725able to figure out dependencies. When running in debug mode (i.e. using
1726``-DDD``), BitBake produces output when it discovers something for which
1727it cannot figure out dependencies. The Yocto Project team has currently
1728not managed to cover those dependencies in detail and is aware of the
1729need to fix this situation.
1730
1731Thus far, this section has limited discussion to the direct inputs into
1732a task. Information based on direct inputs is referred to as the
1733"basehash" in the code. However, the question of a task's indirect
1734inputs still exits - items already built and present in the `Build
1735Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. The checksum (or
1736signature) for a particular task needs to add the hashes of all the
1737tasks on which the particular task depends. Choosing which dependencies
1738to add is a policy decision. However, the effect is to generate a master
1739checksum that combines the basehash and the hashes of the task's
1740dependencies.
1741
1742At the code level, a variety of ways exist by which both the basehash
1743and the dependent task hashes can be influenced. Within the BitBake
1744configuration file, you can give BitBake some extra information to help
1745it construct the basehash. The following statement effectively results
1746in a list of global variable dependency excludes (i.e. variables never
1747included in any checksum): BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH
1748PWD BB_TASKHASH BBPATH DL_DIR \\ SSTATE_DIR THISDIR FILESEXTRAPATHS
1749FILE_DIRNAME HOME LOGNAME SHELL TERM \\ USER FILESPATH STAGING_DIR_HOST
1750STAGING_DIR_TARGET COREBASE PRSERV_HOST \\ PRSERV_DUMPDIR
1751PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \\ CCACHE_DIR
1752EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX" The
1753previous example excludes
1754```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__ since that variable
1755is actually constructed as a path within
1756```TMPDIR`` <&YOCTO_DOCS_REF_URL;#var-TMPDIR>`__, which is on the
1757whitelist.
1758
1759The rules for deciding which hashes of dependent tasks to include
1760through dependency chains are more complex and are generally
1761accomplished with a Python function. The code in
1762``meta/lib/oe/sstatesig.py`` shows two examples of this and also
1763illustrates how you can insert your own policy into the system if so
1764desired. This file defines the two basic signature generators
1765`OE-Core <&YOCTO_DOCS_REF_URL;#oe-core>`__ uses: "OEBasic" and
1766"OEBasicHash". By default, a dummy "noop" signature handler is enabled
1767in BitBake. This means that behavior is unchanged from previous
1768versions. OE-Core uses the "OEBasicHash" signature handler by default
1769through this setting in the ``bitbake.conf`` file: BB_SIGNATURE_HANDLER
1770?= "OEBasicHash" The "OEBasicHash" ``BB_SIGNATURE_HANDLER`` is the same
1771as the "OEBasic" version but adds the task hash to the `stamp
1772files <#stamp-files-and-the-rerunning-of-tasks>`__. This results in any
1773metadata change that changes the task hash, automatically causing the
1774task to be run again. This removes the need to bump
1775```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__ values, and changes to metadata
1776automatically ripple across the build.
1777
1778It is also worth noting that the end result of these signature
1779generators is to make some dependency and hash information available to
1780the build. This information includes:
1781
1782- ``BB_BASEHASH_task-``\ taskname: The base hashes for each task in the
1783 recipe.
1784
1785- ``BB_BASEHASH_``\ filename\ ``:``\ taskname: The base hashes for each
1786 dependent task.
1787
1788- ``BBHASHDEPS_``\ filename\ ``:``\ taskname: The task dependencies for
1789 each task.
1790
1791- ``BB_TASKHASH``: The hash of the currently running task.
1792
1793Shared State
1794------------
1795
1796Checksums and dependencies, as discussed in the previous section, solve
1797half the problem of supporting a shared state. The other half of the
1798problem is being able to use checksum information during the build and
1799being able to reuse or rebuild specific components.
1800
1801The ```sstate`` <&YOCTO_DOCS_REF_URL;#ref-classes-sstate>`__ class is a
1802relatively generic implementation of how to "capture" a snapshot of a
1803given task. The idea is that the build process does not care about the
1804source of a task's output. Output could be freshly built or it could be
1805downloaded and unpacked from somewhere. In other words, the build
1806process does not need to worry about its origin.
1807
1808Two types of output exist. One type is just about creating a directory
1809in ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__. A good example is
1810the output of either
1811```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ or
1812```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__. The other
1813type of output occurs when a set of data is merged into a shared
1814directory tree such as the sysroot.
1815
1816The Yocto Project team has tried to keep the details of the
1817implementation hidden in ``sstate`` class. From a user's perspective,
1818adding shared state wrapping to a task is as simple as this
1819```do_deploy`` <&YOCTO_DOCS_REF_URL;#ref-tasks-deploy>`__ example taken
1820from the ```deploy`` <&YOCTO_DOCS_REF_URL;#ref-classes-deploy>`__ class:
1821DEPLOYDIR = "${WORKDIR}/deploy-${PN}" SSTATETASKS += "do_deploy"
1822do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
1823do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" python
1824do_deploy_setscene () { sstate_setscene(d) } addtask do_deploy_setscene
1825do_deploy[dirs] = "${DEPLOYDIR} ${B}" do_deploy[stamp-extra-info] =
1826"${MACHINE_ARCH}" The following list explains the previous example:
1827
1828- Adding "do_deploy" to ``SSTATETASKS`` adds some required
1829 sstate-related processing, which is implemented in the
1830 ```sstate`` <&YOCTO_DOCS_REF_URL;#ref-classes-sstate>`__ class, to
1831 before and after the
1832 ```do_deploy`` <&YOCTO_DOCS_REF_URL;#ref-tasks-deploy>`__ task.
1833
1834- The ``do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"`` declares that
1835 ``do_deploy`` places its output in ``${DEPLOYDIR}`` when run normally
1836 (i.e. when not using the sstate cache). This output becomes the input
1837 to the shared state cache.
1838
1839- The ``do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"`` line
1840 causes the contents of the shared state cache to be copied to
1841 ``${DEPLOY_DIR_IMAGE}``.
1842
1843 .. note::
1844
1845 If
1846 do_deploy
1847 is not already in the shared state cache or if its input checksum
1848 (signature) has changed from when the output was cached, the task
1849 runs to populate the shared state cache, after which the contents
1850 of the shared state cache is copied to
1851 ${DEPLOY_DIR_IMAGE}
1852 . If
1853 do_deploy
1854 is in the shared state cache and its signature indicates that the
1855 cached output is still valid (i.e. if no relevant task inputs have
1856 changed), then the contents of the shared state cache copies
1857 directly to
1858 ${DEPLOY_DIR_IMAGE}
1859 by the
1860 do_deploy_setscene
1861 task instead, skipping the
1862 do_deploy
1863 task.
1864
1865- The following task definition is glue logic needed to make the
1866 previous settings effective: python do_deploy_setscene () {
1867 sstate_setscene(d) } addtask do_deploy_setscene ``sstate_setscene()``
1868 takes the flags above as input and accelerates the ``do_deploy`` task
1869 through the shared state cache if possible. If the task was
1870 accelerated, ``sstate_setscene()`` returns True. Otherwise, it
1871 returns False, and the normal ``do_deploy`` task runs. For more
1872 information, see the "`setscene <&YOCTO_DOCS_BB_URL;#setscene>`__"
1873 section in the BitBake User Manual.
1874
1875- The ``do_deploy[dirs] = "${DEPLOYDIR} ${B}"`` line creates
1876 ``${DEPLOYDIR}`` and ``${B}`` before the ``do_deploy`` task runs, and
1877 also sets the current working directory of ``do_deploy`` to ``${B}``.
1878 For more information, see the "`Variable
1879 Flags <&YOCTO_DOCS_BB_URL;#variable-flags>`__" section in the BitBake
1880 User Manual.
1881
1882 .. note::
1883
1884 In cases where
1885 sstate-inputdirs
1886 and
1887 sstate-outputdirs
1888 would be the same, you can use
1889 sstate-plaindirs
1890 . For example, to preserve the
1891 ${PKGD}
1892 and
1893 ${PKGDEST}
1894 output from the
1895 do_package
1896 task, use the following:
1897 ::
1898
1899 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
1900
1901
1902- The ``do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"`` line appends
1903 extra metadata to the `stamp
1904 file <#stamp-files-and-the-rerunning-of-tasks>`__. In this case, the
1905 metadata makes the task specific to a machine's architecture. See
1906 "`The Task List <&YOCTO_DOCS_BB_URL;#ref-bitbake-tasklist>`__"
1907 section in the BitBake User Manual for more information on the
1908 ``stamp-extra-info`` flag.
1909
1910- ``sstate-inputdirs`` and ``sstate-outputdirs`` can also be used with
1911 multiple directories. For example, the following declares
1912 ``PKGDESTWORK`` and ``SHLIBWORK`` as shared state input directories,
1913 which populates the shared state cache, and ``PKGDATA_DIR`` and
1914 ``SHLIBSDIR`` as the corresponding shared state output directories:
1915 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
1916 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
1917
1918- These methods also include the ability to take a lockfile when
1919 manipulating shared state directory structures, for cases where file
1920 additions or removals are sensitive: do_package[sstate-lockfile] =
1921 "${PACKAGELOCK}"
1922
1923Behind the scenes, the shared state code works by looking in
1924```SSTATE_DIR`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR>`__ and
1925```SSTATE_MIRRORS`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_MIRRORS>`__ for
1926shared state files. Here is an example: SSTATE_MIRRORS ?= "\\ file://.\*
1927http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \\n \\
1928file://.\* file:///some/local/dir/sstate/PATH"
1929
1930.. note::
1931
1932 The shared state directory (
1933 SSTATE_DIR
1934 ) is organized into two-character subdirectories, where the
1935 subdirectory names are based on the first two characters of the hash.
1936 If the shared state directory structure for a mirror has the same
1937 structure as
1938 SSTATE_DIR
1939 , you must specify "PATH" as part of the URI to enable the build
1940 system to map to the appropriate subdirectory.
1941
1942The shared state package validity can be detected just by looking at the
1943filename since the filename contains the task checksum (or signature) as
1944described earlier in this section. If a valid shared state package is
1945found, the build process downloads it and uses it to accelerate the
1946task.
1947
1948The build processes use the ``*_setscene`` tasks for the task
1949acceleration phase. BitBake goes through this phase before the main
1950execution code and tries to accelerate any tasks for which it can find
1951shared state packages. If a shared state package for a task is
1952available, the shared state package is used. This means the task and any
1953tasks on which it is dependent are not executed.
1954
1955As a real world example, the aim is when building an IPK-based image,
1956only the
1957```do_package_write_ipk`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_ipk>`__
1958tasks would have their shared state packages fetched and extracted.
1959Since the sysroot is not used, it would never get extracted. This is
1960another reason why a task-based approach is preferred over a
1961recipe-based approach, which would have to install the output from every
1962task.
1963
1964Automatically Added Runtime Dependencies
1965========================================
1966
1967The OpenEmbedded build system automatically adds common types of runtime
1968dependencies between packages, which means that you do not need to
1969explicitly declare the packages using
1970```RDEPENDS`` <&YOCTO_DOCS_REF_URL;#var-RDEPENDS>`__. Three automatic
1971mechanisms exist (``shlibdeps``, ``pcdeps``, and ``depchains``) that
1972handle shared libraries, package configuration (pkg-config) modules, and
1973``-dev`` and ``-dbg`` packages, respectively. For other types of runtime
1974dependencies, you must manually declare the dependencies.
1975
1976- ``shlibdeps``: During the
1977 ```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__ task of
1978 each recipe, all shared libraries installed by the recipe are
1979 located. For each shared library, the package that contains the
1980 shared library is registered as providing the shared library. More
1981 specifically, the package is registered as providing the
1982 `soname <https://en.wikipedia.org/wiki/Soname>`__ of the library. The
1983 resulting shared-library-to-package mapping is saved globally in
1984 ```PKGDATA_DIR`` <&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR>`__ by the
1985 ```do_packagedata`` <&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata>`__
1986 task.
1987
1988 Simultaneously, all executables and shared libraries installed by the
1989 recipe are inspected to see what shared libraries they link against.
1990 For each shared library dependency that is found, ``PKGDATA_DIR`` is
1991 queried to see if some package (likely from a different recipe)
1992 contains the shared library. If such a package is found, a runtime
1993 dependency is added from the package that depends on the shared
1994 library to the package that contains the library.
1995
1996 The automatically added runtime dependency also includes a version
1997 restriction. This version restriction specifies that at least the
1998 current version of the package that provides the shared library must
1999 be used, as if "package (>= version)" had been added to ``RDEPENDS``.
2000 This forces an upgrade of the package containing the shared library
2001 when installing the package that depends on the library, if needed.
2002
2003 If you want to avoid a package being registered as providing a
2004 particular shared library (e.g. because the library is for internal
2005 use only), then add the library to
2006 ```PRIVATE_LIBS`` <&YOCTO_DOCS_REF_URL;#var-PRIVATE_LIBS>`__ inside
2007 the package's recipe.
2008
2009- ``pcdeps``: During the ``do_package`` task of each recipe, all
2010 pkg-config modules (``*.pc`` files) installed by the recipe are
2011 located. For each module, the package that contains the module is
2012 registered as providing the module. The resulting module-to-package
2013 mapping is saved globally in ``PKGDATA_DIR`` by the
2014 ``do_packagedata`` task.
2015
2016 Simultaneously, all pkg-config modules installed by the recipe are
2017 inspected to see what other pkg-config modules they depend on. A
2018 module is seen as depending on another module if it contains a
2019 "Requires:" line that specifies the other module. For each module
2020 dependency, ``PKGDATA_DIR`` is queried to see if some package
2021 contains the module. If such a package is found, a runtime dependency
2022 is added from the package that depends on the module to the package
2023 that contains the module.
2024
2025 .. note::
2026
2027 The
2028 pcdeps
2029 mechanism most often infers dependencies between
2030 -dev
2031 packages.
2032
2033- ``depchains``: If a package ``foo`` depends on a package ``bar``,
2034 then ``foo-dev`` and ``foo-dbg`` are also made to depend on
2035 ``bar-dev`` and ``bar-dbg``, respectively. Taking the ``-dev``
2036 packages as an example, the ``bar-dev`` package might provide headers
2037 and shared library symlinks needed by ``foo-dev``, which shows the
2038 need for a dependency between the packages.
2039
2040 The dependencies added by ``depchains`` are in the form of
2041 ```RRECOMMENDS`` <&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS>`__.
2042
2043 .. note::
2044
2045 By default,
2046 foo-dev
2047 also has an
2048 RDEPENDS
2049 -style dependency on
2050 foo
2051 , because the default value of
2052 RDEPENDS_${PN}-dev
2053 (set in
2054 bitbake.conf
2055 ) includes "${PN}".
2056
2057 To ensure that the dependency chain is never broken, ``-dev`` and
2058 ``-dbg`` packages are always generated by default, even if the
2059 packages turn out to be empty. See the
2060 ```ALLOW_EMPTY`` <&YOCTO_DOCS_REF_URL;#var-ALLOW_EMPTY>`__ variable
2061 for more information.
2062
2063The ``do_package`` task depends on the ``do_packagedata`` task of each
2064recipe in ```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ through use
2065of a ``[``\ ```deptask`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__\ ``]``
2066declaration, which guarantees that the required
2067shared-library/module-to-package mapping information will be available
2068when needed as long as ``DEPENDS`` has been correctly set.
2069
2070Fakeroot and Pseudo
2071===================
2072
2073Some tasks are easier to implement when allowed to perform certain
2074operations that are normally reserved for the root user (e.g.
2075```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__,
2076```do_package_write*`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb>`__,
2077```do_rootfs`` <&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs>`__, and
2078```do_image*`` <&YOCTO_DOCS_REF_URL;#ref-tasks-image>`__). For example,
2079the ``do_install`` task benefits from being able to set the UID and GID
2080of installed files to arbitrary values.
2081
2082One approach to allowing tasks to perform root-only operations would be
2083to require `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ to run as
2084root. However, this method is cumbersome and has security issues. The
2085approach that is actually used is to run tasks that benefit from root
2086privileges in a "fake" root environment. Within this environment, the
2087task and its child processes believe that they are running as the root
2088user, and see an internally consistent view of the filesystem. As long
2089as generating the final output (e.g. a package or an image) does not
2090require root privileges, the fact that some earlier steps ran in a fake
2091root environment does not cause problems.
2092
2093The capability to run tasks in a fake root environment is known as
2094"`fakeroot <http://man.he.net/man1/fakeroot>`__", which is derived from
2095the BitBake keyword/variable flag that requests a fake root environment
2096for a task.
2097
2098In the `OpenEmbedded build
2099system <&YOCTO_DOCS_REF_URL;#build-system-term>`__, the program that
2100implements fakeroot is known as
2101`Pseudo <https://www.yoctoproject.org/software-item/pseudo/>`__. Pseudo
2102overrides system calls by using the environment variable ``LD_PRELOAD``,
2103which results in the illusion of running as root. To keep track of
2104"fake" file ownership and permissions resulting from operations that
2105require root permissions, Pseudo uses an SQLite 3 database. This
2106database is stored in
2107``${``\ ```WORKDIR`` <&YOCTO_DOCS_REF_URL;#var-WORKDIR>`__\ ``}/pseudo/files.db``
2108for individual recipes. Storing the database in a file as opposed to in
2109memory gives persistence between tasks and builds, which is not
2110accomplished using fakeroot.
2111
2112.. note::
2113
2114 If you add your own task that manipulates the same files or
2115 directories as a fakeroot task, then that task also needs to run
2116 under fakeroot. Otherwise, the task cannot run root-only operations,
2117 and cannot see the fake file ownership and permissions set by the
2118 other task. You need to also add a dependency on
2119 virtual/fakeroot-native:do_populate_sysroot
2120 , giving the following:
2121 ::
2122
2123 fakeroot do_mytask () {
2124 ...
2125 }
2126 do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot"
2127
2128
2129For more information, see the
2130```FAKEROOT*`` <&YOCTO_DOCS_BB_URL;#var-FAKEROOT>`__ variables in the
2131BitBake User Manual. You can also reference the "`Why Not
2132Fakeroot? <https://github.com/wrpseudo/pseudo/wiki/WhyNotFakeroot>`__"
2133article for background information on Fakeroot and Pseudo.
diff --git a/documentation/overview-manual/overview-manual-development-environment.rst b/documentation/overview-manual/overview-manual-development-environment.rst
new file mode 100644
index 0000000000..4e6770c4f4
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-development-environment.rst
@@ -0,0 +1,656 @@
1*****************************************
2The Yocto Project Development Environment
3*****************************************
4
5This chapter takes a look at the Yocto Project development environment.
6The chapter provides Yocto Project Development environment concepts that
7help you understand how work is accomplished in an open source
8environment, which is very different as compared to work accomplished in
9a closed, proprietary environment.
10
11Specifically, this chapter addresses open source philosophy, source
12repositories, workflows, Git, and licensing.
13
14Open Source Philosophy
15======================
16
17Open source philosophy is characterized by software development directed
18by peer production and collaboration through an active community of
19developers. Contrast this to the more standard centralized development
20models used by commercial software companies where a finite set of
21developers produces a product for sale using a defined set of procedures
22that ultimately result in an end product whose architecture and source
23material are closed to the public.
24
25Open source projects conceptually have differing concurrent agendas,
26approaches, and production. These facets of the development process can
27come from anyone in the public (community) who has a stake in the
28software project. The open source environment contains new copyright,
29licensing, domain, and consumer issues that differ from the more
30traditional development environment. In an open source environment, the
31end product, source material, and documentation are all available to the
32public at no cost.
33
34A benchmark example of an open source project is the Linux kernel, which
35was initially conceived and created by Finnish computer science student
36Linus Torvalds in 1991. Conversely, a good example of a non-open source
37project is the Windows family of operating systems developed by
38Microsoft Corporation.
39
40Wikipedia has a good historical description of the Open Source
41Philosophy `here <http://en.wikipedia.org/wiki/Open_source>`__. You can
42also find helpful information on how to participate in the Linux
43Community
44`here <http://ldn.linuxfoundation.org/book/how-participate-linux-community>`__.
45
46.. _gs-the-development-host:
47
48The Development Host
49====================
50
51A development host or `build
52host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ is key to
53using the Yocto Project. Because the goal of the Yocto Project is to
54develop images or applications that run on embedded hardware,
55development of those images and applications generally takes place on a
56system not intended to run the software - the development host.
57
58You need to set up a development host in order to use it with the Yocto
59Project. Most find that it is best to have a native Linux machine
60function as the development host. However, it is possible to use a
61system that does not run Linux as its operating system as your
62development host. When you have a Mac or Windows-based system, you can
63set it up as the development host by using
64`CROPS <https://github.com/crops/poky-container>`__, which leverages
65`Docker Containers <https://www.docker.com/>`__. Once you take the steps
66to set up a CROPS machine, you effectively have access to a shell
67environment that is similar to what you see when using a Linux-based
68development host. For the steps needed to set up a system using CROPS,
69see the "`Setting Up to Use CROss PlatformS
70(CROPS) <&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops>`__" section in
71the Yocto Project Development Tasks Manual.
72
73If your development host is going to be a system that runs a Linux
74distribution, steps still exist that you must take to prepare the system
75for use with the Yocto Project. You need to be sure that the Linux
76distribution on the system is one that supports the Yocto Project. You
77also need to be sure that the correct set of host packages are installed
78that allow development using the Yocto Project. For the steps needed to
79set up a development host that runs Linux, see the "`Setting Up a Native
80Linux Host <&YOCTO_DOCS_DEV_URL;#setting-up-a-native-linux-host>`__"
81section in the Yocto Project Development Tasks Manual.
82
83Once your development host is set up to use the Yocto Project, several
84methods exist for you to do work in the Yocto Project environment:
85
86- *Command Lines, BitBake, and Shells:* Traditional development in the
87 Yocto Project involves using the `OpenEmbedded build
88 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__, which uses
89 BitBake, in a command-line environment from a shell on your
90 development host. You can accomplish this from a host that is a
91 native Linux machine or from a host that has been set up with CROPS.
92 Either way, you create, modify, and build images and applications all
93 within a shell-based environment using components and tools available
94 through your Linux distribution and the Yocto Project.
95
96 For a general flow of the build procedures, see the "`Building a
97 Simple Image <&YOCTO_DOCS_DEV_URL;#dev-building-a-simple-image>`__"
98 section in the Yocto Project Development Tasks Manual.
99
100- *Board Support Package (BSP) Development:* Development of BSPs
101 involves using the Yocto Project to create and test layers that allow
102 easy development of images and applications targeted for specific
103 hardware. To development BSPs, you need to take some additional steps
104 beyond what was described in setting up a development host.
105
106 The `Yocto Project Board Support Package (BSP) Developer's
107 Guide <&YOCTO_DOCS_BSP_URL;>`__ provides BSP-related development
108 information. For specifics on development host preparation, see the
109 "`Preparing Your Build Host to Work With BSP
110 Layers <&YOCTO_DOCS_BSP_URL;#preparing-your-build-host-to-work-with-bsp-layers>`__"
111 section in the Yocto Project Board Support Package (BSP) Developer's
112 Guide.
113
114- *Kernel Development:* If you are going to be developing kernels using
115 the Yocto Project you likely will be using ``devtool``. A workflow
116 using ``devtool`` makes kernel development quicker by reducing
117 iteration cycle times.
118
119 The `Yocto Project Linux Kernel Development
120 Manual <&YOCTO_DOCS_KERNEL_DEV_URL;>`__ provides kernel-related
121 development information. For specifics on development host
122 preparation, see the "`Preparing the Build Host to Work on the
123 Kernel <&YOCTO_DOCS_KERNEL_DEV_URL;#preparing-the-build-host-to-work-on-the-kernel>`__"
124 section in the Yocto Project Linux Kernel Development Manual.
125
126- *Using Toaster:* The other Yocto Project development method that
127 involves an interface that effectively puts the Yocto Project into
128 the background is Toaster. Toaster provides an interface to the
129 OpenEmbedded build system. The interface enables you to configure and
130 run your builds. Information about builds is collected and stored in
131 a database. You can use Toaster to configure and start builds on
132 multiple remote build servers.
133
134 For steps that show you how to set up your development host to use
135 Toaster and on how to use Toaster in general, see the `Toaster User
136 Manual <&YOCTO_DOCS_TOAST_URL;>`__.
137
138.. _yocto-project-repositories:
139
140Yocto Project Source Repositories
141=================================
142
143The Yocto Project team maintains complete source repositories for all
144Yocto Project files at ` <&YOCTO_GIT_URL;>`__. This web-based source
145code browser is organized into categories by function such as IDE
146Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth. From the
147interface, you can click on any particular item in the "Name" column and
148see the URL at the bottom of the page that you need to clone a Git
149repository for that particular item. Having a local Git repository of
150the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__, which
151is usually named "poky", allows you to make changes, contribute to the
152history, and ultimately enhance the Yocto Project's tools, Board Support
153Packages, and so forth.
154
155For any supported release of Yocto Project, you can also go to the
156`Yocto Project Website <&YOCTO_HOME_URL;>`__ and select the "DOWNLOADS"
157item from the "SOFTWARE" menu and get a released tarball of the ``poky``
158repository, any supported BSP tarball, or Yocto Project tools. Unpacking
159these tarballs gives you a snapshot of the released files.
160
161.. note::
162
163 - The recommended method for setting up the Yocto Project `Source
164 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ and the files
165 for supported BSPs (e.g., ``meta-intel``) is to use `Git <#git>`__
166 to create a local copy of the upstream repositories.
167
168 - Be sure to always work in matching branches for both the selected
169 BSP repository and the Source Directory (i.e. ``poky``)
170 repository. For example, if you have checked out the "master"
171 branch of ``poky`` and you are going to use ``meta-intel``, be
172 sure to checkout the "master" branch of ``meta-intel``.
173
174In summary, here is where you can get the project files needed for
175development:
176
177- `Source Repositories: <&YOCTO_GIT_URL;>`__ This area contains IDE
178 Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and
179 Yocto Metadata Layers. You can create local copies of Git
180 repositories for each of these areas.
181
182 For steps on how to view and access these upstream Git repositories,
183 see the "`Accessing Source
184 Repositories <&YOCTO_DOCS_DEV_URL;#accessing-source-repositories>`__"
185 Section in the Yocto Project Development Tasks Manual.
186
187- `Index of /releases: <&YOCTO_DL_URL;/releases/>`__ This is an index
188 of releases such as Poky, Pseudo, installers for cross-development
189 toolchains, miscellaneous support and all released versions of Yocto
190 Project in the form of images or tarballs. Downloading and extracting
191 these files does not produce a local copy of the Git repository but
192 rather a snapshot of a particular release or image.
193
194 For steps on how to view and access these files, see the "`Accessing
195 Index of
196 Releases <&YOCTO_DOCS_DEV_URL;#accessing-index-of-releases>`__"
197 section in the Yocto Project Development Tasks Manual.
198
199- *"DOWNLOADS" page for the*\ `Yocto Project
200 Website <&YOCTO_HOME_URL;>`__\ *:*
201
202 The Yocto Project website includes a "DOWNLOADS" page accessible
203 through the "SOFTWARE" menu that allows you to download any Yocto
204 Project release, tool, and Board Support Package (BSP) in tarball
205 form. The tarballs are similar to those found in the `Index of
206 /releases: <&YOCTO_DL_URL;/releases/>`__ area.
207
208 For steps on how to use the "DOWNLOADS" page, see the "`Using the
209 Downloads Page <&YOCTO_DOCS_DEV_URL;#using-the-downloads-page>`__"
210 section in the Yocto Project Development Tasks Manual.
211
212.. _gs-git-workflows-and-the-yocto-project:
213
214Git Workflows and the Yocto Project
215===================================
216
217Developing using the Yocto Project likely requires the use of
218`Git <#git>`__. Git is a free, open source distributed version control
219system used as part of many collaborative design environments. This
220section provides workflow concepts using the Yocto Project and Git. In
221particular, the information covers basic practices that describe roles
222and actions in a collaborative development environment.
223
224.. note::
225
226 If you are familiar with this type of development environment, you
227 might not want to read this section.
228
229The Yocto Project files are maintained using Git in "branches" whose Git
230histories track every change and whose structures provide branches for
231all diverging functionality. Although there is no need to use Git, many
232open source projects do so.
233
234For the Yocto Project, a key individual called the "maintainer" is
235responsible for the integrity of the "master" branch of a given Git
236repository. The "master" branch is the “upstream” repository from which
237final or most recent builds of a project occur. The maintainer is
238responsible for accepting changes from other developers and for
239organizing the underlying branch structure to reflect release strategies
240and so forth.
241
242.. note::
243
244 For information on finding out who is responsible for (maintains) a
245 particular area of code in the Yocto Project, see the "
246 Submitting a Change to the Yocto Project
247 " section of the Yocto Project Development Tasks Manual.
248
249The Yocto Project ``poky`` Git repository also has an upstream
250contribution Git repository named ``poky-contrib``. You can see all the
251branches in this repository using the web interface of the `Source
252Repositories <&YOCTO_GIT_URL;>`__ organized within the "Poky Support"
253area. These branches hold changes (commits) to the project that have
254been submitted or committed by the Yocto Project development team and by
255community members who contribute to the project. The maintainer
256determines if the changes are qualified to be moved from the "contrib"
257branches into the "master" branch of the Git repository.
258
259Developers (including contributing community members) create and
260maintain cloned repositories of upstream branches. The cloned
261repositories are local to their development platforms and are used to
262develop changes. When a developer is satisfied with a particular feature
263or change, they "push" the change to the appropriate "contrib"
264repository.
265
266Developers are responsible for keeping their local repository up-to-date
267with whatever upstream branch they are working against. They are also
268responsible for straightening out any conflicts that might arise within
269files that are being worked on simultaneously by more than one person.
270All this work is done locally on the development host before anything is
271pushed to a "contrib" area and examined at the maintainer’s level.
272
273A somewhat formal method exists by which developers commit changes and
274push them into the "contrib" area and subsequently request that the
275maintainer include them into an upstream branch. This process is called
276“submitting a patch” or "submitting a change." For information on
277submitting patches and changes, see the "`Submitting a Change to the
278Yocto Project <&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change>`__" section
279in the Yocto Project Development Tasks Manual.
280
281In summary, a single point of entry exists for changes into a "master"
282or development branch of the Git repository, which is controlled by the
283project’s maintainer. And, a set of developers exist who independently
284develop, test, and submit changes to "contrib" areas for the maintainer
285to examine. The maintainer then chooses which changes are going to
286become a permanent part of the project.
287
288While each development environment is unique, there are some best
289practices or methods that help development run smoothly. The following
290list describes some of these practices. For more information about Git
291workflows, see the workflow topics in the `Git Community
292Book <http://book.git-scm.com>`__.
293
294- *Make Small Changes:* It is best to keep the changes you commit small
295 as compared to bundling many disparate changes into a single commit.
296 This practice not only keeps things manageable but also allows the
297 maintainer to more easily include or refuse changes.
298
299- *Make Complete Changes:* It is also good practice to leave the
300 repository in a state that allows you to still successfully build
301 your project. In other words, do not commit half of a feature, then
302 add the other half as a separate, later commit. Each commit should
303 take you from one buildable project state to another buildable state.
304
305- *Use Branches Liberally:* It is very easy to create, use, and delete
306 local branches in your working Git repository on the development
307 host. You can name these branches anything you like. It is helpful to
308 give them names associated with the particular feature or change on
309 which you are working. Once you are done with a feature or change and
310 have merged it into your local master branch, simply discard the
311 temporary branch.
312
313- *Merge Changes:* The ``git merge`` command allows you to take the
314 changes from one branch and fold them into another branch. This
315 process is especially helpful when more than a single developer might
316 be working on different parts of the same feature. Merging changes
317 also automatically identifies any collisions or "conflicts" that
318 might happen as a result of the same lines of code being altered by
319 two different developers.
320
321- *Manage Branches:* Because branches are easy to use, you should use a
322 system where branches indicate varying levels of code readiness. For
323 example, you can have a "work" branch to develop in, a "test" branch
324 where the code or change is tested, a "stage" branch where changes
325 are ready to be committed, and so forth. As your project develops,
326 you can merge code across the branches to reflect ever-increasing
327 stable states of the development.
328
329- *Use Push and Pull:* The push-pull workflow is based on the concept
330 of developers "pushing" local commits to a remote repository, which
331 is usually a contribution repository. This workflow is also based on
332 developers "pulling" known states of the project down into their
333 local development repositories. The workflow easily allows you to
334 pull changes submitted by other developers from the upstream
335 repository into your work area ensuring that you have the most recent
336 software on which to develop. The Yocto Project has two scripts named
337 ``create-pull-request`` and ``send-pull-request`` that ship with the
338 release to facilitate this workflow. You can find these scripts in
339 the ``scripts`` folder of the `Source
340 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__. For information
341 on how to use these scripts, see the "`Using Scripts to Push a Change
342 Upstream and Request a
343 Pull <&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream>`__" section in
344 the Yocto Project Development Tasks Manual.
345
346- *Patch Workflow:* This workflow allows you to notify the maintainer
347 through an email that you have a change (or patch) you would like
348 considered for the "master" branch of the Git repository. To send
349 this type of change, you format the patch and then send the email
350 using the Git commands ``git format-patch`` and ``git send-email``.
351 For information on how to use these scripts, see the "`Submitting a
352 Change to the Yocto
353 Project <&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change>`__" section in
354 the Yocto Project Development Tasks Manual.
355
356Git
357===
358
359The Yocto Project makes extensive use of Git, which is a free, open
360source distributed version control system. Git supports distributed
361development, non-linear development, and can handle large projects. It
362is best that you have some fundamental understanding of how Git tracks
363projects and how to work with Git if you are going to use the Yocto
364Project for development. This section provides a quick overview of how
365Git works and provides you with a summary of some essential Git
366commands.
367
368.. note::
369
370 - For more information on Git, see
371 ` <http://git-scm.com/documentation>`__.
372
373 - If you need to download Git, it is recommended that you add Git to
374 your system through your distribution's "software store" (e.g. for
375 Ubuntu, use the Ubuntu Software feature). For the Git download
376 page, see ` <http://git-scm.com/download>`__.
377
378 - For information beyond the introductory nature in this section,
379 see the "`Locating Yocto Project Source
380 Files <&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files>`__"
381 section in the Yocto Project Development Tasks Manual.
382
383Repositories, Tags, and Branches
384--------------------------------
385
386As mentioned briefly in the previous section and also in the "`Git
387Workflows and the Yocto
388Project <#gs-git-workflows-and-the-yocto-project>`__" section, the Yocto
389Project maintains source repositories at ` <&YOCTO_GIT_URL;>`__. If you
390look at this web-interface of the repositories, each item is a separate
391Git repository.
392
393Git repositories use branching techniques that track content change (not
394files) within a project (e.g. a new feature or updated documentation).
395Creating a tree-like structure based on project divergence allows for
396excellent historical information over the life of a project. This
397methodology also allows for an environment from which you can do lots of
398local experimentation on projects as you develop changes or new
399features.
400
401A Git repository represents all development efforts for a given project.
402For example, the Git repository ``poky`` contains all changes and
403developments for that repository over the course of its entire life.
404That means that all changes that make up all releases are captured. The
405repository maintains a complete history of changes.
406
407You can create a local copy of any repository by "cloning" it with the
408``git clone`` command. When you clone a Git repository, you end up with
409an identical copy of the repository on your development system. Once you
410have a local copy of a repository, you can take steps to develop
411locally. For examples on how to clone Git repositories, see the
412"`Locating Yocto Project Source
413Files <&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files>`__"
414section in the Yocto Project Development Tasks Manual.
415
416It is important to understand that Git tracks content change and not
417files. Git uses "branches" to organize different development efforts.
418For example, the ``poky`` repository has several branches that include
419the current "DISTRO_NAME_NO_CAP" branch, the "master" branch, and many
420branches for past Yocto Project releases. You can see all the branches
421by going to ` <&YOCTO_GIT_URL;/cgit.cgi/poky/>`__ and clicking on the
422``[...]`` link beneath the "Branch" heading.
423
424Each of these branches represents a specific area of development. The
425"master" branch represents the current or most recent development. All
426other branches represent offshoots of the "master" branch.
427
428When you create a local copy of a Git repository, the copy has the same
429set of branches as the original. This means you can use Git to create a
430local working area (also called a branch) that tracks a specific
431development branch from the upstream source Git repository. in other
432words, you can define your local Git environment to work on any
433development branch in the repository. To help illustrate, consider the
434following example Git commands: $ cd ~ $ git clone
435git://git.yoctoproject.org/poky $ cd poky $ git checkout -b
436DISTRO_NAME_NO_CAP origin/DISTRO_NAME_NO_CAP In the previous example
437after moving to the home directory, the ``git clone`` command creates a
438local copy of the upstream ``poky`` Git repository. By default, Git
439checks out the "master" branch for your work. After changing the working
440directory to the new local repository (i.e. ``poky``), the
441``git checkout`` command creates and checks out a local branch named
442"DISTRO_NAME_NO_CAP", which tracks the upstream
443"origin/DISTRO_NAME_NO_CAP" branch. Changes you make while in this
444branch would ultimately affect the upstream "DISTRO_NAME_NO_CAP" branch
445of the ``poky`` repository.
446
447It is important to understand that when you create and checkout a local
448working branch based on a branch name, your local environment matches
449the "tip" of that particular development branch at the time you created
450your local branch, which could be different from the files in the
451"master" branch of the upstream repository. In other words, creating and
452checking out a local branch based on the "DISTRO_NAME_NO_CAP" branch
453name is not the same as checking out the "master" branch in the
454repository. Keep reading to see how you create a local snapshot of a
455Yocto Project Release.
456
457Git uses "tags" to mark specific changes in a repository branch
458structure. Typically, a tag is used to mark a special point such as the
459final change (or commit) before a project is released. You can see the
460tags used with the ``poky`` Git repository by going to
461` <&YOCTO_GIT_URL;/cgit.cgi/poky/>`__ and clicking on the ``[...]`` link
462beneath the "Tag" heading.
463
464Some key tags for the ``poky`` repository are ``jethro-14.0.3``,
465``morty-16.0.1``, ``pyro-17.0.0``, and
466``DISTRO_NAME_NO_CAP-POKYVERSION``. These tags represent Yocto Project
467releases.
468
469When you create a local copy of the Git repository, you also have access
470to all the tags in the upstream repository. Similar to branches, you can
471create and checkout a local working Git branch based on a tag name. When
472you do this, you get a snapshot of the Git repository that reflects the
473state of the files when the change was made associated with that tag.
474The most common use is to checkout a working branch that matches a
475specific Yocto Project release. Here is an example: $ cd ~ $ git clone
476git://git.yoctoproject.org/poky $ cd poky $ git fetch --tags $ git
477checkout tags/rocko-18.0.0 -b my_rocko-18.0.0 In this example, the name
478of the top-level directory of your local Yocto Project repository is
479``poky``. After moving to the ``poky`` directory, the ``git fetch``
480command makes all the upstream tags available locally in your
481repository. Finally, the ``git checkout`` command creates and checks out
482a branch named "my-rocko-18.0.0" that is based on the upstream branch
483whose "HEAD" matches the commit in the repository associated with the
484"rocko-18.0.0" tag. The files in your repository now exactly match that
485particular Yocto Project release as it is tagged in the upstream Git
486repository. It is important to understand that when you create and
487checkout a local working branch based on a tag, your environment matches
488a specific point in time and not the entire development branch (i.e.
489from the "tip" of the branch backwards).
490
491Basic Commands
492--------------
493
494Git has an extensive set of commands that lets you manage changes and
495perform collaboration over the life of a project. Conveniently though,
496you can manage with a small set of basic operations and workflows once
497you understand the basic philosophy behind Git. You do not have to be an
498expert in Git to be functional. A good place to look for instruction on
499a minimal set of Git commands is
500`here <http://git-scm.com/documentation>`__.
501
502The following list of Git commands briefly describes some basic Git
503operations as a way to get started. As with any set of commands, this
504list (in most cases) simply shows the base command and omits the many
505arguments it supports. See the Git documentation for complete
506descriptions and strategies on how to use these commands:
507
508- *``git init``:* Initializes an empty Git repository. You cannot use
509 Git commands unless you have a ``.git`` repository.
510
511- *``git clone``:* Creates a local clone of a Git repository that is on
512 equal footing with a fellow developer’s Git repository or an upstream
513 repository.
514
515- *``git add``:* Locally stages updated file contents to the index that
516 Git uses to track changes. You must stage all files that have changed
517 before you can commit them.
518
519- *``git commit``:* Creates a local "commit" that documents the changes
520 you made. Only changes that have been staged can be committed.
521 Commits are used for historical purposes, for determining if a
522 maintainer of a project will allow the change, and for ultimately
523 pushing the change from your local Git repository into the project’s
524 upstream repository.
525
526- *``git status``:* Reports any modified files that possibly need to be
527 staged and gives you a status of where you stand regarding local
528 commits as compared to the upstream repository.
529
530- *``git checkout`` branch-name:* Changes your local working branch and
531 in this form assumes the local branch already exists. This command is
532 analogous to "cd".
533
534- *``git checkout –b`` working-branch upstream-branch:* Creates and
535 checks out a working branch on your local machine. The local branch
536 tracks the upstream branch. You can use your local branch to isolate
537 your work. It is a good idea to use local branches when adding
538 specific features or changes. Using isolated branches facilitates
539 easy removal of changes if they do not work out.
540
541- *``git branch``:* Displays the existing local branches associated
542 with your local repository. The branch that you have currently
543 checked out is noted with an asterisk character.
544
545- *``git branch -D`` branch-name:* Deletes an existing local branch.
546 You need to be in a local branch other than the one you are deleting
547 in order to delete branch-name.
548
549- *``git pull --rebase``:* Retrieves information from an upstream Git
550 repository and places it in your local Git repository. You use this
551 command to make sure you are synchronized with the repository from
552 which you are basing changes (.e.g. the "master" branch). The
553 "--rebase" option ensures that any local commits you have in your
554 branch are preserved at the top of your local branch.
555
556- *``git push`` repo-name local-branch\ ``:``\ upstream-branch:* Sends
557 all your committed local changes to the upstream Git repository that
558 your local repository is tracking (e.g. a contribution repository).
559 The maintainer of the project draws from these repositories to merge
560 changes (commits) into the appropriate branch of project's upstream
561 repository.
562
563- *``git merge``:* Combines or adds changes from one local branch of
564 your repository with another branch. When you create a local Git
565 repository, the default branch is named "master". A typical workflow
566 is to create a temporary branch that is based off "master" that you
567 would use for isolated work. You would make your changes in that
568 isolated branch, stage and commit them locally, switch to the
569 "master" branch, and then use the ``git merge`` command to apply the
570 changes from your isolated branch into the currently checked out
571 branch (e.g. "master"). After the merge is complete and if you are
572 done with working in that isolated branch, you can safely delete the
573 isolated branch.
574
575- *``git cherry-pick`` commits:* Choose and apply specific commits from
576 one branch into another branch. There are times when you might not be
577 able to merge all the changes in one branch with another but need to
578 pick out certain ones.
579
580- *``gitk``:* Provides a GUI view of the branches and changes in your
581 local Git repository. This command is a good way to graphically see
582 where things have diverged in your local repository.
583
584 .. note::
585
586 You need to install the
587 gitk
588 package on your development system to use this command.
589
590- *``git log``:* Reports a history of your commits to the repository.
591 This report lists all commits regardless of whether you have pushed
592 them upstream or not.
593
594- *``git diff``:* Displays line-by-line differences between a local
595 working file and the same file as understood by Git. This command is
596 useful to see what you have changed in any given file.
597
598Licensing
599=========
600
601Because open source projects are open to the public, they have different
602licensing structures in place. License evolution for both Open Source
603and Free Software has an interesting history. If you are interested in
604this history, you can find basic information here:
605
606- `Open source license
607 history <http://en.wikipedia.org/wiki/Open-source_license>`__
608
609- `Free software license
610 history <http://en.wikipedia.org/wiki/Free_software_license>`__
611
612In general, the Yocto Project is broadly licensed under the
613Massachusetts Institute of Technology (MIT) License. MIT licensing
614permits the reuse of software within proprietary software as long as the
615license is distributed with that software. MIT is also compatible with
616the GNU General Public License (GPL). Patches to the Yocto Project
617follow the upstream licensing scheme. You can find information on the
618MIT license
619`here <http://www.opensource.org/licenses/mit-license.php>`__. You can
620find information on the GNU GPL
621`here <http://www.opensource.org/licenses/LGPL-3.0>`__.
622
623When you build an image using the Yocto Project, the build process uses
624a known list of licenses to ensure compliance. You can find this list in
625the `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ at
626``meta/files/common-licenses``. Once the build completes, the list of
627all licenses found and used during that build are kept in the `Build
628Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ at
629``tmp/deploy/licenses``.
630
631If a module requires a license that is not in the base list, the build
632process generates a warning during the build. These tools make it easier
633for a developer to be certain of the licenses with which their shipped
634products must comply. However, even with these tools it is still up to
635the developer to resolve potential licensing issues.
636
637The base list of licenses used by the build process is a combination of
638the Software Package Data Exchange (SPDX) list and the Open Source
639Initiative (OSI) projects. `SPDX Group <http://spdx.org>`__ is a working
640group of the Linux Foundation that maintains a specification for a
641standard format for communicating the components, licenses, and
642copyrights associated with a software package.
643`OSI <http://opensource.org>`__ is a corporation dedicated to the Open
644Source Definition and the effort for reviewing and approving licenses
645that conform to the Open Source Definition (OSD).
646
647You can find a list of the combined SPDX and OSI licenses that the Yocto
648Project uses in the ``meta/files/common-licenses`` directory in your
649`Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__.
650
651For information that can help you maintain compliance with various open
652source licensing during the lifecycle of a product created using the
653Yocto Project, see the "`Maintaining Open Source License Compliance
654During Your Product's
655Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
656section in the Yocto Project Development Tasks Manual.
diff --git a/documentation/overview-manual/overview-manual-intro.rst b/documentation/overview-manual/overview-manual-intro.rst
new file mode 100644
index 0000000000..82c0051c47
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-intro.rst
@@ -0,0 +1,74 @@
1**********************************************
2The Yocto Project Overview and Concepts Manual
3**********************************************
4
5.. _overview-manual-welcome:
6
7Welcome
8=======
9
10Welcome to the Yocto Project Overview and Concepts Manual! This manual
11introduces the Yocto Project by providing concepts, software overviews,
12best-known-methods (BKMs), and any other high-level introductory
13information suitable for a new Yocto Project user.
14
15The following list describes what you can get from this manual:
16
17- `Introducing the Yocto Project <#overview-yp>`__\ *:* This chapter
18 provides an introduction to the Yocto Project. You will learn about
19 features and challenges of the Yocto Project, the layer model,
20 components and tools, development methods, the
21 `Poky <&YOCTO_DOCS_REF_URL;#poky>`__ reference distribution, the
22 OpenEmbedded build system workflow, and some basic Yocto terms.
23
24- `The Yocto Project Development
25 Environment <#overview-development-environment>`__\ *:* This chapter
26 helps you get started understanding the Yocto Project development
27 environment. You will learn about open source, development hosts,
28 Yocto Project source repositories, workflows using Git and the Yocto
29 Project, a Git primer, and information about licensing.
30
31- `Yocto Project Concepts <#overview-manual-concepts>`__\ *:* This
32 chapter presents various concepts regarding the Yocto Project. You
33 can find conceptual information about components, development,
34 cross-toolchains, and so forth.
35
36This manual does not give you the following:
37
38- *Step-by-step Instructions for Development Tasks:* Instructional
39 procedures reside in other manuals within the Yocto Project
40 documentation set. For example, the `Yocto Project Development Tasks
41 Manual <&YOCTO_DOCS_DEV_URL;>`__ provides examples on how to perform
42 various development tasks. As another example, the `Yocto Project
43 Application Development and the Extensible Software Development Kit
44 (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual contains detailed
45 instructions on how to install an SDK, which is used to develop
46 applications for target hardware.
47
48- *Reference Material:* This type of material resides in an appropriate
49 reference manual. For example, system variables are documented in the
50 `Yocto Project Reference Manual <&YOCTO_DOCS_REF_URL;>`__. As another
51 example, the `Yocto Project Board Support Package (BSP) Developer's
52 Guide <&YOCTO_DOCS_BSP_URL;>`__ contains reference information on
53 BSPs.
54
55- *Detailed Public Information Not Specific to the Yocto Project:* For
56 example, exhaustive information on how to use the Source Control
57 Manager Git is better covered with Internet searches and official Git
58 Documentation than through the Yocto Project documentation.
59
60.. _overview-manual-other-information:
61
62Other Information
63=================
64
65Because this manual presents information for many different topics,
66supplemental information is recommended for full comprehension. For
67additional introductory information on the Yocto Project, see the `Yocto
68Project Website <&YOCTO_HOME_URL;>`__. If you want to build an image
69with no knowledge of Yocto Project as a way of quickly testing it out,
70see the `Yocto Project Quick Build <&YOCTO_DOCS_BRIEF_URL;>`__ document.
71For a comprehensive list of links and other documentation, see the
72"`Links and Related
73Documentation <&YOCTO_DOCS_REF_URL;#resources-links-and-related-documentation>`__"
74section in the Yocto Project Reference Manual.
diff --git a/documentation/overview-manual/overview-manual-yp-intro.rst b/documentation/overview-manual/overview-manual-yp-intro.rst
new file mode 100644
index 0000000000..62257c26b9
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-yp-intro.rst
@@ -0,0 +1,941 @@
1*****************************
2Introducing the Yocto Project
3*****************************
4
5What is the Yocto Project?
6==========================
7
8The Yocto Project is an open source collaboration project that helps
9developers create custom Linux-based systems that are designed for
10embedded products regardless of the product's hardware architecture.
11Yocto Project provides a flexible toolset and a development environment
12that allows embedded device developers across the world to collaborate
13through shared technologies, software stacks, configurations, and best
14practices used to create these tailored Linux images.
15
16Thousands of developers worldwide have discovered that Yocto Project
17provides advantages in both systems and applications development,
18archival and management benefits, and customizations used for speed,
19footprint, and memory utilization. The project is a standard when it
20comes to delivering embedded software stacks. The project allows
21software customizations and build interchange for multiple hardware
22platforms as well as software stacks that can be maintained and scaled.
23
24For further introductory information on the Yocto Project, you might be
25interested in this
26`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
27by Drew Moseley and in this short introductory
28`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
29
30The remainder of this section overviews advantages and challenges tied
31to the Yocto Project.
32
33.. _gs-features:
34
35Features
36--------
37
38The following list describes features and advantages of the Yocto
39Project:
40
41- *Widely Adopted Across the Industry:* Semiconductor, operating
42 system, software, and service vendors exist whose products and
43 services adopt and support the Yocto Project. For a look at the Yocto
44 Project community and the companies involved with the Yocto Project,
45 see the "COMMUNITY" and "ECOSYSTEM" tabs on the `Yocto
46 Project <&YOCTO_HOME_URL;>`__ home page.
47
48- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
49 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
50 create and supply BSPs that support their hardware. If you have
51 custom silicon, you can create a BSP that supports that architecture.
52
53 Aside from lots of architecture support, the Yocto Project fully
54 supports a wide range of device emulation through the Quick EMUlator
55 (QEMU).
56
57- *Images and Code Transfer Easily:* Yocto Project output can easily
58 move between architectures without moving to new development
59 environments. Additionally, if you have used the Yocto Project to
60 create an image or application and you find yourself not able to
61 support it, commercial Linux vendors such as Wind River, Mentor
62 Graphics, Timesys, and ENEA could take it and provide ongoing
63 support. These vendors have offerings that are built using the Yocto
64 Project.
65
66- *Flexibility:* Corporations use the Yocto Project many different
67 ways. One example is to create an internal Linux distribution as a
68 code base the corporation can use across multiple product groups.
69 Through customization and layering, a project group can leverage the
70 base Linux distribution to create a distribution that works for their
71 product needs.
72
73- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
74 distribution, you can use the Yocto Project to create exactly what
75 you need for embedded devices. You only add the feature support or
76 packages that you absolutely need for the device. For devices that
77 have display hardware, you can use available system components such
78 as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich
79 user experience. For devices that do not have a display or where you
80 want to use alternative UI frameworks, you can choose to not install
81 these components.
82
83- *Comprehensive Toolchain Capabilities:* Toolchains for supported
84 architectures satisfy most use cases. However, if your hardware
85 supports features that are not part of a standard toolchain, you can
86 easily customize that toolchain through specification of
87 platform-specific tuning parameters. And, should you need to use a
88 third-party toolchain, mechanisms built into the Yocto Project allow
89 for that.
90
91- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
92 policy ensures that you are free to set policies based on the needs
93 of your design instead of adopting decisions enforced by some system
94 software provider.
95
96- *Uses a Layer Model:* The Yocto Project `layer
97 infrastructure <#the-yocto-project-layer-model>`__ groups related
98 functionality into separate bundles. You can incrementally add these
99 grouped functionalities to your project as needed. Using layers to
100 isolate and group functionality reduces project complexity and
101 redundancy, allows you to easily extend the system, make
102 customizations, and keep functionality organized.
103
104- *Supports Partial Builds:* You can build and rebuild individual
105 packages as needed. Yocto Project accomplishes this through its
106 `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being
107 able to build and debug components individually eases project
108 development.
109
110- *Releases According to a Strict Schedule:* Major releases occur on a
111 `six-month cycle <&YOCTO_DOCS_REF_URL;#ref-release-process>`__
112 predictably in October and April. The most recent two releases
113 support point releases to address common vulnerabilities and
114 exposures. This predictability is crucial for projects based on the
115 Yocto Project and allows development teams to plan activities.
116
117- *Rich Ecosystem of Individuals and Organizations:* For open source
118 projects, the value of community is very important. Support forums,
119 expertise, and active developers who continue to push the Yocto
120 Project forward are readily available.
121
122- *Binary Reproducibility:* The Yocto Project allows you to be very
123 specific about dependencies and achieves very high percentages of
124 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
125 distributions are not specific about which packages are pulled in and
126 in what order to support dependencies, other build systems can
127 arbitrarily include packages.
128
129- *License Manifest:* The Yocto Project provides a `license
130 manifest <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__
131 for review by people who need to track the use of open source
132 licenses (e.g.legal teams).
133
134.. _gs-challenges:
135
136Challenges
137----------
138
139The following list presents challenges you might encounter when
140developing using the Yocto Project:
141
142- *Steep Learning Curve:* The Yocto Project has a steep learning curve
143 and has many different ways to accomplish similar tasks. It can be
144 difficult to choose how to proceed when varying methods exist by
145 which to accomplish a given task.
146
147- *Understanding What Changes You Need to Make For Your Design Requires
148 Some Research:* Beyond the simple tutorial stage, understanding what
149 changes need to be made for your particular design can require a
150 significant amount of research and investigation. For information
151 that helps you transition from trying out the Yocto Project to using
152 it for your project, see the "`What I wish I'd
153 Known <&YOCTO_DOCS_URL;/what-i-wish-id-known/>`__" and
154 "`Transitioning to a Custom Environment for Systems
155 Development <&YOCTO_DOCS_URL;/transitioning-to-a-custom-environment/>`__"
156 documents on the Yocto Project website.
157
158- *Project Workflow Could Be Confusing:* The `Yocto Project
159 workflow <#overview-development-environment>`__ could be confusing if
160 you are used to traditional desktop and server software development.
161 In a desktop development environment, mechanisms exist to easily pull
162 and install new packages, which are typically pre-compiled binaries
163 from servers accessible over the Internet. Using the Yocto Project,
164 you must modify your configuration and rebuild to add additional
165 packages.
166
167- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
168 developing code to run on a target, compilation, execution, and
169 testing done on the actual target can be faster than running a
170 BitBake build on a development host and then deploying binaries to
171 the target for test. While the Yocto Project does support development
172 tools on the target, the additional step of integrating your changes
173 back into the Yocto Project build environment would be required.
174 Yocto Project supports an intermediate approach that involves making
175 changes on the development system within the BitBake environment and
176 then deploying only the updated packages to the target.
177
178 The Yocto Project `OpenEmbedded build
179 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ produces packages
180 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
181 these packages into the running system on the target by using
182 utilities on the target such as ``rpm`` or ``ipk``.
183
184- *Initial Build Times Can be Significant:* Long initial build times
185 are unfortunately unavoidable due to the large number of packages
186 initially built from scratch for a fully functioning Linux system.
187 Once that initial build is completed, however, the shared-state
188 (sstate) cache mechanism Yocto Project uses keeps the system from
189 rebuilding packages that have not been "touched" since the last
190 build. The sstate mechanism significantly reduces times for
191 successive builds.
192
193The Yocto Project Layer Model
194=============================
195
196The Yocto Project's "Layer Model" is a development model for embedded
197and IoT Linux creation that distinguishes the Yocto Project from other
198simple build systems. The Layer Model simultaneously supports
199collaboration and customization. Layers are repositories that contain
200related sets of instructions that tell the `OpenEmbedded build
201system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ what to do. You can
202collaborate, share, and reuse layers.
203
204Layers can contain changes to previous instructions or settings at any
205time. This powerful override capability is what allows you to customize
206previously supplied collaborative or community layers to suit your
207product requirements.
208
209You use different layers to logically separate information in your
210build. As an example, you could have BSP, GUI, distro configuration,
211middleware, or application layers. Putting your entire build into one
212layer limits and complicates future customization and reuse. Isolating
213information into layers, on the other hand, helps simplify future
214customizations and reuse. You might find it tempting to keep everything
215in one layer when working on a single project. However, the more modular
216your Metadata, the easier it is to cope with future changes.
217
218.. note::
219
220 - Use Board Support Package (BSP) layers from silicon vendors when
221 possible.
222
223 - Familiarize yourself with the `Yocto Project curated layer
224 index <https://caffelli-staging.yoctoproject.org/software-overview/layers/>`__
225 or the `OpenEmbedded layer
226 index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
227 The latter contains more layers but they are less universally
228 validated.
229
230 - Layers support the inclusion of technologies, hardware components,
231 and software components. The `Yocto Project
232 Compatible <&YOCTO_DOCS_DEV_URL;#making-sure-your-layer-is-compatible-with-yocto-project>`__
233 designation provides a minimum level of standardization that
234 contributes to a strong ecosystem. "YP Compatible" is applied to
235 appropriate products and software components such as BSPs, other
236 OE-compatible layers, and related open-source projects, allowing
237 the producer to use Yocto Project badges and branding assets.
238
239To illustrate how layers are used to keep things modular, consider
240machine customizations. These types of customizations typically reside
241in a special layer, rather than a general layer, called a BSP Layer.
242Furthermore, the machine customizations should be isolated from recipes
243and Metadata that support a new GUI environment, for example. This
244situation gives you a couple of layers: one for the machine
245configurations, and one for the GUI environment. It is important to
246understand, however, that the BSP layer can still make machine-specific
247additions to recipes within the GUI environment layer without polluting
248the GUI layer itself with those machine-specific changes. You can
249accomplish this through a recipe that is a BitBake append
250(``.bbappend``) file, which is described later in this section.
251
252.. note::
253
254 For general information on BSP layer structure, see the
255 Yocto Project Board Support Packages (BSP) Developer's Guide
256 .
257
258The `Source Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__
259contains both general layers and BSP layers right out of the box. You
260can easily identify layers that ship with a Yocto Project release in the
261Source Directory by their names. Layers typically have names that begin
262with the string ``meta-``.
263
264.. note::
265
266 It is not a requirement that a layer name begin with the prefix
267 meta-
268 , but it is a commonly accepted standard in the Yocto Project
269 community.
270
271For example, if you were to examine the `tree
272view <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/>`__ of the
273``poky`` repository, you will see several layers: ``meta``,
274``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
275``meta-yocto-bsp``. Each of these repositories represents a distinct
276layer.
277
278For procedures on how to create layers, see the "`Understanding and
279Creating
280Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
281section in the Yocto Project Development Tasks Manual.
282
283Components and Tools
284====================
285
286The Yocto Project employs a collection of components and tools used by
287the project itself, by project developers, and by those using the Yocto
288Project. These components and tools are open source projects and
289metadata that are separate from the reference distribution
290(`Poky <&YOCTO_DOCS_REF_URL;#poky>`__) and the `OpenEmbedded build
291system <&YOCTO_DOCS_REF_URL;#build-system-term>`__. Most of the
292components and tools are downloaded separately.
293
294This section provides brief overviews of the components and tools
295associated with the Yocto Project.
296
297.. _gs-development-tools:
298
299Development Tools
300-----------------
301
302The following list consists of tools that help you develop images and
303applications using the Yocto Project:
304
305- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
306 open source, cross-platform development framework that leverages
307 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
308 easily managed, extensible environment that allows you to build
309 binaries for a variety of architectures on Windows, Linux and Mac OS
310 X hosts.
311
312- *``devtool``:* This command-line tool is available as part of the
313 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
314 to help build, test, and package software within the eSDK. You can
315 use the tool to optionally integrate what you build into an image
316 built by the OpenEmbedded build system.
317
318 The ``devtool`` command employs a number of sub-commands that allow
319 you to add, modify, and upgrade recipes. As with the OpenEmbedded
320 build system, “recipes” represent software packages within
321 ``devtool``. When you use ``devtool add``, a recipe is automatically
322 created. When you use ``devtool modify``, the specified existing
323 recipe is used in order to determine where to get the source code and
324 how to patch it. In both cases, an environment is set up so that when
325 you build the recipe a source tree that is under your control is used
326 in order to allow you to make changes to the source as desired. By
327 default, both new recipes and the source go into a “workspace”
328 directory under the eSDK. The ``devtool upgrade`` command updates an
329 existing recipe so that you can build it for an updated set of source
330 files.
331
332 You can read about the ``devtool`` workflow in the Yocto Project
333 Application Development and Extensible Software Development Kit
334 (eSDK) Manual in the "`Using ``devtool`` in Your SDK
335 Workflow' <&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow>`__"
336 section.
337
338- *Extensible Software Development Kit (eSDK):* The eSDK provides a
339 cross-development toolchain and libraries tailored to the contents of
340 a specific image. The eSDK makes it easy to add new applications and
341 libraries to an image, modify the source for an existing component,
342 test changes on the target hardware, and integrate into the rest of
343 the OpenEmbedded build system. The eSDK gives you a toolchain
344 experience supplemented with the powerful set of ``devtool`` commands
345 tailored for the Yocto Project environment.
346
347 For information on the eSDK, see the `Yocto Project Application
348 Development and the Extensible Software Development Kit
349 (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ Manual.
350
351- *Toaster:* Toaster is a web interface to the Yocto Project
352 OpenEmbedded build system. Toaster allows you to configure, run, and
353 view information about builds. For information on Toaster, see the
354 `Toaster User Manual <&YOCTO_DOCS_TOAST_URL;>`__.
355
356.. _gs-production-tools:
357
358Production Tools
359----------------
360
361The following list consists of tools that help production related
362activities using the Yocto Project:
363
364- *Auto Upgrade Helper:* This utility when used in conjunction with the
365 `OpenEmbedded build
366 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ (BitBake and
367 OE-Core) automatically generates upgrades for recipes that are based
368 on new versions of the recipes published upstream.
369
370- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
371 versions available for Yocto Project. The main purpose of the system
372 is to help you manage the recipes you maintain and to offer a dynamic
373 overview of the project. The Recipe Reporting System is built on top
374 of the `OpenEmbedded Layer
375 Index <http://layers.openembedded.org/layerindex/layers/>`__, which
376 is a website that indexes OpenEmbedded-Core layers.
377
378- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
379 is a fork of a project originally started by
380 `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking
381 system designed to streamline the process of bringing contributions
382 into a project. The Yocto Project uses Patchwork as an organizational
383 tool to handle patches, which number in the thousands for every
384 release.
385
386- *AutoBuilder:* AutoBuilder is a project that automates build tests
387 and quality assurance (QA). By using the public AutoBuilder, anyone
388 can determine the status of the current "master" branch of Poky.
389
390 .. note::
391
392 AutoBuilder is based on
393 buildbot
394 .
395
396 A goal of the Yocto Project is to lead the open source industry with
397 a project that automates testing and QA procedures. In doing so, the
398 project encourages a development community that publishes QA and test
399 plans, publicly demonstrates QA and test plans, and encourages
400 development of tools that automate and test and QA procedures for the
401 benefit of the development community.
402
403 You can learn more about the AutoBuilder used by the Yocto Project
404 `here <&YOCTO_AB_URL;>`__.
405
406- *Cross-Prelink:* Prelinking is the process of pre-computing the load
407 addresses and link tables generated by the dynamic linker as compared
408 to doing this at runtime. Doing this ahead of time results in
409 performance improvements when the application is launched and reduced
410 memory usage for libraries shared by many applications.
411
412 Historically, cross-prelink is a variant of prelink, which was
413 conceived by `Jakub
414 Jelínek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of
415 years ago. Both prelink and cross-prelink are maintained in the same
416 repository albeit on separate branches. By providing an emulated
417 runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation),
418 the cross-prelink project extends the prelink software’s ability to
419 prelink a sysroot environment. Additionally, the cross-prelink
420 software enables the ability to work in sysroot style environments.
421
422 The dynamic linker determines standard load address calculations
423 based on a variety of factors such as mapping addresses, library
424 usage, and library function conflicts. The prelink tool uses this
425 information, from the dynamic linker, to determine unique load
426 addresses for executable and linkable format (ELF) binaries that are
427 shared libraries and dynamically linked. The prelink tool modifies
428 these ELF binaries with the pre-computed information. The result is
429 faster loading and often lower memory consumption because more of the
430 library code can be re-used from shared Copy-On-Write (COW) pages.
431
432 The original upstream prelink project only supports running prelink
433 on the end target device due to the reliance on the target device’s
434 dynamic linker. This restriction causes issues when developing a
435 cross-compiled system. The cross-prelink adds a synthesized dynamic
436 loader that runs on the host, thus permitting cross-prelinking
437 without ever having to run on a read-write target filesystem.
438
439- *Pseudo:* Pseudo is the Yocto Project implementation of
440 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
441 commands in an environment that seemingly has root privileges.
442
443 During a build, it can be necessary to perform operations that
444 require system administrator privileges. For example, file ownership
445 or permissions might need definition. Pseudo is a tool that you can
446 either use directly or through the environment variable
447 ``LD_PRELOAD``. Either method allows these operations to succeed as
448 if system administrator privileges exist even when they do not.
449
450 You can read more about Pseudo in the "`Fakeroot and
451 Pseudo <#fakeroot-and-pseudo>`__" section.
452
453.. _gs-openembedded-build-system:
454
455Open-Embedded Build System Components
456-------------------------------------
457
458The following list consists of components associated with the
459`OpenEmbedded build system <&YOCTO_DOCS_REF_URL;#build-system-term>`__:
460
461- *BitBake:* BitBake is a core component of the Yocto Project and is
462 used by the OpenEmbedded build system to build images. While BitBake
463 is key to the build system, BitBake is maintained separately from the
464 Yocto Project.
465
466 BitBake is a generic task execution engine that allows shell and
467 Python tasks to be run efficiently and in parallel while working
468 within complex inter-task dependency constraints. In short, BitBake
469 is a build engine that works through recipes written in a specific
470 format in order to perform sets of tasks.
471
472 You can learn more about BitBake in the `BitBake User
473 Manual <&YOCTO_DOCS_BB_URL;>`__.
474
475- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
476 metadata (i.e. recipes, classes, and associated files) used by
477 OpenEmbedded-derived systems, which includes the Yocto Project. The
478 Yocto Project and the OpenEmbedded Project both maintain the
479 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
480 Project `Source
481 Repositories <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta>`__.
482
483 Historically, the Yocto Project integrated the OE-Core metadata
484 throughout the Yocto Project source repository reference system
485 (Poky). After Yocto Project Version 1.0, the Yocto Project and
486 OpenEmbedded agreed to work together and share a common core set of
487 metadata (OE-Core), which contained much of the functionality
488 previously found in Poky. This collaboration achieved a long-standing
489 OpenEmbedded objective for having a more tightly controlled and
490 quality-assured core. The results also fit well with the Yocto
491 Project objective of achieving a smaller number of fully featured
492 tools as compared to many different ones.
493
494 Sharing a core set of metadata results in Poky as an integration
495 layer on top of OE-Core. You can see that in this
496 `figure <#yp-key-dev-elements>`__. The Yocto Project combines various
497 components such as BitBake, OE-Core, script “glue”, and documentation
498 for its build system.
499
500.. _gs-reference-distribution-poky:
501
502Reference Distribution (Poky)
503-----------------------------
504
505Poky is the Yocto Project reference distribution. It contains the
506`Open-Embedded build system <&YOCTO_DOCS_REF_URL;#build-system-term>`__
507(BitBake and OE-Core) as well as a set of metadata to get you started
508building your own distribution. See the
509`figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?"
510section for an illustration that shows Poky and its relationship with
511other parts of the Yocto Project.
512
513To use the Yocto Project tools and components, you can download
514(``clone``) Poky and use it to bootstrap your own distribution.
515
516.. note::
517
518 Poky does not contain binary files. It is a working example of how to
519 build your own custom Linux distribution from source.
520
521You can read more about Poky in the "`Reference Embedded Distribution
522(Poky) <#reference-embedded-distribution>`__" section.
523
524.. _gs-packages-for-finished-targets:
525
526Packages for Finished Targets
527-----------------------------
528
529The following lists components associated with packages for finished
530targets:
531
532- *Matchbox:* Matchbox is an Open Source, base environment for the X
533 Window System running on non-desktop, embedded platforms such as
534 handhelds, set-top boxes, kiosks, and anything else for which screen
535 space, input mechanisms, or system resources are limited.
536
537 Matchbox consists of a number of interchangeable and optional
538 applications that you can tailor to a specific, non-desktop platform
539 to enhance usability in constrained environments.
540
541 You can find the Matchbox source in the Yocto Project `Source
542 Repositories <&YOCTO_GIT_URL;>`__.
543
544- *Opkg* Open PacKaGe management (opkg) is a lightweight package
545 management system based on the itsy package (ipkg) management system.
546 Opkg is written in C and resembles Advanced Package Tool (APT) and
547 Debian Package (dpkg) in operation.
548
549 Opkg is intended for use on embedded Linux devices and is used in
550 this capacity in the
551 `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and
552 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
553 Project.
554
555 .. note::
556
557 As best it can, opkg maintains backwards compatibility with ipkg
558 and conforms to a subset of Debian’s policy manual regarding
559 control files.
560
561.. _gs-archived-components:
562
563Archived Components
564-------------------
565
566The Build Appliance is a virtual machine image that enables you to build
567and boot a custom embedded Linux image with the Yocto Project using a
568non-Linux development system.
569
570Historically, the Build Appliance was the second of three methods by
571which you could use the Yocto Project on a system that was not native to
572Linux.
573
5741. *Hob:* Hob, which is now deprecated and is no longer available since
575 the 2.1 release of the Yocto Project provided a rudimentary,
576 GUI-based interface to the Yocto Project. Toaster has fully replaced
577 Hob.
578
5792. *Build Appliance:* Post Hob, the Build Appliance became available. It
580 was never recommended that you use the Build Appliance as a
581 day-to-day production development environment with the Yocto Project.
582 Build Appliance was useful as a way to try out development in the
583 Yocto Project environment.
584
5853. *CROPS:* The final and best solution available now for developing
586 using the Yocto Project on a system not native to Linux is with
587 `CROPS <#gs-crops-overview>`__.
588
589.. _gs-development-methods:
590
591Development Methods
592===================
593
594The Yocto Project development environment usually involves a `Build
595Host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ and target
596hardware. You use the Build Host to build images and develop
597applications, while you use the target hardware to test deployed
598software.
599
600This section provides an introduction to the choices or development
601methods you have when setting up your Build Host. Depending on the your
602particular workflow preference and the type of operating system your
603Build Host runs, several choices exist that allow you to use the Yocto
604Project.
605
606.. note::
607
608 For additional detail about the Yocto Project development
609 environment, see the "
610 The Yocto Project Development Environment
611 " chapter.
612
613- *Native Linux Host:* By far the best option for a Build Host. A
614 system running Linux as its native operating system allows you to
615 develop software by directly using the
616 `BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ tool. You can
617 accomplish all aspects of development from a familiar shell of a
618 supported Linux distribution.
619
620 For information on how to set up a Build Host on a system running
621 Linux as its native operating system, see the "`Setting Up a Native
622 Linux Host <&YOCTO_DOCS_DEV_URL;#setting-up-a-native-linux-host>`__"
623 section in the Yocto Project Development Tasks Manual.
624
625- *CROss PlatformS (CROPS):* Typically, you use
626 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
627 `Docker Containers <https://www.docker.com/>`__, to set up a Build
628 Host that is not running Linux (e.g. Microsoft Windows or macOS).
629
630 .. note::
631
632 You can, however, use CROPS on a Linux-based system.
633
634 CROPS is an open source, cross-platform development framework that
635 provides an easily managed, extensible environment for building
636 binaries targeted for a variety of architectures on Windows, macOS,
637 or Linux hosts. Once the Build Host is set up using CROPS, you can
638 prepare a shell environment to mimic that of a shell being used on a
639 system natively running Linux.
640
641 For information on how to set up a Build Host with CROPS, see the
642 "`Setting Up to Use CROss PlatformS
643 (CROPS) <&YOCTO_DOCS_DEV_URL;#setting-up-to-use-crops>`__" section in
644 the Yocto Project Development Tasks Manual.
645
646- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
647 For Linux v2 to set up a build host using Windows 10.
648
649 .. note::
650
651 The Yocto Project is not compatible with WSLv1, it is compatible
652 but not officially supported nor validated with WSLv2, if you
653 still decide to use WSL please upgrade to WSLv2.
654
655 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
656 kernel inside of a lightweight utility virtual machine (VM) using
657 virtualization technology.
658
659 For information on how to set up a Build Host with WSLv2, see the
660 "`Setting Up to Use Windows Subsystem For
661 Linux <&YOCTO_DOCS_DEV_URL;#setting-up-to-use-wsl>`__" section in the
662 Yocto Project Development Tasks Manual.
663
664- *Toaster:* Regardless of what your Build Host is running, you can use
665 Toaster to develop software using the Yocto Project. Toaster is a web
666 interface to the Yocto Project's `Open-Embedded build
667 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__. The interface
668 enables you to configure and run your builds. Information about
669 builds is collected and stored in a database. You can use Toaster to
670 configure and start builds on multiple remote build servers.
671
672 For information about and how to use Toaster, see the `Toaster User
673 Manual <&YOCTO_DOCS_TOAST_URL;>`__.
674
675.. _reference-embedded-distribution:
676
677Reference Embedded Distribution (Poky)
678======================================
679
680"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
681Project's reference distribution or Reference OS Kit. Poky contains the
682`OpenEmbedded Build System <&YOCTO_DOCS_REF_URL;#build-system-term>`__
683(`BitBake <&YOCTO_DOCS_REF_URL;#bitbake-term>`__ and
684`OpenEmbedded-Core <&YOCTO_DOCS_REF_URL;#oe-core>`__) as well as a set
685of `metadata <&YOCTO_DOCS_REF_URL;#metadata>`__ to get you started
686building your own distro. In other words, Poky is a base specification
687of the functionality needed for a typical embedded system as well as the
688components from the Yocto Project that allow you to build a distribution
689into a usable binary image.
690
691Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
692found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
693provided all together and known to work well together. You can view
694these items that make up the Poky repository in the `Source
695Repositories <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/>`__.
696
697.. note::
698
699 If you are interested in all the contents of the
700 poky
701 Git repository, see the "
702 Top-Level Core Components
703 " section in the Yocto Project Reference Manual.
704
705The following figure illustrates what generally comprises Poky:
706
707- BitBake is a task executor and scheduler that is the heart of the
708 OpenEmbedded build system.
709
710- ``meta-poky``, which is Poky-specific metadata.
711
712- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
713 Packages (BSPs).
714
715- OpenEmbedded-Core (OE-Core) metadata, which includes shared
716 configurations, global variable definitions, shared classes,
717 packaging, and recipes. Classes define the encapsulation and
718 inheritance of build logic. Recipes are the logical units of software
719 and images to be built.
720
721- Documentation, which contains the Yocto Project source files used to
722 make the set of user manuals.
723
724.. note::
725
726 While Poky is a "complete" distribution specification and is tested
727 and put through QA, you cannot use it as a product "out of the box"
728 in its current form.
729
730To use the Yocto Project tools, you can use Git to clone (download) the
731Poky repository then use your local copy of the reference distribution
732to bootstrap your own distribution.
733
734.. note::
735
736 Poky does not contain binary files. It is a working example of how to
737 build your own custom Linux distribution from source.
738
739Poky has a regular, well established, six-month release cycle under its
740own version. Major releases occur at the same time major releases (point
741releases) occur for the Yocto Project, which are typically in the Spring
742and Fall. For more information on the Yocto Project release schedule and
743cadence, see the "`Yocto Project Releases and the Stable Release
744Process <&YOCTO_DOCS_REF_URL;#ref-release-process>`__" chapter in the
745Yocto Project Reference Manual.
746
747Much has been said about Poky being a "default configuration." A default
748configuration provides a starting image footprint. You can use Poky out
749of the box to create an image ranging from a shell-accessible minimal
750image all the way up to a Linux Standard Base-compliant image that uses
751a GNOME Mobile and Embedded (GMAE) based reference user interface called
752Sato.
753
754One of the most powerful properties of Poky is that every aspect of a
755build is controlled by the metadata. You can use metadata to augment
756these base image types by adding metadata
757`layers <#the-yocto-project-layer-model>`__ that extend functionality.
758These layers can provide, for example, an additional software stack for
759an image type, add a board support package (BSP) for additional
760hardware, or even create a new image type.
761
762Metadata is loosely grouped into configuration files or package recipes.
763A recipe is a collection of non-executable metadata used by BitBake to
764set variables or define additional build-time tasks. A recipe contains
765fields such as the recipe description, the recipe version, the license
766of the package and the upstream source repository. A recipe might also
767indicate that the build process uses autotools, make, distutils or any
768other build process, in which case the basic functionality can be
769defined by the classes it inherits from the OE-Core layer's class
770definitions in ``./meta/classes``. Within a recipe you can also define
771additional tasks as well as task prerequisites. Recipe syntax through
772BitBake also supports both ``_prepend`` and ``_append`` operators as a
773method of extending task functionality. These operators inject code into
774the beginning or end of a task. For information on these BitBake
775operators, see the "`Appending and Prepending (Override Style
776Syntax) <&YOCTO_DOCS_BB_URL;#appending-and-prepending-override-style-syntax>`__"
777section in the BitBake User's Manual.
778
779.. _openembedded-build-system-workflow:
780
781The OpenEmbedded Build System Workflow
782======================================
783
784The `OpenEmbedded build
785system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ uses a "workflow" to
786accomplish image and SDK generation. The following figure overviews that
787workflow: Following is a brief summary of the "workflow":
788
7891. Developers specify architecture, policies, patches and configuration
790 details.
791
7922. The build system fetches and downloads the source code from the
793 specified location. The build system supports standard methods such
794 as tarballs or source code repositories systems such as Git.
795
7963. Once source code is downloaded, the build system extracts the sources
797 into a local work area where patches are applied and common steps for
798 configuring and compiling the software are run.
799
8004. The build system then installs the software into a temporary staging
801 area where the binary package format you select (DEB, RPM, or IPK) is
802 used to roll up the software.
803
8045. Different QA and sanity checks run throughout entire build process.
805
8066. After the binaries are created, the build system generates a binary
807 package feed that is used to create the final root file image.
808
8097. The build system generates the file system image and a customized
810 Extensible SDK (eSDK) for application development in parallel.
811
812For a very detailed look at this workflow, see the "`OpenEmbedded Build
813System Concepts <#openembedded-build-system-build-concepts>`__" section.
814
815Some Basic Terms
816================
817
818It helps to understand some basic fundamental terms when learning the
819Yocto Project. Although a list of terms exists in the "`Yocto Project
820Terms <&YOCTO_DOCS_REF_URL;#ref-terms>`__" section of the Yocto Project
821Reference Manual, this section provides the definitions of some terms
822helpful for getting started:
823
824- *Configuration Files:* Files that hold global definitions of
825 variables, user-defined variables, and hardware configuration
826 information. These files tell the `Open-Embedded build
827 system <&YOCTO_DOCS_REF_URL;#build-system-term>`__ what to build and
828 what to put into the image to support a particular platform.
829
830- *Extensible Software Development Kit (eSDK):* A custom SDK for
831 application developers. This eSDK allows developers to incorporate
832 their library and programming changes back into the image to make
833 their code available to other application developers. For information
834 on the eSDK, see the `Yocto Project Application Development and the
835 Extensible Software Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__
836 manual.
837
838- *Layer:* A collection of related recipes. Layers allow you to
839 consolidate related metadata to customize your build. Layers also
840 isolate information used when building for multiple architectures.
841 Layers are hierarchical in their ability to override previous
842 specifications. You can include any number of available layers from
843 the Yocto Project and customize the build by adding your layers after
844 them. You can search the Layer Index for layers used within Yocto
845 Project.
846
847 For more detailed information on layers, see the "`Understanding and
848 Creating
849 Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
850 section in the Yocto Project Development Tasks Manual. For a
851 discussion specifically on BSP Layers, see the "`BSP
852 Layers <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__" section in the Yocto
853 Project Board Support Packages (BSP) Developer's Guide.
854
855- *Metadata:* A key element of the Yocto Project is the Metadata that
856 is used to construct a Linux distribution and is contained in the
857 files that the OpenEmbedded build system parses when building an
858 image. In general, Metadata includes recipes, configuration files,
859 and other information that refers to the build instructions
860 themselves, as well as the data used to control what things get built
861 and the effects of the build. Metadata also includes commands and
862 data used to indicate what versions of software are used, from where
863 they are obtained, and changes or additions to the software itself
864 (patches or auxiliary files) that are used to fix bugs or customize
865 the software for use in a particular situation. OpenEmbedded-Core is
866 an important set of validated metadata.
867
868- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
869 are sometimes used for the OpenEmbedded Build System.
870
871 BitBake is a task scheduler and execution engine that parses
872 instructions (i.e. recipes) and configuration data. After a parsing
873 phase, BitBake creates a dependency tree to order the compilation,
874 schedules the compilation of the included code, and finally executes
875 the building of the specified custom Linux image (distribution).
876 BitBake is similar to the ``make`` tool.
877
878 During a build process, the build system tracks dependencies and
879 performs a native or cross-compilation of the package. As a first
880 step in a cross-build setup, the framework attempts to create a
881 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
882 platform.
883
884- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
885 foundation recipes, classes, and associated files that are meant to
886 be common among many different OpenEmbedded-derived systems,
887 including the Yocto Project. OE-Core is a curated subset of an
888 original repository developed by the OpenEmbedded community that has
889 been pared down into a smaller, core set of continuously validated
890 recipes. The result is a tightly controlled and quality-assured core
891 set of recipes.
892
893 You can see the Metadata in the ``meta`` directory of the Yocto
894 Project `Source
895 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
896
897- *Packages:* In the context of the Yocto Project, this term refers to
898 a recipe's packaged output produced by BitBake (i.e. a "baked
899 recipe"). A package is generally the compiled binaries produced from
900 the recipe's sources. You "bake" something by running it through
901 BitBake.
902
903 It is worth noting that the term "package" can, in general, have
904 subtle meanings. For example, the packages referred to in the
905 "`Required Packages for the Build
906 Host <&YOCTO_DOCS_REF_URL;#required-packages-for-the-build-host>`__"
907 section in the Yocto Project Reference Manual are compiled binaries
908 that, when installed, add functionality to your Linux distribution.
909
910 Another point worth noting is that historically within the Yocto
911 Project, recipes were referred to as packages - thus, the existence
912 of several BitBake variables that are seemingly mis-named, (e.g.
913 ```PR`` <&YOCTO_DOCS_REF_URL;#var-PR>`__,
914 ```PV`` <&YOCTO_DOCS_REF_URL;#var-PV>`__, and
915 ```PE`` <&YOCTO_DOCS_REF_URL;#var-PE>`__).
916
917- *Poky:* Poky is a reference embedded distribution and a reference
918 test configuration. Poky provides the following:
919
920 - A base-level functional distro used to illustrate how to customize
921 a distribution.
922
923 - A means by which to test the Yocto Project components (i.e. Poky
924 is used to validate the Yocto Project).
925
926 - A vehicle through which you can download the Yocto Project.
927
928 Poky is not a product level distro. Rather, it is a good starting
929 point for customization.
930
931 .. note::
932
933 Poky is an integration layer on top of OE-Core.
934
935- *Recipe:* The most common form of metadata. A recipe contains a list
936 of settings and tasks (i.e. instructions) for building packages that
937 are then used to build the binary image. A recipe describes where you
938 get source code and which patches to apply. Recipes describe
939 dependencies for libraries or for other recipes as well as
940 configuration and compilation options. Related recipes are
941 consolidated into a layer.
diff --git a/documentation/overview-manual/overview-manual.rst b/documentation/overview-manual/overview-manual.rst
new file mode 100644
index 0000000000..e6cd07d34d
--- /dev/null
+++ b/documentation/overview-manual/overview-manual.rst
@@ -0,0 +1,12 @@
1==========================================
2Yocto Project Overview and Concepts Manual
3==========================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 overview-manual-intro
10 overview-manual-yp-intro
11 overview-manual-development-environment
12 overview-manual-concepts
diff --git a/documentation/profile-manual/profile-manual-arch.rst b/documentation/profile-manual/profile-manual-arch.rst
new file mode 100644
index 0000000000..26b6ff0d74
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-arch.rst
@@ -0,0 +1,28 @@
1*************************************************************
2Overall Architecture of the Linux Tracing and Profiling Tools
3*************************************************************
4
5Architecture of the Tracing and Profiling Tools
6===============================================
7
8It may seem surprising to see a section covering an 'overall
9architecture' for what seems to be a random collection of tracing tools
10that together make up the Linux tracing and profiling space. The fact
11is, however, that in recent years this seemingly disparate set of tools
12has started to converge on a 'core' set of underlying mechanisms:
13
14- static tracepoints
15- dynamic tracepoints
16
17 - kprobes
18 - uprobes
19
20- the perf_events subsystem
21- debugfs
22
23.. container:: informalexample
24
25 Tying it Together:
26 Rather than enumerating here how each tool makes use of these common
27 mechanisms, textboxes like this will make note of the specific usages
28 in each tool as they come up in the course of the text.
diff --git a/documentation/profile-manual/profile-manual-examples.rst b/documentation/profile-manual/profile-manual-examples.rst
new file mode 100644
index 0000000000..15b0696366
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-examples.rst
@@ -0,0 +1,20 @@
1*******************
2Real-World Examples
3*******************
4
5This chapter contains real-world examples.
6
7Slow Write Speed on Live Images
8===============================
9
10In one of our previous releases (denzil), users noticed that booting off
11of a live image and writing to disk was noticeably slower. This included
12the boot itself, especially the first one, since first boots tend to do
13a significant amount of writing due to certain post-install scripts.
14
15The problem (and solution) was discovered by using the Yocto tracing
16tools, in this case 'perf stat', 'perf script', 'perf record' and 'perf
17report'.
18
19See all the unvarnished details of how this bug was diagnosed and solved
20here: Yocto Bug #3049
diff --git a/documentation/profile-manual/profile-manual-intro.rst b/documentation/profile-manual/profile-manual-intro.rst
new file mode 100644
index 0000000000..40febd8b4e
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-intro.rst
@@ -0,0 +1,67 @@
1******************************************
2Yocto Project Profiling and Tracing Manual
3******************************************
4
5.. _profile-intro:
6
7Introduction
8============
9
10Yocto bundles a number of tracing and profiling tools - this 'HOWTO'
11describes their basic usage and shows by example how to make use of them
12to examine application and system behavior.
13
14The tools presented are for the most part completely open-ended and have
15quite good and/or extensive documentation of their own which can be used
16to solve just about any problem you might come across in Linux. Each
17section that describes a particular tool has links to that tool's
18documentation and website.
19
20The purpose of this 'HOWTO' is to present a set of common and generally
21useful tracing and profiling idioms along with their application (as
22appropriate) to each tool, in the context of a general-purpose
23'drill-down' methodology that can be applied to solving a large number
24(90%?) of problems. For help with more advanced usages and problems,
25please see the documentation and/or websites listed for each tool.
26
27The final section of this 'HOWTO' is a collection of real-world examples
28which we'll be continually adding to as we solve more problems using the
29tools - feel free to add your own examples to the list!
30
31.. _profile-manual-general-setup:
32
33General Setup
34=============
35
36Most of the tools are available only in 'sdk' images or in images built
37after adding 'tools-profile' to your local.conf. So, in order to be able
38to access all of the tools described here, please first build and boot
39an 'sdk' image e.g. $ bitbake core-image-sato-sdk or alternatively by
40adding 'tools-profile' to the EXTRA_IMAGE_FEATURES line in your
41local.conf: EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile" If you
42use the 'tools-profile' method, you don't need to build an sdk image -
43the tracing and profiling tools will be included in non-sdk images as
44well e.g.: $ bitbake core-image-sato
45
46.. note::
47
48 By default, the Yocto build system strips symbols from the binaries
49 it packages, which makes it difficult to use some of the tools.
50
51 You can prevent that by setting the
52 ```INHIBIT_PACKAGE_STRIP`` <&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_STRIP>`__
53 variable to "1" in your ``local.conf`` when you build the image:
54
55INHIBIT_PACKAGE_STRIP = "1" The above setting will noticeably increase
56the size of your image.
57
58If you've already built a stripped image, you can generate debug
59packages (xxx-dbg) which you can manually install as needed.
60
61To generate debug info for packages, you can add dbg-pkgs to
62EXTRA_IMAGE_FEATURES in local.conf. For example: EXTRA_IMAGE_FEATURES =
63"debug-tweaks tools-profile dbg-pkgs" Additionally, in order to generate
64the right type of debuginfo, we also need to set
65```PACKAGE_DEBUG_SPLIT_STYLE`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_DEBUG_SPLIT_STYLE>`__
66in the ``local.conf`` file: PACKAGE_DEBUG_SPLIT_STYLE =
67'debug-file-directory'
diff --git a/documentation/profile-manual/profile-manual-usage.rst b/documentation/profile-manual/profile-manual-usage.rst
new file mode 100644
index 0000000000..0f9f8925ea
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-usage.rst
@@ -0,0 +1,2018 @@
1***************************************************************
2Basic Usage (with examples) for each of the Yocto Tracing Tools
3***************************************************************
4
5This chapter presents basic usage examples for each of the tracing
6tools.
7
8.. _profile-manual-perf:
9
10perf
11====
12
13The 'perf' tool is the profiling and tracing tool that comes bundled
14with the Linux kernel.
15
16Don't let the fact that it's part of the kernel fool you into thinking
17that it's only for tracing and profiling the kernel - you can indeed use
18it to trace and profile just the kernel, but you can also use it to
19profile specific applications separately (with or without kernel
20context), and you can also use it to trace and profile the kernel and
21all applications on the system simultaneously to gain a system-wide view
22of what's going on.
23
24In many ways, perf aims to be a superset of all the tracing and
25profiling tools available in Linux today, including all the other tools
26covered in this HOWTO. The past couple of years have seen perf subsume a
27lot of the functionality of those other tools and, at the same time,
28those other tools have removed large portions of their previous
29functionality and replaced it with calls to the equivalent functionality
30now implemented by the perf subsystem. Extrapolation suggests that at
31some point those other tools will simply become completely redundant and
32go away; until then, we'll cover those other tools in these pages and in
33many cases show how the same things can be accomplished in perf and the
34other tools when it seems useful to do so.
35
36The coverage below details some of the most common ways you'll likely
37want to apply the tool; full documentation can be found either within
38the tool itself or in the man pages at
39`perf(1) <http://linux.die.net/man/1/perf>`__.
40
41.. _perf-setup:
42
43Setup
44-----
45
46For this section, we'll assume you've already performed the basic setup
47outlined in the General Setup section.
48
49In particular, you'll get the most mileage out of perf if you profile an
50image built with the following in your ``local.conf`` file:
51`INHIBIT_PACKAGE_STRIP <&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_STRIP>`__
52= "1"
53
54perf runs on the target system for the most part. You can archive
55profile data and copy it to the host for analysis, but for the rest of
56this document we assume you've ssh'ed to the host and will be running
57the perf commands on the target.
58
59.. _perf-basic-usage:
60
61Basic Usage
62-----------
63
64The perf tool is pretty much self-documenting. To remind yourself of the
65available commands, simply type 'perf', which will show you basic usage
66along with the available perf subcommands: root@crownbay:~# perf usage:
67perf [--version] [--help] COMMAND [ARGS] The most commonly used perf
68commands are: annotate Read perf.data (created by perf record) and
69display annotated code archive Create archive with object files with
70build-ids found in perf.data file bench General framework for benchmark
71suites buildid-cache Manage build-id cache. buildid-list List the
72buildids in a perf.data file diff Read two perf.data files and display
73the differential profile evlist List the event names in a perf.data file
74inject Filter to augment the events stream with additional information
75kmem Tool to trace/measure kernel memory(slab) properties kvm Tool to
76trace/measure kvm guest os list List all symbolic event types lock
77Analyze lock events probe Define new dynamic tracepoints record Run a
78command and record its profile into perf.data report Read perf.data
79(created by perf record) and display the profile sched Tool to
80trace/measure scheduler properties (latencies) script Read perf.data
81(created by perf record) and display trace output stat Run a command and
82gather performance counter statistics test Runs sanity tests. timechart
83Tool to visualize total system behavior during a workload top System
84profiling tool. See 'perf help COMMAND' for more information on a
85specific command.
86
87Using perf to do Basic Profiling
88~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
89
90As a simple test case, we'll profile the 'wget' of a fairly large file,
91which is a minimally interesting case because it has both file and
92network I/O aspects, and at least in the case of standard Yocto images,
93it's implemented as part of busybox, so the methods we use to analyze it
94can be used in a very similar way to the whole host of supported busybox
95applets in Yocto. root@crownbay:~# rm linux-2.6.19.2.tar.bz2; \\ wget
96http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
97The quickest and easiest way to get some basic overall data about what's
98going on for a particular workload is to profile it using 'perf stat'.
99'perf stat' basically profiles using a few default counters and displays
100the summed counts at the end of the run: root@crownbay:~# perf stat wget
101http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
102Connecting to downloads.yoctoproject.org (140.211.169.59:80)
103linux-2.6.19.2.tar.b 100%
104\|***************************************************\| 41727k 0:00:00
105ETA Performance counter stats for 'wget
106http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
1074597.223902 task-clock # 0.077 CPUs utilized 23568 context-switches #
1080.005 M/sec 68 CPU-migrations # 0.015 K/sec 241 page-faults # 0.052
109K/sec 3045817293 cycles # 0.663 GHz <not supported>
110stalled-cycles-frontend <not supported> stalled-cycles-backend 858909167
111instructions # 0.28 insns per cycle 165441165 branches # 35.987 M/sec
11219550329 branch-misses # 11.82% of all branches 59.836627620 seconds
113time elapsed Many times such a simple-minded test doesn't yield much of
114interest, but sometimes it does (see Real-world Yocto bug (slow
115loop-mounted write speed)).
116
117Also, note that 'perf stat' isn't restricted to a fixed set of counters
118- basically any event listed in the output of 'perf list' can be tallied
119by 'perf stat'. For example, suppose we wanted to see a summary of all
120the events related to kernel memory allocation/freeing along with cache
121hits and misses: root@crownbay:~# perf stat -e kmem:\* -e
122cache-references -e cache-misses wget
123http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
124Connecting to downloads.yoctoproject.org (140.211.169.59:80)
125linux-2.6.19.2.tar.b 100%
126\|***************************************************\| 41727k 0:00:00
127ETA Performance counter stats for 'wget
128http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
1295566 kmem:kmalloc 125517 kmem:kmem_cache_alloc 0 kmem:kmalloc_node 0
130kmem:kmem_cache_alloc_node 34401 kmem:kfree 69920 kmem:kmem_cache_free
131133 kmem:mm_page_free 41 kmem:mm_page_free_batched 11502
132kmem:mm_page_alloc 11375 kmem:mm_page_alloc_zone_locked 0
133kmem:mm_page_pcpu_drain 0 kmem:mm_page_alloc_extfrag 66848602
134cache-references 2917740 cache-misses # 4.365 % of all cache refs
13544.831023415 seconds time elapsed So 'perf stat' gives us a nice easy
136way to get a quick overview of what might be happening for a set of
137events, but normally we'd need a little more detail in order to
138understand what's going on in a way that we can act on in a useful way.
139
140To dive down into a next level of detail, we can use 'perf record'/'perf
141report' which will collect profiling data and present it to use using an
142interactive text-based UI (or simply as text if we specify --stdio to
143'perf report').
144
145As our first attempt at profiling this workload, we'll simply run 'perf
146record', handing it the workload we want to profile (everything after
147'perf record' and any perf options we hand it - here none - will be
148executed in a new shell). perf collects samples until the process exits
149and records them in a file named 'perf.data' in the current working
150directory. root@crownbay:~# perf record wget
151http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
152Connecting to downloads.yoctoproject.org (140.211.169.59:80)
153linux-2.6.19.2.tar.b 100%
154\|************************************************\| 41727k 0:00:00 ETA
155[ perf record: Woken up 1 times to write data ] [ perf record: Captured
156and wrote 0.176 MB perf.data (~7700 samples) ] To see the results in a
157'text-based UI' (tui), simply run 'perf report', which will read the
158perf.data file in the current working directory and display the results
159in an interactive UI: root@crownbay:~# perf report
160
161The above screenshot displays a 'flat' profile, one entry for each
162'bucket' corresponding to the functions that were profiled during the
163profiling run, ordered from the most popular to the least (perf has
164options to sort in various orders and keys as well as display entries
165only above a certain threshold and so on - see the perf documentation
166for details). Note that this includes both userspace functions (entries
167containing a [.]) and kernel functions accounted to the process (entries
168containing a [k]). (perf has command-line modifiers that can be used to
169restrict the profiling to kernel or userspace, among others).
170
171Notice also that the above report shows an entry for 'busybox', which is
172the executable that implements 'wget' in Yocto, but that instead of a
173useful function name in that entry, it displays a not-so-friendly hex
174value instead. The steps below will show how to fix that problem.
175
176Before we do that, however, let's try running a different profile, one
177which shows something a little more interesting. The only difference
178between the new profile and the previous one is that we'll add the -g
179option, which will record not just the address of a sampled function,
180but the entire callchain to the sampled function as well:
181root@crownbay:~# perf record -g wget
182http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
183Connecting to downloads.yoctoproject.org (140.211.169.59:80)
184linux-2.6.19.2.tar.b 100%
185\|************************************************\| 41727k 0:00:00 ETA
186[ perf record: Woken up 3 times to write data ] [ perf record: Captured
187and wrote 0.652 MB perf.data (~28476 samples) ] root@crownbay:~# perf
188report
189
190Using the callgraph view, we can actually see not only which functions
191took the most time, but we can also see a summary of how those functions
192were called and learn something about how the program interacts with the
193kernel in the process.
194
195Notice that each entry in the above screenshot now contains a '+' on the
196left-hand side. This means that we can expand the entry and drill down
197into the callchains that feed into that entry. Pressing 'enter' on any
198one of them will expand the callchain (you can also press 'E' to expand
199them all at the same time or 'C' to collapse them all).
200
201In the screenshot above, we've toggled the \__copy_to_user_ll() entry
202and several subnodes all the way down. This lets us see which callchains
203contributed to the profiled \__copy_to_user_ll() function which
204contributed 1.77% to the total profile.
205
206As a bit of background explanation for these callchains, think about
207what happens at a high level when you run wget to get a file out on the
208network. Basically what happens is that the data comes into the kernel
209via the network connection (socket) and is passed to the userspace
210program 'wget' (which is actually a part of busybox, but that's not
211important for now), which takes the buffers the kernel passes to it and
212writes it to a disk file to save it.
213
214The part of this process that we're looking at in the above call stacks
215is the part where the kernel passes the data it's read from the socket
216down to wget i.e. a copy-to-user.
217
218Notice also that here there's also a case where the hex value is
219displayed in the callstack, here in the expanded sys_clock_gettime()
220function. Later we'll see it resolve to a userspace function call in
221busybox.
222
223The above screenshot shows the other half of the journey for the data -
224from the wget program's userspace buffers to disk. To get the buffers to
225disk, the wget program issues a write(2), which does a copy-from-user to
226the kernel, which then takes care via some circuitous path (probably
227also present somewhere in the profile data), to get it safely to disk.
228
229Now that we've seen the basic layout of the profile data and the basics
230of how to extract useful information out of it, let's get back to the
231task at hand and see if we can get some basic idea about where the time
232is spent in the program we're profiling, wget. Remember that wget is
233actually implemented as an applet in busybox, so while the process name
234is 'wget', the executable we're actually interested in is busybox. So
235let's expand the first entry containing busybox:
236
237Again, before we expanded we saw that the function was labeled with a
238hex value instead of a symbol as with most of the kernel entries.
239Expanding the busybox entry doesn't make it any better.
240
241The problem is that perf can't find the symbol information for the
242busybox binary, which is actually stripped out by the Yocto build
243system.
244
245One way around that is to put the following in your ``local.conf`` file
246when you build the image:
247`INHIBIT_PACKAGE_STRIP <&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_STRIP>`__
248= "1" However, we already have an image with the binaries stripped, so
249what can we do to get perf to resolve the symbols? Basically we need to
250install the debuginfo for the busybox package.
251
252To generate the debug info for the packages in the image, we can add
253dbg-pkgs to EXTRA_IMAGE_FEATURES in local.conf. For example:
254EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs"
255Additionally, in order to generate the type of debuginfo that perf
256understands, we also need to set
257```PACKAGE_DEBUG_SPLIT_STYLE`` <&YOCTO_DOCS_REF_URL;#var-PACKAGE_DEBUG_SPLIT_STYLE>`__
258in the ``local.conf`` file: PACKAGE_DEBUG_SPLIT_STYLE =
259'debug-file-directory' Once we've done that, we can install the
260debuginfo for busybox. The debug packages once built can be found in
261build/tmp/deploy/rpm/\* on the host system. Find the busybox-dbg-...rpm
262file and copy it to the target. For example: [trz@empanada core2]$ scp
263/home/trz/yocto/crownbay-tracing-dbg/build/tmp/deploy/rpm/core2_32/busybox-dbg-1.20.2-r2.core2_32.rpm
264root@192.168.1.31: root@192.168.1.31's password:
265busybox-dbg-1.20.2-r2.core2_32.rpm 100% 1826KB 1.8MB/s 00:01 Now install
266the debug rpm on the target: root@crownbay:~# rpm -i
267busybox-dbg-1.20.2-r2.core2_32.rpm Now that the debuginfo is installed,
268we see that the busybox entries now display their functions
269symbolically:
270
271If we expand one of the entries and press 'enter' on a leaf node, we're
272presented with a menu of actions we can take to get more information
273related to that entry:
274
275One of these actions allows us to show a view that displays a
276busybox-centric view of the profiled functions (in this case we've also
277expanded all the nodes using the 'E' key):
278
279Finally, we can see that now that the busybox debuginfo is installed,
280the previously unresolved symbol in the sys_clock_gettime() entry
281mentioned previously is now resolved, and shows that the
282sys_clock_gettime system call that was the source of 6.75% of the
283copy-to-user overhead was initiated by the handle_input() busybox
284function:
285
286At the lowest level of detail, we can dive down to the assembly level
287and see which instructions caused the most overhead in a function.
288Pressing 'enter' on the 'udhcpc_main' function, we're again presented
289with a menu:
290
291Selecting 'Annotate udhcpc_main', we get a detailed listing of
292percentages by instruction for the udhcpc_main function. From the
293display, we can see that over 50% of the time spent in this function is
294taken up by a couple tests and the move of a constant (1) to a register:
295
296As a segue into tracing, let's try another profile using a different
297counter, something other than the default 'cycles'.
298
299The tracing and profiling infrastructure in Linux has become unified in
300a way that allows us to use the same tool with a completely different
301set of counters, not just the standard hardware counters that
302traditional tools have had to restrict themselves to (of course the
303traditional tools can also make use of the expanded possibilities now
304available to them, and in some cases have, as mentioned previously).
305
306We can get a list of the available events that can be used to profile a
307workload via 'perf list': root@crownbay:~# perf list List of pre-defined
308events (to be used in -e): cpu-cycles OR cycles [Hardware event]
309stalled-cycles-frontend OR idle-cycles-frontend [Hardware event]
310stalled-cycles-backend OR idle-cycles-backend [Hardware event]
311instructions [Hardware event] cache-references [Hardware event]
312cache-misses [Hardware event] branch-instructions OR branches [Hardware
313event] branch-misses [Hardware event] bus-cycles [Hardware event]
314ref-cycles [Hardware event] cpu-clock [Software event] task-clock
315[Software event] page-faults OR faults [Software event] minor-faults
316[Software event] major-faults [Software event] context-switches OR cs
317[Software event] cpu-migrations OR migrations [Software event]
318alignment-faults [Software event] emulation-faults [Software event]
319L1-dcache-loads [Hardware cache event] L1-dcache-load-misses [Hardware
320cache event] L1-dcache-prefetch-misses [Hardware cache event]
321L1-icache-loads [Hardware cache event] L1-icache-load-misses [Hardware
322cache event] . . . rNNN [Raw hardware event descriptor]
323cpu/t1=v1[,t2=v2,t3 ...]/modifier [Raw hardware event descriptor] (see
324'perf list --help' on how to encode it) mem:<addr>[:access] [Hardware
325breakpoint] sunrpc:rpc_call_status [Tracepoint event]
326sunrpc:rpc_bind_status [Tracepoint event] sunrpc:rpc_connect_status
327[Tracepoint event] sunrpc:rpc_task_begin [Tracepoint event]
328skb:kfree_skb [Tracepoint event] skb:consume_skb [Tracepoint event]
329skb:skb_copy_datagram_iovec [Tracepoint event] net:net_dev_xmit
330[Tracepoint event] net:net_dev_queue [Tracepoint event]
331net:netif_receive_skb [Tracepoint event] net:netif_rx [Tracepoint event]
332napi:napi_poll [Tracepoint event] sock:sock_rcvqueue_full [Tracepoint
333event] sock:sock_exceed_buf_limit [Tracepoint event]
334udp:udp_fail_queue_rcv_skb [Tracepoint event] hda:hda_send_cmd
335[Tracepoint event] hda:hda_get_response [Tracepoint event]
336hda:hda_bus_reset [Tracepoint event] scsi:scsi_dispatch_cmd_start
337[Tracepoint event] scsi:scsi_dispatch_cmd_error [Tracepoint event]
338scsi:scsi_eh_wakeup [Tracepoint event] drm:drm_vblank_event [Tracepoint
339event] drm:drm_vblank_event_queued [Tracepoint event]
340drm:drm_vblank_event_delivered [Tracepoint event] random:mix_pool_bytes
341[Tracepoint event] random:mix_pool_bytes_nolock [Tracepoint event]
342random:credit_entropy_bits [Tracepoint event] gpio:gpio_direction
343[Tracepoint event] gpio:gpio_value [Tracepoint event]
344block:block_rq_abort [Tracepoint event] block:block_rq_requeue
345[Tracepoint event] block:block_rq_issue [Tracepoint event]
346block:block_bio_bounce [Tracepoint event] block:block_bio_complete
347[Tracepoint event] block:block_bio_backmerge [Tracepoint event] . .
348writeback:writeback_wake_thread [Tracepoint event]
349writeback:writeback_wake_forker_thread [Tracepoint event]
350writeback:writeback_bdi_register [Tracepoint event] . .
351writeback:writeback_single_inode_requeue [Tracepoint event]
352writeback:writeback_single_inode [Tracepoint event] kmem:kmalloc
353[Tracepoint event] kmem:kmem_cache_alloc [Tracepoint event]
354kmem:mm_page_alloc [Tracepoint event] kmem:mm_page_alloc_zone_locked
355[Tracepoint event] kmem:mm_page_pcpu_drain [Tracepoint event]
356kmem:mm_page_alloc_extfrag [Tracepoint event]
357vmscan:mm_vmscan_kswapd_sleep [Tracepoint event]
358vmscan:mm_vmscan_kswapd_wake [Tracepoint event]
359vmscan:mm_vmscan_wakeup_kswapd [Tracepoint event]
360vmscan:mm_vmscan_direct_reclaim_begin [Tracepoint event] . .
361module:module_get [Tracepoint event] module:module_put [Tracepoint
362event] module:module_request [Tracepoint event] sched:sched_kthread_stop
363[Tracepoint event] sched:sched_wakeup [Tracepoint event]
364sched:sched_wakeup_new [Tracepoint event] sched:sched_process_fork
365[Tracepoint event] sched:sched_process_exec [Tracepoint event]
366sched:sched_stat_runtime [Tracepoint event] rcu:rcu_utilization
367[Tracepoint event] workqueue:workqueue_queue_work [Tracepoint event]
368workqueue:workqueue_execute_end [Tracepoint event]
369signal:signal_generate [Tracepoint event] signal:signal_deliver
370[Tracepoint event] timer:timer_init [Tracepoint event] timer:timer_start
371[Tracepoint event] timer:hrtimer_cancel [Tracepoint event]
372timer:itimer_state [Tracepoint event] timer:itimer_expire [Tracepoint
373event] irq:irq_handler_entry [Tracepoint event] irq:irq_handler_exit
374[Tracepoint event] irq:softirq_entry [Tracepoint event] irq:softirq_exit
375[Tracepoint event] irq:softirq_raise [Tracepoint event] printk:console
376[Tracepoint event] task:task_newtask [Tracepoint event] task:task_rename
377[Tracepoint event] syscalls:sys_enter_socketcall [Tracepoint event]
378syscalls:sys_exit_socketcall [Tracepoint event] . . .
379syscalls:sys_enter_unshare [Tracepoint event] syscalls:sys_exit_unshare
380[Tracepoint event] raw_syscalls:sys_enter [Tracepoint event]
381raw_syscalls:sys_exit [Tracepoint event]
382
383.. container:: informalexample
384
385 Tying it Together:
386 These are exactly the same set of events defined by the trace event
387 subsystem and exposed by ftrace/tracecmd/kernelshark as files in
388 /sys/kernel/debug/tracing/events, by SystemTap as
389 kernel.trace("tracepoint_name") and (partially) accessed by LTTng.
390
391Only a subset of these would be of interest to us when looking at this
392workload, so let's choose the most likely subsystems (identified by the
393string before the colon in the Tracepoint events) and do a 'perf stat'
394run using only those wildcarded subsystems: root@crownbay:~# perf stat
395-e skb:\* -e net:\* -e napi:\* -e sched:\* -e workqueue:\* -e irq:\* -e
396syscalls:\* wget
397http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
398Performance counter stats for 'wget
399http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
40023323 skb:kfree_skb 0 skb:consume_skb 49897 skb:skb_copy_datagram_iovec
4016217 net:net_dev_xmit 6217 net:net_dev_queue 7962 net:netif_receive_skb
4022 net:netif_rx 8340 napi:napi_poll 0 sched:sched_kthread_stop 0
403sched:sched_kthread_stop_ret 3749 sched:sched_wakeup 0
404sched:sched_wakeup_new 0 sched:sched_switch 29 sched:sched_migrate_task
4050 sched:sched_process_free 1 sched:sched_process_exit 0
406sched:sched_wait_task 0 sched:sched_process_wait 0
407sched:sched_process_fork 1 sched:sched_process_exec 0
408sched:sched_stat_wait 2106519415641 sched:sched_stat_sleep 0
409sched:sched_stat_iowait 147453613 sched:sched_stat_blocked 12903026955
410sched:sched_stat_runtime 0 sched:sched_pi_setprio 3574
411workqueue:workqueue_queue_work 3574 workqueue:workqueue_activate_work 0
412workqueue:workqueue_execute_start 0 workqueue:workqueue_execute_end
41316631 irq:irq_handler_entry 16631 irq:irq_handler_exit 28521
414irq:softirq_entry 28521 irq:softirq_exit 28728 irq:softirq_raise 1
415syscalls:sys_enter_sendmmsg 1 syscalls:sys_exit_sendmmsg 0
416syscalls:sys_enter_recvmmsg 0 syscalls:sys_exit_recvmmsg 14
417syscalls:sys_enter_socketcall 14 syscalls:sys_exit_socketcall . . .
41816965 syscalls:sys_enter_read 16965 syscalls:sys_exit_read 12854
419syscalls:sys_enter_write 12854 syscalls:sys_exit_write . . .
42058.029710972 seconds time elapsed Let's pick one of these tracepoints
421and tell perf to do a profile using it as the sampling event:
422root@crownbay:~# perf record -g -e sched:sched_wakeup wget
423http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
424
425The screenshot above shows the results of running a profile using
426sched:sched_switch tracepoint, which shows the relative costs of various
427paths to sched_wakeup (note that sched_wakeup is the name of the
428tracepoint - it's actually defined just inside ttwu_do_wakeup(), which
429accounts for the function name actually displayed in the profile: /\* \*
430Mark the task runnable and perform wakeup-preemption. \*/ static void
431ttwu_do_wakeup(struct rq \*rq, struct task_struct \*p, int wake_flags) {
432trace_sched_wakeup(p, true); . . . } A couple of the more interesting
433callchains are expanded and displayed above, basically some network
434receive paths that presumably end up waking up wget (busybox) when
435network data is ready.
436
437Note that because tracepoints are normally used for tracing, the default
438sampling period for tracepoints is 1 i.e. for tracepoints perf will
439sample on every event occurrence (this can be changed using the -c
440option). This is in contrast to hardware counters such as for example
441the default 'cycles' hardware counter used for normal profiling, where
442sampling periods are much higher (in the thousands) because profiling
443should have as low an overhead as possible and sampling on every cycle
444would be prohibitively expensive.
445
446Using perf to do Basic Tracing
447~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
448
449Profiling is a great tool for solving many problems or for getting a
450high-level view of what's going on with a workload or across the system.
451It is however by definition an approximation, as suggested by the most
452prominent word associated with it, 'sampling'. On the one hand, it
453allows a representative picture of what's going on in the system to be
454cheaply taken, but on the other hand, that cheapness limits its utility
455when that data suggests a need to 'dive down' more deeply to discover
456what's really going on. In such cases, the only way to see what's really
457going on is to be able to look at (or summarize more intelligently) the
458individual steps that go into the higher-level behavior exposed by the
459coarse-grained profiling data.
460
461As a concrete example, we can trace all the events we think might be
462applicable to our workload: root@crownbay:~# perf record -g -e skb:\* -e
463net:\* -e napi:\* -e sched:sched_switch -e sched:sched_wakeup -e irq:\*
464-e syscalls:sys_enter_read -e syscalls:sys_exit_read -e
465syscalls:sys_enter_write -e syscalls:sys_exit_write wget
466http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
467We can look at the raw trace output using 'perf script' with no
468arguments: root@crownbay:~# perf script perf 1262 [000] 11624.857082:
469sys_exit_read: 0x0 perf 1262 [000] 11624.857193: sched_wakeup:
470comm=migration/0 pid=6 prio=0 success=1 target_cpu=000 wget 1262 [001]
47111624.858021: softirq_raise: vec=1 [action=TIMER] wget 1262 [001]
47211624.858074: softirq_entry: vec=1 [action=TIMER] wget 1262 [001]
47311624.858081: softirq_exit: vec=1 [action=TIMER] wget 1262 [001]
47411624.858166: sys_enter_read: fd: 0x0003, buf: 0xbf82c940, count: 0x0200
475wget 1262 [001] 11624.858177: sys_exit_read: 0x200 wget 1262 [001]
47611624.858878: kfree_skb: skbaddr=0xeb248d80 protocol=0
477location=0xc15a5308 wget 1262 [001] 11624.858945: kfree_skb:
478skbaddr=0xeb248000 protocol=0 location=0xc15a5308 wget 1262 [001]
47911624.859020: softirq_raise: vec=1 [action=TIMER] wget 1262 [001]
48011624.859076: softirq_entry: vec=1 [action=TIMER] wget 1262 [001]
48111624.859083: softirq_exit: vec=1 [action=TIMER] wget 1262 [001]
48211624.859167: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
483wget 1262 [001] 11624.859192: sys_exit_read: 0x1d7 wget 1262 [001]
48411624.859228: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
485wget 1262 [001] 11624.859233: sys_exit_read: 0x0 wget 1262 [001]
48611624.859573: sys_enter_read: fd: 0x0003, buf: 0xbf82c580, count: 0x0200
487wget 1262 [001] 11624.859584: sys_exit_read: 0x200 wget 1262 [001]
48811624.859864: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
489wget 1262 [001] 11624.859888: sys_exit_read: 0x400 wget 1262 [001]
49011624.859935: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
491wget 1262 [001] 11624.859944: sys_exit_read: 0x400 This gives us a
492detailed timestamped sequence of events that occurred within the
493workload with respect to those events.
494
495In many ways, profiling can be viewed as a subset of tracing -
496theoretically, if you have a set of trace events that's sufficient to
497capture all the important aspects of a workload, you can derive any of
498the results or views that a profiling run can.
499
500Another aspect of traditional profiling is that while powerful in many
501ways, it's limited by the granularity of the underlying data. Profiling
502tools offer various ways of sorting and presenting the sample data,
503which make it much more useful and amenable to user experimentation, but
504in the end it can't be used in an open-ended way to extract data that
505just isn't present as a consequence of the fact that conceptually, most
506of it has been thrown away.
507
508Full-blown detailed tracing data does however offer the opportunity to
509manipulate and present the information collected during a tracing run in
510an infinite variety of ways.
511
512Another way to look at it is that there are only so many ways that the
513'primitive' counters can be used on their own to generate interesting
514output; to get anything more complicated than simple counts requires
515some amount of additional logic, which is typically very specific to the
516problem at hand. For example, if we wanted to make use of a 'counter'
517that maps to the value of the time difference between when a process was
518scheduled to run on a processor and the time it actually ran, we
519wouldn't expect such a counter to exist on its own, but we could derive
520one called say 'wakeup_latency' and use it to extract a useful view of
521that metric from trace data. Likewise, we really can't figure out from
522standard profiling tools how much data every process on the system reads
523and writes, along with how many of those reads and writes fail
524completely. If we have sufficient trace data, however, we could with the
525right tools easily extract and present that information, but we'd need
526something other than pre-canned profiling tools to do that.
527
528Luckily, there is a general-purpose way to handle such needs, called
529'programming languages'. Making programming languages easily available
530to apply to such problems given the specific format of data is called a
531'programming language binding' for that data and language. Perf supports
532two programming language bindings, one for Python and one for Perl.
533
534.. container:: informalexample
535
536 Tying it Together:
537 Language bindings for manipulating and aggregating trace data are of
538 course not a new idea. One of the first projects to do this was IBM's
539 DProbes dpcc compiler, an ANSI C compiler which targeted a low-level
540 assembly language running on an in-kernel interpreter on the target
541 system. This is exactly analogous to what Sun's DTrace did, except
542 that DTrace invented its own language for the purpose. Systemtap,
543 heavily inspired by DTrace, also created its own one-off language,
544 but rather than running the product on an in-kernel interpreter,
545 created an elaborate compiler-based machinery to translate its
546 language into kernel modules written in C.
547
548Now that we have the trace data in perf.data, we can use 'perf script
549-g' to generate a skeleton script with handlers for the read/write
550entry/exit events we recorded: root@crownbay:~# perf script -g python
551generated Python script: perf-script.py The skeleton script simply
552creates a python function for each event type in the perf.data file. The
553body of each function simply prints the event name along with its
554parameters. For example: def net__netif_rx(event_name, context,
555common_cpu, common_secs, common_nsecs, common_pid, common_comm, skbaddr,
556len, name): print_header(event_name, common_cpu, common_secs,
557common_nsecs, common_pid, common_comm) print "skbaddr=%u, len=%u,
558name=%s\n" % (skbaddr, len, name), We can run that script directly to
559print all of the events contained in the perf.data file:
560root@crownbay:~# perf script -s perf-script.py in trace_begin
561syscalls__sys_exit_read 0 11624.857082795 1262 perf nr=3, ret=0
562sched__sched_wakeup 0 11624.857193498 1262 perf comm=migration/0, pid=6,
563prio=0, success=1, target_cpu=0 irq__softirq_raise 1 11624.858021635
5641262 wget vec=TIMER irq__softirq_entry 1 11624.858074075 1262 wget
565vec=TIMER irq__softirq_exit 1 11624.858081389 1262 wget vec=TIMER
566syscalls__sys_enter_read 1 11624.858166434 1262 wget nr=3, fd=3,
567buf=3213019456, count=512 syscalls__sys_exit_read 1 11624.858177924 1262
568wget nr=3, ret=512 skb__kfree_skb 1 11624.858878188 1262 wget
569skbaddr=3945041280, location=3243922184, protocol=0 skb__kfree_skb 1
57011624.858945608 1262 wget skbaddr=3945037824, location=3243922184,
571protocol=0 irq__softirq_raise 1 11624.859020942 1262 wget vec=TIMER
572irq__softirq_entry 1 11624.859076935 1262 wget vec=TIMER
573irq__softirq_exit 1 11624.859083469 1262 wget vec=TIMER
574syscalls__sys_enter_read 1 11624.859167565 1262 wget nr=3, fd=3,
575buf=3077701632, count=1024 syscalls__sys_exit_read 1 11624.859192533
5761262 wget nr=3, ret=471 syscalls__sys_enter_read 1 11624.859228072 1262
577wget nr=3, fd=3, buf=3077701632, count=1024 syscalls__sys_exit_read 1
57811624.859233707 1262 wget nr=3, ret=0 syscalls__sys_enter_read 1
57911624.859573008 1262 wget nr=3, fd=3, buf=3213018496, count=512
580syscalls__sys_exit_read 1 11624.859584818 1262 wget nr=3, ret=512
581syscalls__sys_enter_read 1 11624.859864562 1262 wget nr=3, fd=3,
582buf=3077701632, count=1024 syscalls__sys_exit_read 1 11624.859888770
5831262 wget nr=3, ret=1024 syscalls__sys_enter_read 1 11624.859935140 1262
584wget nr=3, fd=3, buf=3077701632, count=1024 syscalls__sys_exit_read 1
58511624.859944032 1262 wget nr=3, ret=1024 That in itself isn't very
586useful; after all, we can accomplish pretty much the same thing by
587simply running 'perf script' without arguments in the same directory as
588the perf.data file.
589
590We can however replace the print statements in the generated function
591bodies with whatever we want, and thereby make it infinitely more
592useful.
593
594As a simple example, let's just replace the print statements in the
595function bodies with a simple function that does nothing but increment a
596per-event count. When the program is run against a perf.data file, each
597time a particular event is encountered, a tally is incremented for that
598event. For example: def net__netif_rx(event_name, context, common_cpu,
599common_secs, common_nsecs, common_pid, common_comm, skbaddr, len, name):
600inc_counts(event_name) Each event handler function in the generated code
601is modified to do this. For convenience, we define a common function
602called inc_counts() that each handler calls; inc_counts() simply tallies
603a count for each event using the 'counts' hash, which is a specialized
604hash function that does Perl-like autovivification, a capability that's
605extremely useful for kinds of multi-level aggregation commonly used in
606processing traces (see perf's documentation on the Python language
607binding for details): counts = autodict() def inc_counts(event_name):
608try: counts[event_name] += 1 except TypeError: counts[event_name] = 1
609Finally, at the end of the trace processing run, we want to print the
610result of all the per-event tallies. For that, we use the special
611'trace_end()' function: def trace_end(): for event_name, count in
612counts.iteritems(): print "%-40s %10s\n" % (event_name, count) The end
613result is a summary of all the events recorded in the trace:
614skb__skb_copy_datagram_iovec 13148 irq__softirq_entry 4796
615irq__irq_handler_exit 3805 irq__softirq_exit 4795
616syscalls__sys_enter_write 8990 net__net_dev_xmit 652 skb__kfree_skb 4047
617sched__sched_wakeup 1155 irq__irq_handler_entry 3804 irq__softirq_raise
6184799 net__net_dev_queue 652 syscalls__sys_enter_read 17599
619net__netif_receive_skb 1743 syscalls__sys_exit_read 17598 net__netif_rx
6202 napi__napi_poll 1877 syscalls__sys_exit_write 8990 Note that this is
621pretty much exactly the same information we get from 'perf stat', which
622goes a little way to support the idea mentioned previously that given
623the right kind of trace data, higher-level profiling-type summaries can
624be derived from it.
625
626Documentation on using the `'perf script' python
627binding <http://linux.die.net/man/1/perf-script-python>`__.
628
629System-Wide Tracing and Profiling
630~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
631
632The examples so far have focused on tracing a particular program or
633workload - in other words, every profiling run has specified the program
634to profile in the command-line e.g. 'perf record wget ...'.
635
636It's also possible, and more interesting in many cases, to run a
637system-wide profile or trace while running the workload in a separate
638shell.
639
640To do system-wide profiling or tracing, you typically use the -a flag to
641'perf record'.
642
643To demonstrate this, open up one window and start the profile using the
644-a flag (press Ctrl-C to stop tracing): root@crownbay:~# perf record -g
645-a ^C[ perf record: Woken up 6 times to write data ] [ perf record:
646Captured and wrote 1.400 MB perf.data (~61172 samples) ] In another
647window, run the wget test: root@crownbay:~# wget
648http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
649Connecting to downloads.yoctoproject.org (140.211.169.59:80)
650linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k
6510:00:00 ETA Here we see entries not only for our wget load, but for
652other processes running on the system as well:
653
654In the snapshot above, we can see callchains that originate in libc, and
655a callchain from Xorg that demonstrates that we're using a proprietary X
656driver in userspace (notice the presence of 'PVR' and some other
657unresolvable symbols in the expanded Xorg callchain).
658
659Note also that we have both kernel and userspace entries in the above
660snapshot. We can also tell perf to focus on userspace but providing a
661modifier, in this case 'u', to the 'cycles' hardware counter when we
662record a profile: root@crownbay:~# perf record -g -a -e cycles:u ^C[
663perf record: Woken up 2 times to write data ] [ perf record: Captured
664and wrote 0.376 MB perf.data (~16443 samples) ]
665
666Notice in the screenshot above, we see only userspace entries ([.])
667
668Finally, we can press 'enter' on a leaf node and select the 'Zoom into
669DSO' menu item to show only entries associated with a specific DSO. In
670the screenshot below, we've zoomed into the 'libc' DSO which shows all
671the entries associated with the libc-xxx.so DSO.
672
673We can also use the system-wide -a switch to do system-wide tracing.
674Here we'll trace a couple of scheduler events: root@crownbay:~# perf
675record -a -e sched:sched_switch -e sched:sched_wakeup ^C[ perf record:
676Woken up 38 times to write data ] [ perf record: Captured and wrote
6779.780 MB perf.data (~427299 samples) ] We can look at the raw output
678using 'perf script' with no arguments: root@crownbay:~# perf script perf
6791383 [001] 6171.460045: sched_wakeup: comm=kworker/1:1 pid=21 prio=120
680success=1 target_cpu=001 perf 1383 [001] 6171.460066: sched_switch:
681prev_comm=perf prev_pid=1383 prev_prio=120 prev_state=R+ ==>
682next_comm=kworker/1:1 next_pid=21 next_prio=120 kworker/1:1 21 [001]
6836171.460093: sched_switch: prev_comm=kworker/1:1 prev_pid=21
684prev_prio=120 prev_state=S ==> next_comm=perf next_pid=1383
685next_prio=120 swapper 0 [000] 6171.468063: sched_wakeup:
686comm=kworker/0:3 pid=1209 prio=120 success=1 target_cpu=000 swapper 0
687[000] 6171.468107: sched_switch: prev_comm=swapper/0 prev_pid=0
688prev_prio=120 prev_state=R ==> next_comm=kworker/0:3 next_pid=1209
689next_prio=120 kworker/0:3 1209 [000] 6171.468143: sched_switch:
690prev_comm=kworker/0:3 prev_pid=1209 prev_prio=120 prev_state=S ==>
691next_comm=swapper/0 next_pid=0 next_prio=120 perf 1383 [001]
6926171.470039: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1
693target_cpu=001 perf 1383 [001] 6171.470058: sched_switch: prev_comm=perf
694prev_pid=1383 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1
695next_pid=21 next_prio=120 kworker/1:1 21 [001] 6171.470082:
696sched_switch: prev_comm=kworker/1:1 prev_pid=21 prev_prio=120
697prev_state=S ==> next_comm=perf next_pid=1383 next_prio=120 perf 1383
698[001] 6171.480035: sched_wakeup: comm=kworker/1:1 pid=21 prio=120
699success=1 target_cpu=001
700
701.. _perf-filtering:
702
703Filtering
704^^^^^^^^^
705
706Notice that there are a lot of events that don't really have anything to
707do with what we're interested in, namely events that schedule 'perf'
708itself in and out or that wake perf up. We can get rid of those by using
709the '--filter' option - for each event we specify using -e, we can add a
710--filter after that to filter out trace events that contain fields with
711specific values: root@crownbay:~# perf record -a -e sched:sched_switch
712--filter 'next_comm != perf && prev_comm != perf' -e sched:sched_wakeup
713--filter 'comm != perf' ^C[ perf record: Woken up 38 times to write data
714] [ perf record: Captured and wrote 9.688 MB perf.data (~423279 samples)
715] root@crownbay:~# perf script swapper 0 [000] 7932.162180:
716sched_switch: prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=R
717==> next_comm=kworker/0:3 next_pid=1209 next_prio=120 kworker/0:3 1209
718[000] 7932.162236: sched_switch: prev_comm=kworker/0:3 prev_pid=1209
719prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0
720next_prio=120 perf 1407 [001] 7932.170048: sched_wakeup:
721comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001 perf 1407
722[001] 7932.180044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120
723success=1 target_cpu=001 perf 1407 [001] 7932.190038: sched_wakeup:
724comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001 perf 1407
725[001] 7932.200044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120
726success=1 target_cpu=001 perf 1407 [001] 7932.210044: sched_wakeup:
727comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001 perf 1407
728[001] 7932.220044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120
729success=1 target_cpu=001 swapper 0 [001] 7932.230111: sched_wakeup:
730comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001 swapper 0
731[001] 7932.230146: sched_switch: prev_comm=swapper/1 prev_pid=0
732prev_prio=120 prev_state=R ==> next_comm=kworker/1:1 next_pid=21
733next_prio=120 kworker/1:1 21 [001] 7932.230205: sched_switch:
734prev_comm=kworker/1:1 prev_pid=21 prev_prio=120 prev_state=S ==>
735next_comm=swapper/1 next_pid=0 next_prio=120 swapper 0 [000]
7367932.326109: sched_wakeup: comm=kworker/0:3 pid=1209 prio=120 success=1
737target_cpu=000 swapper 0 [000] 7932.326171: sched_switch:
738prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=R ==>
739next_comm=kworker/0:3 next_pid=1209 next_prio=120 kworker/0:3 1209 [000]
7407932.326214: sched_switch: prev_comm=kworker/0:3 prev_pid=1209
741prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0
742next_prio=120 In this case, we've filtered out all events that have
743'perf' in their 'comm' or 'comm_prev' or 'comm_next' fields. Notice that
744there are still events recorded for perf, but notice that those events
745don't have values of 'perf' for the filtered fields. To completely
746filter out anything from perf will require a bit more work, but for the
747purpose of demonstrating how to use filters, it's close enough.
748
749.. container:: informalexample
750
751 Tying it Together:
752 These are exactly the same set of event filters defined by the trace
753 event subsystem. See the ftrace/tracecmd/kernelshark section for more
754 discussion about these event filters.
755
756.. container:: informalexample
757
758 Tying it Together:
759 These event filters are implemented by a special-purpose
760 pseudo-interpreter in the kernel and are an integral and
761 indispensable part of the perf design as it relates to tracing.
762 kernel-based event filters provide a mechanism to precisely throttle
763 the event stream that appears in user space, where it makes sense to
764 provide bindings to real programming languages for postprocessing the
765 event stream. This architecture allows for the intelligent and
766 flexible partitioning of processing between the kernel and user
767 space. Contrast this with other tools such as SystemTap, which does
768 all of its processing in the kernel and as such requires a special
769 project-defined language in order to accommodate that design, or
770 LTTng, where everything is sent to userspace and as such requires a
771 super-efficient kernel-to-userspace transport mechanism in order to
772 function properly. While perf certainly can benefit from for instance
773 advances in the design of the transport, it doesn't fundamentally
774 depend on them. Basically, if you find that your perf tracing
775 application is causing buffer I/O overruns, it probably means that
776 you aren't taking enough advantage of the kernel filtering engine.
777
778Using Dynamic Tracepoints
779~~~~~~~~~~~~~~~~~~~~~~~~~
780
781perf isn't restricted to the fixed set of static tracepoints listed by
782'perf list'. Users can also add their own 'dynamic' tracepoints anywhere
783in the kernel. For instance, suppose we want to define our own
784tracepoint on do_fork(). We can do that using the 'perf probe' perf
785subcommand: root@crownbay:~# perf probe do_fork Added new event:
786probe:do_fork (on do_fork) You can now use it in all perf tools, such
787as: perf record -e probe:do_fork -aR sleep 1 Adding a new tracepoint via
788'perf probe' results in an event with all the expected files and format
789in /sys/kernel/debug/tracing/events, just the same as for static
790tracepoints (as discussed in more detail in the trace events subsystem
791section: root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork#
792ls -al drwxr-xr-x 2 root root 0 Oct 28 11:42 . drwxr-xr-x 3 root root 0
793Oct 28 11:42 .. -rw-r--r-- 1 root root 0 Oct 28 11:42 enable -rw-r--r--
7941 root root 0 Oct 28 11:42 filter -r--r--r-- 1 root root 0 Oct 28 11:42
795format -r--r--r-- 1 root root 0 Oct 28 11:42 id
796root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork# cat format
797name: do_fork ID: 944 format: field:unsigned short common_type;
798offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2;
799size:1; signed:0; field:unsigned char common_preempt_count; offset:3;
800size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1;
801field:int common_padding; offset:8; size:4; signed:1; field:unsigned
802long \__probe_ip; offset:12; size:4; signed:0; print fmt: "(%lx)",
803REC->__probe_ip We can list all dynamic tracepoints currently in
804existence: root@crownbay:~# perf probe -l probe:do_fork (on do_fork)
805probe:schedule (on schedule) Let's record system-wide ('sleep 30' is a
806trick for recording system-wide but basically do nothing and then wake
807up after 30 seconds): root@crownbay:~# perf record -g -a -e
808probe:do_fork sleep 30 [ perf record: Woken up 1 times to write data ] [
809perf record: Captured and wrote 0.087 MB perf.data (~3812 samples) ]
810Using 'perf script' we can see each do_fork event that fired:
811root@crownbay:~# perf script # ======== # captured on: Sun Oct 28
81211:55:18 2012 # hostname : crownbay # os release : 3.4.11-yocto-standard
813# perf version : 3.4.11 # arch : i686 # nrcpus online : 2 # nrcpus avail
814: 2 # cpudesc : Intel(R) Atom(TM) CPU E660 @ 1.30GHz # cpuid :
815GenuineIntel,6,38,1 # total memory : 1017184 kB # cmdline :
816/usr/bin/perf record -g -a -e probe:do_fork sleep 30 # event : name =
817probe:do_fork, type = 2, config = 0x3b0, config1 = 0x0, config2 = 0x0,
818excl_usr = 0, excl_kern = 0, id = { 5, 6 } # HEADER_CPU_TOPOLOGY info
819available, use -I to display # ======== # matchbox-deskto 1197 [001]
82034211.378318: do_fork: (c1028460) matchbox-deskto 1295 [001]
82134211.380388: do_fork: (c1028460) pcmanfm 1296 [000] 34211.632350:
822do_fork: (c1028460) pcmanfm 1296 [000] 34211.639917: do_fork: (c1028460)
823matchbox-deskto 1197 [001] 34217.541603: do_fork: (c1028460)
824matchbox-deskto 1299 [001] 34217.543584: do_fork: (c1028460) gthumb 1300
825[001] 34217.697451: do_fork: (c1028460) gthumb 1300 [001] 34219.085734:
826do_fork: (c1028460) gthumb 1300 [000] 34219.121351: do_fork: (c1028460)
827gthumb 1300 [001] 34219.264551: do_fork: (c1028460) pcmanfm 1296 [000]
82834219.590380: do_fork: (c1028460) matchbox-deskto 1197 [001]
82934224.955965: do_fork: (c1028460) matchbox-deskto 1306 [001]
83034224.957972: do_fork: (c1028460) matchbox-termin 1307 [000]
83134225.038214: do_fork: (c1028460) matchbox-termin 1307 [001]
83234225.044218: do_fork: (c1028460) matchbox-termin 1307 [000]
83334225.046442: do_fork: (c1028460) matchbox-deskto 1197 [001]
83434237.112138: do_fork: (c1028460) matchbox-deskto 1311 [001]
83534237.114106: do_fork: (c1028460) gaku 1312 [000] 34237.202388: do_fork:
836(c1028460) And using 'perf report' on the same file, we can see the
837callgraphs from starting a few programs during those 30 seconds:
838
839.. container:: informalexample
840
841 Tying it Together:
842 The trace events subsystem accommodate static and dynamic tracepoints
843 in exactly the same way - there's no difference as far as the
844 infrastructure is concerned. See the ftrace section for more details
845 on the trace event subsystem.
846
847.. container:: informalexample
848
849 Tying it Together:
850 Dynamic tracepoints are implemented under the covers by kprobes and
851 uprobes. kprobes and uprobes are also used by and in fact are the
852 main focus of SystemTap.
853
854.. _perf-documentation:
855
856Documentation
857-------------
858
859Online versions of the man pages for the commands discussed in this
860section can be found here:
861
862- The `'perf stat' manpage <http://linux.die.net/man/1/perf-stat>`__.
863
864- The `'perf record'
865 manpage <http://linux.die.net/man/1/perf-record>`__.
866
867- The `'perf report'
868 manpage <http://linux.die.net/man/1/perf-report>`__.
869
870- The `'perf probe' manpage <http://linux.die.net/man/1/perf-probe>`__.
871
872- The `'perf script'
873 manpage <http://linux.die.net/man/1/perf-script>`__.
874
875- Documentation on using the `'perf script' python
876 binding <http://linux.die.net/man/1/perf-script-python>`__.
877
878- The top-level `perf(1) manpage <http://linux.die.net/man/1/perf>`__.
879
880Normally, you should be able to invoke the man pages via perf itself
881e.g. 'perf help' or 'perf help record'.
882
883However, by default Yocto doesn't install man pages, but perf invokes
884the man pages for most help functionality. This is a bug and is being
885addressed by a Yocto bug: `Bug 3388 - perf: enable man pages for basic
886'help'
887functionality <https://bugzilla.yoctoproject.org/show_bug.cgi?id=3388>`__.
888
889The man pages in text form, along with some other files, such as a set
890of examples, can be found in the 'perf' directory of the kernel tree:
891tools/perf/Documentation There's also a nice perf tutorial on the perf
892wiki that goes into more detail than we do here in certain areas: `Perf
893Tutorial <https://perf.wiki.kernel.org/index.php/Tutorial>`__
894
895.. _profile-manual-ftrace:
896
897ftrace
898======
899
900'ftrace' literally refers to the 'ftrace function tracer' but in reality
901this encompasses a number of related tracers along with the
902infrastructure that they all make use of.
903
904.. _ftrace-setup:
905
906Setup
907-----
908
909For this section, we'll assume you've already performed the basic setup
910outlined in the General Setup section.
911
912ftrace, trace-cmd, and kernelshark run on the target system, and are
913ready to go out-of-the-box - no additional setup is necessary. For the
914rest of this section we assume you've ssh'ed to the host and will be
915running ftrace on the target. kernelshark is a GUI application and if
916you use the '-X' option to ssh you can have the kernelshark GUI run on
917the target but display remotely on the host if you want.
918
919Basic ftrace usage
920------------------
921
922'ftrace' essentially refers to everything included in the /tracing
923directory of the mounted debugfs filesystem (Yocto follows the standard
924convention and mounts it at /sys/kernel/debug). Here's a listing of all
925the files found in /sys/kernel/debug/tracing on a Yocto system:
926root@sugarbay:/sys/kernel/debug/tracing# ls README kprobe_events trace
927available_events kprobe_profile trace_clock available_filter_functions
928options trace_marker available_tracers per_cpu trace_options
929buffer_size_kb printk_formats trace_pipe buffer_total_size_kb
930saved_cmdlines tracing_cpumask current_tracer set_event tracing_enabled
931dyn_ftrace_total_info set_ftrace_filter tracing_on enabled_functions
932set_ftrace_notrace tracing_thresh events set_ftrace_pid free_buffer
933set_graph_function The files listed above are used for various purposes
934- some relate directly to the tracers themselves, others are used to set
935tracing options, and yet others actually contain the tracing output when
936a tracer is in effect. Some of the functions can be guessed from their
937names, others need explanation; in any case, we'll cover some of the
938files we see here below but for an explanation of the others, please see
939the ftrace documentation.
940
941We'll start by looking at some of the available built-in tracers.
942
943cat'ing the 'available_tracers' file lists the set of available tracers:
944root@sugarbay:/sys/kernel/debug/tracing# cat available_tracers blk
945function_graph function nop The 'current_tracer' file contains the
946tracer currently in effect: root@sugarbay:/sys/kernel/debug/tracing# cat
947current_tracer nop The above listing of current_tracer shows that the
948'nop' tracer is in effect, which is just another way of saying that
949there's actually no tracer currently in effect.
950
951echo'ing one of the available_tracers into current_tracer makes the
952specified tracer the current tracer:
953root@sugarbay:/sys/kernel/debug/tracing# echo function > current_tracer
954root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer function The
955above sets the current tracer to be the 'function tracer'. This tracer
956traces every function call in the kernel and makes it available as the
957contents of the 'trace' file. Reading the 'trace' file lists the
958currently buffered function calls that have been traced by the function
959tracer: root@sugarbay:/sys/kernel/debug/tracing# cat trace \| less #
960tracer: function # # entries-in-buffer/entries-written: 310629/766471
961#P:8 # # \_-----=> irqs-off # / \_----=> need-resched # \| / \_---=>
962hardirq/softirq # \|\| / \_--=> preempt-depth # \||\| / delay # TASK-PID
963CPU# \|||\| TIMESTAMP FUNCTION # \| \| \| \|||\| \| \| <idle>-0 [004]
964d..1 470.867169: ktime_get_real <-intel_idle <idle>-0 [004] d..1
965470.867170: getnstimeofday <-ktime_get_real <idle>-0 [004] d..1
966470.867171: ns_to_timeval <-intel_idle <idle>-0 [004] d..1 470.867171:
967ns_to_timespec <-ns_to_timeval <idle>-0 [004] d..1 470.867172:
968smp_apic_timer_interrupt <-apic_timer_interrupt <idle>-0 [004] d..1
969470.867172: native_apic_mem_write <-smp_apic_timer_interrupt <idle>-0
970[004] d..1 470.867172: irq_enter <-smp_apic_timer_interrupt <idle>-0
971[004] d..1 470.867172: rcu_irq_enter <-irq_enter <idle>-0 [004] d..1
972470.867173: rcu_idle_exit_common.isra.33 <-rcu_irq_enter <idle>-0 [004]
973d..1 470.867173: local_bh_disable <-irq_enter <idle>-0 [004] d..1
974470.867173: add_preempt_count <-local_bh_disable <idle>-0 [004] d.s1
975470.867174: tick_check_idle <-irq_enter <idle>-0 [004] d.s1 470.867174:
976tick_check_oneshot_broadcast <-tick_check_idle <idle>-0 [004] d.s1
977470.867174: ktime_get <-tick_check_idle <idle>-0 [004] d.s1 470.867174:
978tick_nohz_stop_idle <-tick_check_idle <idle>-0 [004] d.s1 470.867175:
979update_ts_time_stats <-tick_nohz_stop_idle <idle>-0 [004] d.s1
980470.867175: nr_iowait_cpu <-update_ts_time_stats <idle>-0 [004] d.s1
981470.867175: tick_do_update_jiffies64 <-tick_check_idle <idle>-0 [004]
982d.s1 470.867175: \_raw_spin_lock <-tick_do_update_jiffies64 <idle>-0
983[004] d.s1 470.867176: add_preempt_count <-_raw_spin_lock <idle>-0 [004]
984d.s2 470.867176: do_timer <-tick_do_update_jiffies64 <idle>-0 [004] d.s2
985470.867176: \_raw_spin_lock <-do_timer <idle>-0 [004] d.s2 470.867176:
986add_preempt_count <-_raw_spin_lock <idle>-0 [004] d.s3 470.867177:
987ntp_tick_length <-do_timer <idle>-0 [004] d.s3 470.867177:
988\_raw_spin_lock_irqsave <-ntp_tick_length . . . Each line in the trace
989above shows what was happening in the kernel on a given cpu, to the
990level of detail of function calls. Each entry shows the function called,
991followed by its caller (after the arrow).
992
993The function tracer gives you an extremely detailed idea of what the
994kernel was doing at the point in time the trace was taken, and is a
995great way to learn about how the kernel code works in a dynamic sense.
996
997.. container:: informalexample
998
999 Tying it Together:
1000 The ftrace function tracer is also available from within perf, as the
1001 ftrace:function tracepoint.
1002
1003It is a little more difficult to follow the call chains than it needs to
1004be - luckily there's a variant of the function tracer that displays the
1005callchains explicitly, called the 'function_graph' tracer:
1006root@sugarbay:/sys/kernel/debug/tracing# echo function_graph >
1007current_tracer root@sugarbay:/sys/kernel/debug/tracing# cat trace \|
1008less tracer: function_graph CPU DURATION FUNCTION CALLS \| \| \| \| \|
1009\| \| 7) 0.046 us \| pick_next_task_fair(); 7) 0.043 us \|
1010pick_next_task_stop(); 7) 0.042 us \| pick_next_task_rt(); 7) 0.032 us
1011\| pick_next_task_fair(); 7) 0.030 us \| pick_next_task_idle(); 7) \|
1012\_raw_spin_unlock_irq() { 7) 0.033 us \| sub_preempt_count(); 7) 0.258
1013us \| } 7) 0.032 us \| sub_preempt_count(); 7) + 13.341 us \| } /\*
1014\__schedule \*/ 7) 0.095 us \| } /\* sub_preempt_count \*/ 7) \|
1015schedule() { 7) \| \__schedule() { 7) 0.060 us \| add_preempt_count();
10167) 0.044 us \| rcu_note_context_switch(); 7) \| \_raw_spin_lock_irq() {
10177) 0.033 us \| add_preempt_count(); 7) 0.247 us \| } 7) \|
1018idle_balance() { 7) \| \_raw_spin_unlock() { 7) 0.031 us \|
1019sub_preempt_count(); 7) 0.246 us \| } 7) \| update_shares() { 7) 0.030
1020us \| \__rcu_read_lock(); 7) 0.029 us \| \__rcu_read_unlock(); 7) 0.484
1021us \| } 7) 0.030 us \| \__rcu_read_lock(); 7) \| load_balance() { 7) \|
1022find_busiest_group() { 7) 0.031 us \| idle_cpu(); 7) 0.029 us \|
1023idle_cpu(); 7) 0.035 us \| idle_cpu(); 7) 0.906 us \| } 7) 1.141 us \| }
10247) 0.022 us \| msecs_to_jiffies(); 7) \| load_balance() { 7) \|
1025find_busiest_group() { 7) 0.031 us \| idle_cpu(); . . . 4) 0.062 us \|
1026msecs_to_jiffies(); 4) 0.062 us \| \__rcu_read_unlock(); 4) \|
1027\_raw_spin_lock() { 4) 0.073 us \| add_preempt_count(); 4) 0.562 us \| }
10284) + 17.452 us \| } 4) 0.108 us \| put_prev_task_fair(); 4) 0.102 us \|
1029pick_next_task_fair(); 4) 0.084 us \| pick_next_task_stop(); 4) 0.075 us
1030\| pick_next_task_rt(); 4) 0.062 us \| pick_next_task_fair(); 4) 0.066
1031us \| pick_next_task_idle(); ------------------------------------------
10324) kworker-74 => <idle>-0 ------------------------------------------ 4)
1033\| finish_task_switch() { 4) \| \_raw_spin_unlock_irq() { 4) 0.100 us \|
1034sub_preempt_count(); 4) 0.582 us \| } 4) 1.105 us \| } 4) 0.088 us \|
1035sub_preempt_count(); 4) ! 100.066 us \| } . . . 3) \| sys_ioctl() { 3)
10360.083 us \| fget_light(); 3) \| security_file_ioctl() { 3) 0.066 us \|
1037cap_file_ioctl(); 3) 0.562 us \| } 3) \| do_vfs_ioctl() { 3) \|
1038drm_ioctl() { 3) 0.075 us \| drm_ut_debug_printk(); 3) \|
1039i915_gem_pwrite_ioctl() { 3) \| i915_mutex_lock_interruptible() { 3)
10400.070 us \| mutex_lock_interruptible(); 3) 0.570 us \| } 3) \|
1041drm_gem_object_lookup() { 3) \| \_raw_spin_lock() { 3) 0.080 us \|
1042add_preempt_count(); 3) 0.620 us \| } 3) \| \_raw_spin_unlock() { 3)
10430.085 us \| sub_preempt_count(); 3) 0.562 us \| } 3) 2.149 us \| } 3)
10440.133 us \| i915_gem_object_pin(); 3) \|
1045i915_gem_object_set_to_gtt_domain() { 3) 0.065 us \|
1046i915_gem_object_flush_gpu_write_domain(); 3) 0.065 us \|
1047i915_gem_object_wait_rendering(); 3) 0.062 us \|
1048i915_gem_object_flush_cpu_write_domain(); 3) 1.612 us \| } 3) \|
1049i915_gem_object_put_fence() { 3) 0.097 us \|
1050i915_gem_object_flush_fence.constprop.36(); 3) 0.645 us \| } 3) 0.070 us
1051\| add_preempt_count(); 3) 0.070 us \| sub_preempt_count(); 3) 0.073 us
1052\| i915_gem_object_unpin(); 3) 0.068 us \| mutex_unlock(); 3) 9.924 us
1053\| } 3) + 11.236 us \| } 3) + 11.770 us \| } 3) + 13.784 us \| } 3) \|
1054sys_ioctl() { As you can see, the function_graph display is much easier
1055to follow. Also note that in addition to the function calls and
1056associated braces, other events such as scheduler events are displayed
1057in context. In fact, you can freely include any tracepoint available in
1058the trace events subsystem described in the next section by simply
1059enabling those events, and they'll appear in context in the function
1060graph display. Quite a powerful tool for understanding kernel dynamics.
1061
1062Also notice that there are various annotations on the left hand side of
1063the display. For example if the total time it took for a given function
1064to execute is above a certain threshold, an exclamation point or plus
1065sign appears on the left hand side. Please see the ftrace documentation
1066for details on all these fields.
1067
1068The 'trace events' Subsystem
1069----------------------------
1070
1071One especially important directory contained within the
1072/sys/kernel/debug/tracing directory is the 'events' subdirectory, which
1073contains representations of every tracepoint in the system. Listing out
1074the contents of the 'events' subdirectory, we see mainly another set of
1075subdirectories: root@sugarbay:/sys/kernel/debug/tracing# cd events
1076root@sugarbay:/sys/kernel/debug/tracing/events# ls -al drwxr-xr-x 38
1077root root 0 Nov 14 23:19 . drwxr-xr-x 5 root root 0 Nov 14 23:19 ..
1078drwxr-xr-x 19 root root 0 Nov 14 23:19 block drwxr-xr-x 32 root root 0
1079Nov 14 23:19 btrfs drwxr-xr-x 5 root root 0 Nov 14 23:19 drm -rw-r--r--
10801 root root 0 Nov 14 23:19 enable drwxr-xr-x 40 root root 0 Nov 14 23:19
1081ext3 drwxr-xr-x 79 root root 0 Nov 14 23:19 ext4 drwxr-xr-x 14 root root
10820 Nov 14 23:19 ftrace drwxr-xr-x 8 root root 0 Nov 14 23:19 hda
1083-r--r--r-- 1 root root 0 Nov 14 23:19 header_event -r--r--r-- 1 root
1084root 0 Nov 14 23:19 header_page drwxr-xr-x 25 root root 0 Nov 14 23:19
1085i915 drwxr-xr-x 7 root root 0 Nov 14 23:19 irq drwxr-xr-x 12 root root 0
1086Nov 14 23:19 jbd drwxr-xr-x 14 root root 0 Nov 14 23:19 jbd2 drwxr-xr-x
108714 root root 0 Nov 14 23:19 kmem drwxr-xr-x 7 root root 0 Nov 14 23:19
1088module drwxr-xr-x 3 root root 0 Nov 14 23:19 napi drwxr-xr-x 6 root root
10890 Nov 14 23:19 net drwxr-xr-x 3 root root 0 Nov 14 23:19 oom drwxr-xr-x
109012 root root 0 Nov 14 23:19 power drwxr-xr-x 3 root root 0 Nov 14 23:19
1091printk drwxr-xr-x 8 root root 0 Nov 14 23:19 random drwxr-xr-x 4 root
1092root 0 Nov 14 23:19 raw_syscalls drwxr-xr-x 3 root root 0 Nov 14 23:19
1093rcu drwxr-xr-x 6 root root 0 Nov 14 23:19 rpm drwxr-xr-x 20 root root 0
1094Nov 14 23:19 sched drwxr-xr-x 7 root root 0 Nov 14 23:19 scsi drwxr-xr-x
10954 root root 0 Nov 14 23:19 signal drwxr-xr-x 5 root root 0 Nov 14 23:19
1096skb drwxr-xr-x 4 root root 0 Nov 14 23:19 sock drwxr-xr-x 10 root root 0
1097Nov 14 23:19 sunrpc drwxr-xr-x 538 root root 0 Nov 14 23:19 syscalls
1098drwxr-xr-x 4 root root 0 Nov 14 23:19 task drwxr-xr-x 14 root root 0 Nov
109914 23:19 timer drwxr-xr-x 3 root root 0 Nov 14 23:19 udp drwxr-xr-x 21
1100root root 0 Nov 14 23:19 vmscan drwxr-xr-x 3 root root 0 Nov 14 23:19
1101vsyscall drwxr-xr-x 6 root root 0 Nov 14 23:19 workqueue drwxr-xr-x 26
1102root root 0 Nov 14 23:19 writeback Each one of these subdirectories
1103corresponds to a 'subsystem' and contains yet again more subdirectories,
1104each one of those finally corresponding to a tracepoint. For example,
1105here are the contents of the 'kmem' subsystem:
1106root@sugarbay:/sys/kernel/debug/tracing/events# cd kmem
1107root@sugarbay:/sys/kernel/debug/tracing/events/kmem# ls -al drwxr-xr-x
110814 root root 0 Nov 14 23:19 . drwxr-xr-x 38 root root 0 Nov 14 23:19 ..
1109-rw-r--r-- 1 root root 0 Nov 14 23:19 enable -rw-r--r-- 1 root root 0
1110Nov 14 23:19 filter drwxr-xr-x 2 root root 0 Nov 14 23:19 kfree
1111drwxr-xr-x 2 root root 0 Nov 14 23:19 kmalloc drwxr-xr-x 2 root root 0
1112Nov 14 23:19 kmalloc_node drwxr-xr-x 2 root root 0 Nov 14 23:19
1113kmem_cache_alloc drwxr-xr-x 2 root root 0 Nov 14 23:19
1114kmem_cache_alloc_node drwxr-xr-x 2 root root 0 Nov 14 23:19
1115kmem_cache_free drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_alloc
1116drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_alloc_extfrag drwxr-xr-x 2
1117root root 0 Nov 14 23:19 mm_page_alloc_zone_locked drwxr-xr-x 2 root
1118root 0 Nov 14 23:19 mm_page_free drwxr-xr-x 2 root root 0 Nov 14 23:19
1119mm_page_free_batched drwxr-xr-x 2 root root 0 Nov 14 23:19
1120mm_page_pcpu_drain Let's see what's inside the subdirectory for a
1121specific tracepoint, in this case the one for kmalloc:
1122root@sugarbay:/sys/kernel/debug/tracing/events/kmem# cd kmalloc
1123root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# ls -al
1124drwxr-xr-x 2 root root 0 Nov 14 23:19 . drwxr-xr-x 14 root root 0 Nov 14
112523:19 .. -rw-r--r-- 1 root root 0 Nov 14 23:19 enable -rw-r--r-- 1 root
1126root 0 Nov 14 23:19 filter -r--r--r-- 1 root root 0 Nov 14 23:19 format
1127-r--r--r-- 1 root root 0 Nov 14 23:19 id The 'format' file for the
1128tracepoint describes the event in memory, which is used by the various
1129tracing tools that now make use of these tracepoint to parse the event
1130and make sense of it, along with a 'print fmt' field that allows tools
1131like ftrace to display the event as text. Here's what the format of the
1132kmalloc event looks like:
1133root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# cat format
1134name: kmalloc ID: 313 format: field:unsigned short common_type;
1135offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2;
1136size:1; signed:0; field:unsigned char common_preempt_count; offset:3;
1137size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1;
1138field:int common_padding; offset:8; size:4; signed:1; field:unsigned
1139long call_site; offset:16; size:8; signed:0; field:const void \* ptr;
1140offset:24; size:8; signed:0; field:size_t bytes_req; offset:32; size:8;
1141signed:0; field:size_t bytes_alloc; offset:40; size:8; signed:0;
1142field:gfp_t gfp_flags; offset:48; size:4; signed:0; print fmt:
1143"call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s",
1144REC->call_site, REC->ptr, REC->bytes_req, REC->bytes_alloc,
1145(REC->gfp_flags) ? \__print_flags(REC->gfp_flags, "|", {(unsigned
1146long)(((( gfp_t)0x10u) \| (( gfp_t)0x40u) \| (( gfp_t)0x80u) \| ((
1147gfp_t)0x20000u) \| (( gfp_t)0x02u) \| (( gfp_t)0x08u)) \| ((
1148gfp_t)0x4000u) \| (( gfp_t)0x10000u) \| (( gfp_t)0x1000u) \| ((
1149gfp_t)0x200u) \| (( gfp_t)0x400000u)), "GFP_TRANSHUGE"}, {(unsigned
1150long)((( gfp_t)0x10u) \| (( gfp_t)0x40u) \| (( gfp_t)0x80u) \| ((
1151gfp_t)0x20000u) \| (( gfp_t)0x02u) \| (( gfp_t)0x08u)),
1152"GFP_HIGHUSER_MOVABLE"}, {(unsigned long)((( gfp_t)0x10u) \| ((
1153gfp_t)0x40u) \| (( gfp_t)0x80u) \| (( gfp_t)0x20000u) \| ((
1154gfp_t)0x02u)), "GFP_HIGHUSER"}, {(unsigned long)((( gfp_t)0x10u) \| ((
1155gfp_t)0x40u) \| (( gfp_t)0x80u) \| (( gfp_t)0x20000u)), "GFP_USER"},
1156{(unsigned long)((( gfp_t)0x10u) \| (( gfp_t)0x40u) \| (( gfp_t)0x80u)
1157\| (( gfp_t)0x80000u)), GFP_TEMPORARY"}, {(unsigned long)(((
1158gfp_t)0x10u) \| (( gfp_t)0x40u) \| (( gfp_t)0x80u)), "GFP_KERNEL"},
1159{(unsigned long)((( gfp_t)0x10u) \| (( gfp_t)0x40u)), "GFP_NOFS"},
1160{(unsigned long)((( gfp_t)0x20u)), "GFP_ATOMIC"}, {(unsigned long)(((
1161gfp_t)0x10u)), "GFP_NOIO"}, {(unsigned long)(( gfp_t)0x20u),
1162"GFP_HIGH"}, {(unsigned long)(( gfp_t)0x10u), "GFP_WAIT"}, {(unsigned
1163long)(( gfp_t)0x40u), "GFP_IO"}, {(unsigned long)(( gfp_t)0x100u),
1164"GFP_COLD"}, {(unsigned long)(( gfp_t)0x200u), "GFP_NOWARN"}, {(unsigned
1165long)(( gfp_t)0x400u), "GFP_REPEAT"}, {(unsigned long)(( gfp_t)0x800u),
1166"GFP_NOFAIL"}, {(unsigned long)(( gfp_t)0x1000u), "GFP_NORETRY"},
1167{(unsigned long)(( gfp_t)0x4000u), "GFP_COMP"}, {(unsigned long)((
1168gfp_t)0x8000u), "GFP_ZERO"}, {(unsigned long)(( gfp_t)0x10000u),
1169"GFP_NOMEMALLOC"}, {(unsigned long)(( gfp_t)0x20000u), "GFP_HARDWALL"},
1170{(unsigned long)(( gfp_t)0x40000u), "GFP_THISNODE"}, {(unsigned long)((
1171gfp_t)0x80000u), "GFP_RECLAIMABLE"}, {(unsigned long)(( gfp_t)0x08u),
1172"GFP_MOVABLE"}, {(unsigned long)(( gfp_t)0), "GFP_NOTRACK"}, {(unsigned
1173long)(( gfp_t)0x400000u), "GFP_NO_KSWAPD"}, {(unsigned long)((
1174gfp_t)0x800000u), "GFP_OTHER_NODE"} ) : "GFP_NOWAIT" The 'enable' file
1175in the tracepoint directory is what allows the user (or tools such as
1176trace-cmd) to actually turn the tracepoint on and off. When enabled, the
1177corresponding tracepoint will start appearing in the ftrace 'trace' file
1178described previously. For example, this turns on the kmalloc tracepoint:
1179root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 1 >
1180enable At the moment, we're not interested in the function tracer or
1181some other tracer that might be in effect, so we first turn it off, but
1182if we do that, we still need to turn tracing on in order to see the
1183events in the output buffer: root@sugarbay:/sys/kernel/debug/tracing#
1184echo nop > current_tracer root@sugarbay:/sys/kernel/debug/tracing# echo
11851 > tracing_on Now, if we look at the the 'trace' file, we see nothing
1186but the kmalloc events we just turned on:
1187root@sugarbay:/sys/kernel/debug/tracing# cat trace \| less # tracer: nop
1188# # entries-in-buffer/entries-written: 1897/1897 #P:8 # # \_-----=>
1189irqs-off # / \_----=> need-resched # \| / \_---=> hardirq/softirq # \|\|
1190/ \_--=> preempt-depth # \||\| / delay # TASK-PID CPU# \|||\| TIMESTAMP
1191FUNCTION # \| \| \| \|||\| \| \| dropbear-1465 [000] ...1 18154.620753:
1192kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048
1193bytes_alloc=2048 gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3 18154.621640:
1194kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1195bytes_alloc=512 gfp_flags=GFP_ATOMIC <idle>-0 [000] ..s3 18154.621656:
1196kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1197bytes_alloc=512 gfp_flags=GFP_ATOMIC matchbox-termin-1361 [001] ...1
119818154.755472: kmalloc: call_site=ffffffff81614050 ptr=ffff88006d5f0e00
1199bytes_req=512 bytes_alloc=512 gfp_flags=GFP_KERNEL|GFP_REPEAT Xorg-1264
1200[002] ...1 18154.755581: kmalloc: call_site=ffffffff8141abe8
1201ptr=ffff8800734f4cc0 bytes_req=168 bytes_alloc=192
1202gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY Xorg-1264 [002] ...1
120318154.755583: kmalloc: call_site=ffffffff814192a3 ptr=ffff88001f822520
1204bytes_req=24 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO Xorg-1264
1205[002] ...1 18154.755589: kmalloc: call_site=ffffffff81419edb
1206ptr=ffff8800721a2f00 bytes_req=64 bytes_alloc=64
1207gfp_flags=GFP_KERNEL|GFP_ZERO matchbox-termin-1361 [001] ...1
120818155.354594: kmalloc: call_site=ffffffff81614050 ptr=ffff88006db35400
1209bytes_req=576 bytes_alloc=1024 gfp_flags=GFP_KERNEL|GFP_REPEAT Xorg-1264
1210[002] ...1 18155.354703: kmalloc: call_site=ffffffff8141abe8
1211ptr=ffff8800734f4cc0 bytes_req=168 bytes_alloc=192
1212gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY Xorg-1264 [002] ...1
121318155.354705: kmalloc: call_site=ffffffff814192a3 ptr=ffff88001f822520
1214bytes_req=24 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO Xorg-1264
1215[002] ...1 18155.354711: kmalloc: call_site=ffffffff81419edb
1216ptr=ffff8800721a2f00 bytes_req=64 bytes_alloc=64
1217gfp_flags=GFP_KERNEL|GFP_ZERO <idle>-0 [000] ..s3 18155.673319: kmalloc:
1218call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1219bytes_alloc=512 gfp_flags=GFP_ATOMIC dropbear-1465 [000] ...1
122018155.673525: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000
1221bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3
122218155.674821: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800
1223bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC <idle>-0 [000] ..s3
122418155.793014: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800
1225bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC dropbear-1465 [000]
1226...1 18155.793219: kmalloc: call_site=ffffffff816650d4
1227ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048
1228gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3 18155.794147: kmalloc:
1229call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1230bytes_alloc=512 gfp_flags=GFP_ATOMIC <idle>-0 [000] ..s3 18155.936705:
1231kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1232bytes_alloc=512 gfp_flags=GFP_ATOMIC dropbear-1465 [000] ...1
123318155.936910: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000
1234bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3
123518155.937869: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800
1236bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC matchbox-termin-1361
1237[001] ...1 18155.953667: kmalloc: call_site=ffffffff81614050
1238ptr=ffff88006d5f2000 bytes_req=512 bytes_alloc=512
1239gfp_flags=GFP_KERNEL|GFP_REPEAT Xorg-1264 [002] ...1 18155.953775:
1240kmalloc: call_site=ffffffff8141abe8 ptr=ffff8800734f4cc0 bytes_req=168
1241bytes_alloc=192 gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY Xorg-1264
1242[002] ...1 18155.953777: kmalloc: call_site=ffffffff814192a3
1243ptr=ffff88001f822520 bytes_req=24 bytes_alloc=32
1244gfp_flags=GFP_KERNEL|GFP_ZERO Xorg-1264 [002] ...1 18155.953783:
1245kmalloc: call_site=ffffffff81419edb ptr=ffff8800721a2f00 bytes_req=64
1246bytes_alloc=64 gfp_flags=GFP_KERNEL|GFP_ZERO <idle>-0 [000] ..s3
124718156.176053: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800
1248bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC dropbear-1465 [000]
1249...1 18156.176257: kmalloc: call_site=ffffffff816650d4
1250ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048
1251gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3 18156.177717: kmalloc:
1252call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1253bytes_alloc=512 gfp_flags=GFP_ATOMIC <idle>-0 [000] ..s3 18156.399229:
1254kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512
1255bytes_alloc=512 gfp_flags=GFP_ATOMIC dropbear-1465 [000] ...1
125618156.399434: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000
1257bytes_http://rostedt.homelinux.com/kernelshark/req=2048 bytes_alloc=2048
1258gfp_flags=GFP_KERNEL <idle>-0 [000] ..s3 18156.400660: kmalloc:
1259call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512
1260bytes_alloc=512 gfp_flags=GFP_ATOMIC matchbox-termin-1361 [001] ...1
126118156.552800: kmalloc: call_site=ffffffff81614050 ptr=ffff88006db34800
1262bytes_req=576 bytes_alloc=1024 gfp_flags=GFP_KERNEL|GFP_REPEAT To again
1263disable the kmalloc event, we need to send 0 to the enable file:
1264root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 0 >
1265enable You can enable any number of events or complete subsystems (by
1266using the 'enable' file in the subsystem directory) and get an
1267arbitrarily fine-grained idea of what's going on in the system by
1268enabling as many of the appropriate tracepoints as applicable.
1269
1270A number of the tools described in this HOWTO do just that, including
1271trace-cmd and kernelshark in the next section.
1272
1273.. container:: informalexample
1274
1275 Tying it Together:
1276 These tracepoints and their representation are used not only by
1277 ftrace, but by many of the other tools covered in this document and
1278 they form a central point of integration for the various tracers
1279 available in Linux. They form a central part of the instrumentation
1280 for the following tools: perf, lttng, ftrace, blktrace and SystemTap
1281
1282.. container:: informalexample
1283
1284 Tying it Together:
1285 Eventually all the special-purpose tracers currently available in
1286 /sys/kernel/debug/tracing will be removed and replaced with
1287 equivalent tracers based on the 'trace events' subsystem.
1288
1289.. _trace-cmd-kernelshark:
1290
1291trace-cmd/kernelshark
1292---------------------
1293
1294trace-cmd is essentially an extensive command-line 'wrapper' interface
1295that hides the details of all the individual files in
1296/sys/kernel/debug/tracing, allowing users to specify specific particular
1297events within the /sys/kernel/debug/tracing/events/ subdirectory and to
1298collect traces and avoid having to deal with those details directly.
1299
1300As yet another layer on top of that, kernelshark provides a GUI that
1301allows users to start and stop traces and specify sets of events using
1302an intuitive interface, and view the output as both trace events and as
1303a per-CPU graphical display. It directly uses 'trace-cmd' as the
1304plumbing that accomplishes all that underneath the covers (and actually
1305displays the trace-cmd command it uses, as we'll see).
1306
1307To start a trace using kernelshark, first start kernelshark:
1308root@sugarbay:~# kernelshark Then bring up the 'Capture' dialog by
1309choosing from the kernelshark menu: Capture \| Record That will display
1310the following dialog, which allows you to choose one or more events (or
1311even one or more complete subsystems) to trace:
1312
1313Note that these are exactly the same sets of events described in the
1314previous trace events subsystem section, and in fact is where trace-cmd
1315gets them for kernelshark.
1316
1317In the above screenshot, we've decided to explore the graphics subsystem
1318a bit and so have chosen to trace all the tracepoints contained within
1319the 'i915' and 'drm' subsystems.
1320
1321After doing that, we can start and stop the trace using the 'Run' and
1322'Stop' button on the lower right corner of the dialog (the same button
1323will turn into the 'Stop' button after the trace has started):
1324
1325Notice that the right-hand pane shows the exact trace-cmd command-line
1326that's used to run the trace, along with the results of the trace-cmd
1327run.
1328
1329Once the 'Stop' button is pressed, the graphical view magically fills up
1330with a colorful per-cpu display of the trace data, along with the
1331detailed event listing below that:
1332
1333Here's another example, this time a display resulting from tracing 'all
1334events':
1335
1336The tool is pretty self-explanatory, but for more detailed information
1337on navigating through the data, see the `kernelshark
1338website <http://rostedt.homelinux.com/kernelshark/>`__.
1339
1340.. _ftrace-documentation:
1341
1342Documentation
1343-------------
1344
1345The documentation for ftrace can be found in the kernel Documentation
1346directory: Documentation/trace/ftrace.txt The documentation for the
1347trace event subsystem can also be found in the kernel Documentation
1348directory: Documentation/trace/events.txt There is a nice series of
1349articles on using ftrace and trace-cmd at LWN:
1350
1351- `Debugging the kernel using Ftrace - part
1352 1 <http://lwn.net/Articles/365835/>`__
1353
1354- `Debugging the kernel using Ftrace - part
1355 2 <http://lwn.net/Articles/366796/>`__
1356
1357- `Secrets of the Ftrace function
1358 tracer <http://lwn.net/Articles/370423/>`__
1359
1360- `trace-cmd: A front-end for
1361 Ftrace <https://lwn.net/Articles/410200/>`__
1362
1363There's more detailed documentation kernelshark usage here:
1364`KernelShark <http://rostedt.homelinux.com/kernelshark/>`__
1365
1366An amusing yet useful README (a tracing mini-HOWTO) can be found in
1367/sys/kernel/debug/tracing/README.
1368
1369.. _profile-manual-systemtap:
1370
1371systemtap
1372=========
1373
1374SystemTap is a system-wide script-based tracing and profiling tool.
1375
1376SystemTap scripts are C-like programs that are executed in the kernel to
1377gather/print/aggregate data extracted from the context they end up being
1378invoked under.
1379
1380For example, this probe from the `SystemTap
1381tutorial <http://sourceware.org/systemtap/tutorial/>`__ simply prints a
1382line every time any process on the system open()s a file. For each line,
1383it prints the executable name of the program that opened the file, along
1384with its PID, and the name of the file it opened (or tried to open),
1385which it extracts from the open syscall's argstr. probe syscall.open {
1386printf ("%s(%d) open (%s)\n", execname(), pid(), argstr) } probe
1387timer.ms(4000) # after 4 seconds { exit () } Normally, to execute this
1388probe, you'd simply install systemtap on the system you want to probe,
1389and directly run the probe on that system e.g. assuming the name of the
1390file containing the above text is trace_open.stp: # stap trace_open.stp
1391What systemtap does under the covers to run this probe is 1) parse and
1392convert the probe to an equivalent 'C' form, 2) compile the 'C' form
1393into a kernel module, 3) insert the module into the kernel, which arms
1394it, and 4) collect the data generated by the probe and display it to the
1395user.
1396
1397In order to accomplish steps 1 and 2, the 'stap' program needs access to
1398the kernel build system that produced the kernel that the probed system
1399is running. In the case of a typical embedded system (the 'target'), the
1400kernel build system unfortunately isn't typically part of the image
1401running on the target. It is normally available on the 'host' system
1402that produced the target image however; in such cases, steps 1 and 2 are
1403executed on the host system, and steps 3 and 4 are executed on the
1404target system, using only the systemtap 'runtime'.
1405
1406The systemtap support in Yocto assumes that only steps 3 and 4 are run
1407on the target; it is possible to do everything on the target, but this
1408section assumes only the typical embedded use-case.
1409
1410So basically what you need to do in order to run a systemtap script on
1411the target is to 1) on the host system, compile the probe into a kernel
1412module that makes sense to the target, 2) copy the module onto the
1413target system and 3) insert the module into the target kernel, which
1414arms it, and 4) collect the data generated by the probe and display it
1415to the user.
1416
1417.. _systemtap-setup:
1418
1419Setup
1420-----
1421
1422Those are a lot of steps and a lot of details, but fortunately Yocto
1423includes a script called 'crosstap' that will take care of those
1424details, allowing you to simply execute a systemtap script on the remote
1425target, with arguments if necessary.
1426
1427In order to do this from a remote host, however, you need to have access
1428to the build for the image you booted. The 'crosstap' script provides
1429details on how to do this if you run the script on the host without
1430having done a build:
1431
1432.. note::
1433
1434 SystemTap, which uses 'crosstap', assumes you can establish an ssh
1435 connection to the remote target. Please refer to the crosstap wiki
1436 page for details on verifying ssh connections at
1437 . Also, the ability to ssh into the target system is not enabled by
1438 default in \*-minimal images.
1439
1440$ crosstap root@192.168.1.88 trace_open.stp Error: No target kernel
1441build found. Did you forget to create a local build of your image?
1442'crosstap' requires a local sdk build of the target system (or a build
1443that includes 'tools-profile') in order to build kernel modules that can
1444probe the target system. Practically speaking, that means you need to do
1445the following: - If you're running a pre-built image, download the
1446release and/or BSP tarballs used to build the image. - If you're working
1447from git sources, just clone the metadata and BSP layers needed to build
1448the image you'll be booting. - Make sure you're properly set up to build
1449a new image (see the BSP README and/or the widely available basic
1450documentation that discusses how to build images). - Build an -sdk
1451version of the image e.g.: $ bitbake core-image-sato-sdk OR - Build a
1452non-sdk image but include the profiling tools: [ edit local.conf and add
1453'tools-profile' to the end of the EXTRA_IMAGE_FEATURES variable ] $
1454bitbake core-image-sato Once you've build the image on the host system,
1455you're ready to boot it (or the equivalent pre-built image) and use
1456'crosstap' to probe it (you need to source the environment as usual
1457first): $ source oe-init-build-env $ cd ~/my/systemtap/scripts $
1458crosstap root@192.168.1.xxx myscript.stp So essentially what you need to
1459do is build an SDK image or image with 'tools-profile' as detailed in
1460the "`General Setup <#profile-manual-general-setup>`__" section of this
1461manual, and boot the resulting target image.
1462
1463.. note::
1464
1465 If you have a build directory containing multiple machines, you need
1466 to have the MACHINE you're connecting to selected in local.conf, and
1467 the kernel in that machine's build directory must match the kernel on
1468 the booted system exactly, or you'll get the above 'crosstap' message
1469 when you try to invoke a script.
1470
1471Running a Script on a Target
1472----------------------------
1473
1474Once you've done that, you should be able to run a systemtap script on
1475the target: $ cd /path/to/yocto $ source oe-init-build-env ### Shell
1476environment set up for builds. ### You can now run 'bitbake <target>'
1477Common targets are: core-image-minimal core-image-sato meta-toolchain
1478meta-ide-support You can also run generated qemu images with a command
1479like 'runqemu qemux86-64' Once you've done that, you can cd to whatever
1480directory contains your scripts and use 'crosstap' to run the script: $
1481cd /path/to/my/systemap/script $ crosstap root@192.168.7.2
1482trace_open.stp If you get an error connecting to the target e.g.: $
1483crosstap root@192.168.7.2 trace_open.stp error establishing ssh
1484connection on remote 'root@192.168.7.2' Try ssh'ing to the target and
1485see what happens: $ ssh root@192.168.7.2 A lot of the time, connection
1486problems are due specifying a wrong IP address or having a 'host key
1487verification error'.
1488
1489If everything worked as planned, you should see something like this
1490(enter the password when prompted, or press enter if it's set up to use
1491no password): $ crosstap root@192.168.7.2 trace_open.stp
1492root@192.168.7.2's password: matchbox-termin(1036) open
1493("/tmp/vte3FS2LW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
1494matchbox-termin(1036) open ("/tmp/vteJMC7LW",
1495O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
1496
1497.. _systemtap-documentation:
1498
1499Documentation
1500-------------
1501
1502The SystemTap language reference can be found here: `SystemTap Language
1503Reference <http://sourceware.org/systemtap/langref/>`__
1504
1505Links to other SystemTap documents, tutorials, and examples can be found
1506here: `SystemTap documentation
1507page <http://sourceware.org/systemtap/documentation.html>`__
1508
1509.. _profile-manual-sysprof:
1510
1511Sysprof
1512=======
1513
1514Sysprof is a very easy to use system-wide profiler that consists of a
1515single window with three panes and a few buttons which allow you to
1516start, stop, and view the profile from one place.
1517
1518.. _sysprof-setup:
1519
1520Setup
1521-----
1522
1523For this section, we'll assume you've already performed the basic setup
1524outlined in the General Setup section.
1525
1526Sysprof is a GUI-based application that runs on the target system. For
1527the rest of this document we assume you've ssh'ed to the host and will
1528be running Sysprof on the target (you can use the '-X' option to ssh and
1529have the Sysprof GUI run on the target but display remotely on the host
1530if you want).
1531
1532.. _sysprof-basic-usage:
1533
1534Basic Usage
1535-----------
1536
1537To start profiling the system, you simply press the 'Start' button. To
1538stop profiling and to start viewing the profile data in one easy step,
1539press the 'Profile' button.
1540
1541Once you've pressed the profile button, the three panes will fill up
1542with profiling data:
1543
1544The left pane shows a list of functions and processes. Selecting one of
1545those expands that function in the right pane, showing all its callees.
1546Note that this caller-oriented display is essentially the inverse of
1547perf's default callee-oriented callchain display.
1548
1549In the screenshot above, we're focusing on \__copy_to_user_ll() and
1550looking up the callchain we can see that one of the callers of
1551\__copy_to_user_ll is sys_read() and the complete callpath between them.
1552Notice that this is essentially a portion of the same information we saw
1553in the perf display shown in the perf section of this page.
1554
1555Similarly, the above is a snapshot of the Sysprof display of a
1556copy-from-user callchain.
1557
1558Finally, looking at the third Sysprof pane in the lower left, we can see
1559a list of all the callers of a particular function selected in the top
1560left pane. In this case, the lower pane is showing all the callers of
1561\__mark_inode_dirty:
1562
1563Double-clicking on one of those functions will in turn change the focus
1564to the selected function, and so on.
1565
1566.. container:: informalexample
1567
1568 Tying it Together:
1569 If you like sysprof's 'caller-oriented' display, you may be able to
1570 approximate it in other tools as well. For example, 'perf report' has
1571 the -g (--call-graph) option that you can experiment with; one of the
1572 options is 'caller' for an inverted caller-based callgraph display.
1573
1574.. _sysprof-documentation:
1575
1576Documentation
1577-------------
1578
1579There doesn't seem to be any documentation for Sysprof, but maybe that's
1580because it's pretty self-explanatory. The Sysprof website, however, is
1581here: `Sysprof, System-wide Performance Profiler for
1582Linux <http://sysprof.com/>`__
1583
1584LTTng (Linux Trace Toolkit, next generation)
1585============================================
1586
1587.. _lttng-setup:
1588
1589Setup
1590-----
1591
1592For this section, we'll assume you've already performed the basic setup
1593outlined in the General Setup section. LTTng is run on the target system
1594by ssh'ing to it.
1595
1596Collecting and Viewing Traces
1597-----------------------------
1598
1599Once you've applied the above commits and built and booted your image
1600(you need to build the core-image-sato-sdk image or use one of the other
1601methods described in the General Setup section), you're ready to start
1602tracing.
1603
1604Collecting and viewing a trace on the target (inside a shell)
1605~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1606
1607First, from the host, ssh to the target: $ ssh -l root 192.168.1.47 The
1608authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
1609RSA key fingerprint is 23:bd:c8:b1:a8:71:52:00:ee:00:4f:64:9e:10:b9:7e.
1610Are you sure you want to continue connecting (yes/no)? yes Warning:
1611Permanently added '192.168.1.47' (RSA) to the list of known hosts.
1612root@192.168.1.47's password: Once on the target, use these steps to
1613create a trace: root@crownbay:~# lttng create Spawning a session daemon
1614Session auto-20121015-232120 created. Traces will be written in
1615/home/root/lttng-traces/auto-20121015-232120 Enable the events you want
1616to trace (in this case all kernel events): root@crownbay:~# lttng
1617enable-event --kernel --all All kernel events are enabled in channel
1618channel0 Start the trace: root@crownbay:~# lttng start Tracing started
1619for session auto-20121015-232120 And then stop the trace after awhile or
1620after running a particular workload that you want to trace:
1621root@crownbay:~# lttng stop Tracing stopped for session
1622auto-20121015-232120 You can now view the trace in text form on the
1623target: root@crownbay:~# lttng view [23:21:56.989270399] (+?.?????????)
1624sys_geteuid: { 1 }, { } [23:21:56.989278081] (+0.000007682)
1625exit_syscall: { 1 }, { ret = 0 } [23:21:56.989286043] (+0.000007962)
1626sys_pipe: { 1 }, { fildes = 0xB77B9E8C } [23:21:56.989321802]
1627(+0.000035759) exit_syscall: { 1 }, { ret = 0 } [23:21:56.989329345]
1628(+0.000007543) sys_mmap_pgoff: { 1 }, { addr = 0x0, len = 10485760, prot
1629= 3, flags = 131362, fd = 4294967295, pgoff = 0 } [23:21:56.989351694]
1630(+0.000022349) exit_syscall: { 1 }, { ret = -1247805440 }
1631[23:21:56.989432989] (+0.000081295) sys_clone: { 1 }, { clone_flags =
16320x411, newsp = 0xB5EFFFE4, parent_tid = 0xFFFFFFFF, child_tid = 0x0 }
1633[23:21:56.989477129] (+0.000044140) sched_stat_runtime: { 1 }, { comm =
1634"lttng-consumerd", tid = 1193, runtime = 681660, vruntime = 43367983388
1635} [23:21:56.989486697] (+0.000009568) sched_migrate_task: { 1 }, { comm
1636= "lttng-consumerd", tid = 1193, prio = 20, orig_cpu = 1, dest_cpu = 1 }
1637[23:21:56.989508418] (+0.000021721) hrtimer_init: { 1 }, { hrtimer =
16383970832076, clockid = 1, mode = 1 } [23:21:56.989770462] (+0.000262044)
1639hrtimer_cancel: { 1 }, { hrtimer = 3993865440 } [23:21:56.989771580]
1640(+0.000001118) hrtimer_cancel: { 0 }, { hrtimer = 3993812192 }
1641[23:21:56.989776957] (+0.000005377) hrtimer_expire_entry: { 1 }, {
1642hrtimer = 3993865440, now = 79815980007057, function = 3238465232 }
1643[23:21:56.989778145] (+0.000001188) hrtimer_expire_entry: { 0 }, {
1644hrtimer = 3993812192, now = 79815980008174, function = 3238465232 }
1645[23:21:56.989791695] (+0.000013550) softirq_raise: { 1 }, { vec = 1 }
1646[23:21:56.989795396] (+0.000003701) softirq_raise: { 0 }, { vec = 1 }
1647[23:21:56.989800635] (+0.000005239) softirq_raise: { 0 }, { vec = 9 }
1648[23:21:56.989807130] (+0.000006495) sched_stat_runtime: { 1 }, { comm =
1649"lttng-consumerd", tid = 1193, runtime = 330710, vruntime = 43368314098
1650} [23:21:56.989809993] (+0.000002863) sched_stat_runtime: { 0 }, { comm
1651= "lttng-sessiond", tid = 1181, runtime = 1015313, vruntime =
165236976733240 } [23:21:56.989818514] (+0.000008521) hrtimer_expire_exit: {
16530 }, { hrtimer = 3993812192 } [23:21:56.989819631] (+0.000001117)
1654hrtimer_expire_exit: { 1 }, { hrtimer = 3993865440 }
1655[23:21:56.989821866] (+0.000002235) hrtimer_start: { 0 }, { hrtimer =
16563993812192, function = 3238465232, expires = 79815981000000, softexpires
1657= 79815981000000 } [23:21:56.989822984] (+0.000001118) hrtimer_start: {
16581 }, { hrtimer = 3993865440, function = 3238465232, expires =
165979815981000000, softexpires = 79815981000000 } [23:21:56.989832762]
1660(+0.000009778) softirq_entry: { 1 }, { vec = 1 } [23:21:56.989833879]
1661(+0.000001117) softirq_entry: { 0 }, { vec = 1 } [23:21:56.989838069]
1662(+0.000004190) timer_cancel: { 1 }, { timer = 3993871956 }
1663[23:21:56.989839187] (+0.000001118) timer_cancel: { 0 }, { timer =
16643993818708 } [23:21:56.989841492] (+0.000002305) timer_expire_entry: { 1
1665}, { timer = 3993871956, now = 79515980, function = 3238277552 }
1666[23:21:56.989842819] (+0.000001327) timer_expire_entry: { 0 }, { timer =
16673993818708, now = 79515980, function = 3238277552 } [23:21:56.989854831]
1668(+0.000012012) sched_stat_runtime: { 1 }, { comm = "lttng-consumerd",
1669tid = 1193, runtime = 49237, vruntime = 43368363335 }
1670[23:21:56.989855949] (+0.000001118) sched_stat_runtime: { 0 }, { comm =
1671"lttng-sessiond", tid = 1181, runtime = 45121, vruntime = 36976778361 }
1672[23:21:56.989861257] (+0.000005308) sched_stat_sleep: { 1 }, { comm =
1673"kworker/1:1", tid = 21, delay = 9451318 } [23:21:56.989862374]
1674(+0.000001117) sched_stat_sleep: { 0 }, { comm = "kworker/0:0", tid = 4,
1675delay = 9958820 } [23:21:56.989868241] (+0.000005867) sched_wakeup: { 0
1676}, { comm = "kworker/0:0", tid = 4, prio = 120, success = 1, target_cpu
1677= 0 } [23:21:56.989869358] (+0.000001117) sched_wakeup: { 1 }, { comm =
1678"kworker/1:1", tid = 21, prio = 120, success = 1, target_cpu = 1 }
1679[23:21:56.989877460] (+0.000008102) timer_expire_exit: { 1 }, { timer =
16803993871956 } [23:21:56.989878577] (+0.000001117) timer_expire_exit: { 0
1681}, { timer = 3993818708 } . . . You can now safely destroy the trace
1682session (note that this doesn't delete the trace - it's still there in
1683~/lttng-traces): root@crownbay:~# lttng destroy Session
1684auto-20121015-232120 destroyed at /home/root Note that the trace is
1685saved in a directory of the same name as returned by 'lttng create',
1686under the ~/lttng-traces directory (note that you can change this by
1687supplying your own name to 'lttng create'): root@crownbay:~# ls -al
1688~/lttng-traces drwxrwx--- 3 root root 1024 Oct 15 23:21 . drwxr-xr-x 5
1689root root 1024 Oct 15 23:57 .. drwxrwx--- 3 root root 1024 Oct 15 23:21
1690auto-20121015-232120
1691
1692Collecting and viewing a userspace trace on the target (inside a shell)
1693~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1694
1695For LTTng userspace tracing, you need to have a properly instrumented
1696userspace program. For this example, we'll use the 'hello' test program
1697generated by the lttng-ust build.
1698
1699The 'hello' test program isn't installed on the rootfs by the lttng-ust
1700build, so we need to copy it over manually. First cd into the build
1701directory that contains the hello executable: $ cd
1702build/tmp/work/core2_32-poky-linux/lttng-ust/2.0.5-r0/git/tests/hello/.libs
1703Copy that over to the target machine: $ scp hello root@192.168.1.20: You
1704now have the instrumented lttng 'hello world' test program on the
1705target, ready to test.
1706
1707First, from the host, ssh to the target: $ ssh -l root 192.168.1.47 The
1708authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
1709RSA key fingerprint is 23:bd:c8:b1:a8:71:52:00:ee:00:4f:64:9e:10:b9:7e.
1710Are you sure you want to continue connecting (yes/no)? yes Warning:
1711Permanently added '192.168.1.47' (RSA) to the list of known hosts.
1712root@192.168.1.47's password: Once on the target, use these steps to
1713create a trace: root@crownbay:~# lttng create Session
1714auto-20190303-021943 created. Traces will be written in
1715/home/root/lttng-traces/auto-20190303-021943 Enable the events you want
1716to trace (in this case all userspace events): root@crownbay:~# lttng
1717enable-event --userspace --all All UST events are enabled in channel
1718channel0 Start the trace: root@crownbay:~# lttng start Tracing started
1719for session auto-20190303-021943 Run the instrumented hello world
1720program: root@crownbay:~# ./hello Hello, World! Tracing... done. And
1721then stop the trace after awhile or after running a particular workload
1722that you want to trace: root@crownbay:~# lttng stop Tracing stopped for
1723session auto-20190303-021943 You can now view the trace in text form on
1724the target: root@crownbay:~# lttng view [02:31:14.906146544]
1725(+?.?????????) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, {
1726intfield = 0, intfield2 = 0x0, longfield = 0, netintfield = 0,
1727netintfieldhex = 0x0, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ],
1728arrfield2 = "test", \_seqfield1_length = 4, seqfield1 = [ [0] = 116, [1]
1729= 101, [2] = 115, [3] = 116 ], \_seqfield2_length = 4, seqfield2 =
1730"test", stringfield = "test", floatfield = 2222, doublefield = 2,
1731boolfield = 1 } [02:31:14.906170360] (+0.000023816) hello:1424
1732ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 1, intfield2 = 0x1,
1733longfield = 1, netintfield = 1, netintfieldhex = 0x1, arrfield1 = [ [0]
1734= 1, [1] = 2, [2] = 3 ], arrfield2 = "test", \_seqfield1_length = 4,
1735seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ],
1736\_seqfield2_length = 4, seqfield2 = "test", stringfield = "test",
1737floatfield = 2222, doublefield = 2, boolfield = 1 } [02:31:14.906183140]
1738(+0.000012780) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, {
1739intfield = 2, intfield2 = 0x2, longfield = 2, netintfield = 2,
1740netintfieldhex = 0x2, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ],
1741arrfield2 = "test", \_seqfield1_length = 4, seqfield1 = [ [0] = 116, [1]
1742= 101, [2] = 115, [3] = 116 ], \_seqfield2_length = 4, seqfield2 =
1743"test", stringfield = "test", floatfield = 2222, doublefield = 2,
1744boolfield = 1 } [02:31:14.906194385] (+0.000011245) hello:1424
1745ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 3, intfield2 = 0x3,
1746longfield = 3, netintfield = 3, netintfieldhex = 0x3, arrfield1 = [ [0]
1747= 1, [1] = 2, [2] = 3 ], arrfield2 = "test", \_seqfield1_length = 4,
1748seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ],
1749\_seqfield2_length = 4, seqfield2 = "test", stringfield = "test",
1750floatfield = 2222, doublefield = 2, boolfield = 1 } . . . You can now
1751safely destroy the trace session (note that this doesn't delete the
1752trace - it's still there in ~/lttng-traces): root@crownbay:~# lttng
1753destroy Session auto-20190303-021943 destroyed at /home/root
1754
1755.. _lltng-documentation:
1756
1757Documentation
1758-------------
1759
1760You can find the primary LTTng Documentation on the `LTTng
1761Documentation <https://lttng.org/docs/>`__ site. The documentation on
1762this site is appropriate for intermediate to advanced software
1763developers who are working in a Linux environment and are interested in
1764efficient software tracing.
1765
1766For information on LTTng in general, visit the `LTTng
1767Project <http://lttng.org/lttng2.0>`__ site. You can find a "Getting
1768Started" link on this site that takes you to an LTTng Quick Start.
1769
1770.. _profile-manual-blktrace:
1771
1772blktrace
1773========
1774
1775blktrace is a tool for tracing and reporting low-level disk I/O.
1776blktrace provides the tracing half of the equation; its output can be
1777piped into the blkparse program, which renders the data in a
1778human-readable form and does some basic analysis:
1779
1780.. _blktrace-setup:
1781
1782Setup
1783-----
1784
1785For this section, we'll assume you've already performed the basic setup
1786outlined in the "`General Setup <#profile-manual-general-setup>`__"
1787section.
1788
1789blktrace is an application that runs on the target system. You can run
1790the entire blktrace and blkparse pipeline on the target, or you can run
1791blktrace in 'listen' mode on the target and have blktrace and blkparse
1792collect and analyze the data on the host (see the "`Using blktrace
1793Remotely <#using-blktrace-remotely>`__" section below). For the rest of
1794this section we assume you've ssh'ed to the host and will be running
1795blkrace on the target.
1796
1797.. _blktrace-basic-usage:
1798
1799Basic Usage
1800-----------
1801
1802To record a trace, simply run the 'blktrace' command, giving it the name
1803of the block device you want to trace activity on: root@crownbay:~#
1804blktrace /dev/sdc In another shell, execute a workload you want to
1805trace. root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget
1806http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2;
1807sync Connecting to downloads.yoctoproject.org (140.211.169.59:80)
1808linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k
18090:00:00 ETA Press Ctrl-C in the blktrace shell to stop the trace. It
1810will display how many events were logged, along with the per-cpu file
1811sizes (blktrace records traces in per-cpu kernel buffers and simply
1812dumps them to userspace for blkparse to merge and sort later). ^C=== sdc
1813=== CPU 0: 7082 events, 332 KiB data CPU 1: 1578 events, 74 KiB data
1814Total: 8660 events (dropped 0), 406 KiB data If you examine the files
1815saved to disk, you see multiple files, one per CPU and with the device
1816name as the first part of the filename: root@crownbay:~# ls -al
1817drwxr-xr-x 6 root root 1024 Oct 27 22:39 . drwxr-sr-x 4 root root 1024
1818Oct 26 18:24 .. -rw-r--r-- 1 root root 339938 Oct 27 22:40
1819sdc.blktrace.0 -rw-r--r-- 1 root root 75753 Oct 27 22:40 sdc.blktrace.1
1820To view the trace events, simply invoke 'blkparse' in the directory
1821containing the trace files, giving it the device name that forms the
1822first part of the filenames: root@crownbay:~# blkparse sdc 8,32 1 1
18230.000000000 1225 Q WS 3417048 + 8 [jbd2/sdc-8] 8,32 1 2 0.000025213 1225
1824G WS 3417048 + 8 [jbd2/sdc-8] 8,32 1 3 0.000033384 1225 P N [jbd2/sdc-8]
18258,32 1 4 0.000043301 1225 I WS 3417048 + 8 [jbd2/sdc-8] 8,32 1 0
18260.000057270 0 m N cfq1225 insert_request 8,32 1 0 0.000064813 0 m N
1827cfq1225 add_to_rr 8,32 1 5 0.000076336 1225 U N [jbd2/sdc-8] 1 8,32 1 0
18280.000088559 0 m N cfq workload slice:150 8,32 1 0 0.000097359 0 m N
1829cfq1225 set_active wl_prio:0 wl_type:1 8,32 1 0 0.000104063 0 m N
1830cfq1225 Not idling. st->count:1 8,32 1 0 0.000112584 0 m N cfq1225 fifo=
1831(null) 8,32 1 0 0.000118730 0 m N cfq1225 dispatch_insert 8,32 1 0
18320.000127390 0 m N cfq1225 dispatched a request 8,32 1 0 0.000133536 0 m
1833N cfq1225 activate rq, drv=1 8,32 1 6 0.000136889 1225 D WS 3417048 + 8
1834[jbd2/sdc-8] 8,32 1 7 0.000360381 1225 Q WS 3417056 + 8 [jbd2/sdc-8]
18358,32 1 8 0.000377422 1225 G WS 3417056 + 8 [jbd2/sdc-8] 8,32 1 9
18360.000388876 1225 P N [jbd2/sdc-8] 8,32 1 10 0.000397886 1225 Q WS
18373417064 + 8 [jbd2/sdc-8] 8,32 1 11 0.000404800 1225 M WS 3417064 + 8
1838[jbd2/sdc-8] 8,32 1 12 0.000412343 1225 Q WS 3417072 + 8 [jbd2/sdc-8]
18398,32 1 13 0.000416533 1225 M WS 3417072 + 8 [jbd2/sdc-8] 8,32 1 14
18400.000422121 1225 Q WS 3417080 + 8 [jbd2/sdc-8] 8,32 1 15 0.000425194
18411225 M WS 3417080 + 8 [jbd2/sdc-8] 8,32 1 16 0.000431968 1225 Q WS
18423417088 + 8 [jbd2/sdc-8] 8,32 1 17 0.000435251 1225 M WS 3417088 + 8
1843[jbd2/sdc-8] 8,32 1 18 0.000440279 1225 Q WS 3417096 + 8 [jbd2/sdc-8]
18448,32 1 19 0.000443911 1225 M WS 3417096 + 8 [jbd2/sdc-8] 8,32 1 20
18450.000450336 1225 Q WS 3417104 + 8 [jbd2/sdc-8] 8,32 1 21 0.000454038
18461225 M WS 3417104 + 8 [jbd2/sdc-8] 8,32 1 22 0.000462070 1225 Q WS
18473417112 + 8 [jbd2/sdc-8] 8,32 1 23 0.000465422 1225 M WS 3417112 + 8
1848[jbd2/sdc-8] 8,32 1 24 0.000474222 1225 I WS 3417056 + 64 [jbd2/sdc-8]
18498,32 1 0 0.000483022 0 m N cfq1225 insert_request 8,32 1 25 0.000489727
18501225 U N [jbd2/sdc-8] 1 8,32 1 0 0.000498457 0 m N cfq1225 Not idling.
1851st->count:1 8,32 1 0 0.000503765 0 m N cfq1225 dispatch_insert 8,32 1 0
18520.000512914 0 m N cfq1225 dispatched a request 8,32 1 0 0.000518851 0 m
1853N cfq1225 activate rq, drv=2 . . . 8,32 0 0 58.515006138 0 m N cfq3551
1854complete rqnoidle 1 8,32 0 2024 58.516603269 3 C WS 3156992 + 16 [0]
18558,32 0 0 58.516626736 0 m N cfq3551 complete rqnoidle 1 8,32 0 0
185658.516634558 0 m N cfq3551 arm_idle: 8 group_idle: 0 8,32 0 0
185758.516636933 0 m N cfq schedule dispatch 8,32 1 0 58.516971613 0 m N
1858cfq3551 slice expired t=0 8,32 1 0 58.516982089 0 m N cfq3551 sl_used=13
1859disp=6 charge=13 iops=0 sect=80 8,32 1 0 58.516985511 0 m N cfq3551
1860del_from_rr 8,32 1 0 58.516990819 0 m N cfq3551 put_queue CPU0 (sdc):
1861Reads Queued: 0, 0KiB Writes Queued: 331, 26,284KiB Read Dispatches: 0,
18620KiB Write Dispatches: 485, 40,484KiB Reads Requeued: 0 Writes Requeued:
18630 Reads Completed: 0, 0KiB Writes Completed: 511, 41,000KiB Read Merges:
18640, 0KiB Write Merges: 13, 160KiB Read depth: 0 Write depth: 2 IO
1865unplugs: 23 Timer unplugs: 0 CPU1 (sdc): Reads Queued: 0, 0KiB Writes
1866Queued: 249, 15,800KiB Read Dispatches: 0, 0KiB Write Dispatches: 42,
18671,600KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB
1868Writes Completed: 16, 1,084KiB Read Merges: 0, 0KiB Write Merges: 40,
1869276KiB Read depth: 0 Write depth: 2 IO unplugs: 30 Timer unplugs: 1
1870Total (sdc): Reads Queued: 0, 0KiB Writes Queued: 580, 42,084KiB Read
1871Dispatches: 0, 0KiB Write Dispatches: 527, 42,084KiB Reads Requeued: 0
1872Writes Requeued: 0 Reads Completed: 0, 0KiB Writes Completed: 527,
187342,084KiB Read Merges: 0, 0KiB Write Merges: 53, 436KiB IO unplugs: 53
1874Timer unplugs: 1 Throughput (R/W): 0KiB/s / 719KiB/s Events (sdc): 6,592
1875entries Skips: 0 forward (0 - 0.0%) Input file sdc.blktrace.0 added
1876Input file sdc.blktrace.1 added The report shows each event that was
1877found in the blktrace data, along with a summary of the overall block
1878I/O traffic during the run. You can look at the
1879`blkparse <http://linux.die.net/man/1/blkparse>`__ manpage to learn the
1880meaning of each field displayed in the trace listing.
1881
1882.. _blktrace-live-mode:
1883
1884Live Mode
1885~~~~~~~~~
1886
1887blktrace and blkparse are designed from the ground up to be able to
1888operate together in a 'pipe mode' where the stdout of blktrace can be
1889fed directly into the stdin of blkparse: root@crownbay:~# blktrace
1890/dev/sdc -o - \| blkparse -i - This enables long-lived tracing sessions
1891to run without writing anything to disk, and allows the user to look for
1892certain conditions in the trace data in 'real-time' by viewing the trace
1893output as it scrolls by on the screen or by passing it along to yet
1894another program in the pipeline such as grep which can be used to
1895identify and capture conditions of interest.
1896
1897There's actually another blktrace command that implements the above
1898pipeline as a single command, so the user doesn't have to bother typing
1899in the above command sequence: root@crownbay:~# btrace /dev/sdc
1900
1901Using blktrace Remotely
1902~~~~~~~~~~~~~~~~~~~~~~~
1903
1904Because blktrace traces block I/O and at the same time normally writes
1905its trace data to a block device, and in general because it's not really
1906a great idea to make the device being traced the same as the device the
1907tracer writes to, blktrace provides a way to trace without perturbing
1908the traced device at all by providing native support for sending all
1909trace data over the network.
1910
1911To have blktrace operate in this mode, start blktrace on the target
1912system being traced with the -l option, along with the device to trace:
1913root@crownbay:~# blktrace -l /dev/sdc server: waiting for connections...
1914On the host system, use the -h option to connect to the target system,
1915also passing it the device to trace: $ blktrace -d /dev/sdc -h
1916192.168.1.43 blktrace: connecting to 192.168.1.43 blktrace: connected!
1917On the target system, you should see this: server: connection from
1918192.168.1.43 In another shell, execute a workload you want to trace.
1919root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget
1920http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2;
1921sync Connecting to downloads.yoctoproject.org (140.211.169.59:80)
1922linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k
19230:00:00 ETA When it's done, do a Ctrl-C on the host system to stop the
1924trace: ^C=== sdc === CPU 0: 7691 events, 361 KiB data CPU 1: 4109
1925events, 193 KiB data Total: 11800 events (dropped 0), 554 KiB data On
1926the target system, you should also see a trace summary for the trace
1927just ended: server: end of run for 192.168.1.43:sdc === sdc === CPU 0:
19287691 events, 361 KiB data CPU 1: 4109 events, 193 KiB data Total: 11800
1929events (dropped 0), 554 KiB data The blktrace instance on the host will
1930save the target output inside a hostname-timestamp directory: $ ls -al
1931drwxr-xr-x 10 root root 1024 Oct 28 02:40 . drwxr-sr-x 4 root root 1024
1932Oct 26 18:24 .. drwxr-xr-x 2 root root 1024 Oct 28 02:40
1933192.168.1.43-2012-10-28-02:40:56 cd into that directory to see the
1934output files: $ ls -l -rw-r--r-- 1 root root 369193 Oct 28 02:44
1935sdc.blktrace.0 -rw-r--r-- 1 root root 197278 Oct 28 02:44 sdc.blktrace.1
1936And run blkparse on the host system using the device name: $ blkparse
1937sdc 8,32 1 1 0.000000000 1263 Q RM 6016 + 8 [ls] 8,32 1 0 0.000036038 0
1938m N cfq1263 alloced 8,32 1 2 0.000039390 1263 G RM 6016 + 8 [ls] 8,32 1
19393 0.000049168 1263 I RM 6016 + 8 [ls] 8,32 1 0 0.000056152 0 m N cfq1263
1940insert_request 8,32 1 0 0.000061600 0 m N cfq1263 add_to_rr 8,32 1 0
19410.000075498 0 m N cfq workload slice:300 . . . 8,32 0 0 177.266385696 0
1942m N cfq1267 arm_idle: 8 group_idle: 0 8,32 0 0 177.266388140 0 m N cfq
1943schedule dispatch 8,32 1 0 177.266679239 0 m N cfq1267 slice expired t=0
19448,32 1 0 177.266689297 0 m N cfq1267 sl_used=9 disp=6 charge=9 iops=0
1945sect=56 8,32 1 0 177.266692649 0 m N cfq1267 del_from_rr 8,32 1 0
1946177.266696560 0 m N cfq1267 put_queue CPU0 (sdc): Reads Queued: 0, 0KiB
1947Writes Queued: 270, 21,708KiB Read Dispatches: 59, 2,628KiB Write
1948Dispatches: 495, 39,964KiB Reads Requeued: 0 Writes Requeued: 0 Reads
1949Completed: 90, 2,752KiB Writes Completed: 543, 41,596KiB Read Merges: 0,
19500KiB Write Merges: 9, 344KiB Read depth: 2 Write depth: 2 IO unplugs: 20
1951Timer unplugs: 1 CPU1 (sdc): Reads Queued: 688, 2,752KiB Writes Queued:
1952381, 20,652KiB Read Dispatches: 31, 124KiB Write Dispatches: 59,
19532,396KiB Reads Requeued: 0 Writes Requeued: 0 Reads Completed: 0, 0KiB
1954Writes Completed: 11, 764KiB Read Merges: 598, 2,392KiB Write Merges:
195588, 448KiB Read depth: 2 Write depth: 2 IO unplugs: 52 Timer unplugs: 0
1956Total (sdc): Reads Queued: 688, 2,752KiB Writes Queued: 651, 42,360KiB
1957Read Dispatches: 90, 2,752KiB Write Dispatches: 554, 42,360KiB Reads
1958Requeued: 0 Writes Requeued: 0 Reads Completed: 90, 2,752KiB Writes
1959Completed: 554, 42,360KiB Read Merges: 598, 2,392KiB Write Merges: 97,
1960792KiB IO unplugs: 72 Timer unplugs: 1 Throughput (R/W): 15KiB/s /
1961238KiB/s Events (sdc): 9,301 entries Skips: 0 forward (0 - 0.0%) You
1962should see the trace events and summary just as you would have if you'd
1963run the same command on the target.
1964
1965Tracing Block I/O via 'ftrace'
1966~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1967
1968It's also possible to trace block I/O using only `trace events
1969subsystem <#the-trace-events-subsystem>`__, which can be useful for
1970casual tracing if you don't want to bother dealing with the userspace
1971tools.
1972
1973To enable tracing for a given device, use /sys/block/xxx/trace/enable,
1974where xxx is the device name. This for example enables tracing for
1975/dev/sdc: root@crownbay:/sys/kernel/debug/tracing# echo 1 >
1976/sys/block/sdc/trace/enable Once you've selected the device(s) you want
1977to trace, selecting the 'blk' tracer will turn the blk tracer on:
1978root@crownbay:/sys/kernel/debug/tracing# cat available_tracers blk
1979function_graph function nop root@crownbay:/sys/kernel/debug/tracing#
1980echo blk > current_tracer Execute the workload you're interested in:
1981root@crownbay:/sys/kernel/debug/tracing# cat /media/sdc/testfile.txt And
1982look at the output (note here that we're using 'trace_pipe' instead of
1983trace to capture this trace - this allows us to wait around on the pipe
1984for data to appear): root@crownbay:/sys/kernel/debug/tracing# cat
1985trace_pipe cat-3587 [001] d..1 3023.276361: 8,32 Q R 1699848 + 8 [cat]
1986cat-3587 [001] d..1 3023.276410: 8,32 m N cfq3587 alloced cat-3587 [001]
1987d..1 3023.276415: 8,32 G R 1699848 + 8 [cat] cat-3587 [001] d..1
19883023.276424: 8,32 P N [cat] cat-3587 [001] d..2 3023.276432: 8,32 I R
19891699848 + 8 [cat] cat-3587 [001] d..1 3023.276439: 8,32 m N cfq3587
1990insert_request cat-3587 [001] d..1 3023.276445: 8,32 m N cfq3587
1991add_to_rr cat-3587 [001] d..2 3023.276454: 8,32 U N [cat] 1 cat-3587
1992[001] d..1 3023.276464: 8,32 m N cfq workload slice:150 cat-3587 [001]
1993d..1 3023.276471: 8,32 m N cfq3587 set_active wl_prio:0 wl_type:2
1994cat-3587 [001] d..1 3023.276478: 8,32 m N cfq3587 fifo= (null) cat-3587
1995[001] d..1 3023.276483: 8,32 m N cfq3587 dispatch_insert cat-3587 [001]
1996d..1 3023.276490: 8,32 m N cfq3587 dispatched a request cat-3587 [001]
1997d..1 3023.276497: 8,32 m N cfq3587 activate rq, drv=1 cat-3587 [001]
1998d..2 3023.276500: 8,32 D R 1699848 + 8 [cat] And this turns off tracing
1999for the specified device: root@crownbay:/sys/kernel/debug/tracing# echo
20000 > /sys/block/sdc/trace/enable
2001
2002.. _blktrace-documentation:
2003
2004Documentation
2005-------------
2006
2007Online versions of the man pages for the commands discussed in this
2008section can be found here:
2009
2010- http://linux.die.net/man/8/blktrace
2011
2012- http://linux.die.net/man/1/blkparse
2013
2014- http://linux.die.net/man/8/btrace
2015
2016The above manpages, along with manpages for the other blktrace utilities
2017(btt, blkiomon, etc) can be found in the /doc directory of the blktrace
2018tools git repo: $ git clone git://git.kernel.dk/blktrace.git
diff --git a/documentation/profile-manual/profile-manual.rst b/documentation/profile-manual/profile-manual.rst
new file mode 100644
index 0000000000..b4361d39dc
--- /dev/null
+++ b/documentation/profile-manual/profile-manual.rst
@@ -0,0 +1,12 @@
1==========================================
2Yocto Project Profiling and Tracing Manual
3==========================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 profile-manual-intro
10 profile-manual-arch
11 profile-manual-usage
12 profile-manual-examples
diff --git a/documentation/ref-manual/faq.rst b/documentation/ref-manual/faq.rst
new file mode 100644
index 0000000000..6d26537980
--- /dev/null
+++ b/documentation/ref-manual/faq.rst
@@ -0,0 +1,418 @@
1***
2FAQ
3***
4
5**Q:** How does Poky differ from `OpenEmbedded <&OE_HOME_URL;>`__?
6
7**A:** The term "`Poky <#>`__" refers to the specific reference build
8system that the Yocto Project provides. Poky is based on
9`OE-Core <#oe-core>`__ and `BitBake <#bitbake-term>`__. Thus, the
10generic term used here for the build system is the "OpenEmbedded build
11system." Development in the Yocto Project using Poky is closely tied to
12OpenEmbedded, with changes always being merged to OE-Core or BitBake
13first before being pulled back into Poky. This practice benefits both
14projects immediately.
15
16**Q:** My development system does not meet the required Git, tar, and
17Python versions. In particular, I do not have Python 3.5.0 or greater.
18Can I still use the Yocto Project?
19
20**A:** You can get the required tools on your host development system a
21couple different ways (i.e. building a tarball or downloading a
22tarball). See the "`Required Git, tar, Python and gcc
23Versions <#required-git-tar-python-and-gcc-versions>`__" section for
24steps on how to update your build tools.
25
26**Q:** How can you claim Poky / OpenEmbedded-Core is stable?
27
28**A:** There are three areas that help with stability;
29
30- The Yocto Project team keeps `OE-Core <#oe-core>`__ small and
31 focused, containing around 830 recipes as opposed to the thousands
32 available in other OpenEmbedded community layers. Keeping it small
33 makes it easy to test and maintain.
34
35- The Yocto Project team runs manual and automated tests using a small,
36 fixed set of reference hardware as well as emulated targets.
37
38- The Yocto Project uses an autobuilder, which provides continuous
39 build and integration tests.
40
41**Q:** How do I get support for my board added to the Yocto Project?
42
43**A:** Support for an additional board is added by creating a Board
44Support Package (BSP) layer for it. For more information on how to
45create a BSP layer, see the "`Understanding and Creating
46Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
47section in the Yocto Project Development Tasks Manual and the `Yocto
48Project Board Support Package (BSP) Developer's
49Guide <&YOCTO_DOCS_BSP_URL;>`__.
50
51Usually, if the board is not completely exotic, adding support in the
52Yocto Project is fairly straightforward.
53
54**Q:** Are there any products built using the OpenEmbedded build system?
55
56**A:** The software running on the `Vernier
57LabQuest <http://vernier.com/labquest/>`__ is built using the
58OpenEmbedded build system. See the `Vernier
59LabQuest <http://www.vernier.com/products/interfaces/labq/>`__ website
60for more information. There are a number of pre-production devices using
61the OpenEmbedded build system and the Yocto Project team announces them
62as soon as they are released.
63
64**Q:** What does the OpenEmbedded build system produce as output?
65
66**A:** Because you can use the same set of recipes to create output of
67various formats, the output of an OpenEmbedded build depends on how you
68start it. Usually, the output is a flashable image ready for the target
69device.
70
71**Q:** How do I add my package to the Yocto Project?
72
73**A:** To add a package, you need to create a BitBake recipe. For
74information on how to create a BitBake recipe, see the "`Writing a New
75Recipe <&YOCTO_DOCS_DEV_URL;#new-recipe-writing-a-new-recipe>`__"
76section in the Yocto Project Development Tasks Manual.
77
78**Q:** Do I have to reflash my entire board with a new Yocto Project
79image when recompiling a package?
80
81**A:** The OpenEmbedded build system can build packages in various
82formats such as IPK for OPKG, Debian package (``.deb``), or RPM. You can
83then upgrade the packages using the package tools on the device, much
84like on a desktop distribution such as Ubuntu or Fedora. However,
85package management on the target is entirely optional.
86
87**Q:** I see the error
88'``chmod: XXXXX new permissions are r-xrwxrwx, not r-xr-xr-x``'. What is
89wrong?
90
91**A:** You are probably running the build on an NTFS filesystem. Use
92``ext2``, ``ext3``, or ``ext4`` instead.
93
94**Q:** I see lots of 404 responses for files when the OpenEmbedded build
95system is trying to download sources. Is something wrong?
96
97**A:** Nothing is wrong. The OpenEmbedded build system checks any
98configured source mirrors before downloading from the upstream sources.
99The build system does this searching for both source archives and
100pre-checked out versions of SCM-managed software. These checks help in
101large installations because it can reduce load on the SCM servers
102themselves. The address above is one of the default mirrors configured
103into the build system. Consequently, if an upstream source disappears,
104the team can place sources there so builds continue to work.
105
106**Q:** I have machine-specific data in a package for one machine only
107but the package is being marked as machine-specific in all cases, how do
108I prevent this?
109
110**A:** Set ``SRC_URI_OVERRIDES_PACKAGE_ARCH`` = "0" in the ``.bb`` file
111but make sure the package is manually marked as machine-specific for the
112case that needs it. The code that handles
113``SRC_URI_OVERRIDES_PACKAGE_ARCH`` is in the
114``meta/classes/base.bbclass`` file.
115
116**Q:** I'm behind a firewall and need to use a proxy server. How do I do
117that?
118
119**A:** Most source fetching by the OpenEmbedded build system is done by
120``wget`` and you therefore need to specify the proxy settings in a
121``.wgetrc`` file, which can be in your home directory if you are a
122single user or can be in ``/usr/local/etc/wgetrc`` as a global user
123file.
124
125Following is the applicable code for setting various proxy types in the
126``.wgetrc`` file. By default, these settings are disabled with comments.
127To use them, remove the comments: # You can set the default proxies for
128Wget to use for http, https, and ftp. # They will override the value in
129the environment. #https_proxy = http://proxy.yoyodyne.com:18023/
130#http_proxy = http://proxy.yoyodyne.com:18023/ #ftp_proxy =
131http://proxy.yoyodyne.com:18023/ # If you do not want to use proxy at
132all, set this to off. #use_proxy = on The Yocto Project also includes a
133``meta-poky/conf/site.conf.sample`` file that shows how to configure CVS
134and Git proxy servers if needed. For more information on setting up
135various proxy types and configuring proxy servers, see the "`Working
136Behind a Network
137Proxy <&YOCTO_WIKI_URL;/wiki/Working_Behind_a_Network_Proxy>`__" Wiki
138page.
139
140**Q:** What’s the difference between target and target\ ``-native``?
141
142**A:** The ``*-native`` targets are designed to run on the system being
143used for the build. These are usually tools that are needed to assist
144the build in some way such as ``quilt-native``, which is used to apply
145patches. The non-native version is the one that runs on the target
146device.
147
148**Q:** I'm seeing random build failures. Help?!
149
150**A:** If the same build is failing in totally different and random
151ways, the most likely explanation is:
152
153- The hardware you are running the build on has some problem.
154
155- You are running the build under virtualization, in which case the
156 virtualization probably has bugs.
157
158The OpenEmbedded build system processes a massive amount of data that
159causes lots of network, disk and CPU activity and is sensitive to even
160single-bit failures in any of these areas. True random failures have
161always been traced back to hardware or virtualization issues.
162
163**Q:** When I try to build a native recipe, the build fails with
164``iconv.h`` problems.
165
166**A:** If you get an error message that indicates GNU ``libiconv`` is
167not in use but ``iconv.h`` has been included from ``libiconv``, you need
168to check to see if you have a previously installed version of the header
169file in ``/usr/local/include``. #error GNU libiconv not in use but
170included iconv.h is from libiconv If you find a previously installed
171file, you should either uninstall it or temporarily rename it and try
172the build again.
173
174This issue is just a single manifestation of "system leakage" issues
175caused when the OpenEmbedded build system finds and uses previously
176installed files during a native build. This type of issue might not be
177limited to ``iconv.h``. Be sure that leakage cannot occur from
178``/usr/local/include`` and ``/opt`` locations.
179
180**Q:** What do we need to ship for license compliance?
181
182**A:** This is a difficult question and you need to consult your lawyer
183for the answer for your specific case. It is worth bearing in mind that
184for GPL compliance, there needs to be enough information shipped to
185allow someone else to rebuild and produce the same end result you are
186shipping. This means sharing the source code, any patches applied to it,
187and also any configuration information about how that package was
188configured and built.
189
190You can find more information on licensing in the
191"`Licensing <&YOCTO_DOCS_OM_URL;#licensing>`__" section in the Yocto
192Project Overview and Concepts Manual and also in the "`Maintaining Open
193Source License Compliance During Your Product's
194Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
195section in the Yocto Project Development Tasks Manual.
196
197**Q:** How do I disable the cursor on my touchscreen device?
198
199**A:** You need to create a form factor file as described in the
200"`Miscellaneous BSP-Specific Recipe
201Files <&YOCTO_DOCS_BSP_URL;#bsp-filelayout-misc-recipes>`__" section in
202the Yocto Project Board Support Packages (BSP) Developer's Guide. Set
203the ``HAVE_TOUCHSCREEN`` variable equal to one as follows:
204HAVE_TOUCHSCREEN=1
205
206**Q:** How do I make sure connected network interfaces are brought up by
207default?
208
209**A:** The default interfaces file provided by the netbase recipe does
210not automatically bring up network interfaces. Therefore, you will need
211to add a BSP-specific netbase that includes an interfaces file. See the
212"`Miscellaneous BSP-Specific Recipe
213Files <&YOCTO_DOCS_BSP_URL;#bsp-filelayout-misc-recipes>`__" section in
214the Yocto Project Board Support Packages (BSP) Developer's Guide for
215information on creating these types of miscellaneous recipe files.
216
217For example, add the following files to your layer:
218meta-MACHINE/recipes-bsp/netbase/netbase/MACHINE/interfaces
219meta-MACHINE/recipes-bsp/netbase/netbase_5.0.bbappend
220
221**Q:** How do I create images with more free space?
222
223**A:** By default, the OpenEmbedded build system creates images that are
2241.3 times the size of the populated root filesystem. To affect the image
225size, you need to set various configurations:
226
227- *Image Size:* The OpenEmbedded build system uses the
228 ```IMAGE_ROOTFS_SIZE`` <#var-IMAGE_ROOTFS_SIZE>`__ variable to define
229 the size of the image in Kbytes. The build system determines the size
230 by taking into account the initial root filesystem size before any
231 modifications such as requested size for the image and any requested
232 additional free disk space to be added to the image.
233
234- *Overhead:* Use the
235 ```IMAGE_OVERHEAD_FACTOR`` <#var-IMAGE_OVERHEAD_FACTOR>`__ variable
236 to define the multiplier that the build system applies to the initial
237 image size, which is 1.3 by default.
238
239- *Additional Free Space:* Use the
240 ```IMAGE_ROOTFS_EXTRA_SPACE`` <#var-IMAGE_ROOTFS_EXTRA_SPACE>`__
241 variable to add additional free space to the image. The build system
242 adds this space to the image after it determines its
243 ``IMAGE_ROOTFS_SIZE``.
244
245**Q:** Why don't you support directories with spaces in the pathnames?
246
247**A:** The Yocto Project team has tried to do this before but too many
248of the tools the OpenEmbedded build system depends on, such as
249``autoconf``, break when they find spaces in pathnames. Until that
250situation changes, the team will not support spaces in pathnames.
251
252**Q:** How do I use an external toolchain?
253
254**A:** The toolchain configuration is very flexible and customizable. It
255is primarily controlled with the ``TCMODE`` variable. This variable
256controls which ``tcmode-*.inc`` file to include from the
257``meta/conf/distro/include`` directory within the `Source
258Directory <#source-directory>`__.
259
260The default value of ``TCMODE`` is "default", which tells the
261OpenEmbedded build system to use its internally built toolchain (i.e.
262``tcmode-default.inc``). However, other patterns are accepted. In
263particular, "external-*" refers to external toolchains. One example is
264the Sourcery G++ Toolchain. The support for this toolchain resides in
265the separate ``meta-sourcery`` layer at
266` <http://github.com/MentorEmbedded/meta-sourcery/>`__.
267
268In addition to the toolchain configuration, you also need a
269corresponding toolchain recipe file. This recipe file needs to package
270up any pre-built objects in the toolchain such as ``libgcc``,
271``libstdcc++``, any locales, and ``libc``.
272
273**Q:** How does the OpenEmbedded build system obtain source code and
274will it work behind my firewall or proxy server?
275
276**A:** The way the build system obtains source code is highly
277configurable. You can setup the build system to get source code in most
278environments if HTTP transport is available.
279
280When the build system searches for source code, it first tries the local
281download directory. If that location fails, Poky tries
282```PREMIRRORS`` <#var-PREMIRRORS>`__, the upstream source, and then
283```MIRRORS`` <#var-MIRRORS>`__ in that order.
284
285Assuming your distribution is "poky", the OpenEmbedded build system uses
286the Yocto Project source ``PREMIRRORS`` by default for SCM-based
287sources, upstreams for normal tarballs, and then falls back to a number
288of other mirrors including the Yocto Project source mirror if those
289fail.
290
291As an example, you could add a specific server for the build system to
292attempt before any others by adding something like the following to the
293``local.conf`` configuration file: PREMIRRORS_prepend = "\\ git://.*/.\*
294http://www.yoctoproject.org/sources/ \\n \\ ftp://.*/.\*
295http://www.yoctoproject.org/sources/ \\n \\ http://.*/.\*
296http://www.yoctoproject.org/sources/ \\n \\ https://.*/.\*
297http://www.yoctoproject.org/sources/ \\n"
298
299These changes cause the build system to intercept Git, FTP, HTTP, and
300HTTPS requests and direct them to the ``http://`` sources mirror. You
301can use ``file://`` URLs to point to local directories or network shares
302as well.
303
304Aside from the previous technique, these options also exist:
305BB_NO_NETWORK = "1" This statement tells BitBake to issue an error
306instead of trying to access the Internet. This technique is useful if
307you want to ensure code builds only from local sources.
308
309Here is another technique: BB_FETCH_PREMIRRORONLY = "1" This statement
310limits the build system to pulling source from the ``PREMIRRORS`` only.
311Again, this technique is useful for reproducing builds.
312
313Here is another technique: BB_GENERATE_MIRROR_TARBALLS = "1" This
314statement tells the build system to generate mirror tarballs. This
315technique is useful if you want to create a mirror server. If not,
316however, the technique can simply waste time during the build.
317
318Finally, consider an example where you are behind an HTTP-only firewall.
319You could make the following changes to the ``local.conf`` configuration
320file as long as the ``PREMIRRORS`` server is current: PREMIRRORS_prepend
321= "\\ ftp://.*/.\* http://www.yoctoproject.org/sources/ \\n \\
322http://.*/.\* http://www.yoctoproject.org/sources/ \\n \\ https://.*/.\*
323http://www.yoctoproject.org/sources/ \\n" BB_FETCH_PREMIRRORONLY = "1"
324These changes would cause the build system to successfully fetch source
325over HTTP and any network accesses to anything other than the
326``PREMIRRORS`` would fail.
327
328The build system also honors the standard shell environment variables
329``http_proxy``, ``ftp_proxy``, ``https_proxy``, and ``all_proxy`` to
330redirect requests through proxy servers.
331
332.. note::
333
334 You can find more information on the "
335 Working Behind a Network Proxy
336 " Wiki page.
337
338**Q:** Can I get rid of build output so I can start over?
339
340**A:** Yes - you can easily do this. When you use BitBake to build an
341image, all the build output goes into the directory created when you run
342the build environment setup script (i.e.
343````` <#structure-core-script>`__). By default, this `Build
344Directory <#build-directory>`__ is named ``build`` but can be named
345anything you want.
346
347Within the Build Directory, is the ``tmp`` directory. To remove all the
348build output yet preserve any source code or downloaded files from
349previous builds, simply remove the ``tmp`` directory.
350
351**Q:** Why do ``${bindir}`` and ``${libdir}`` have strange values for
352``-native`` recipes?
353
354**A:** Executables and libraries might need to be used from a directory
355other than the directory into which they were initially installed.
356Complicating this situation is the fact that sometimes these executables
357and libraries are compiled with the expectation of being run from that
358initial installation target directory. If this is the case, moving them
359causes problems.
360
361This scenario is a fundamental problem for package maintainers of
362mainstream Linux distributions as well as for the OpenEmbedded build
363system. As such, a well-established solution exists. Makefiles,
364Autotools configuration scripts, and other build systems are expected to
365respect environment variables such as ``bindir``, ``libdir``, and
366``sysconfdir`` that indicate where executables, libraries, and data
367reside when a program is actually run. They are also expected to respect
368a ``DESTDIR`` environment variable, which is prepended to all the other
369variables when the build system actually installs the files. It is
370understood that the program does not actually run from within
371``DESTDIR``.
372
373When the OpenEmbedded build system uses a recipe to build a
374target-architecture program (i.e. one that is intended for inclusion on
375the image being built), that program eventually runs from the root file
376system of that image. Thus, the build system provides a value of
377"/usr/bin" for ``bindir``, a value of "/usr/lib" for ``libdir``, and so
378forth.
379
380Meanwhile, ``DESTDIR`` is a path within the `Build
381Directory <#build-directory>`__. However, when the recipe builds a
382native program (i.e. one that is intended to run on the build machine),
383that program is never installed directly to the build machine's root
384file system. Consequently, the build system uses paths within the Build
385Directory for ``DESTDIR``, ``bindir`` and related variables. To better
386understand this, consider the following two paths where the first is
387relatively normal and the second is not:
388
389.. note::
390
391 Due to these lengthy examples, the paths are artificially broken
392 across lines for readability.
393
394/home/maxtothemax/poky-bootchart2/build/tmp/work/i586-poky-linux/zlib/
3951.2.8-r0/sysroot-destdir/usr/bin
396/home/maxtothemax/poky-bootchart2/build/tmp/work/x86_64-linux/
397zlib-native/1.2.8-r0/sysroot-destdir/home/maxtothemax/poky-bootchart2/
398build/tmp/sysroots/x86_64-linux/usr/bin Even if the paths look unusual,
399they both are correct - the first for a target and the second for a
400native recipe. These paths are a consequence of the ``DESTDIR``
401mechanism and while they appear strange, they are correct and in
402practice very effective.
403
404**Q:** The files provided by my ``*-native`` recipe do not appear to be
405available to other recipes. Files are missing from the native sysroot,
406my recipe is installing to the wrong place, or I am getting permissions
407errors during the do_install task in my recipe! What is wrong?
408
409**A:** This situation results when a build system does not recognize the
410environment variables supplied to it by `BitBake <#bitbake-term>`__. The
411incident that prompted this FAQ entry involved a Makefile that used an
412environment variable named ``BINDIR`` instead of the more standard
413variable ``bindir``. The makefile's hardcoded default value of
414"/usr/bin" worked most of the time, but not for the recipe's ``-native``
415variant. For another example, permissions errors might be caused by a
416Makefile that ignores ``DESTDIR`` or uses a different name for that
417environment variable. Check the the build system to see if these kinds
418of issues exist.
diff --git a/documentation/ref-manual/migration.rst b/documentation/ref-manual/migration.rst
new file mode 100644
index 0000000000..6ddfa93833
--- /dev/null
+++ b/documentation/ref-manual/migration.rst
@@ -0,0 +1,5081 @@
1******************************************
2Migrating to a Newer Yocto Project Release
3******************************************
4
5This chapter provides information you can use to migrate work to a newer
6Yocto Project release. You can find the same information in the release
7notes for a given release.
8
9General Migration Considerations
10================================
11
12Some considerations are not tied to a specific Yocto Project release.
13This section presents information you should consider when migrating to
14any new Yocto Project release.
15
16- *Dealing with Customized Recipes*: Issues could arise if you take
17 older recipes that contain customizations and simply copy them
18 forward expecting them to work after you migrate to new Yocto Project
19 metadata. For example, suppose you have a recipe in your layer that
20 is a customized version of a core recipe copied from the earlier
21 release, rather than through the use of an append file. When you
22 migrate to a newer version of Yocto Project, the metadata (e.g.
23 perhaps an include file used by the recipe) could have changed in a
24 way that would break the build. Say, for example, a function is
25 removed from an include file and the customized recipe tries to call
26 that function.
27
28 You could "forward-port" all your customizations in your recipe so
29 that everything works for the new release. However, this is not the
30 optimal solution as you would have to repeat this process with each
31 new release if changes occur that give rise to problems.
32
33 The better solution (where practical) is to use append files
34 (``*.bbappend``) to capture any customizations you want to make to a
35 recipe. Doing so, isolates your changes from the main recipe making
36 them much more manageable. However, sometimes it is not practical to
37 use an append file. A good example of this is when introducing a
38 newer or older version of a recipe in another layer.
39
40- *Updating Append Files*: Since append files generally only contain
41 your customizations, they often do not need to be adjusted for new
42 releases. However, if the ``.bbappend`` file is specific to a
43 particular version of the recipe (i.e. its name does not use the %
44 wildcard) and the version of the recipe to which it is appending has
45 changed, then you will at a minimum need to rename the append file to
46 match the name of the recipe file. A mismatch between an append file
47 and its corresponding recipe file (``.bb``) will trigger an error
48 during parsing.
49
50 Depending on the type of customization the append file applies, other
51 incompatibilities might occur when you upgrade. For example, if your
52 append file applies a patch and the recipe to which it is appending
53 is updated to a newer version, the patch might no longer apply. If
54 this is the case and assuming the patch is still needed, you must
55 modify the patch file so that it does apply.
56
57Moving to the Yocto Project 1.3 Release
58=======================================
59
60This section provides migration information for moving to the Yocto
61Project 1.3 Release from the prior release.
62
63.. _1.3-local-configuration:
64
65Local Configuration
66-------------------
67
68Differences include changes for
69```SSTATE_MIRRORS`` <#var-SSTATE_MIRRORS>`__ and ``bblayers.conf``.
70
71.. _migration-1.3-sstate-mirrors:
72
73SSTATE_MIRRORS
74~~~~~~~~~~~~~~
75
76The shared state cache (sstate-cache), as pointed to by
77```SSTATE_DIR`` <#var-SSTATE_DIR>`__, by default now has two-character
78subdirectories to prevent issues arising from too many files in the same
79directory. Also, native sstate-cache packages, which are built to run on
80the host system, will go into a subdirectory named using the distro ID
81string. If you copy the newly structured sstate-cache to a mirror
82location (either local or remote) and then point to it in
83```SSTATE_MIRRORS`` <#var-SSTATE_MIRRORS>`__, you need to append "PATH"
84to the end of the mirror URL so that the path used by BitBake before the
85mirror substitution is appended to the path used to access the mirror.
86Here is an example: SSTATE_MIRRORS = "file://.\*
87http://someserver.tld/share/sstate/PATH"
88
89.. _migration-1.3-bblayers-conf:
90
91bblayers.conf
92~~~~~~~~~~~~~
93
94The ``meta-yocto`` layer consists of two parts that correspond to the
95Poky reference distribution and the reference hardware Board Support
96Packages (BSPs), respectively: ``meta-yocto`` and ``meta-yocto-bsp``.
97When running BitBake for the first time after upgrading, your
98``conf/bblayers.conf`` file will be updated to handle this change and
99you will be asked to re-run or restart for the changes to take effect.
100
101.. _1.3-recipes:
102
103Recipes
104-------
105
106Differences include changes for the following:
107
108- Python function whitespace
109
110- ``proto=`` in ``SRC_URI``
111
112- ``nativesdk``
113
114- Task recipes
115
116- ``IMAGE_FEATURES``
117
118- Removed recipes
119
120.. _migration-1.3-python-function-whitespace:
121
122Python Function Whitespace
123~~~~~~~~~~~~~~~~~~~~~~~~~~
124
125All Python functions must now use four spaces for indentation.
126Previously, an inconsistent mix of spaces and tabs existed, which made
127extending these functions using ``_append`` or ``_prepend`` complicated
128given that Python treats whitespace as syntactically significant. If you
129are defining or extending any Python functions (e.g.
130``populate_packages``, ``do_unpack``, ``do_patch`` and so forth) in
131custom recipes or classes, you need to ensure you are using consistent
132four-space indentation.
133
134.. _migration-1.3-proto=-in-src-uri:
135
136proto= in SRC_URI
137~~~~~~~~~~~~~~~~~
138
139Any use of ``proto=`` in ```SRC_URI`` <#var-SRC_URI>`__ needs to be
140changed to ``protocol=``. In particular, this applies to the following
141URIs:
142
143- ``svn://``
144
145- ``bzr://``
146
147- ``hg://``
148
149- ``osc://``
150
151Other URIs were already using ``protocol=``. This change improves
152consistency.
153
154.. _migration-1.3-nativesdk:
155
156nativesdk
157~~~~~~~~~
158
159The suffix ``nativesdk`` is now implemented as a prefix, which
160simplifies a lot of the packaging code for ``nativesdk`` recipes. All
161custom ``nativesdk`` recipes, which are relocatable packages that are
162native to ```SDK_ARCH`` <#var-SDK_ARCH>`__, and any references need to
163be updated to use ``nativesdk-*`` instead of ``*-nativesdk``.
164
165.. _migration-1.3-task-recipes:
166
167Task Recipes
168~~~~~~~~~~~~
169
170"Task" recipes are now known as "Package groups" and have been renamed
171from ``task-*.bb`` to ``packagegroup-*.bb``. Existing references to the
172previous ``task-*`` names should work in most cases as there is an
173automatic upgrade path for most packages. However, you should update
174references in your own recipes and configurations as they could be
175removed in future releases. You should also rename any custom ``task-*``
176recipes to ``packagegroup-*``, and change them to inherit
177``packagegroup`` instead of ``task``, as well as taking the opportunity
178to remove anything now handled by ``packagegroup.bbclass``, such as
179providing ``-dev`` and ``-dbg`` packages, setting
180```LIC_FILES_CHKSUM`` <#var-LIC_FILES_CHKSUM>`__, and so forth. See the
181"```packagegroup.bbclass`` <#ref-classes-packagegroup>`__" section for
182further details.
183
184.. _migration-1.3-image-features:
185
186IMAGE_FEATURES
187~~~~~~~~~~~~~~
188
189Image recipes that previously included "apps-console-core" in
190```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ should now include "splash"
191instead to enable the boot-up splash screen. Retaining
192"apps-console-core" will still include the splash screen but generates a
193warning. The "apps-x11-core" and "apps-x11-games" ``IMAGE_FEATURES``
194features have been removed.
195
196.. _migration-1.3-removed-recipes:
197
198Removed Recipes
199~~~~~~~~~~~~~~~
200
201The following recipes have been removed. For most of them, it is
202unlikely that you would have any references to them in your own
203`Metadata <#metadata>`__. However, you should check your metadata
204against this list to be sure:
205
206- *``libx11-trim``*: Replaced by ``libx11``, which has a negligible
207 size difference with modern Xorg.
208
209- *``xserver-xorg-lite``*: Use ``xserver-xorg``, which has a negligible
210 size difference when DRI and GLX modules are not installed.
211
212- *``xserver-kdrive``*: Effectively unmaintained for many years.
213
214- *``mesa-xlib``*: No longer serves any purpose.
215
216- *``galago``*: Replaced by telepathy.
217
218- *``gail``*: Functionality was integrated into GTK+ 2.13.
219
220- *``eggdbus``*: No longer needed.
221
222- *``gcc-*-intermediate``*: The build has been restructured to avoid
223 the need for this step.
224
225- *``libgsmd``*: Unmaintained for many years. Functionality now
226 provided by ``ofono`` instead.
227
228- *contacts, dates, tasks, eds-tools*: Largely unmaintained PIM
229 application suite. It has been moved to ``meta-gnome`` in
230 ``meta-openembedded``.
231
232In addition to the previously listed changes, the ``meta-demoapps``
233directory has also been removed because the recipes in it were not being
234maintained and many had become obsolete or broken. Additionally, these
235recipes were not parsed in the default configuration. Many of these
236recipes are already provided in an updated and maintained form within
237the OpenEmbedded community layers such as ``meta-oe`` and
238``meta-gnome``. For the remainder, you can now find them in the
239``meta-extras`` repository, which is in the Yocto Project `Source
240Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__.
241
242.. _1.3-linux-kernel-naming:
243
244Linux Kernel Naming
245-------------------
246
247The naming scheme for kernel output binaries has been changed to now
248include ```PE`` <#var-PE>`__ as part of the filename:
249KERNEL_IMAGE_BASE_NAME ?=
250"${KERNEL_IMAGETYPE}-${PE}-${PV}-${PR}-${MACHINE}-${DATETIME}"
251
252Because the ``PE`` variable is not set by default, these binary files
253could result with names that include two dash characters. Here is an
254example:
255bzImage--3.10.9+git0+cd502a8814_7144bcc4b8-r0-qemux86-64-20130830085431.bin
256
257Moving to the Yocto Project 1.4 Release
258=======================================
259
260This section provides migration information for moving to the Yocto
261Project 1.4 Release from the prior release.
262
263.. _migration-1.4-bitbake:
264
265BitBake
266-------
267
268Differences include the following:
269
270- *Comment Continuation:* If a comment ends with a line continuation
271 (\) character, then the next line must also be a comment. Any
272 instance where this is not the case, now triggers a warning. You must
273 either remove the continuation character, or be sure the next line is
274 a comment.
275
276- *Package Name Overrides:* The runtime package specific variables
277 ```RDEPENDS`` <#var-RDEPENDS>`__,
278 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__,
279 ```RSUGGESTS`` <#var-RSUGGESTS>`__,
280 ```RPROVIDES`` <#var-RPROVIDES>`__,
281 ```RCONFLICTS`` <#var-RCONFLICTS>`__,
282 ```RREPLACES`` <#var-RREPLACES>`__, ```FILES`` <#var-FILES>`__,
283 ```ALLOW_EMPTY`` <#var-ALLOW_EMPTY>`__, and the pre, post, install,
284 and uninstall script functions ``pkg_preinst``, ``pkg_postinst``,
285 ``pkg_prerm``, and ``pkg_postrm`` should always have a package name
286 override. For example, use ``RDEPENDS_${PN}`` for the main package
287 instead of ``RDEPENDS``. BitBake uses more strict checks when it
288 parses recipes.
289
290.. _migration-1.4-build-behavior:
291
292Build Behavior
293--------------
294
295Differences include the following:
296
297- *Shared State Code:* The shared state code has been optimized to
298 avoid running unnecessary tasks. For example, the following no longer
299 populates the target sysroot since that is not necessary: $ bitbake
300 -c rootfs some-image Instead, the system just needs to extract the
301 output package contents, re-create the packages, and construct the
302 root filesystem. This change is unlikely to cause any problems unless
303 you have missing declared dependencies.
304
305- *Scanning Directory Names:* When scanning for files in
306 ```SRC_URI`` <#var-SRC_URI>`__, the build system now uses
307 ```FILESOVERRIDES`` <#var-FILESOVERRIDES>`__ instead of
308 ```OVERRIDES`` <#var-OVERRIDES>`__ for the directory names. In
309 general, the values previously in ``OVERRIDES`` are now in
310 ``FILESOVERRIDES`` as well. However, if you relied upon an additional
311 value you previously added to ``OVERRIDES``, you might now need to
312 add it to ``FILESOVERRIDES`` unless you are already adding it through
313 the ```MACHINEOVERRIDES`` <#var-MACHINEOVERRIDES>`__ or
314 ```DISTROOVERRIDES`` <#var-DISTROOVERRIDES>`__ variables, as
315 appropriate. For more related changes, see the
316 "`Variables <#migration-1.4-variables>`__" section.
317
318.. _migration-1.4-proxies-and-fetching-source:
319
320Proxies and Fetching Source
321---------------------------
322
323A new ``oe-git-proxy`` script has been added to replace previous methods
324of handling proxies and fetching source from Git. See the
325``meta-yocto/conf/site.conf.sample`` file for information on how to use
326this script.
327
328.. _migration-1.4-custom-interfaces-file-netbase-change:
329
330Custom Interfaces File (netbase change)
331---------------------------------------
332
333If you have created your own custom ``etc/network/interfaces`` file by
334creating an append file for the ``netbase`` recipe, you now need to
335create an append file for the ``init-ifupdown`` recipe instead, which
336you can find in the `Source Directory <#source-directory>`__ at
337``meta/recipes-core/init-ifupdown``. For information on how to use
338append files, see the "`Using .bbappend
339Files <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
340Yocto Project Development Tasks Manual.
341
342.. _migration-1.4-remote-debugging:
343
344Remote Debugging
345----------------
346
347Support for remote debugging with the Eclipse IDE is now separated into
348an image feature (``eclipse-debug``) that corresponds to the
349``packagegroup-core-eclipse-debug`` package group. Previously, the
350debugging feature was included through the ``tools-debug`` image
351feature, which corresponds to the ``packagegroup-core-tools-debug``
352package group.
353
354.. _migration-1.4-variables:
355
356Variables
357---------
358
359The following variables have changed:
360
361- *``SANITY_TESTED_DISTROS``:* This variable now uses a distribution
362 ID, which is composed of the host distributor ID followed by the
363 release. Previously,
364 ```SANITY_TESTED_DISTROS`` <#var-SANITY_TESTED_DISTROS>`__ was
365 composed of the description field. For example, "Ubuntu 12.10"
366 becomes "Ubuntu-12.10". You do not need to worry about this change if
367 you are not specifically setting this variable, or if you are
368 specifically setting it to "".
369
370- *``SRC_URI``:* The ``${``\ ```PN`` <#var-PN>`__\ ``}``,
371 ``${``\ ```PF`` <#var-PF>`__\ ``}``,
372 ``${``\ ```P`` <#var-P>`__\ ``}``, and ``FILE_DIRNAME`` directories
373 have been dropped from the default value of the
374 ```FILESPATH`` <#var-FILESPATH>`__ variable, which is used as the
375 search path for finding files referred to in
376 ```SRC_URI`` <#var-SRC_URI>`__. If you have a recipe that relied upon
377 these directories, which would be unusual, then you will need to add
378 the appropriate paths within the recipe or, alternatively, rearrange
379 the files. The most common locations are still covered by ``${BP}``,
380 ``${BPN}``, and "files", which all remain in the default value of
381 ```FILESPATH`` <#var-FILESPATH>`__.
382
383.. _migration-target-package-management-with-rpm:
384
385Target Package Management with RPM
386----------------------------------
387
388If runtime package management is enabled and the RPM backend is
389selected, Smart is now installed for package download, dependency
390resolution, and upgrades instead of Zypper. For more information on how
391to use Smart, run the following command on the target: smart --help
392
393.. _migration-1.4-recipes-moved:
394
395Recipes Moved
396-------------
397
398The following recipes were moved from their previous locations because
399they are no longer used by anything in the OpenEmbedded-Core:
400
401- *``clutter-box2d``:* Now resides in the ``meta-oe`` layer.
402
403- *``evolution-data-server``:* Now resides in the ``meta-gnome`` layer.
404
405- *``gthumb``:* Now resides in the ``meta-gnome`` layer.
406
407- *``gtkhtml2``:* Now resides in the ``meta-oe`` layer.
408
409- *``gupnp``:* Now resides in the ``meta-multimedia`` layer.
410
411- *``gypsy``:* Now resides in the ``meta-oe`` layer.
412
413- *``libcanberra``:* Now resides in the ``meta-gnome`` layer.
414
415- *``libgdata``:* Now resides in the ``meta-gnome`` layer.
416
417- *``libmusicbrainz``:* Now resides in the ``meta-multimedia`` layer.
418
419- *``metacity``:* Now resides in the ``meta-gnome`` layer.
420
421- *``polkit``:* Now resides in the ``meta-oe`` layer.
422
423- *``zeroconf``:* Now resides in the ``meta-networking`` layer.
424
425.. _migration-1.4-removals-and-renames:
426
427Removals and Renames
428--------------------
429
430The following list shows what has been removed or renamed:
431
432- *``evieext``:* Removed because it has been removed from ``xserver``
433 since 2008.
434
435- *Gtk+ DirectFB:* Removed support because upstream Gtk+ no longer
436 supports it as of version 2.18.
437
438- *``libxfontcache / xfontcacheproto``:* Removed because they were
439 removed from the Xorg server in 2008.
440
441- *``libxp / libxprintapputil / libxprintutil / printproto``:* Removed
442 because the XPrint server was removed from Xorg in 2008.
443
444- *``libxtrap / xtrapproto``:* Removed because their functionality was
445 broken upstream.
446
447- *linux-yocto 3.0 kernel:* Removed with linux-yocto 3.8 kernel being
448 added. The linux-yocto 3.2 and linux-yocto 3.4 kernels remain as part
449 of the release.
450
451- *``lsbsetup``:* Removed with functionality now provided by
452 ``lsbtest``.
453
454- *``matchbox-stroke``:* Removed because it was never more than a
455 proof-of-concept.
456
457- *``matchbox-wm-2 / matchbox-theme-sato-2``:* Removed because they are
458 not maintained. However, ``matchbox-wm`` and ``matchbox-theme-sato``
459 are still provided.
460
461- *``mesa-dri``:* Renamed to ``mesa``.
462
463- *``mesa-xlib``:* Removed because it was no longer useful.
464
465- *``mutter``:* Removed because nothing ever uses it and the recipe is
466 very old.
467
468- *``orinoco-conf``:* Removed because it has become obsolete.
469
470- *``update-modules``:* Removed because it is no longer used. The
471 kernel module ``postinstall`` and ``postrm`` scripts can now do the
472 same task without the use of this script.
473
474- *``web``:* Removed because it is not maintained. Superseded by
475 ``web-webkit``.
476
477- *``xf86bigfontproto``:* Removed because upstream it has been disabled
478 by default since 2007. Nothing uses ``xf86bigfontproto``.
479
480- *``xf86rushproto``:* Removed because its dependency in ``xserver``
481 was spurious and it was removed in 2005.
482
483- *``zypper / libzypp / sat-solver``:* Removed and been functionally
484 replaced with Smart (``python-smartpm``) when RPM packaging is used
485 and package management is enabled on the target.
486
487Moving to the Yocto Project 1.5 Release
488=======================================
489
490This section provides migration information for moving to the Yocto
491Project 1.5 Release from the prior release.
492
493.. _migration-1.5-host-dependency-changes:
494
495Host Dependency Changes
496-----------------------
497
498The OpenEmbedded build system now has some additional requirements on
499the host system:
500
501- Python 2.7.3+
502
503- Tar 1.24+
504
505- Git 1.7.8+
506
507- Patched version of Make if you are using 3.82. Most distributions
508 that provide Make 3.82 use the patched version.
509
510If the Linux distribution you are using on your build host does not
511provide packages for these, you can install and use the Buildtools
512tarball, which provides an SDK-like environment containing them.
513
514For more information on this requirement, see the "`Required Git, tar,
515Python and gcc Versions <#required-git-tar-python-and-gcc-versions>`__"
516section.
517
518.. _migration-1.5-atom-pc-bsp:
519
520``atom-pc`` Board Support Package (BSP)
521---------------------------------------
522
523The ``atom-pc`` hardware reference BSP has been replaced by a
524``genericx86`` BSP. This BSP is not necessarily guaranteed to work on
525all x86 hardware, but it will run on a wider range of systems than the
526``atom-pc`` did.
527
528.. note::
529
530 Additionally, a
531 genericx86-64
532 BSP has been added for 64-bit Atom systems.
533
534.. _migration-1.5-bitbake:
535
536BitBake
537-------
538
539The following changes have been made that relate to BitBake:
540
541- BitBake now supports a ``_remove`` operator. The addition of this
542 operator means you will have to rename any items in recipe space
543 (functions, variables) whose names currently contain ``_remove_`` or
544 end with ``_remove`` to avoid unexpected behavior.
545
546- BitBake's global method pool has been removed. This method is not
547 particularly useful and led to clashes between recipes containing
548 functions that had the same name.
549
550- The "none" server backend has been removed. The "process" server
551 backend has been serving well as the default for a long time now.
552
553- The ``bitbake-runtask`` script has been removed.
554
555- ``${``\ ```P`` <#var-P>`__\ ``}`` and
556 ``${``\ ```PF`` <#var-PF>`__\ ``}`` are no longer added to
557 ```PROVIDES`` <#var-PROVIDES>`__ by default in ``bitbake.conf``.
558 These version-specific ``PROVIDES`` items were seldom used.
559 Attempting to use them could result in two versions being built
560 simultaneously rather than just one version due to the way BitBake
561 resolves dependencies.
562
563.. _migration-1.5-qa-warnings:
564
565QA Warnings
566-----------
567
568The following changes have been made to the package QA checks:
569
570- If you have customized ```ERROR_QA`` <#var-ERROR_QA>`__ or
571 ```WARN_QA`` <#var-WARN_QA>`__ values in your configuration, check
572 that they contain all of the issues that you wish to be reported.
573 Previous Yocto Project versions contained a bug that meant that any
574 item not mentioned in ``ERROR_QA`` or ``WARN_QA`` would be treated as
575 a warning. Consequently, several important items were not already in
576 the default value of ``WARN_QA``. All of the possible QA checks are
577 now documented in the "```insane.bbclass`` <#ref-classes-insane>`__"
578 section.
579
580- An additional QA check has been added to check if
581 ``/usr/share/info/dir`` is being installed. Your recipe should delete
582 this file within ```do_install`` <#ref-tasks-install>`__ if "make
583 install" is installing it.
584
585- If you are using the buildhistory class, the check for the package
586 version going backwards is now controlled using a standard QA check.
587 Thus, if you have customized your ``ERROR_QA`` or ``WARN_QA`` values
588 and still wish to have this check performed, you should add
589 "version-going-backwards" to your value for one or the other
590 variables depending on how you wish it to be handled. See the
591 documented QA checks in the
592 "```insane.bbclass`` <#ref-classes-insane>`__" section.
593
594.. _migration-1.5-directory-layout-changes:
595
596Directory Layout Changes
597------------------------
598
599The following directory changes exist:
600
601- Output SDK installer files are now named to include the image name
602 and tuning architecture through the ```SDK_NAME`` <#var-SDK_NAME>`__
603 variable.
604
605- Images and related files are now installed into a directory that is
606 specific to the machine, instead of a parent directory containing
607 output files for multiple machines. The
608 ```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__ variable continues
609 to point to the directory containing images for the current
610 ```MACHINE`` <#var-MACHINE>`__ and should be used anywhere there is a
611 need to refer to this directory. The ``runqemu`` script now uses this
612 variable to find images and kernel binaries and will use BitBake to
613 determine the directory. Alternatively, you can set the
614 ``DEPLOY_DIR_IMAGE`` variable in the external environment.
615
616- When buildhistory is enabled, its output is now written under the
617 `Build Directory <#build-directory>`__ rather than
618 ```TMPDIR`` <#var-TMPDIR>`__. Doing so makes it easier to delete
619 ``TMPDIR`` and preserve the build history. Additionally, data for
620 produced SDKs is now split by ```IMAGE_NAME`` <#var-IMAGE_NAME>`__.
621
622- The ``pkgdata`` directory produced as part of the packaging process
623 has been collapsed into a single machine-specific directory. This
624 directory is located under ``sysroots`` and uses a machine-specific
625 name (i.e. ``tmp/sysroots/machine/pkgdata``).
626
627.. _migration-1.5-shortened-git-srcrev-values:
628
629Shortened Git ``SRCREV`` Values
630-------------------------------
631
632BitBake will now shorten revisions from Git repositories from the normal
63340 characters down to 10 characters within ```SRCPV`` <#var-SRCPV>`__
634for improved usability in path and file names. This change should be
635safe within contexts where these revisions are used because the chances
636of spatially close collisions is very low. Distant collisions are not a
637major issue in the way the values are used.
638
639.. _migration-1.5-image-features:
640
641``IMAGE_FEATURES``
642------------------
643
644The following changes have been made that relate to
645```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__:
646
647- The value of ``IMAGE_FEATURES`` is now validated to ensure invalid
648 feature items are not added. Some users mistakenly add package names
649 to this variable instead of using
650 ```IMAGE_INSTALL`` <#var-IMAGE_INSTALL>`__ in order to have the
651 package added to the image, which does not work. This change is
652 intended to catch those kinds of situations. Valid ``IMAGE_FEATURES``
653 are drawn from ``PACKAGE_GROUP`` definitions,
654 ```COMPLEMENTARY_GLOB`` <#var-COMPLEMENTARY_GLOB>`__ and a new
655 "validitems" varflag on ``IMAGE_FEATURES``. The "validitems" varflag
656 change allows additional features to be added if they are not
657 provided using the previous two mechanisms.
658
659- The previously deprecated "apps-console-core" ``IMAGE_FEATURES`` item
660 is no longer supported. Add "splash" to ``IMAGE_FEATURES`` if you
661 wish to have the splash screen enabled, since this is all that
662 apps-console-core was doing.
663
664.. _migration-1.5-run:
665
666``/run``
667--------
668
669The ``/run`` directory from the Filesystem Hierarchy Standard 3.0 has
670been introduced. You can find some of the implications for this change
671`here <http://cgit.openembedded.org/openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873>`__.
672The change also means that recipes that install files to ``/var/run``
673must be changed. You can find a guide on how to make these changes
674`here <http://permalink.gmane.org/gmane.comp.handhelds.openembedded/58530>`__.
675
676.. _migration-1.5-removal-of-package-manager-database-within-image-recipes:
677
678Removal of Package Manager Database Within Image Recipes
679--------------------------------------------------------
680
681The image ``core-image-minimal`` no longer adds
682``remove_packaging_data_files`` to
683```ROOTFS_POSTPROCESS_COMMAND`` <#var-ROOTFS_POSTPROCESS_COMMAND>`__.
684This addition is now handled automatically when "package-management" is
685not in ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__. If you have custom
686image recipes that make this addition, you should remove the lines, as
687they are not needed and might interfere with correct operation of
688postinstall scripts.
689
690.. _migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time:
691
692Images Now Rebuild Only on Changes Instead of Every Time
693--------------------------------------------------------
694
695The ```do_rootfs`` <#ref-tasks-rootfs>`__ and other related image
696construction tasks are no longer marked as "nostamp". Consequently, they
697will only be re-executed when their inputs have changed. Previous
698versions of the OpenEmbedded build system always rebuilt the image when
699requested rather when necessary.
700
701.. _migration-1.5-task-recipes:
702
703Task Recipes
704------------
705
706The previously deprecated ``task.bbclass`` has now been dropped. For
707recipes that previously inherited from this class, you should rename
708them from ``task-*`` to ``packagegroup-*`` and inherit packagegroup
709instead.
710
711For more information, see the
712"```packagegroup.bbclass`` <#ref-classes-packagegroup>`__" section.
713
714.. _migration-1.5-busybox:
715
716BusyBox
717-------
718
719By default, we now split BusyBox into two binaries: one that is suid
720root for those components that need it, and another for the rest of the
721components. Splitting BusyBox allows for optimization that eliminates
722the ``tinylogin`` recipe as recommended by upstream. You can disable
723this split by setting
724```BUSYBOX_SPLIT_SUID`` <#var-BUSYBOX_SPLIT_SUID>`__ to "0".
725
726.. _migration-1.5-automated-image-testing:
727
728Automated Image Testing
729-----------------------
730
731A new automated image testing framework has been added through the
732```testimage.bbclass`` <#ref-classes-testimage*>`__ class. This
733framework replaces the older ``imagetest-qemu`` framework.
734
735You can learn more about performing automated image tests in the
736"`Performing Automated Runtime
737Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
738section in the Yocto Project Development Tasks Manual.
739
740.. _migration-1.5-build-history:
741
742Build History
743-------------
744
745Following are changes to Build History:
746
747- Installed package sizes: ``installed-package-sizes.txt`` for an image
748 now records the size of the files installed by each package instead
749 of the size of each compressed package archive file.
750
751- The dependency graphs (``depends*.dot``) now use the actual package
752 names instead of replacing dashes, dots and plus signs with
753 underscores.
754
755- The ``buildhistory-diff`` and ``buildhistory-collect-srcrevs``
756 utilities have improved command-line handling. Use the ``--help``
757 option for each utility for more information on the new syntax.
758
759For more information on Build History, see the "`Maintaining Build
760Output
761Quality <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__"
762section in the Yocto Project Development Tasks Manual.
763
764.. _migration-1.5-udev:
765
766``udev``
767--------
768
769Following are changes to ``udev``:
770
771- ``udev`` no longer brings in ``udev-extraconf`` automatically through
772 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__, since this was originally
773 intended to be optional. If you need the extra rules, then add
774 ``udev-extraconf`` to your image.
775
776- ``udev`` no longer brings in ``pciutils-ids`` or ``usbutils-ids``
777 through ``RRECOMMENDS``. These are not needed by ``udev`` itself and
778 removing them saves around 350KB.
779
780.. _migration-1.5-removed-renamed-recipes:
781
782Removed and Renamed Recipes
783---------------------------
784
785- The ``linux-yocto`` 3.2 kernel has been removed.
786
787- ``libtool-nativesdk`` has been renamed to ``nativesdk-libtool``.
788
789- ``tinylogin`` has been removed. It has been replaced by a suid
790 portion of Busybox. See the "`BusyBox <#migration-1.5-busybox>`__"
791 section for more information.
792
793- ``external-python-tarball`` has been renamed to
794 ``buildtools-tarball``.
795
796- ``web-webkit`` has been removed. It has been functionally replaced by
797 ``midori``.
798
799- ``imake`` has been removed. It is no longer needed by any other
800 recipe.
801
802- ``transfig-native`` has been removed. It is no longer needed by any
803 other recipe.
804
805- ``anjuta-remote-run`` has been removed. Anjuta IDE integration has
806 not been officially supported for several releases.
807
808.. _migration-1.5-other-changes:
809
810Other Changes
811-------------
812
813Following is a list of short entries describing other changes:
814
815- ``run-postinsts``: Make this generic.
816
817- ``base-files``: Remove the unnecessary ``media/``\ xxx directories.
818
819- ``alsa-state``: Provide an empty ``asound.conf`` by default.
820
821- ``classes/image``: Ensure
822 ```BAD_RECOMMENDATIONS`` <#var-BAD_RECOMMENDATIONS>`__ supports
823 pre-renamed package names.
824
825- ``classes/rootfs_rpm``: Implement ``BAD_RECOMMENDATIONS`` for RPM.
826
827- ``systemd``: Remove ``systemd_unitdir`` if ``systemd`` is not in
828 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__.
829
830- ``systemd``: Remove ``init.d`` dir if ``systemd`` unit file is
831 present and ``sysvinit`` is not a distro feature.
832
833- ``libpam``: Deny all services for the ``OTHER`` entries.
834
835- ``image.bbclass``: Move ``runtime_mapping_rename`` to avoid conflict
836 with ``multilib``. See
837 ```YOCTO #4993`` <https://bugzilla.yoctoproject.org/show_bug.cgi?id=4993>`__
838 in Bugzilla for more information.
839
840- ``linux-dtb``: Use kernel build system to generate the ``dtb`` files.
841
842- ``kern-tools``: Switch from guilt to new ``kgit-s2q`` tool.
843
844Moving to the Yocto Project 1.6 Release
845=======================================
846
847This section provides migration information for moving to the Yocto
848Project 1.6 Release from the prior release.
849
850.. _migration-1.6-archiver-class:
851
852``archiver`` Class
853------------------
854
855The ```archiver`` <#ref-classes-archiver>`__ class has been rewritten
856and its configuration has been simplified. For more details on the
857source archiver, see the "`Maintaining Open Source License Compliance
858During Your Product's
859Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
860section in the Yocto Project Development Tasks Manual.
861
862.. _migration-1.6-packaging-changes:
863
864Packaging Changes
865-----------------
866
867The following packaging changes have been made:
868
869- The ``binutils`` recipe no longer produces a ``binutils-symlinks``
870 package. ``update-alternatives`` is now used to handle the preferred
871 ``binutils`` variant on the target instead.
872
873- The tc (traffic control) utilities have been split out of the main
874 ``iproute2`` package and put into the ``iproute2-tc`` package.
875
876- The ``gtk-engines`` schemas have been moved to a dedicated
877 ``gtk-engines-schemas`` package.
878
879- The ``armv7a`` with thumb package architecture suffix has changed.
880 The suffix for these packages with the thumb optimization enabled is
881 "t2" as it should be. Use of this suffix was not the case in the 1.5
882 release. Architecture names will change within package feeds as a
883 result.
884
885.. _migration-1.6-bitbake:
886
887BitBake
888-------
889
890The following changes have been made to `BitBake <#bitbake-term>`__.
891
892.. _migration-1.6-matching-branch-requirement-for-git-fetching:
893
894Matching Branch Requirement for Git Fetching
895~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
896
897When fetching source from a Git repository using
898```SRC_URI`` <#var-SRC_URI>`__, BitBake will now validate the
899```SRCREV`` <#var-SRCREV>`__ value against the branch. You can specify
900the branch using the following form: SRC_URI =
901"git://server.name/repository;branch=branchname" If you do not specify a
902branch, BitBake looks in the default "master" branch.
903
904Alternatively, if you need to bypass this check (e.g. if you are
905fetching a revision corresponding to a tag that is not on any branch),
906you can add ";nobranch=1" to the end of the URL within ``SRC_URI``.
907
908.. _migration-1.6-bitbake-deps:
909
910Python Definition substitutions
911~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
912
913BitBake had some previously deprecated Python definitions within its
914``bb`` module removed. You should use their sub-module counterparts
915instead:
916
917- ``bb.MalformedUrl``: Use ``bb.fetch.MalformedUrl``.
918
919- ``bb.encodeurl``: Use ``bb.fetch.encodeurl``.
920
921- ``bb.decodeurl``: Use ``bb.fetch.decodeurl``
922
923- ``bb.mkdirhier``: Use ``bb.utils.mkdirhier``.
924
925- ``bb.movefile``: Use ``bb.utils.movefile``.
926
927- ``bb.copyfile``: Use ``bb.utils.copyfile``.
928
929- ``bb.which``: Use ``bb.utils.which``.
930
931- ``bb.vercmp_string``: Use ``bb.utils.vercmp_string``.
932
933- ``bb.vercmp``: Use ``bb.utils.vercmp``.
934
935.. _migration-1.6-bitbake-fetcher:
936
937SVK Fetcher
938~~~~~~~~~~~
939
940The SVK fetcher has been removed from BitBake.
941
942.. _migration-1.6-bitbake-console-output:
943
944Console Output Error Redirection
945~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
946
947The BitBake console UI will now output errors to ``stderr`` instead of
948``stdout``. Consequently, if you are piping or redirecting the output of
949``bitbake`` to somewhere else, and you wish to retain the errors, you
950will need to add ``2>&1`` (or something similar) to the end of your
951``bitbake`` command line.
952
953.. _migration-1.6-task-taskname-overrides:
954
955``task-``\ taskname Overrides
956~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
957
958``task-``\ taskname overrides have been adjusted so that tasks whose
959names contain underscores have the underscores replaced by hyphens for
960the override so that they now function properly. For example, the task
961override for ```do_populate_sdk`` <#ref-tasks-populate_sdk>`__ is
962``task-populate-sdk``.
963
964.. _migration-1.6-variable-changes:
965
966Changes to Variables
967--------------------
968
969The following variables have changed. For information on the
970OpenEmbedded build system variables, see the "`Variables
971Glossary <#ref-variables-glos>`__" Chapter.
972
973.. _migration-1.6-variable-changes-TMPDIR:
974
975``TMPDIR``
976~~~~~~~~~~
977
978```TMPDIR`` <#var-TMPDIR>`__ can no longer be on an NFS mount. NFS does
979not offer full POSIX locking and inode consistency and can cause
980unexpected issues if used to store ``TMPDIR``.
981
982The check for this occurs on startup. If ``TMPDIR`` is detected on an
983NFS mount, an error occurs.
984
985.. _migration-1.6-variable-changes-PRINC:
986
987``PRINC``
988~~~~~~~~~
989
990The ``PRINC`` variable has been deprecated and triggers a warning if
991detected during a build. For ```PR`` <#var-PR>`__ increments on changes,
992use the PR service instead. You can find out more about this service in
993the "`Working With a PR
994Service <&YOCTO_DOCS_DEV_URL;#working-with-a-pr-service>`__" section in
995the Yocto Project Development Tasks Manual.
996
997.. _migration-1.6-variable-changes-IMAGE_TYPES:
998
999``IMAGE_TYPES``
1000~~~~~~~~~~~~~~~
1001
1002The "sum.jffs2" option for ```IMAGE_TYPES`` <#var-IMAGE_TYPES>`__ has
1003been replaced by the "jffs2.sum" option, which fits the processing
1004order.
1005
1006.. _migration-1.6-variable-changes-COPY_LIC_MANIFEST:
1007
1008``COPY_LIC_MANIFEST``
1009~~~~~~~~~~~~~~~~~~~~~
1010
1011The ```COPY_LIC_MANIFEST`` <#var-COPY_LIC_MANIFEST>`__ variable must now
1012be set to "1" rather than any value in order to enable it.
1013
1014.. _migration-1.6-variable-changes-COPY_LIC_DIRS:
1015
1016``COPY_LIC_DIRS``
1017~~~~~~~~~~~~~~~~~
1018
1019The ```COPY_LIC_DIRS`` <#var-COPY_LIC_DIRS>`__ variable must now be set
1020to "1" rather than any value in order to enable it.
1021
1022.. _migration-1.6-variable-changes-PACKAGE_GROUP:
1023
1024``PACKAGE_GROUP``
1025~~~~~~~~~~~~~~~~~
1026
1027The ``PACKAGE_GROUP`` variable has been renamed to
1028```FEATURE_PACKAGES`` <#var-FEATURE_PACKAGES>`__ to more accurately
1029reflect its purpose. You can still use ``PACKAGE_GROUP`` but the
1030OpenEmbedded build system produces a warning message when it encounters
1031the variable.
1032
1033.. _migration-1.6-variable-changes-variable-entry-behavior:
1034
1035Preprocess and Post Process Command Variable Behavior
1036~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1037
1038The following variables now expect a semicolon separated list of
1039functions to call and not arbitrary shell commands:
1040`ROOTFS_PREPROCESS_COMMAND <#var-ROOTFS_PREPROCESS_COMMAND>`__
1041`ROOTFS_POSTPROCESS_COMMAND <#var-ROOTFS_POSTPROCESS_COMMAND>`__
1042`SDK_POSTPROCESS_COMMAND <#var-SDK_POSTPROCESS_COMMAND>`__
1043`POPULATE_SDK_POST_TARGET_COMMAND <#var-POPULATE_SDK_POST_TARGET_COMMAND>`__
1044`POPULATE_SDK_POST_HOST_COMMAND <#var-POPULATE_SDK_POST_HOST_COMMAND>`__
1045`IMAGE_POSTPROCESS_COMMAND <#var-IMAGE_POSTPROCESS_COMMAND>`__
1046`IMAGE_PREPROCESS_COMMAND <#var-IMAGE_PREPROCESS_COMMAND>`__
1047`ROOTFS_POSTUNINSTALL_COMMAND <#var-ROOTFS_POSTUNINSTALL_COMMAND>`__
1048`ROOTFS_POSTINSTALL_COMMAND <#var-ROOTFS_POSTINSTALL_COMMAND>`__ For
1049migration purposes, you can simply wrap shell commands in a shell
1050function and then call the function. Here is an example:
1051my_postprocess_function() { echo "hello" > ${IMAGE_ROOTFS}/hello.txt }
1052ROOTFS_POSTPROCESS_COMMAND += "my_postprocess_function; "
1053
1054.. _migration-1.6-package-test-ptest:
1055
1056Package Test (ptest)
1057--------------------
1058
1059Package Tests (ptest) are built but not installed by default. For
1060information on using Package Tests, see the "`Testing Packages with
1061ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section in
1062the Yocto Project Development Tasks Manual. For information on the
1063``ptest`` class, see the "```ptest.bbclass`` <#ref-classes-ptest>`__"
1064section.
1065
1066.. _migration-1.6-build-changes:
1067
1068Build Changes
1069-------------
1070
1071Separate build and source directories have been enabled by default for
1072selected recipes where it is known to work (a whitelist) and for all
1073recipes that inherit the ```cmake`` <#ref-classes-cmake>`__ class. In
1074future releases the ```autotools`` <#ref-classes-autotools>`__ class
1075will enable a separate build directory by default as well. Recipes
1076building Autotools-based software that fails to build with a separate
1077build directory should be changed to inherit from the
1078```autotools-brokensep`` <#ref-classes-autotools>`__ class instead of
1079the ``autotools`` or ``autotools_stage``\ classes.
1080
1081.. _migration-1.6-building-qemu-native:
1082
1083``qemu-native``
1084---------------
1085
1086``qemu-native`` now builds without SDL-based graphical output support by
1087default. The following additional lines are needed in your
1088``local.conf`` to enable it: PACKAGECONFIG_pn-qemu-native = "sdl"
1089ASSUME_PROVIDED += "libsdl-native"
1090
1091.. note::
1092
1093 The default
1094 local.conf
1095 contains these statements. Consequently, if you are building a
1096 headless system and using a default
1097 local.conf
1098 file, you will need comment these two lines out.
1099
1100.. _migration-1.6-core-image-basic:
1101
1102``core-image-basic``
1103--------------------
1104
1105``core-image-basic`` has been renamed to ``core-image-full-cmdline``.
1106
1107In addition to ``core-image-basic`` being renamed,
1108``packagegroup-core-basic`` has been renamed to
1109``packagegroup-core-full-cmdline`` to match.
1110
1111.. _migration-1.6-licensing:
1112
1113Licensing
1114---------
1115
1116The top-level ``LICENSE`` file has been changed to better describe the
1117license of the various components of `OE-Core <#oe-core>`__. However,
1118the licensing itself remains unchanged.
1119
1120Normally, this change would not cause any side-effects. However, some
1121recipes point to this file within
1122```LIC_FILES_CHKSUM`` <#var-LIC_FILES_CHKSUM>`__ (as
1123``${COREBASE}/LICENSE``) and thus the accompanying checksum must be
1124changed from 3f40d7994397109285ec7b81fdeb3b58 to
11254d92cd373abda3937c2bc47fbc49d690. A better alternative is to have
1126``LIC_FILES_CHKSUM`` point to a file describing the license that is
1127distributed with the source that the recipe is building, if possible,
1128rather than pointing to ``${COREBASE}/LICENSE``.
1129
1130.. _migration-1.6-cflags-options:
1131
1132``CFLAGS`` Options
1133------------------
1134
1135The "-fpermissive" option has been removed from the default
1136```CFLAGS`` <#var-CFLAGS>`__ value. You need to take action on
1137individual recipes that fail when building with this option. You need to
1138either patch the recipes to fix the issues reported by the compiler, or
1139you need to add "-fpermissive" to ``CFLAGS`` in the recipes.
1140
1141.. _migration-1.6-custom-images:
1142
1143Custom Image Output Types
1144-------------------------
1145
1146Custom image output types, as selected using
1147```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__, must declare their
1148dependencies on other image types (if any) using a new
1149```IMAGE_TYPEDEP`` <#var-IMAGE_TYPEDEP>`__ variable.
1150
1151.. _migration-1.6-do-package-write-task:
1152
1153Tasks
1154-----
1155
1156The ``do_package_write`` task has been removed. The task is no longer
1157needed.
1158
1159.. _migration-1.6-update-alternatives-provider:
1160
1161``update-alternative`` Provider
1162-------------------------------
1163
1164The default ``update-alternatives`` provider has been changed from
1165``opkg`` to ``opkg-utils``. This change resolves some troublesome
1166circular dependencies. The runtime package has also been renamed from
1167``update-alternatives-cworth`` to ``update-alternatives-opkg``.
1168
1169.. _migration-1.6-virtclass-overrides:
1170
1171``virtclass`` Overrides
1172-----------------------
1173
1174The ``virtclass`` overrides are now deprecated. Use the equivalent class
1175overrides instead (e.g. ``virtclass-native`` becomes ``class-native``.)
1176
1177.. _migration-1.6-removed-renamed-recipes:
1178
1179Removed and Renamed Recipes
1180---------------------------
1181
1182The following recipes have been removed:
1183
1184- ``packagegroup-toolset-native`` - This recipe is largely unused.
1185
1186- ``linux-yocto-3.8`` - Support for the Linux yocto 3.8 kernel has been
1187 dropped. Support for the 3.10 and 3.14 kernels have been added with
1188 the ``linux-yocto-3.10`` and ``linux-yocto-3.14`` recipes.
1189
1190- ``ocf-linux`` - This recipe has been functionally replaced using
1191 ``cryptodev-linux``.
1192
1193- ``genext2fs`` - ``genext2fs`` is no longer used by the build system
1194 and is unmaintained upstream.
1195
1196- ``js`` - This provided an ancient version of Mozilla's javascript
1197 engine that is no longer needed.
1198
1199- ``zaurusd`` - The recipe has been moved to the ``meta-handheld``
1200 layer.
1201
1202- ``eglibc 2.17`` - Replaced by the ``eglibc 2.19`` recipe.
1203
1204- ``gcc 4.7.2`` - Replaced by the now stable ``gcc 4.8.2``.
1205
1206- ``external-sourcery-toolchain`` - this recipe is now maintained in
1207 the ``meta-sourcery`` layer.
1208
1209- ``linux-libc-headers-yocto 3.4+git`` - Now using version 3.10 of the
1210 ``linux-libc-headers`` by default.
1211
1212- ``meta-toolchain-gmae`` - This recipe is obsolete.
1213
1214- ``packagegroup-core-sdk-gmae`` - This recipe is obsolete.
1215
1216- ``packagegroup-core-standalone-gmae-sdk-target`` - This recipe is
1217 obsolete.
1218
1219.. _migration-1.6-removed-classes:
1220
1221Removed Classes
1222---------------
1223
1224The following classes have become obsolete and have been removed:
1225
1226- ``module_strip``
1227
1228- ``pkg_metainfo``
1229
1230- ``pkg_distribute``
1231
1232- ``image-empty``
1233
1234.. _migration-1.6-reference-bsps:
1235
1236Reference Board Support Packages (BSPs)
1237---------------------------------------
1238
1239The following reference BSPs changes occurred:
1240
1241- The BeagleBoard (``beagleboard``) ARM reference hardware has been
1242 replaced by the BeagleBone (``beaglebone``) hardware.
1243
1244- The RouterStation Pro (``routerstationpro``) MIPS reference hardware
1245 has been replaced by the EdgeRouter Lite (``edgerouter``) hardware.
1246
1247The previous reference BSPs for the ``beagleboard`` and
1248``routerstationpro`` machines are still available in a new
1249``meta-yocto-bsp-old`` layer in the `Source
1250Repositories <&YOCTO_GIT_URL;>`__ at
1251http://git.yoctoproject.org/cgit/cgit.cgi/meta-yocto-bsp-old/.
1252
1253Moving to the Yocto Project 1.7 Release
1254=======================================
1255
1256This section provides migration information for moving to the Yocto
1257Project 1.7 Release from the prior release.
1258
1259.. _migration-1.7-changes-to-setting-qemu-packageconfig-options:
1260
1261Changes to Setting QEMU ``PACKAGECONFIG`` Options in ``local.conf``
1262-------------------------------------------------------------------
1263
1264The QEMU recipe now uses a number of
1265```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__ options to enable various
1266optional features. The method used to set defaults for these options
1267means that existing ``local.conf`` files will need to be be modified to
1268append to ``PACKAGECONFIG`` for ``qemu-native`` and ``nativesdk-qemu``
1269instead of setting it. In other words, to enable graphical output for
1270QEMU, you should now have these lines in ``local.conf``:
1271PACKAGECONFIG_append_pn-qemu-native = " sdl"
1272PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
1273
1274.. _migration-1.7-minimum-git-version:
1275
1276Minimum Git version
1277-------------------
1278
1279The minimum `Git <&YOCTO_DOCS_OM_URL;#git>`__ version required on the
1280build host is now 1.7.8 because the ``--list`` option is now required by
1281BitBake's Git fetcher. As always, if your host distribution does not
1282provide a version of Git that meets this requirement, you can use the
1283``buildtools-tarball`` that does. See the "`Required Git, tar, Python
1284and gcc Versions <#required-git-tar-python-and-gcc-versions>`__" section
1285for more information.
1286
1287.. _migration-1.7-autotools-class-changes:
1288
1289Autotools Class Changes
1290-----------------------
1291
1292The following ```autotools`` <#ref-classes-autotools>`__ class changes
1293occurred:
1294
1295- *A separate build directory is now used by default:* The
1296 ``autotools`` class has been changed to use a directory for building
1297 (```B`` <#var-B>`__), which is separate from the source directory
1298 (```S`` <#var-S>`__). This is commonly referred to as ``B != S``, or
1299 an out-of-tree build.
1300
1301 If the software being built is already capable of building in a
1302 directory separate from the source, you do not need to do anything.
1303 However, if the software is not capable of being built in this
1304 manner, you will need to either patch the software so that it can
1305 build separately, or you will need to change the recipe to inherit
1306 the ```autotools-brokensep`` <#ref-classes-autotools>`__ class
1307 instead of the ``autotools`` or ``autotools_stage`` classes.
1308
1309- *The ``--foreign`` option is no longer passed to ``automake`` when
1310 running ``autoconf``:* This option tells ``automake`` that a
1311 particular software package does not follow the GNU standards and
1312 therefore should not be expected to distribute certain files such as
1313 ``ChangeLog``, ``AUTHORS``, and so forth. Because the majority of
1314 upstream software packages already tell ``automake`` to enable
1315 foreign mode themselves, the option is mostly superfluous. However,
1316 some recipes will need patches for this change. You can easily make
1317 the change by patching ``configure.ac`` so that it passes "foreign"
1318 to ``AM_INIT_AUTOMAKE()``. See `this
1319 commit <http://cgit.openembedded.org/openembedded-core/commit/?id=01943188f85ce6411717fb5bf702d609f55813f2>`__
1320 for an example showing how to make the patch.
1321
1322.. _migration-1.7-binary-configuration-scripts-disabled:
1323
1324Binary Configuration Scripts Disabled
1325-------------------------------------
1326
1327Some of the core recipes that package binary configuration scripts now
1328disable the scripts due to the scripts previously requiring error-prone
1329path substitution. Software that links against these libraries using
1330these scripts should use the much more robust ``pkg-config`` instead.
1331The list of recipes changed in this version (and their configuration
1332scripts) is as follows: directfb (directfb-config) freetype
1333(freetype-config) gpgme (gpgme-config) libassuan (libassuan-config)
1334libcroco (croco-6.0-config) libgcrypt (libgcrypt-config) libgpg-error
1335(gpg-error-config) libksba (ksba-config) libpcap (pcap-config) libpcre
1336(pcre-config) libpng (libpng-config, libpng16-config) libsdl
1337(sdl-config) libusb-compat (libusb-config) libxml2 (xml2-config) libxslt
1338(xslt-config) ncurses (ncurses-config) neon (neon-config) npth
1339(npth-config) pth (pth-config) taglib (taglib-config) Additionally,
1340support for ``pkg-config`` has been added to some recipes in the
1341previous list in the rare cases where the upstream software package does
1342not already provide it.
1343
1344.. _migration-1.7-glibc-replaces-eglibc:
1345
1346``eglibc 2.19`` Replaced with ``glibc 2.20``
1347--------------------------------------------
1348
1349Because ``eglibc`` and ``glibc`` were already fairly close, this
1350replacement should not require any significant changes to other software
1351that links to ``eglibc``. However, there were a number of minor changes
1352in ``glibc 2.20`` upstream that could require patching some software
1353(e.g. the removal of the ``_BSD_SOURCE`` feature test macro).
1354
1355``glibc 2.20`` requires version 2.6.32 or greater of the Linux kernel.
1356Thus, older kernels will no longer be usable in conjunction with it.
1357
1358For full details on the changes in ``glibc 2.20``, see the upstream
1359release notes
1360`here <https://sourceware.org/ml/libc-alpha/2014-09/msg00088.html>`__.
1361
1362.. _migration-1.7-kernel-module-autoloading:
1363
1364Kernel Module Autoloading
1365-------------------------
1366
1367The ```module_autoload_*`` <#var-module_autoload>`__ variable is now
1368deprecated and a new
1369```KERNEL_MODULE_AUTOLOAD`` <#var-KERNEL_MODULE_AUTOLOAD>`__ variable
1370should be used instead. Also, ```module_conf_*`` <#var-module_conf>`__
1371must now be used in conjunction with a new
1372```KERNEL_MODULE_PROBECONF`` <#var-KERNEL_MODULE_PROBECONF>`__ variable.
1373The new variables no longer require you to specify the module name as
1374part of the variable name. This change not only simplifies usage but
1375also allows the values of these variables to be appropriately
1376incorporated into task signatures and thus trigger the appropriate tasks
1377to re-execute when changed. You should replace any references to
1378``module_autoload_*`` with ``KERNEL_MODULE_AUTOLOAD``, and add any
1379modules for which ``module_conf_*`` is specified to
1380``KERNEL_MODULE_PROBECONF``.
1381
1382.. _migration-1.7-qa-check-changes:
1383
1384QA Check Changes
1385----------------
1386
1387The following changes have occurred to the QA check process:
1388
1389- Additional QA checks ``file-rdeps`` and ``build-deps`` have been
1390 added in order to verify that file dependencies are satisfied (e.g.
1391 package contains a script requiring ``/bin/bash``) and build-time
1392 dependencies are declared, respectively. For more information, please
1393 see the "`QA Error and Warning Messages <#ref-qa-checks>`__" chapter.
1394
1395- Package QA checks are now performed during a new
1396 ```do_package_qa`` <#ref-tasks-package_qa>`__ task rather than being
1397 part of the ```do_package`` <#ref-tasks-package>`__ task. This allows
1398 more parallel execution. This change is unlikely to be an issue
1399 except for highly customized recipes that disable packaging tasks
1400 themselves by marking them as ``noexec``. For those packages, you
1401 will need to disable the ``do_package_qa`` task as well.
1402
1403- Files being overwritten during the
1404 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task now
1405 trigger an error instead of a warning. Recipes should not be
1406 overwriting files written to the sysroot by other recipes. If you
1407 have these types of recipes, you need to alter them so that they do
1408 not overwrite these files.
1409
1410 You might now receive this error after changes in configuration or
1411 metadata resulting in orphaned files being left in the sysroot. If
1412 you do receive this error, the way to resolve the issue is to delete
1413 your ```TMPDIR`` <#var-TMPDIR>`__ or to move it out of the way and
1414 then re-start the build. Anything that has been fully built up to
1415 that point and does not need rebuilding will be restored from the
1416 shared state cache and the rest of the build will be able to proceed
1417 as normal.
1418
1419.. _migration-1.7-removed-recipes:
1420
1421Removed Recipes
1422---------------
1423
1424The following recipes have been removed:
1425
1426- ``x-load``: This recipe has been superseded by U-boot SPL for all
1427 Cortex-based TI SoCs. For legacy boards, the ``meta-ti`` layer, which
1428 contains a maintained recipe, should be used instead.
1429
1430- ``ubootchart``: This recipe is obsolete. A ``bootchart2`` recipe has
1431 been added to functionally replace it.
1432
1433- ``linux-yocto 3.4``: Support for the linux-yocto 3.4 kernel has been
1434 dropped. Support for the 3.10 and 3.14 kernels remains, while support
1435 for version 3.17 has been added.
1436
1437- ``eglibc`` has been removed in favor of ``glibc``. See the
1438 "```eglibc 2.19`` Replaced with
1439 ``glibc 2.20`` <#migration-1.7-glibc-replaces-eglibc>`__" section for
1440 more information.
1441
1442.. _migration-1.7-miscellaneous-changes:
1443
1444Miscellaneous Changes
1445---------------------
1446
1447The following miscellaneous change occurred:
1448
1449- The build history feature now writes ``build-id.txt`` instead of
1450 ``build-id``. Additionally, ``build-id.txt`` now contains the full
1451 build header as printed by BitBake upon starting the build. You
1452 should manually remove old "build-id" files from your existing build
1453 history repositories to avoid confusion. For information on the build
1454 history feature, see the "`Maintaining Build Output
1455 Quality <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__"
1456 section in the Yocto Project Development Tasks Manual.
1457
1458Moving to the Yocto Project 1.8 Release
1459=======================================
1460
1461This section provides migration information for moving to the Yocto
1462Project 1.8 Release from the prior release.
1463
1464.. _migration-1.8-removed-recipes:
1465
1466Removed Recipes
1467---------------
1468
1469The following recipes have been removed:
1470
1471- ``owl-video``: Functionality replaced by ``gst-player``.
1472
1473- ``gaku``: Functionality replaced by ``gst-player``.
1474
1475- ``gnome-desktop``: This recipe is now available in ``meta-gnome`` and
1476 is no longer needed.
1477
1478- ``gsettings-desktop-schemas``: This recipe is now available in
1479 ``meta-gnome`` and is no longer needed.
1480
1481- ``python-argparse``: The ``argparse`` module is already provided in
1482 the default Python distribution in a package named
1483 ``python-argparse``. Consequently, the separate ``python-argparse``
1484 recipe is no longer needed.
1485
1486- ``telepathy-python, libtelepathy, telepathy-glib, telepathy-idle, telepathy-mission-control``:
1487 All these recipes have moved to ``meta-oe`` and are consequently no
1488 longer needed by any recipes in OpenEmbedded-Core.
1489
1490- ``linux-yocto_3.10`` and ``linux-yocto_3.17``: Support for the
1491 linux-yocto 3.10 and 3.17 kernels has been dropped. Support for the
1492 3.14 kernel remains, while support for 3.19 kernel has been added.
1493
1494- ``poky-feed-config-opkg``: This recipe has become obsolete and is no
1495 longer needed. Use ``distro-feed-config`` from ``meta-oe`` instead.
1496
1497- ``libav 0.8.x``: ``libav 9.x`` is now used.
1498
1499- ``sed-native``: No longer needed. A working version of ``sed`` is
1500 expected to be provided by the host distribution.
1501
1502.. _migration-1.8-bluez:
1503
1504BlueZ 4.x / 5.x Selection
1505-------------------------
1506
1507Proper built-in support for selecting BlueZ 5.x in preference to the
1508default of 4.x now exists. To use BlueZ 5.x, simply add "bluez5" to your
1509```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ value. If you had
1510previously added append files (``*.bbappend``) to make this selection,
1511you can now remove them.
1512
1513Additionally, a ``bluetooth`` class has been added to make selection of
1514the appropriate bluetooth support within a recipe a little easier. If
1515you wish to make use of this class in a recipe, add something such as
1516the following: inherit bluetooth PACKAGECONFIG ??=
1517"${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', '${BLUEZ}', '',
1518d)}" PACKAGECONFIG[bluez4] =
1519"--enable-bluetooth,--disable-bluetooth,bluez4" PACKAGECONFIG[bluez5] =
1520"--enable-bluez5,--disable-bluez5,bluez5"
1521
1522.. _migration-1.8-kernel-build-changes:
1523
1524Kernel Build Changes
1525--------------------
1526
1527The kernel build process was changed to place the source in a common
1528shared work area and to place build artifacts separately in the source
1529code tree. In theory, migration paths have been provided for most common
1530usages in kernel recipes but this might not work in all cases. In
1531particular, users need to ensure that ``${S}`` (source files) and
1532``${B}`` (build artifacts) are used correctly in functions such as
1533```do_configure`` <#ref-tasks-configure>`__ and
1534```do_install`` <#ref-tasks-install>`__. For kernel recipes that do not
1535inherit from ``kernel-yocto`` or include ``linux-yocto.inc``, you might
1536wish to refer to the ``linux.inc`` file in the ``meta-oe`` layer for the
1537kinds of changes you need to make. For reference, here is the
1538`commit <http://cgit.openembedded.org/meta-openembedded/commit/meta-oe/recipes-kernel/linux/linux.inc?id=fc7132ede27ac67669448d3d2845ce7d46c6a1ee>`__
1539where the ``linux.inc`` file in ``meta-oe`` was updated.
1540
1541Recipes that rely on the kernel source code and do not inherit the
1542module classes might need to add explicit dependencies on the
1543``do_shared_workdir`` kernel task, for example: do_configure[depends] +=
1544"virtual/kernel:do_shared_workdir"
1545
1546.. _migration-1.8-ssl:
1547
1548SSL 3.0 is Now Disabled in OpenSSL
1549----------------------------------
1550
1551SSL 3.0 is now disabled when building OpenSSL. Disabling SSL 3.0 avoids
1552any lingering instances of the POODLE vulnerability. If you feel you
1553must re-enable SSL 3.0, then you can add an append file (``*.bbappend``)
1554for the ``openssl`` recipe to remove "-no-ssl3" from
1555```EXTRA_OECONF`` <#var-EXTRA_OECONF>`__.
1556
1557.. _migration-1.8-default-sysroot-poisoning:
1558
1559Default Sysroot Poisoning
1560-------------------------
1561
1562``gcc's`` default sysroot and include directories are now "poisoned". In
1563other words, the sysroot and include directories are being redirected to
1564a non-existent location in order to catch when host directories are
1565being used due to the correct options not being passed. This poisoning
1566applies both to the cross-compiler used within the build and to the
1567cross-compiler produced in the SDK.
1568
1569If this change causes something in the build to fail, it almost
1570certainly means the various compiler flags and commands are not being
1571passed correctly to the underlying piece of software. In such cases, you
1572need to take corrective steps.
1573
1574.. _migration-1.8-rebuild-improvements:
1575
1576Rebuild Improvements
1577--------------------
1578
1579Changes have been made to the ```base`` <#ref-classes-base>`__,
1580```autotools`` <#ref-classes-autotools>`__, and
1581```cmake`` <#ref-classes-cmake>`__ classes to clean out generated files
1582when the ```do_configure`` <#ref-tasks-configure>`__ task needs to be
1583re-executed.
1584
1585One of the improvements is to attempt to run "make clean" during the
1586``do_configure`` task if a ``Makefile`` exists. Some software packages
1587do not provide a working clean target within their make files. If you
1588have such recipes, you need to set
1589```CLEANBROKEN`` <#var-CLEANBROKEN>`__ to "1" within the recipe, for
1590example: CLEANBROKEN = "1"
1591
1592.. _migration-1.8-qa-check-and-validation-changes:
1593
1594QA Check and Validation Changes
1595-------------------------------
1596
1597The following QA Check and Validation Changes have occurred:
1598
1599- Usage of ``PRINC`` previously triggered a warning. It now triggers an
1600 error. You should remove any remaining usage of ``PRINC`` in any
1601 recipe or append file.
1602
1603- An additional QA check has been added to detect usage of ``${D}`` in
1604 ```FILES`` <#var-FILES>`__ values where ```D`` <#var-D>`__ values
1605 should not be used at all. The same check ensures that ``$D`` is used
1606 in ``pkg_preinst/pkg_postinst/pkg_prerm/pkg_postrm`` functions
1607 instead of ``${D}``.
1608
1609- ```S`` <#var-S>`__ now needs to be set to a valid value within a
1610 recipe. If ``S`` is not set in the recipe, the directory is not
1611 automatically created. If ``S`` does not point to a directory that
1612 exists at the time the ```do_unpack`` <#ref-tasks-unpack>`__ task
1613 finishes, a warning will be shown.
1614
1615- ```LICENSE`` <#var-LICENSE>`__ is now validated for correct
1616 formatting of multiple licenses. If the format is invalid (e.g.
1617 multiple licenses are specified with no operators to specify how the
1618 multiple licenses interact), then a warning will be shown.
1619
1620.. _migration-1.8-miscellaneous-changes:
1621
1622Miscellaneous Changes
1623---------------------
1624
1625The following miscellaneous changes have occurred:
1626
1627- The ``send-error-report`` script now expects a "-s" option to be
1628 specified before the server address. This assumes a server address is
1629 being specified.
1630
1631- The ``oe-pkgdata-util`` script now expects a "-p" option to be
1632 specified before the ``pkgdata`` directory, which is now optional. If
1633 the ``pkgdata`` directory is not specified, the script will run
1634 BitBake to query ```PKGDATA_DIR`` <#var-PKGDATA_DIR>`__ from the
1635 build environment.
1636
1637Moving to the Yocto Project 2.0 Release
1638=======================================
1639
1640This section provides migration information for moving to the Yocto
1641Project 2.0 Release from the prior release.
1642
1643.. _migration-2.0-gcc-5:
1644
1645GCC 5
1646-----
1647
1648The default compiler is now GCC 5.2. This change has required fixes for
1649compilation errors in a number of other recipes.
1650
1651One important example is a fix for when the Linux kernel freezes at boot
1652time on ARM when built with GCC 5. If you are using your own kernel
1653recipe or source tree and building for ARM, you will likely need to
1654apply this
1655`patch <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a077224fd35b2f7fbc93f14cf67074fc792fbac2>`__.
1656The standard ``linux-yocto`` kernel source tree already has a workaround
1657for the same issue.
1658
1659For further details, see ` <https://gcc.gnu.org/gcc-5/changes.html>`__
1660and the porting guide at
1661` <https://gcc.gnu.org/gcc-5/porting_to.html>`__.
1662
1663Alternatively, you can switch back to GCC 4.9 or 4.8 by setting
1664``GCCVERSION`` in your configuration, as follows: GCCVERSION = "4.9%"
1665
1666.. _migration-2.0-Gstreamer-0.10-removed:
1667
1668Gstreamer 0.10 Removed
1669----------------------
1670
1671Gstreamer 0.10 has been removed in favor of Gstreamer 1.x. As part of
1672the change, recipes for Gstreamer 0.10 and related software are now
1673located in ``meta-multimedia``. This change results in Qt4 having Phonon
1674and Gstreamer support in QtWebkit disabled by default.
1675
1676.. _migration-2.0-removed-recipes:
1677
1678Removed Recipes
1679---------------
1680
1681The following recipes have been moved or removed:
1682
1683- ``bluez4``: The recipe is obsolete and has been moved due to
1684 ``bluez5`` becoming fully integrated. The ``bluez4`` recipe now
1685 resides in ``meta-oe``.
1686
1687- ``gamin``: The recipe is obsolete and has been removed.
1688
1689- ``gnome-icon-theme``: The recipe's functionally has been replaced by
1690 ``adwaita-icon-theme``.
1691
1692- Gstreamer 0.10 Recipes: Recipes for Gstreamer 0.10 have been removed
1693 in favor of the recipes for Gstreamer 1.x.
1694
1695- ``insserv``: The recipe is obsolete and has been removed.
1696
1697- ``libunique``: The recipe is no longer used and has been moved to
1698 ``meta-oe``.
1699
1700- ``midori``: The recipe's functionally has been replaced by
1701 ``epiphany``.
1702
1703- ``python-gst``: The recipe is obsolete and has been removed since it
1704 only contains bindings for Gstreamer 0.10.
1705
1706- ``qt-mobility``: The recipe is obsolete and has been removed since it
1707 requires ``Gstreamer 0.10``, which has been replaced.
1708
1709- ``subversion``: All 1.6.x versions of this recipe have been removed.
1710
1711- ``webkit-gtk``: The older 1.8.3 version of this recipe has been
1712 removed in favor of ``webkitgtk``.
1713
1714.. _migration-2.0-bitbake-datastore-improvements:
1715
1716BitBake datastore improvements
1717------------------------------
1718
1719The method by which BitBake's datastore handles overrides has changed.
1720Overrides are now applied dynamically and ``bb.data.update_data()`` is
1721now a no-op. Thus, ``bb.data.update_data()`` is no longer required in
1722order to apply the correct overrides. In practice, this change is
1723unlikely to require any changes to Metadata. However, these minor
1724changes in behavior exist:
1725
1726- All potential overrides are now visible in the variable history as
1727 seen when you run the following: $ bitbake -e
1728
1729- ``d.delVar('``\ VARNAME\ ``')`` and
1730 ``d.setVar('``\ VARNAME\ ``', None)`` result in the variable and all
1731 of its overrides being cleared out. Before the change, only the
1732 non-overridden values were cleared.
1733
1734.. _migration-2.0-shell-message-function-changes:
1735
1736Shell Message Function Changes
1737------------------------------
1738
1739The shell versions of the BitBake message functions (i.e. ``bbdebug``,
1740``bbnote``, ``bbwarn``, ``bbplain``, ``bberror``, and ``bbfatal``) are
1741now connected through to their BitBake equivalents ``bb.debug()``,
1742``bb.note()``, ``bb.warn()``, ``bb.plain()``, ``bb.error()``, and
1743``bb.fatal()``, respectively. Thus, those message functions that you
1744would expect to be printed by the BitBake UI are now actually printed.
1745In practice, this change means two things:
1746
1747- If you now see messages on the console that you did not previously
1748 see as a result of this change, you might need to clean up the calls
1749 to ``bbwarn``, ``bberror``, and so forth. Or, you might want to
1750 simply remove the calls.
1751
1752- The ``bbfatal`` message function now suppresses the full error log in
1753 the UI, which means any calls to ``bbfatal`` where you still wish to
1754 see the full error log should be replaced by ``die`` or
1755 ``bbfatal_log``.
1756
1757.. _migration-2.0-extra-development-debug-package-cleanup:
1758
1759Extra Development/Debug Package Cleanup
1760---------------------------------------
1761
1762The following recipes have had extra ``dev/dbg`` packages removed:
1763
1764- ``acl``
1765
1766- ``apmd``
1767
1768- ``aspell``
1769
1770- ``attr``
1771
1772- ``augeas``
1773
1774- ``bzip2``
1775
1776- ``cogl``
1777
1778- ``curl``
1779
1780- ``elfutils``
1781
1782- ``gcc-target``
1783
1784- ``libgcc``
1785
1786- ``libtool``
1787
1788- ``libxmu``
1789
1790- ``opkg``
1791
1792- ``pciutils``
1793
1794- ``rpm``
1795
1796- ``sysfsutils``
1797
1798- ``tiff``
1799
1800- ``xz``
1801
1802All of the above recipes now conform to the standard packaging scheme
1803where a single ``-dev``, ``-dbg``, and ``-staticdev`` package exists per
1804recipe.
1805
1806.. _migration-2.0-recipe-maintenance-tracking-data-moved-to-oe-core:
1807
1808Recipe Maintenance Tracking Data Moved to OE-Core
1809-------------------------------------------------
1810
1811Maintenance tracking data for recipes that was previously part of
1812``meta-yocto`` has been moved to `OE-Core <#oe-core>`__. The change
1813includes ``package_regex.inc`` and ``distro_alias.inc``, which are
1814typically enabled when using the ``distrodata`` class. Additionally, the
1815contents of ``upstream_tracking.inc`` has now been split out to the
1816relevant recipes.
1817
1818.. _migration-2.0-automatic-stale-sysroot-file-cleanup:
1819
1820Automatic Stale Sysroot File Cleanup
1821------------------------------------
1822
1823Stale files from recipes that no longer exist in the current
1824configuration are now automatically removed from sysroot as well as
1825removed from any other place managed by shared state. This automatic
1826cleanup means that the build system now properly handles situations such
1827as renaming the build system side of recipes, removal of layers from
1828``bblayers.conf``, and ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__
1829changes.
1830
1831Additionally, work directories for old versions of recipes are now
1832pruned. If you wish to disable pruning old work directories, you can set
1833the following variable in your configuration:
1834SSTATE_PRUNE_OBSOLETEWORKDIR = "0"
1835
1836.. _migration-2.0-linux-yocto-kernel-metadata-repository-now-split-from-source:
1837
1838``linux-yocto`` Kernel Metadata Repository Now Split from Source
1839----------------------------------------------------------------
1840
1841The ``linux-yocto`` tree has up to now been a combined set of kernel
1842changes and configuration (meta) data carried in a single tree. While
1843this format is effective at keeping kernel configuration and source
1844modifications synchronized, it is not always obvious to developers how
1845to manipulate the Metadata as compared to the source.
1846
1847Metadata processing has now been removed from the
1848```kernel-yocto`` <#ref-classes-kernel-yocto>`__ class and the external
1849Metadata repository ``yocto-kernel-cache``, which has always been used
1850to seed the ``linux-yocto`` "meta" branch. This separate ``linux-yocto``
1851cache repository is now the primary location for this data. Due to this
1852change, ``linux-yocto`` is no longer able to process combined trees.
1853Thus, if you need to have your own combined kernel repository, you must
1854do the split there as well and update your recipes accordingly. See the
1855``meta/recipes-kernel/linux/linux-yocto_4.1.bb`` recipe for an example.
1856
1857.. _migration-2.0-additional-qa-checks:
1858
1859Additional QA checks
1860--------------------
1861
1862The following QA checks have been added:
1863
1864- Added a "host-user-contaminated" check for ownership issues for
1865 packaged files outside of ``/home``. The check looks for files that
1866 are incorrectly owned by the user that ran BitBake instead of owned
1867 by a valid user in the target system.
1868
1869- Added an "invalid-chars" check for invalid (non-UTF8) characters in
1870 recipe metadata variable values (i.e.
1871 ```DESCRIPTION`` <#var-DESCRIPTION>`__,
1872 ```SUMMARY`` <#var-SUMMARY>`__, ```LICENSE`` <#var-LICENSE>`__, and
1873 ```SECTION`` <#var-SECTION>`__). Some package managers do not support
1874 these characters.
1875
1876- Added an "invalid-packageconfig" check for any options specified in
1877 ```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__ that do not match any
1878 ``PACKAGECONFIG`` option defined for the recipe.
1879
1880.. _migration-2.0-miscellaneous:
1881
1882Miscellaneous Changes
1883---------------------
1884
1885These additional changes exist:
1886
1887- ``gtk-update-icon-cache`` has been renamed to ``gtk-icon-utils``.
1888
1889- The ``tools-profile`` ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__
1890 item as well as its corresponding packagegroup and
1891 ``packagegroup-core-tools-profile`` no longer bring in ``oprofile``.
1892 Bringing in ``oprofile`` was originally added to aid compilation on
1893 resource-constrained targets. However, this aid has not been widely
1894 used and is not likely to be used going forward due to the more
1895 powerful target platforms and the existence of better
1896 cross-compilation tools.
1897
1898- The ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable's default
1899 value now specifies ``ext4`` instead of ``ext3``.
1900
1901- All support for the ``PRINC`` variable has been removed.
1902
1903- The ``packagegroup-core-full-cmdline`` packagegroup no longer brings
1904 in ``lighttpd`` due to the fact that bringing in ``lighttpd`` is not
1905 really in line with the packagegroup's purpose, which is to add full
1906 versions of command-line tools that by default are provided by
1907 ``busybox``.
1908
1909Moving to the Yocto Project 2.1 Release
1910=======================================
1911
1912This section provides migration information for moving to the Yocto
1913Project 2.1 Release from the prior release.
1914
1915.. _migration-2.1-variable-expansion-in-python-functions:
1916
1917Variable Expansion in Python Functions
1918--------------------------------------
1919
1920Variable expressions, such as ``${``\ VARNAME\ ``}`` no longer expand
1921automatically within Python functions. Suppressing expansion was done to
1922allow Python functions to construct shell scripts or other code for
1923situations in which you do not want such expressions expanded. For any
1924existing code that relies on these expansions, you need to change the
1925expansions to expand the value of individual variables through
1926``d.getVar()``. To alternatively expand more complex expressions, use
1927``d.expand()``.
1928
1929.. _migration-2.1-overrides-must-now-be-lower-case:
1930
1931Overrides Must Now be Lower-Case
1932--------------------------------
1933
1934The convention for overrides has always been for them to be lower-case
1935characters. This practice is now a requirement as BitBake's datastore
1936now assumes lower-case characters in order to give a slight performance
1937boost during parsing. In practical terms, this requirement means that
1938anything that ends up in ```OVERRIDES`` <#var-OVERRIDES>`__ must now
1939appear in lower-case characters (e.g. values for ``MACHINE``,
1940``TARGET_ARCH``, ``DISTRO``, and also recipe names if
1941``_pn-``\ recipename overrides are to be effective).
1942
1943.. _migration-2.1-expand-parameter-to-getvar-and-getvarflag-now-mandatory:
1944
1945Expand Parameter to ``getVar()`` and ``getVarFlag()`` is Now Mandatory
1946----------------------------------------------------------------------
1947
1948The expand parameter to ``getVar()`` and ``getVarFlag()`` previously
1949defaulted to False if not specified. Now, however, no default exists so
1950one must be specified. You must change any ``getVar()`` calls that do
1951not specify the final expand parameter to calls that do specify the
1952parameter. You can run the following ``sed`` command at the base of a
1953layer to make this change: sed -e 's:\(\.getVar([^,()]*\)):\1, False):g'
1954-i \`grep -ril getVar \*\` sed -e 's:\(\.getVarFlag([^,()]*,
1955[^,()]*\)):\1, False):g' -i \`grep -ril getVarFlag \*\`
1956
1957.. note::
1958
1959 The reason for this change is that it prepares the way for changing
1960 the default to True in a future Yocto Project release. This future
1961 change is a much more sensible default than False. However, the
1962 change needs to be made gradually as a sudden change of the default
1963 would potentially cause side-effects that would be difficult to
1964 detect.
1965
1966.. _migration-2.1-makefile-environment-changes:
1967
1968Makefile Environment Changes
1969----------------------------
1970
1971```EXTRA_OEMAKE`` <#var-EXTRA_OEMAKE>`__ now defaults to "" instead of
1972"-e MAKEFLAGS=". Setting ``EXTRA_OEMAKE`` to "-e MAKEFLAGS=" by default
1973was a historical accident that has required many classes (e.g.
1974``autotools``, ``module``) and recipes to override this default in order
1975to work with sensible build systems. When upgrading to the release, you
1976must edit any recipe that relies upon this old default by either setting
1977``EXTRA_OEMAKE`` back to "-e MAKEFLAGS=" or by explicitly setting any
1978required variable value overrides using ``EXTRA_OEMAKE``, which is
1979typically only needed when a Makefile sets a default value for a
1980variable that is inappropriate for cross-compilation using the "="
1981operator rather than the "?=" operator.
1982
1983.. _migration-2.1-libexecdir-reverted-to-prefix-libexec:
1984
1985``libexecdir`` Reverted to ``${prefix}/libexec``
1986------------------------------------------------
1987
1988The use of ``${libdir}/${BPN}`` as ``libexecdir`` is different as
1989compared to all other mainstream distributions, which either uses
1990``${prefix}/libexec`` or ``${libdir}``. The use is also contrary to the
1991GNU Coding Standards (i.e.
1992` <https://www.gnu.org/prep/standards/html_node/Directory-Variables.html>`__)
1993that suggest ``${prefix}/libexec`` and also notes that any
1994package-specific nesting should be done by the package itself. Finally,
1995having ``libexecdir`` change between recipes makes it very difficult for
1996different recipes to invoke binaries that have been installed into
1997``libexecdir``. The Filesystem Hierarchy Standard (i.e.
1998` <http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html>`__) now
1999recognizes the use of ``${prefix}/libexec/``, giving distributions the
2000choice between ``${prefix}/lib`` or ``${prefix}/libexec`` without
2001breaking FHS.
2002
2003.. _migration-2.1-ac-cv-sizeof-off-t-no-longer-cached-in-site-files:
2004
2005``ac_cv_sizeof_off_t`` is No Longer Cached in Site Files
2006--------------------------------------------------------
2007
2008For recipes inheriting the ```autotools`` <#ref-classes-autotools>`__
2009class, ``ac_cv_sizeof_off_t`` is no longer cached in the site files for
2010``autoconf``. The reason for this change is because the
2011``ac_cv_sizeof_off_t`` value is not necessarily static per architecture
2012as was previously assumed. Rather, the value changes based on whether
2013large file support is enabled. For most software that uses ``autoconf``,
2014this change should not be a problem. However, if you have a recipe that
2015bypasses the standard ```do_configure`` <#ref-tasks-configure>`__ task
2016from the ``autotools`` class and the software the recipe is building
2017uses a very old version of ``autoconf``, the recipe might be incapable
2018of determining the correct size of ``off_t`` during ``do_configure``.
2019
2020The best course of action is to patch the software as necessary to allow
2021the default implementation from the ``autotools`` class to work such
2022that ``autoreconf`` succeeds and produces a working configure script,
2023and to remove the overridden ``do_configure`` task such that the default
2024implementation does get used.
2025
2026.. _migration-2.1-image-generation-split-out-from-filesystem-generation:
2027
2028Image Generation is Now Split Out from Filesystem Generation
2029------------------------------------------------------------
2030
2031Previously, for image recipes the ```do_rootfs`` <#ref-tasks-rootfs>`__
2032task assembled the filesystem and then from that filesystem generated
2033images. With this Yocto Project release, image generation is split into
2034separate ```do_image_*`` <#ref-tasks-image>`__ tasks for clarity both in
2035operation and in the code.
2036
2037For most cases, this change does not present any problems. However, if
2038you have made customizations that directly modify the ``do_rootfs`` task
2039or that mention ``do_rootfs``, you might need to update those changes.
2040In particular, if you had added any tasks after ``do_rootfs``, you
2041should make edits so that those tasks are after the
2042```do_image_complete`` <#ref-tasks-image-complete>`__ task rather than
2043after ``do_rootfs`` so that the your added tasks run at the correct
2044time.
2045
2046A minor part of this restructuring is that the post-processing
2047definitions and functions have been moved from the
2048```image`` <#ref-classes-image>`__ class to the
2049```rootfs-postcommands`` <#ref-classes-rootfs*>`__ class. Functionally,
2050however, they remain unchanged.
2051
2052.. _migration-2.1-removed-recipes:
2053
2054Removed Recipes
2055---------------
2056
2057The following recipes have been removed in the 2.1 release:
2058
2059- ``gcc`` version 4.8: Versions 4.9 and 5.3 remain.
2060
2061- ``qt4``: All support for Qt 4.x has been moved out to a separate
2062 ``meta-qt4`` layer because Qt 4 is no longer supported upstream.
2063
2064- ``x11vnc``: Moved to the ``meta-oe`` layer.
2065
2066- ``linux-yocto-3.14``: No longer supported.
2067
2068- ``linux-yocto-3.19``: No longer supported.
2069
2070- ``libjpeg``: Replaced by the ``libjpeg-turbo`` recipe.
2071
2072- ``pth``: Became obsolete.
2073
2074- ``liboil``: Recipe is no longer needed and has been moved to the
2075 ``meta-multimedia`` layer.
2076
2077- ``gtk-theme-torturer``: Recipe is no longer needed and has been moved
2078 to the ``meta-gnome`` layer.
2079
2080- ``gnome-mime-data``: Recipe is no longer needed and has been moved to
2081 the ``meta-gnome`` layer.
2082
2083- ``udev``: Replaced by the ``eudev`` recipe for compatibility when
2084 using ``sysvinit`` with newer kernels.
2085
2086- ``python-pygtk``: Recipe became obsolete.
2087
2088- ``adt-installer``: Recipe became obsolete. See the "`ADT
2089 Removed <#migration-2.1-adt-removed>`__" section for more
2090 information.
2091
2092.. _migration-2.1-class-changes:
2093
2094Class Changes
2095-------------
2096
2097The following classes have changed:
2098
2099- ``autotools_stage``: Removed because the
2100 ```autotools`` <#ref-classes-autotools>`__ class now provides its
2101 functionality. Recipes that inherited from ``autotools_stage`` should
2102 now inherit from ``autotools`` instead.
2103
2104- ``boot-directdisk``: Merged into the ``image-vm`` class. The
2105 ``boot-directdisk`` class was rarely directly used. Consequently,
2106 this change should not cause any issues.
2107
2108- ``bootimg``: Merged into the
2109 ```image-live`` <#ref-classes-image-live>`__ class. The ``bootimg``
2110 class was rarely directly used. Consequently, this change should not
2111 cause any issues.
2112
2113- ``packageinfo``: Removed due to its limited use by the Hob UI, which
2114 has itself been removed.
2115
2116.. _migration-2.1-build-system-ui-changes:
2117
2118Build System User Interface Changes
2119-----------------------------------
2120
2121The following changes have been made to the build system user interface:
2122
2123- *Hob GTK+-based UI*: Removed because it is unmaintained and based on
2124 the outdated GTK+ 2 library. The Toaster web-based UI is much more
2125 capable and is actively maintained. See the "`Using the Toaster Web
2126 Interface <&YOCTO_DOCS_TOAST_URL;#using-the-toaster-web-interface>`__"
2127 section in the Toaster User Manual for more information on this
2128 interface.
2129
2130- *"puccho" BitBake UI*: Removed because is unmaintained and no longer
2131 useful.
2132
2133.. _migration-2.1-adt-removed:
2134
2135ADT Removed
2136-----------
2137
2138The Application Development Toolkit (ADT) has been removed because its
2139functionality almost completely overlapped with the `standard
2140SDK <&YOCTO_DOCS_SDK_URL;#sdk-using-the-standard-sdk>`__ and the
2141`extensible SDK <&YOCTO_DOCS_SDK_URL;#sdk-extensible>`__. For
2142information on these SDKs and how to build and use them, see the `Yocto
2143Project Application Development and the Extensible Software Development
2144Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
2145
2146.. note::
2147
2148 The Yocto Project Eclipse IDE Plug-in is still supported and is not
2149 affected by this change.
2150
2151.. _migration-2.1-poky-reference-distribution-changes:
2152
2153Poky Reference Distribution Changes
2154-----------------------------------
2155
2156The following changes have been made for the Poky distribution:
2157
2158- The ``meta-yocto`` layer has been renamed to ``meta-poky`` to better
2159 match its purpose, which is to provide the Poky reference
2160 distribution. The ``meta-yocto-bsp`` layer retains its original name
2161 since it provides reference machines for the Yocto Project and it is
2162 otherwise unrelated to Poky. References to ``meta-yocto`` in your
2163 ``conf/bblayers.conf`` should automatically be updated, so you should
2164 not need to change anything unless you are relying on this naming
2165 elsewhere.
2166
2167- The ```uninative`` <#ref-classes-uninative>`__ class is now enabled
2168 by default in Poky. This class attempts to isolate the build system
2169 from the host distribution's C library and makes re-use of native
2170 shared state artifacts across different host distributions practical.
2171 With this class enabled, a tarball containing a pre-built C library
2172 is downloaded at the start of the build.
2173
2174 The ``uninative`` class is enabled through the
2175 ``meta/conf/distro/include/yocto-uninative.inc`` file, which for
2176 those not using the Poky distribution, can include to easily enable
2177 the same functionality.
2178
2179 Alternatively, if you wish to build your own ``uninative`` tarball,
2180 you can do so by building the ``uninative-tarball`` recipe, making it
2181 available to your build machines (e.g. over HTTP/HTTPS) and setting a
2182 similar configuration as the one set by ``yocto-uninative.inc``.
2183
2184- Static library generation, for most cases, is now disabled by default
2185 in the Poky distribution. Disabling this generation saves some build
2186 time as well as the size used for build output artifacts.
2187
2188 Disabling this library generation is accomplished through a
2189 ``meta/conf/distro/include/no-static-libs.inc``, which for those not
2190 using the Poky distribution can easily include to enable the same
2191 functionality.
2192
2193 Any recipe that needs to opt-out of having the "--disable-static"
2194 option specified on the configure command line either because it is
2195 not a supported option for the configure script or because static
2196 libraries are needed should set the following variable:
2197 DISABLE_STATIC = ""
2198
2199- The separate ``poky-tiny`` distribution now uses the musl C library
2200 instead of a heavily pared down ``glibc``. Using musl results in a
2201 smaller distribution and facilitates much greater maintainability
2202 because musl is designed to have a small footprint.
2203
2204 If you have used ``poky-tiny`` and have customized the ``glibc``
2205 configuration you will need to redo those customizations with musl
2206 when upgrading to the new release.
2207
2208.. _migration-2.1-packaging-changes:
2209
2210Packaging Changes
2211-----------------
2212
2213The following changes have been made to packaging:
2214
2215- The ``runuser`` and ``mountpoint`` binaries, which were previously in
2216 the main ``util-linux`` package, have been split out into the
2217 ``util-linux-runuser`` and ``util-linux-mountpoint`` packages,
2218 respectively.
2219
2220- The ``python-elementtree`` package has been merged into the
2221 ``python-xml`` package.
2222
2223.. _migration-2.1-tuning-file-changes:
2224
2225Tuning File Changes
2226-------------------
2227
2228The following changes have been made to the tuning files:
2229
2230- The "no-thumb-interwork" tuning feature has been dropped from the ARM
2231 tune include files. Because interworking is required for ARM EABI,
2232 attempting to disable it through a tuning feature no longer makes
2233 sense.
2234
2235 .. note::
2236
2237 Support for ARM OABI was deprecated in gcc 4.7.
2238
2239- The ``tune-cortexm*.inc`` and ``tune-cortexr4.inc`` files have been
2240 removed because they are poorly tested. Until the OpenEmbedded build
2241 system officially gains support for CPUs without an MMU, these tuning
2242 files would probably be better maintained in a separate layer if
2243 needed.
2244
2245.. _migration-2.1-supporting-gobject-introspection:
2246
2247Supporting GObject Introspection
2248--------------------------------
2249
2250This release supports generation of GLib Introspective Repository (GIR)
2251files through GObject introspection, which is the standard mechanism for
2252accessing GObject-based software from runtime environments. You can
2253enable, disable, and test the generation of this data. See the
2254"`Enabling GObject Introspection
2255Support <&YOCTO_DOCS_DEV_URL;#enabling-gobject-introspection-support>`__"
2256section in the Yocto Project Development Tasks Manual for more
2257information.
2258
2259.. _migration-2.1-miscellaneous-changes:
2260
2261Miscellaneous Changes
2262---------------------
2263
2264These additional changes exist:
2265
2266- The minimum Git version has been increased to 1.8.3.1. If your host
2267 distribution does not provide a sufficiently recent version, you can
2268 install the buildtools, which will provide it. See the "`Required
2269 Git, tar, Python and gcc
2270 Versions <#required-git-tar-python-and-gcc-versions>`__" section for
2271 more information on the buildtools tarball.
2272
2273- The buggy and incomplete support for the RPM version 4 package
2274 manager has been removed. The well-tested and maintained support for
2275 RPM version 5 remains.
2276
2277- Previously, the following list of packages were removed if
2278 package-management was not in
2279 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__, regardless of any
2280 dependencies: update-rc.d base-passwd shadow update-alternatives
2281 run-postinsts With the Yocto Project 2.1 release, these packages are
2282 only removed if "read-only-rootfs" is in ``IMAGE_FEATURES``, since
2283 they might still be needed for a read-write image even in the absence
2284 of a package manager (e.g. if users need to be added, modified, or
2285 removed at runtime).
2286
2287- The
2288 ```devtool modify`` <&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component>`__
2289 command now defaults to extracting the source since that is most
2290 commonly expected. The "-x" or "--extract" options are now no-ops. If
2291 you wish to provide your own existing source tree, you will now need
2292 to specify either the "-n" or "--no-extract" options when running
2293 ``devtool modify``.
2294
2295- If the formfactor for a machine is either not supplied or does not
2296 specify whether a keyboard is attached, then the default is to assume
2297 a keyboard is attached rather than assume no keyboard. This change
2298 primarily affects the Sato UI.
2299
2300- The ``.debug`` directory packaging is now automatic. If your recipe
2301 builds software that installs binaries into directories other than
2302 the standard ones, you no longer need to take care of setting
2303 ``FILES_${PN}-dbg`` to pick up the resulting ``.debug`` directories
2304 as these directories are automatically found and added.
2305
2306- Inaccurate disk and CPU percentage data has been dropped from
2307 ``buildstats`` output. This data has been replaced with
2308 ``getrusage()`` data and corrected IO statistics. You will probably
2309 need to update any custom code that reads the ``buildstats`` data.
2310
2311- The ``meta/conf/distro/include/package_regex.inc`` is now deprecated.
2312 The contents of this file have been moved to individual recipes.
2313
2314 .. note::
2315
2316 Because this file will likely be removed in a future Yocto Project
2317 release, it is suggested that you remove any references to the
2318 file that might be in your configuration.
2319
2320- The ``v86d/uvesafb`` has been removed from the ``genericx86`` and
2321 ``genericx86-64`` reference machines, which are provided by the
2322 ``meta-yocto-bsp`` layer. Most modern x86 boards do not rely on this
2323 file and it only adds kernel error messages during startup. If you do
2324 still need to support ``uvesafb``, you can simply add ``v86d`` to
2325 your image.
2326
2327- Build sysroot paths are now removed from debug symbol files. Removing
2328 these paths means that remote GDB using an unstripped build system
2329 sysroot will no longer work (although this was never documented to
2330 work). The supported method to accomplish something similar is to set
2331 ``IMAGE_GEN_DEBUGFS`` to "1", which will generate a companion debug
2332 image containing unstripped binaries and associated debug sources
2333 alongside the image.
2334
2335Moving to the Yocto Project 2.2 Release
2336=======================================
2337
2338This section provides migration information for moving to the Yocto
2339Project 2.2 Release from the prior release.
2340
2341.. _migration-2.2-minimum-kernel-version:
2342
2343Minimum Kernel Version
2344----------------------
2345
2346The minimum kernel version for the target system and for SDK is now
23473.2.0, due to the upgrade to ``glibc 2.24``. Specifically, for
2348AArch64-based targets the version is 3.14. For Nios II-based targets,
2349the minimum kernel version is 3.19.
2350
2351.. note::
2352
2353 For x86 and x86_64, you can reset
2354 OLDEST_KERNEL
2355 to anything down to 2.6.32 if desired.
2356
2357.. _migration-2.2-staging-directories-in-sysroot-simplified:
2358
2359Staging Directories in Sysroot Has Been Simplified
2360--------------------------------------------------
2361
2362The way directories are staged in sysroot has been simplified and
2363introduces the new ```SYSROOT_DIRS`` <#var-SYSROOT_DIRS>`__,
2364```SYSROOT_DIRS_NATIVE`` <#var-SYSROOT_DIRS_NATIVE>`__, and
2365```SYSROOT_DIRS_BLACKLIST`` <#var-SYSROOT_DIRS_BLACKLIST>`__. See the
2366`v2 patch series on the OE-Core Mailing
2367List <http://lists.openembedded.org/pipermail/openembedded-core/2016-May/121365.html>`__
2368for additional information.
2369
2370.. _migration-2.2-removal-of-old-images-from-tmp-deploy-now-enabled:
2371
2372Removal of Old Images and Other Files in ``tmp/deploy`` Now Enabled
2373-------------------------------------------------------------------
2374
2375Removal of old images and other files in ``tmp/deploy/`` is now enabled
2376by default due to a new staging method used for those files. As a result
2377of this change, the ``RM_OLD_IMAGE`` variable is now redundant.
2378
2379.. _migration-2.2-python-changes:
2380
2381Python Changes
2382--------------
2383
2384The following changes for Python occurred:
2385
2386.. _migration-2.2-bitbake-now-requires-python-3.4:
2387
2388BitBake Now Requires Python 3.4+
2389~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2390
2391BitBake requires Python 3.4 or greater.
2392
2393.. _migration-2.2-utf-8-locale-required-on-build-host:
2394
2395UTF-8 Locale Required on Build Host
2396~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2397
2398A UTF-8 locale is required on the build host due to Python 3. Since
2399C.UTF-8 is not a standard, the default is en_US.UTF-8.
2400
2401.. _migration-2.2-metadata-now-must-use-python-3-syntax:
2402
2403Metadata Must Now Use Python 3 Syntax
2404~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2405
2406The metadata is now required to use Python 3 syntax. For help preparing
2407metadata, see any of the many Python 3 porting guides available.
2408Alternatively, you can reference the conversion commits for Bitbake and
2409you can use `OE-Core <#oe-core>`__ as a guide for changes. Following are
2410particular areas of interest: \* subprocess command-line pipes needing
2411locale decoding \* the syntax for octal values changed \* the
2412``iter*()`` functions changed name \* iterators now return views, not
2413lists \* changed names for Python modules
2414
2415.. _migration-2.2-target-python-recipes-switched-to-python-3:
2416
2417Target Python Recipes Switched to Python 3
2418~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2419
2420Most target Python recipes have now been switched to Python 3.
2421Unfortunately, systems using RPM as a package manager and providing
2422online package-manager support through SMART still require Python 2.
2423
2424.. note::
2425
2426 Python 2 and recipes that use it can still be built for the target as
2427 with previous versions.
2428
2429.. _migration-2.2-buildtools-tarball-includes-python-3:
2430
2431``buildtools-tarball`` Includes Python 3
2432~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2433
2434``buildtools-tarball`` now includes Python 3.
2435
2436.. _migration-2.2-uclibc-replaced-by-musl:
2437
2438uClibc Replaced by musl
2439-----------------------
2440
2441uClibc has been removed in favor of musl. Musl has matured, is better
2442maintained, and is compatible with a wider range of applications as
2443compared to uClibc.
2444
2445.. _migration-2.2-B-no-longer-default-working-directory-for-tasks:
2446
2447``${B}`` No Longer Default Working Directory for Tasks
2448------------------------------------------------------
2449
2450``${``\ ```B`` <#var-B>`__\ ``}`` is no longer the default working
2451directory for tasks. Consequently, any custom tasks you define now need
2452to either have the
2453``[``\ ```dirs`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__\ ``]`` flag
2454set, or the task needs to change into the appropriate working directory
2455manually (e.g using ``cd`` for a shell task).
2456
2457.. note::
2458
2459 The preferred method is to use the
2460 [dirs]
2461 flag.
2462
2463.. _migration-2.2-runqemu-ported-to-python:
2464
2465``runqemu`` Ported to Python
2466----------------------------
2467
2468``runqemu`` has been ported to Python and has changed behavior in some
2469cases. Previous usage patterns continue to be supported.
2470
2471The new ``runqemu`` is a Python script. Machine knowledge is no longer
2472hardcoded into ``runqemu``. You can choose to use the ``qemuboot``
2473configuration file to define the BSP's own arguments and to make it
2474bootable with ``runqemu``. If you use a configuration file, use the
2475following form: image-name-machine.qemuboot.conf The configuration file
2476enables fine-grained tuning of options passed to QEMU without the
2477``runqemu`` script hard-coding any knowledge about different machines.
2478Using a configuration file is particularly convenient when trying to use
2479QEMU with machines other than the ``qemu*`` machines in
2480`OE-Core <#oe-core>`__. The ``qemuboot.conf`` file is generated by the
2481``qemuboot`` class when the root filesystem is being build (i.e. build
2482rootfs). QEMU boot arguments can be set in BSP's configuration file and
2483the ``qemuboot`` class will save them to ``qemuboot.conf``.
2484
2485If you want to use ``runqemu`` without a configuration file, use the
2486following command form: $ runqemu machine rootfs kernel [options]
2487Supported machines are as follows: qemuarm qemuarm64 qemux86 qemux86-64
2488qemuppc qemumips qemumips64 qemumipsel qemumips64el Consider the
2489following example, which uses the ``qemux86-64`` machine, provides a
2490root filesystem, provides an image, and uses the ``nographic`` option: $
2491runqemu qemux86-64
2492tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4
2493tmp/deploy/images/qemux86-64/bzImage nographic
2494
2495Following is a list of variables that can be set in configuration files
2496such as ``bsp.conf`` to enable the BSP to be booted by ``runqemu``:
2497
2498.. note::
2499
2500 "QB" means "QEMU Boot".
2501
2502QB_SYSTEM_NAME: QEMU name (e.g. "qemu-system-i386") QB_OPT_APPEND:
2503Options to append to QEMU (e.g. "-show-cursor") QB_DEFAULT_KERNEL:
2504Default kernel to boot (e.g. "bzImage") QB_DEFAULT_FSTYPE: Default
2505FSTYPE to boot (e.g. "ext4") QB_MEM: Memory (e.g. "-m 512") QB_MACHINE:
2506QEMU machine (e.g. "-machine virt") QB_CPU: QEMU cpu (e.g. "-cpu
2507qemu32") QB_CPU_KVM: Similar to QB_CPU except used for kvm support (e.g.
2508"-cpu kvm64") QB_KERNEL_CMDLINE_APPEND: Options to append to the
2509kernel's -append option (e.g. "console=ttyS0 console=tty") QB_DTB: QEMU
2510dtb name QB_AUDIO_DRV: QEMU audio driver (e.g. "alsa", set it when
2511support audio) QB_AUDIO_OPT: QEMU audio option (e.g. "-soundhw
2512ac97,es1370"), which is used when QB_AUDIO_DRV is set. QB_KERNEL_ROOT:
2513Kernel's root (e.g. /dev/vda) QB_TAP_OPT: Network option for 'tap' mode
2514(e.g. "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device
2515virtio-net-device,netdev=net0"). runqemu will replace "@TAP@" with the
2516one that is used, such as tap0, tap1 ... QB_SLIRP_OPT: Network option
2517for SLIRP mode (e.g. "-netdev user,id=net0 -device
2518virtio-net-device,netdev=net0") QB_ROOTFS_OPT: Used as rootfs (e.g.
2519"-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device
2520virtio-blk-device,drive=disk0"). runqemu will replace "@ROOTFS@" with
2521the one which is used, such as core-image-minimal-qemuarm64.ext4.
2522QB_SERIAL_OPT: Serial port (e.g. "-serial mon:stdio") QB_TCPSERIAL_OPT:
2523tcp serial port option (e.g. " -device virtio-serial-device -chardev
2524socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device
2525virtconsole,chardev=virtcon" runqemu will replace "@PORT@" with the port
2526number which is used.
2527
2528To use ``runqemu``, set ```IMAGE_CLASSES`` <#var-IMAGE_CLASSES>`__ as
2529follows and run ``runqemu``:
2530
2531.. note::
2532
2533 For command-line syntax, use
2534 runqemu help
2535 .
2536
2537IMAGE_CLASSES += "qemuboot"
2538
2539.. _migration-2.2-default-linker-hash-style-changed:
2540
2541Default Linker Hash Style Changed
2542---------------------------------
2543
2544The default linker hash style for ``gcc-cross`` is now "sysv" in order
2545to catch recipes that are building software without using the
2546OpenEmbedded ```LDFLAGS`` <#var-LDFLAGS>`__. This change could result in
2547seeing some "No GNU_HASH in the elf binary" QA issues when building such
2548recipes. You need to fix these recipes so that they use the expected
2549``LDFLAGS``. Depending on how the software is built, the build system
2550used by the software (e.g. a Makefile) might need to be patched.
2551However, sometimes making this fix is as simple as adding the following
2552to the recipe: TARGET_CC_ARCH += "${LDFLAGS}"
2553
2554.. _migration-2.2-kernel-image-base-name-no-longer-uses-kernel-imagetype:
2555
2556``KERNEL_IMAGE_BASE_NAME`` no Longer Uses ``KERNEL_IMAGETYPE``
2557--------------------------------------------------------------
2558
2559The ``KERNEL_IMAGE_BASE_NAME`` variable no longer uses the
2560```KERNEL_IMAGETYPE`` <#var-KERNEL_IMAGETYPE>`__ variable to create the
2561image's base name. Because the OpenEmbedded build system can now build
2562multiple kernel image types, this part of the kernel image base name as
2563been removed leaving only the following: KERNEL_IMAGE_BASE_NAME ?=
2564"${PKGE}-${PKGV}-${PKGR}-${MACHINE}-${DATETIME}" If you have recipes or
2565classes that use ``KERNEL_IMAGE_BASE_NAME`` directly, you might need to
2566update the references to ensure they continue to work.
2567
2568.. _migration-2.2-bitbake-changes:
2569
2570BitBake Changes
2571---------------
2572
2573The following changes took place for BitBake:
2574
2575- The "goggle" UI and standalone image-writer tool have been removed as
2576 they both require GTK+ 2.0 and were not being maintained.
2577
2578- The Perforce fetcher now supports ```SRCREV`` <#var-SRCREV>`__ for
2579 specifying the source revision to use, be it
2580 ``${``\ ```AUTOREV`` <#var-AUTOREV>`__\ ``}``, changelist number,
2581 p4date, or label, in preference to separate
2582 ```SRC_URI`` <#var-SRC_URI>`__ parameters to specify these. This
2583 change is more in-line with how the other fetchers work for source
2584 control systems. Recipes that fetch from Perforce will need to be
2585 updated to use ``SRCREV`` in place of specifying the source revision
2586 within ``SRC_URI``.
2587
2588- Some of BitBake's internal code structures for accessing the recipe
2589 cache needed to be changed to support the new multi-configuration
2590 functionality. These changes will affect external tools that use
2591 BitBake's tinfoil module. For information on these changes, see the
2592 changes made to the scripts supplied with OpenEmbedded-Core:
2593 `1 <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=189371f8393971d00bca0fceffd67cc07784f6ee>`__
2594 and
2595 `2 <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=4a5aa7ea4d07c2c90a1654b174873abb018acc67>`__.
2596
2597- The task management code has been rewritten to avoid using ID
2598 indirection in order to improve performance. This change is unlikely
2599 to cause any problems for most users. However, the setscene
2600 verification function as pointed to by
2601 ``BB_SETSCENE_VERIFY_FUNCTION`` needed to change signature.
2602 Consequently, a new variable named ``BB_SETSCENE_VERIFY_FUNCTION2``
2603 has been added allowing multiple versions of BitBake to work with
2604 suitably written metadata, which includes OpenEmbedded-Core and Poky.
2605 Anyone with custom BitBake task scheduler code might also need to
2606 update the code to handle the new structure.
2607
2608.. _migration-2.2-swabber-has-been-removed:
2609
2610Swabber has Been Removed
2611------------------------
2612
2613Swabber, a tool that was intended to detect host contamination in the
2614build process, has been removed, as it has been unmaintained and unused
2615for some time and was never particularly effective. The OpenEmbedded
2616build system has since incorporated a number of mechanisms including
2617enhanced QA checks that mean that there is less of a need for such a
2618tool.
2619
2620.. _migration-2.2-removed-recipes:
2621
2622Removed Recipes
2623---------------
2624
2625The following recipes have been removed:
2626
2627- ``augeas``: No longer needed and has been moved to ``meta-oe``.
2628
2629- ``directfb``: Unmaintained and has been moved to ``meta-oe``.
2630
2631- ``gcc``: Removed 4.9 version. Versions 5.4 and 6.2 are still present.
2632
2633- ``gnome-doc-utils``: No longer needed.
2634
2635- ``gtk-doc-stub``: Replaced by ``gtk-doc``.
2636
2637- ``gtk-engines``: No longer needed and has been moved to
2638 ``meta-gnome``.
2639
2640- ``gtk-sato-engine``: Became obsolete.
2641
2642- ``libglade``: No longer needed and has been moved to ``meta-oe``.
2643
2644- ``libmad``: Unmaintained and functionally replaced by ``libmpg123``.
2645 ``libmad`` has been moved to ``meta-oe``.
2646
2647- ``libowl``: Became obsolete.
2648
2649- ``libxsettings-client``: No longer needed.
2650
2651- ``oh-puzzles``: Functionally replaced by ``puzzles``.
2652
2653- ``oprofileui``: Became obsolete. OProfile has been largely supplanted
2654 by perf.
2655
2656- ``packagegroup-core-directfb.bb``: Removed.
2657
2658- ``core-image-directfb.bb``: Removed.
2659
2660- ``pointercal``: No longer needed and has been moved to ``meta-oe``.
2661
2662- ``python-imaging``: No longer needed and moved to ``meta-python``
2663
2664- ``python-pyrex``: No longer needed and moved to ``meta-python``.
2665
2666- ``sato-icon-theme``: Became obsolete.
2667
2668- ``swabber-native``: Swabber has been removed. See the `entry on
2669 Swabber <#migration-2.2-swabber-has-been-removed>`__.
2670
2671- ``tslib``: No longer needed and has been moved to ``meta-oe``.
2672
2673- ``uclibc``: Removed in favor of musl.
2674
2675- ``xtscal``: No longer needed and moved to ``meta-oe``
2676
2677.. _migration-2.2-removed-classes:
2678
2679Removed Classes
2680---------------
2681
2682The following classes have been removed:
2683
2684- ``distutils-native-base``: No longer needed.
2685
2686- ``distutils3-native-base``: No longer needed.
2687
2688- ``sdl``: Only set ```DEPENDS`` <#var-DEPENDS>`__ and
2689 ```SECTION`` <#var-SECTION>`__, which are better set within the
2690 recipe instead.
2691
2692- ``sip``: Mostly unused.
2693
2694- ``swabber``: See the `entry on
2695 Swabber <#migration-2.2-swabber-has-been-removed>`__.
2696
2697.. _migration-2.2-minor-packaging-changes:
2698
2699Minor Packaging Changes
2700-----------------------
2701
2702The following minor packaging changes have occurred:
2703
2704- ``grub``: Split ``grub-editenv`` into its own package.
2705
2706- ``systemd``: Split container and vm related units into a new package,
2707 systemd-container.
2708
2709- ``util-linux``: Moved ``prlimit`` to a separate
2710 ``util-linux-prlimit`` package.
2711
2712.. _migration-2.2-miscellaneous-changes:
2713
2714Miscellaneous Changes
2715---------------------
2716
2717The following miscellaneous changes have occurred:
2718
2719- ``package_regex.inc``: Removed because the definitions
2720 ``package_regex.inc`` previously contained have been moved to their
2721 respective recipes.
2722
2723- Both ``devtool add`` and ``recipetool create`` now use a fixed
2724 ```SRCREV`` <#var-SRCREV>`__ by default when fetching from a Git
2725 repository. You can override this in either case to use
2726 ``${``\ ```AUTOREV`` <#var-AUTOREV>`__\ ``}`` instead by using the
2727 ``-a`` or ``DASHDASHautorev`` command-line option
2728
2729- ``distcc``: GTK+ UI is now disabled by default.
2730
2731- ``packagegroup-core-tools-testapps``: Removed Piglit.
2732
2733- ``image.bbclass``: Renamed COMPRESS(ION) to CONVERSION. This change
2734 means that ``COMPRESSIONTYPES``, ``COMPRESS_DEPENDS`` and
2735 ``COMPRESS_CMD`` are deprecated in favor of ``CONVERSIONTYPES``,
2736 ``CONVERSION_DEPENDS`` and ``CONVERSION_CMD``. The ``COMPRESS*``
2737 variable names will still work in the 2.2 release but metadata that
2738 does not need to be backwards-compatible should be changed to use the
2739 new names as the ``COMPRESS*`` ones will be removed in a future
2740 release.
2741
2742- ``gtk-doc``: A full version of ``gtk-doc`` is now made available.
2743 However, some old software might not be capable of using the current
2744 version of ``gtk-doc`` to build documentation. You need to change
2745 recipes that build such software so that they explicitly disable
2746 building documentation with ``gtk-doc``.
2747
2748Moving to the Yocto Project 2.3 Release
2749=======================================
2750
2751This section provides migration information for moving to the Yocto
2752Project 2.3 Release from the prior release.
2753
2754.. _migration-2.3-recipe-specific-sysroots:
2755
2756Recipe-specific Sysroots
2757------------------------
2758
2759The OpenEmbedded build system now uses one sysroot per recipe to resolve
2760long-standing issues with configuration script auto-detection of
2761undeclared dependencies. Consequently, you might find that some of your
2762previously written custom recipes are missing declared dependencies,
2763particularly those dependencies that are incidentally built earlier in a
2764typical build process and thus are already likely to be present in the
2765shared sysroot in previous releases.
2766
2767Consider the following:
2768
2769- *Declare Build-Time Dependencies:* Because of this new feature, you
2770 must explicitly declare all build-time dependencies for your recipe.
2771 If you do not declare these dependencies, they are not populated into
2772 the sysroot for the recipe.
2773
2774- *Specify Pre-Installation and Post-Installation Native Tool
2775 Dependencies:* You must specifically specify any special native tool
2776 dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
2777 the ```PACKAGE_WRITE_DEPS`` <#var-PACKAGE_WRITE_DEPS>`__ variable.
2778 Specifying these dependencies ensures that these tools are available
2779 if these scripts need to be run on the build host during the
2780 ```do_rootfs`` <#ref-tasks-rootfs>`__ task.
2781
2782 As an example, see the ``dbus`` recipe. You will see that this recipe
2783 has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
2784 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__. In the example,
2785 ``systemd-systemctl-native`` is added to ``PACKAGE_WRITE_DEPS``,
2786 which is also conditional on "systemd" being in ``DISTRO_FEATURES``.
2787
2788- *Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``:* You need to
2789 examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
2790 steps to take.
2791
2792 Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
2793 were in previous Yocto Project releases. However, since a separate
2794 sysroot is now being populated for every recipe and if existing
2795 functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
2796 relocation, then you will need to change these to use a
2797 post-installation script that is installed by a function added to
2798 ```SYSROOT_PREPROCESS_FUNCS`` <#var-SYSROOT_PREPROCESS_FUNCS>`__.
2799
2800 For an example, see the ``pixbufcache`` class in ``meta/classes/`` in
2801 the Yocto Project `Source
2802 Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__.
2803
2804 .. note::
2805
2806 The
2807 SSTATEPOSTINSTFUNCS
2808 variable itself is now deprecated in favor of the
2809 do_populate_sysroot[postfuncs]
2810 task. Consequently, if you do still have any function or functions
2811 that need to be called after the sysroot component is created for
2812 a recipe, then you would be well advised to take steps to use a
2813 post installation script as described previously. Taking these
2814 steps prepares your code for when
2815 SSTATEPOSTINSTFUNCS
2816 is removed in a future Yocto Project release.
2817
2818- *Specify the Sysroot when Using Certain External Scripts:* Because
2819 the shared sysroot is now gone, the scripts
2820 ``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
2821 such that you need to specify which recipe's
2822 ```STAGING_DIR_NATIVE`` <#var-STAGING_DIR_NATIVE>`__ is used.
2823
2824.. note::
2825
2826 You can find more information on how recipe-specific sysroots work in
2827 the "
2828 staging.bbclass
2829 " section.
2830
2831.. _migration-2.3-path-variable:
2832
2833``PATH`` Variable
2834-----------------
2835
2836Within the environment used to run build tasks, the environment variable
2837``PATH`` is now sanitized such that the normal native binary paths
2838(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
2839directory containing symbolic links linking only to the binaries from
2840the host mentioned in the ```HOSTTOOLS`` <#var-HOSTTOOLS>`__ and
2841```HOSTTOOLS_NONFATAL`` <#var-HOSTTOOLS_NONFATAL>`__ variables is added
2842to ``PATH``.
2843
2844Consequently, any native binaries provided by the host that you need to
2845call needs to be in one of these two variables at the configuration
2846level.
2847
2848Alternatively, you can add a native recipe (i.e. ``-native``) that
2849provides the binary to the recipe's ```DEPENDS`` <#var-DEPENDS>`__
2850value.
2851
2852.. note::
2853
2854 PATH
2855 is not sanitized in the same way within
2856 devshell
2857 . If it were, you would have difficulty running host tools for
2858 development and debugging within the shell.
2859
2860.. _migration-2.3-scripts:
2861
2862Changes to Scripts
2863------------------
2864
2865The following changes to scripts took place:
2866
2867- *``oe-find-native-sysroot``:* The usage for the
2868 ``oe-find-native-sysroot`` script has changed to the following: $ .
2869 oe-find-native-sysroot recipe You must now supply a recipe for recipe
2870 as part of the command. Prior to the Yocto Project DISTRO release, it
2871 was not necessary to provide the script with the command.
2872
2873- *``oe-run-native``:* The usage for the ``oe-run-native`` script has
2874 changed to the following: $ oe-run-native native_recipe tool You must
2875 supply the name of the native recipe and the tool you want to run as
2876 part of the command. Prior to the Yocto Project DISTRO release, it
2877 was not necessary to provide the native recipe with the command.
2878
2879- *``cleanup-workdir``:* The ``cleanup-workdir`` script has been
2880 removed because the script was found to be deleting files it should
2881 not have, which lead to broken build trees. Rather than trying to
2882 delete portions of ```TMPDIR`` <#var-TMPDIR>`__ and getting it wrong,
2883 it is recommended that you delete ``TMPDIR`` and have it restored
2884 from shared state (sstate) on subsequent builds.
2885
2886- *``wipe-sysroot``:* The ``wipe-sysroot`` script has been removed as
2887 it is no longer needed with recipe-specific sysroots.
2888
2889.. _migration-2.3-functions:
2890
2891Changes to Functions
2892--------------------
2893
2894The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
2895and related functions have been removed in favor of ``d.getVar()``,
2896``d.setVar()``, and so forth.
2897
2898You need to fix any references to these old functions.
2899
2900.. _migration-2.3-bitbake-changes:
2901
2902BitBake Changes
2903---------------
2904
2905The following changes took place for BitBake:
2906
2907- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
2908 graphical dependency explorer UI ``depexp`` was replaced by
2909 ``taskexp`` ("Task Explorer"), which provides a graphical way of
2910 exploring the ``task-depends.dot`` file. The data presented by Task
2911 Explorer is much more accurate than the data that was presented by
2912 ``depexp``. Being able to visualize the data is an often requested
2913 feature as standard ``*.dot`` file viewers cannot usual cope with the
2914 size of the ``task-depends.dot`` file.
2915
2916- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
2917 ``pn-depends.dot`` files as previously generated using the
2918 ``bitbake -g`` command have been removed. A ``recipe-depends.dot``
2919 file is now generated as a collapsed version of ``task-depends.dot``
2920 instead.
2921
2922 The reason for this change is because ``package-depends.dot`` and
2923 ``pn-depends.dot`` largely date back to a time before task-based
2924 execution and do not take into account task-level dependencies
2925 between recipes, which could be misleading.
2926
2927- *Mirror Variable Splitting Changes:* Mirror variables including
2928 ```MIRRORS`` <#var-MIRRORS>`__, ```PREMIRRORS`` <#var-PREMIRRORS>`__,
2929 and ```SSTATE_MIRRORS`` <#var-SSTATE_MIRRORS>`__ can now separate
2930 values entirely with spaces. Consequently, you no longer need "\\n".
2931 BitBake looks for pairs of values, which simplifies usage. There
2932 should be no change required to existing mirror variable values
2933 themselves.
2934
2935- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
2936 "rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
2937 instead of an "rsh" parameter. This new optional parameter is used
2938 when the "protocol" parameter is set to "svn+ssh". You can only use
2939 the new parameter to specify the ``ssh`` program used by SVN. The SVN
2940 fetcher passes the new parameter through the ``SVN_SSH`` environment
2941 variable during the ```do_fetch`` <#ref-tasks-fetch>`__ task.
2942
2943 See the "`Subversion (SVN) Fetcher
2944 (svn://) <&YOCTO_DOCS_BB_URL;#svn-fetcher>`__" section in the BitBake
2945 User Manual for additional information.
2946
2947- *``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
2948 Removed:* Because the mechanism they were part of is no longer
2949 necessary with recipe-specific sysroots, the
2950 ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
2951 variables have been removed.
2952
2953.. _migration-2.3-absolute-symlinks:
2954
2955Absolute Symbolic Links
2956-----------------------
2957
2958Absolute symbolic links (symlinks) within staged files are no longer
2959permitted and now trigger an error. Any explicit creation of symlinks
2960can use the ``lnr`` script, which is a replacement for ``ln -r``.
2961
2962If the build scripts in the software that the recipe is building are
2963creating a number of absolute symlinks that need to be corrected, you
2964can inherit ``relative_symlinks`` within the recipe to turn those
2965absolute symlinks into relative symlinks.
2966
2967.. _migration-2.3-gplv2-and-gplv3-moves:
2968
2969GPLv2 Versions of GPLv3 Recipes Moved
2970-------------------------------------
2971
2972Older GPLv2 versions of GPLv3 recipes have moved to a separate
2973``meta-gplv2`` layer.
2974
2975If you use ```INCOMPATIBLE_LICENSE`` <#var-INCOMPATIBLE_LICENSE>`__ to
2976exclude GPLv3 or set ```PREFERRED_VERSION`` <#var-PREFERRED_VERSION>`__
2977to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
2978``meta-gplv2`` layer to your configuration.
2979
2980.. note::
2981
2982 You can find
2983 meta-gplv2
2984 layer in the OpenEmbedded layer index at
2985 .
2986
2987These relocated GPLv2 recipes do not receive the same level of
2988maintenance as other core recipes. The recipes do not get security fixes
2989and upstream no longer maintains them. In fact, the upstream community
2990is actively hostile towards people that use the old versions of the
2991recipes. Moving these recipes into a separate layer both makes the
2992different needs of the recipes clearer and clearly identifies the number
2993of these recipes.
2994
2995.. note::
2996
2997 The long-term solution might be to move to BSD-licensed replacements
2998 of the GPLv3 components for those that need to exclude GPLv3-licensed
2999 components from the target system. This solution will be investigated
3000 for future Yocto Project releases.
3001
3002.. _migration-2.3-package-management-changes:
3003
3004Package Management Changes
3005--------------------------
3006
3007The following package management changes took place:
3008
3009- Smart package manager is replaced by DNF package manager. Smart has
3010 become unmaintained upstream, is not ported to Python 3.x.
3011 Consequently, Smart needed to be replaced. DNF is the only feasible
3012 candidate.
3013
3014 The change in functionality is that the on-target runtime package
3015 management from remote package feeds is now done with a different
3016 tool that has a different set of command-line options. If you have
3017 scripts that call the tool directly, or use its API, they need to be
3018 fixed.
3019
3020 For more information, see the `DNF
3021 Documentation <http://dnf.readthedocs.io/en/latest/>`__.
3022
3023- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
3024
3025 - DNF is API-incompatible with Rpm 5.x and porting it and
3026 maintaining the port is non-trivial.
3027
3028 - Rpm 5.x itself has limited maintenance upstream, and the Yocto
3029 Project is one of the very few remaining users.
3030
3031- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
3032
3033 - Version 6.x of Berkeley DB has largely been rejected by the open
3034 source community due to its AGPLv3 license. As a result, most
3035 mainstream open source projects that require DB are still
3036 developed and tested with DB 5.x.
3037
3038 - In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
3039 Thus, no reason exists to continue carrying DB 6.x in OE-core.
3040
3041- ``createrepo`` is replaced with ``createrepo_c``.
3042
3043 ``createrepo_c`` is the current incarnation of the tool that
3044 generates remote repository metadata. It is written in C as compared
3045 to ``createrepo``, which is written in Python. ``createrepo_c`` is
3046 faster and is maintained.
3047
3048- Architecture-independent RPM packages are "noarch" instead of "all".
3049
3050 This change was made because too many places in DNF/RPM4 stack
3051 already make that assumption. Only the filenames and the architecture
3052 tag has changed. Nothing else has changed in OE-core system,
3053 particularly in the ```allarch.bbclass`` <#ref-classes-allarch>`__
3054 class.
3055
3056- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
3057 currently supported. This issue will be fully addressed in a future
3058 Yocto Project release. See `defect
3059 11209 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11209>`__
3060 for more information on a solution to package feed signing with RPM
3061 in the Yocto Project 2.3 release.
3062
3063- OPKG now uses the libsolv backend for resolving package dependencies
3064 by default. This is vastly superior to OPKG's internal ad-hoc solver
3065 that was previously used. This change does have a small impact on
3066 disk (around 500 KB) and memory footprint.
3067
3068 .. note::
3069
3070 For further details on this change, see the
3071 commit message
3072 .
3073
3074.. _migration-2.3-removed-recipes:
3075
3076Removed Recipes
3077---------------
3078
3079The following recipes have been removed:
3080
3081- *``linux-yocto 4.8:``* Version 4.8 has been removed. Versions 4.1
3082 (LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
3083
3084- *``python-smartpm:``* Functionally replaced by ``dnf``.
3085
3086- *``createrepo:``* Replaced by the ``createrepo-c`` recipe.
3087
3088- *``rpmresolve:``* No longer needed with the move to RPM 4 as RPM
3089 itself is used instead.
3090
3091- *``gstreamer:``* Removed the GStreamer Git version recipes as they
3092 have been stale. ``1.10.``\ x recipes are still present.
3093
3094- *``alsa-conf-base:``* Merged into ``alsa-conf`` since ``libasound``
3095 depended on both. Essentially, no way existed to install only one of
3096 these.
3097
3098- *``tremor:``* Moved to ``meta-multimedia``. Fixed-integer Vorbis
3099 decoding is not needed by current hardware. Thus, GStreamer's ivorbis
3100 plugin has been disabled by default eliminating the need for the
3101 ``tremor`` recipe in `OE-Core <#oe-core>`__.
3102
3103- *``gummiboot:``* Replaced by ``systemd-boot``.
3104
3105.. _migration-2.3-wic-changes:
3106
3107Wic Changes
3108-----------
3109
3110The following changes have been made to Wic:
3111
3112.. note::
3113
3114 For more information on Wic, see the "
3115 Creating Partitioned Images Using Wic
3116 " section in the Yocto Project Development Tasks Manual.
3117
3118- *Default Output Directory Changed:* Wic's default output directory is
3119 now the current directory by default instead of the unusual
3120 ``/var/tmp/wic``.
3121
3122 The "-o" and "--outdir" options remain unchanged and are used to
3123 specify your preferred output directory if you do not want to use the
3124 default directory.
3125
3126- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
3127 it duplicates functionality of the rawcopy plugin.
3128
3129.. _migration-2.3-qa-changes:
3130
3131QA Changes
3132----------
3133
3134The following QA checks have changed:
3135
3136- *``unsafe-references-in-binaries``:* The
3137 ``unsafe-references-in-binaries`` QA check, which was disabled by
3138 default, has now been removed. This check was intended to detect
3139 binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
3140 the case where the user has ``/usr`` on a separate filesystem to
3141 ``/``.
3142
3143 The removed QA check was buggy. Additionally, ``/usr`` residing on a
3144 separate partition from ``/`` is now a rare configuration.
3145 Consequently, ``unsafe-references-in-binaries`` was removed.
3146
3147- *``file-rdeps``:* The ``file-rdeps`` QA check is now an error by
3148 default instead of a warning. Because it is an error instead of a
3149 warning, you need to address missing runtime dependencies.
3150
3151 For additional information, see the
3152 ```insane`` <#ref-classes-insane>`__ class and the "`Errors and
3153 Warnings <#qa-errors-and-warnings>`__" section.
3154
3155.. _migration-2.3-miscellaneous-changes:
3156
3157Miscellaneous Changes
3158---------------------
3159
3160The following miscellaneous changes have occurred:
3161
3162- In this release, a number of recipes have been changed to ignore the
3163 ``largefile`` ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ item,
3164 enabling large file support unconditionally. This feature has always
3165 been enabled by default. Disabling the feature has not been widely
3166 tested.
3167
3168 .. note::
3169
3170 Future releases of the Yocto Project will remove entirely the
3171 ability to disable the
3172 largefile
3173 feature, which would make it unconditionally enabled everywhere.
3174
3175- If the ```DISTRO_VERSION`` <#var-DISTRO_VERSION>`__ value contains
3176 the value of the ```DATE`` <#var-DATE>`__ variable, which is the
3177 default between Poky releases, the ``DATE`` value is explicitly
3178 excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
3179 displayed at the login prompt, in order to avoid conflicts with
3180 Multilib enabled. Regardless, the ``DATE`` value is inaccurate if the
3181 ``base-files`` recipe is restored from shared state (sstate) rather
3182 than rebuilt.
3183
3184 If you need the build date recorded in ``/etc/issue*`` or anywhere
3185 else in your image, a better method is to define a post-processing
3186 function to do it and have the function called from
3187 ```ROOTFS_POSTPROCESS_COMMAND`` <#var-ROOTFS_POSTPROCESS_COMMAND>`__.
3188 Doing so ensures the value is always up-to-date with the created
3189 image.
3190
3191- Dropbear's ``init`` script now disables DSA host keys by default.
3192 This change is in line with the systemd service file, which supports
3193 RSA keys only, and with recent versions of OpenSSH, which deprecates
3194 DSA host keys.
3195
3196- The ```buildhistory`` <#ref-classes-buildhistory>`__ class now
3197 correctly uses tabs as separators between all columns in
3198 ``installed-package-sizes.txt`` in order to aid import into other
3199 tools.
3200
3201- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
3202 ``DISTRO_FEATURES`` feature. Distributions that previously set:
3203 USE_LDCONFIG = "0" should now instead use the following:
3204 DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
3205
3206- The default value of
3207 ```COPYLEFT_LICENSE_INCLUDE`` <#var-COPYLEFT_LICENSE_INCLUDE>`__ now
3208 includes all versions of AGPL licenses in addition to GPL and LGPL.
3209
3210 .. note::
3211
3212 The default list is not intended to be guaranteed as a complete
3213 safe list. You should seek legal advice based on what you are
3214 distributing if you are unsure.
3215
3216- Kernel module packages are now suffixed with the kernel version in
3217 order to allow module packages from multiple kernel versions to
3218 co-exist on a target system. If you wish to return to the previous
3219 naming scheme that does not include the version suffix, use the
3220 following: KERNEL_MODULE_PACKAGE_SUFFIX to ""
3221
3222- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
3223 ``*.la`` files are not actually needed on Linux and relocating them
3224 is an unnecessary burden.
3225
3226 If you need to preserve these ``.la`` files (e.g. in a custom
3227 distribution), you must change
3228 ```INHERIT_DISTRO`` <#var-INHERIT_DISTRO>`__ such that
3229 "remove-libtool" is not included in the value.
3230
3231- Extensible SDKs built for GCC 5+ now refuse to install on a
3232 distribution where the host GCC version is 4.8 or 4.9. This change
3233 resulted from the fact that the installation is known to fail due to
3234 the way the ``uninative`` shared state (sstate) package is built. See
3235 the ```uninative`` <#ref-classes-uninative>`__ class for additional
3236 information.
3237
3238- All native and nativesdk recipes now use a separate
3239 ``DISTRO_FEATURES`` value instead of sharing the value used by
3240 recipes for the target, in order to avoid unnecessary rebuilds.
3241
3242 The ``DISTRO_FEATURES`` for ``native`` recipes is
3243 ```DISTRO_FEATURES_NATIVE`` <#var-DISTRO_FEATURES_NATIVE>`__ added to
3244 an intersection of ``DISTRO_FEATURES`` and
3245 ```DISTRO_FEATURES_FILTER_NATIVE`` <#var-DISTRO_FEATURES_FILTER_NATIVE>`__.
3246
3247 For nativesdk recipes, the corresponding variables are
3248 ```DISTRO_FEATURES_NATIVESDK`` <#var-DISTRO_FEATURES_NATIVESDK>`__
3249 and
3250 ```DISTRO_FEATURES_FILTER_NATIVESDK`` <#var-DISTRO_FEATURES_FILTER_NATIVESDK>`__.
3251
3252- The ``FILESDIR`` variable, which was previously deprecated and rarely
3253 used, has now been removed. You should change any recipes that set
3254 ``FILESDIR`` to set ```FILESPATH`` <#var-FILESPATH>`__ instead.
3255
3256- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
3257 longer needed with recipe-specific sysroots.
3258
3259Moving to the Yocto Project 2.4 Release
3260=======================================
3261
3262This section provides migration information for moving to the Yocto
3263Project 2.4 Release from the prior release.
3264
3265.. _migration-2.4-memory-resident-mode:
3266
3267Memory Resident Mode
3268--------------------
3269
3270A persistent mode is now available in BitBake's default operation,
3271replacing its previous "memory resident mode" (i.e.
3272``oe-init-build-env-memres``). Now you only need to set
3273```BB_SERVER_TIMEOUT`` <#var-BB_SERVER_TIMEOUT>`__ to a timeout (in
3274seconds) and BitBake's server stays resident for that amount of time
3275between invocations. The ``oe-init-build-env-memres`` script has been
3276removed since a separate environment setup script is no longer needed.
3277
3278.. _migration-2.4-packaging-changes:
3279
3280Packaging Changes
3281-----------------
3282
3283This section provides information about packaging changes that have
3284occurred:
3285
3286- *``python3`` Changes:*
3287
3288 - The main "python3" package now brings in all of the standard
3289 Python 3 distribution rather than a subset. This behavior matches
3290 what is expected based on traditional Linux distributions. If you
3291 wish to install a subset of Python 3, specify ``python-core`` plus
3292 one or more of the individual packages that are still produced.
3293
3294 - *``python3``:* The ``bz2.py``, ``lzma.py``, and
3295 ``_compression.py`` scripts have been moved from the
3296 ``python3-misc`` package to the ``python3-compression`` package.
3297
3298- *``binutils``:* The ``libbfd`` library is now packaged in a separate
3299 "libbfd" package. This packaging saves space when certain tools (e.g.
3300 ``perf``) are installed. In such cases, the tools only need
3301 ``libbfd`` rather than all the packages in ``binutils``.
3302
3303- *``util-linux`` Changes:*
3304
3305 - The ``su`` program is now packaged in a separate "util-linux-su"
3306 package, which is only built when "pam" is listed in the
3307 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ variable.
3308 ``util-linux`` should not be installed unless it is needed because
3309 ``su`` is normally provided through the shadow file format. The
3310 main ``util-linux`` package has runtime dependencies (i.e.
3311 ```RDEPENDS`` <#var-RDEPENDS>`__) on the ``util-linux-su`` package
3312 when "pam" is in ``DISTRO_FEATURES``.
3313
3314 - The ``switch_root`` program is now packaged in a separate
3315 "util-linux-switch-root" package for small initramfs images that
3316 do not need the whole ``util-linux`` package or the busybox
3317 binary, which are both much larger than ``switch_root``. The main
3318 ``util-linux`` package has a recommended runtime dependency (i.e.
3319 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__) on the
3320 ``util-linux-switch-root`` package.
3321
3322 - The ``ionice`` program is now packaged in a separate
3323 "util-linux-ionice" package. The main ``util-linux`` package has a
3324 recommended runtime dependency (i.e. ``RRECOMMENDS``) on the
3325 ``util-linux-ionice`` package.
3326
3327- *``initscripts``:* The ``sushell`` program is now packaged in a
3328 separate "initscripts-sushell" package. This packaging change allows
3329 systems to pull ``sushell`` in when ``selinux`` is enabled. The
3330 change also eliminates needing to pull in the entire ``initscripts``
3331 package. The main ``initscripts`` package has a runtime dependency
3332 (i.e. ``RDEPENDS``) on the ``sushell`` package when "selinux" is in
3333 ``DISTRO_FEATURES``.
3334
3335- *``glib-2.0``:* The ``glib-2.0`` package now has a recommended
3336 runtime dependency (i.e. ``RRECOMMENDS``) on the ``shared-mime-info``
3337 package, since large portions of GIO are not useful without the MIME
3338 database. You can remove the dependency by using the
3339 ```BAD_RECOMMENDATIONS`` <#var-BAD_RECOMMENDATIONS>`__ variable if
3340 ``shared-mime-info`` is too large and is not required.
3341
3342- *Go Standard Runtime:* The Go standard runtime has been split out
3343 from the main ``go`` recipe into a separate ``go-runtime`` recipe.
3344
3345.. _migration-2.4-removed-recipes:
3346
3347Removed Recipes
3348---------------
3349
3350The following recipes have been removed:
3351
3352- *``acpitests``:* This recipe is not maintained.
3353
3354- *``autogen-native``:* No longer required by Grub, oe-core, or
3355 meta-oe.
3356
3357- *``bdwgc``:* Nothing in OpenEmbedded-Core requires this recipe. It
3358 has moved to meta-oe.
3359
3360- *``byacc``:* This recipe was only needed by rpm 5.x and has moved to
3361 meta-oe.
3362
3363- *``gcc (5.4)``:* The 5.4 series dropped the recipe in favor of 6.3 /
3364 7.2.
3365
3366- *``gnome-common``:* Deprecated upstream and no longer needed.
3367
3368- *``go-bootstrap-native``:* Go 1.9 does its own bootstrapping so this
3369 recipe has been removed.
3370
3371- *``guile``:* This recipe was only needed by ``autogen-native`` and
3372 ``remake``. The recipe is no longer needed by either of these
3373 programs.
3374
3375- *``libclass-isa-perl``:* This recipe was previously needed for LSB 4,
3376 no longer needed.
3377
3378- *``libdumpvalue-perl``:* This recipe was previously needed for LSB 4,
3379 no longer needed.
3380
3381- *``libenv-perl``:* This recipe was previously needed for LSB 4, no
3382 longer needed.
3383
3384- *``libfile-checktree-perl``:* This recipe was previously needed for
3385 LSB 4, no longer needed.
3386
3387- *``libi18n-collate-perl``:* This recipe was previously needed for LSB
3388 4, no longer needed.
3389
3390- *``libiconv``:* This recipe was only needed for ``uclibc``, which was
3391 removed in the previous release. ``glibc`` and ``musl`` have their
3392 own implementations. ``meta-mingw`` still needs ``libiconv``, so it
3393 has been moved to ``meta-mingw``.
3394
3395- *``libpng12``:* This recipe was previously needed for LSB. The
3396 current ``libpng`` is 1.6.x.
3397
3398- *``libpod-plainer-perl``:* This recipe was previously needed for LSB
3399 4, no longer needed.
3400
3401- *``linux-yocto (4.1)``:* This recipe was removed in favor of 4.4,
3402 4.9, 4.10 and 4.12.
3403
3404- *``mailx``:* This recipe was previously only needed for LSB
3405 compatibility, and upstream is defunct.
3406
3407- *``mesa (git version only)``:* The git version recipe was stale with
3408 respect to the release version.
3409
3410- *``ofono (git version only)``:* The git version recipe was stale with
3411 respect to the release version.
3412
3413- *``portmap``:* This recipe is obsolete and is superseded by
3414 ``rpcbind``.
3415
3416- *``python3-pygpgme``:* This recipe is old and unmaintained. It was
3417 previously required by ``dnf``, which has switched to official
3418 ``gpgme`` Python bindings.
3419
3420- *``python-async``:* This recipe has been removed in favor of the
3421 Python 3 version.
3422
3423- *``python-gitdb``:* This recipe has been removed in favor of the
3424 Python 3 version.
3425
3426- *``python-git``:* This recipe was removed in favor of the Python 3
3427 version.
3428
3429- *``python-mako``:* This recipe was removed in favor of the Python 3
3430 version.
3431
3432- *``python-pexpect``:* This recipe was removed in favor of the Python
3433 3 version.
3434
3435- *``python-ptyprocess``:* This recipe was removed in favor of Python
3436 the 3 version.
3437
3438- *``python-pycurl``:* Nothing is using this recipe in
3439 OpenEmbedded-Core (i.e. ``meta-oe``).
3440
3441- *``python-six``:* This recipe was removed in favor of the Python 3
3442 version.
3443
3444- *``python-smmap``:* This recipe was removed in favor of the Python 3
3445 version.
3446
3447- *``remake``:* Using ``remake`` as the provider of ``virtual/make`` is
3448 broken. Consequently, this recipe is not needed in OpenEmbedded-Core.
3449
3450.. _migration-2.4-kernel-device-tree-move:
3451
3452Kernel Device Tree Move
3453-----------------------
3454
3455Kernel Device Tree support is now easier to enable in a kernel recipe.
3456The Device Tree code has moved to a
3457```kernel-devicetree`` <#ref-classes-kernel-devicetree>`__ class.
3458Functionality is automatically enabled for any recipe that inherits the
3459```kernel`` <#ref-classes-kernel>`__ class and sets the
3460```KERNEL_DEVICETREE`` <#var-KERNEL_DEVICETREE>`__ variable. The
3461previous mechanism for doing this,
3462``meta/recipes-kernel/linux/linux-dtb.inc``, is still available to avoid
3463breakage, but triggers a deprecation warning. Future releases of the
3464Yocto Project will remove ``meta/recipes-kernel/linux/linux-dtb.inc``.
3465It is advisable to remove any ``require`` statements that request
3466``meta/recipes-kernel/linux/linux-dtb.inc`` from any custom kernel
3467recipes you might have. This will avoid breakage in post 2.4 releases.
3468
3469.. _migration-2.4-package-qa-changes:
3470
3471Package QA Changes
3472------------------
3473
3474The following package QA changes took place:
3475
3476- The "unsafe-references-in-scripts" QA check has been removed.
3477
3478- If you refer to ``${COREBASE}/LICENSE`` within
3479 ```LIC_FILES_CHKSUM`` <#var-LIC_FILES_CHKSUM>`__ you receive a
3480 warning because this file is a description of the license for
3481 OE-Core. Use ``${COMMON_LICENSE_DIR}/MIT`` if your recipe is
3482 MIT-licensed and you cannot use the preferred method of referring to
3483 a file within the source tree.
3484
3485.. _migration-2.4-readme-changes:
3486
3487``README`` File Changes
3488-----------------------
3489
3490The following are changes to ``README`` files:
3491
3492- The main Poky ``README`` file has been moved to the ``meta-poky``
3493 layer and has been renamed ``README.poky``. A symlink has been
3494 created so that references to the old location work.
3495
3496- The ``README.hardware`` file has been moved to ``meta-yocto-bsp``. A
3497 symlink has been created so that references to the old location work.
3498
3499- A ``README.qemu`` file has been created with coverage of the
3500 ``qemu*`` machines.
3501
3502.. _migration-2.4-miscellaneous-changes:
3503
3504Miscellaneous Changes
3505---------------------
3506
3507The following are additional changes:
3508
3509- The ``ROOTFS_PKGMANAGE_BOOTSTRAP`` variable and any references to it
3510 have been removed. You should remove this variable from any custom
3511 recipes.
3512
3513- The ``meta-yocto`` directory has been removed.
3514
3515 .. note::
3516
3517 In the Yocto Project 2.1 release
3518 meta-yocto
3519 was renamed to
3520 meta-poky
3521 and the
3522 meta-yocto
3523 subdirectory remained to avoid breaking existing configurations.
3524
3525- The ``maintainers.inc`` file, which tracks maintainers by listing a
3526 primary person responsible for each recipe in OE-Core, has been moved
3527 from ``meta-poky`` to OE-Core (i.e. from
3528 ``meta-poky/conf/distro/include`` to ``meta/conf/distro/include``).
3529
3530- The ```buildhistory`` <#ref-classes-buildhistory>`__ class now makes
3531 a single commit per build rather than one commit per subdirectory in
3532 the repository. This behavior assumes the commits are enabled with
3533 ```BUILDHISTORY_COMMIT`` <#var-BUILDHISTORY_COMMIT>`__ = "1", which
3534 is typical. Previously, the ``buildhistory`` class made one commit
3535 per subdirectory in the repository in order to make it easier to see
3536 the changes for a particular subdirectory. To view a particular
3537 change, specify that subdirectory as the last parameter on the
3538 ``git show`` or ``git diff`` commands.
3539
3540- The ``x86-base.inc`` file, which is included by all x86-based machine
3541 configurations, now sets ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__
3542 using ``?=`` to "live" rather than appending with ``+=``. This change
3543 makes the default easier to override.
3544
3545- BitBake fires multiple "BuildStarted" events when multiconfig is
3546 enabled (one per configuration). For more information, see the
3547 "`Events <&YOCTO_DOCS_BB_URL;#events>`__" section in the BitBake User
3548 Manual.
3549
3550- By default, the ``security_flags.inc`` file sets a
3551 ```GCCPIE`` <#var-GCCPIE>`__ variable with an option to enable
3552 Position Independent Executables (PIE) within ``gcc``. Enabling PIE
3553 in the GNU C Compiler (GCC), makes Return Oriented Programming (ROP)
3554 attacks much more difficult to execute.
3555
3556- OE-Core now provides a ``bitbake-layers`` plugin that implements a
3557 "create-layer" subcommand. The implementation of this subcommand has
3558 resulted in the ``yocto-layer`` script being deprecated and will
3559 likely be removed in the next Yocto Project release.
3560
3561- The ``vmdk``, ``vdi``, and ``qcow2`` image file types are now used in
3562 conjunction with the "wic" image type through ``CONVERSION_CMD``.
3563 Consequently, the equivalent image types are now ``wic.vmdk``,
3564 ``wic.vdi``, and ``wic.qcow2``, respectively.
3565
3566- ``do_image_<type>[depends]`` has replaced ``IMAGE_DEPENDS_<type>``.
3567 If you have your own classes that implement custom image types, then
3568 you need to update them.
3569
3570- OpenSSL 1.1 has been introduced. However, the default is still 1.0.x
3571 through the ```PREFERRED_VERSION`` <#var-PREFERRED_VERSION>`__
3572 variable. This preference is set is due to the remaining
3573 compatibility issues with other software. The
3574 ```PROVIDES`` <#var-PROVIDES>`__ variable in the openssl 1.0 recipe
3575 now includes "openssl10" as a marker that can be used in
3576 ```DEPENDS`` <#var-DEPENDS>`__ within recipes that build software
3577 that still depend on OpenSSL 1.0.
3578
3579- To ensure consistent behavior, BitBake's "-r" and "-R" options (i.e.
3580 prefile and postfile), which are used to read or post-read additional
3581 configuration files from the command line, now only affect the
3582 current BitBake command. Before these BitBake changes, these options
3583 would "stick" for future executions.
3584
3585Moving to the Yocto Project 2.5 Release
3586=======================================
3587
3588This section provides migration information for moving to the Yocto
3589Project 2.5 Release from the prior release.
3590
3591.. _migration-2.5-packaging-changes:
3592
3593Packaging Changes
3594-----------------
3595
3596This section provides information about packaging changes that have
3597occurred:
3598
3599- *``bind-libs``:* The libraries packaged by the bind recipe are in a
3600 separate ``bind-libs`` package.
3601
3602- *``libfm-gtk``:* The ``libfm`` GTK+ bindings are split into a
3603 separate ``libfm-gtk`` package.
3604
3605- *``flex-libfl``:* The flex recipe splits out libfl into a separate
3606 ``flex-libfl`` package to avoid too many dependencies being pulled in
3607 where only the library is needed.
3608
3609- *``grub-efi``:* The ``grub-efi`` configuration is split into a
3610 separate ``grub-bootconf`` recipe. However, the dependency
3611 relationship from ``grub-efi`` is through a virtual/grub-bootconf
3612 provider making it possible to have your own recipe provide the
3613 dependency. Alternatively, you can use a BitBake append file to bring
3614 the configuration back into the ``grub-efi`` recipe.
3615
3616- *armv7a Legacy Package Feed Support:* Legacy support is removed for
3617 transitioning from ``armv7a`` to ``armv7a-vfp-neon`` in package
3618 feeds, which was previously enabled by setting
3619 ``PKGARCHCOMPAT_ARMV7A``. This transition occurred in 2011 and active
3620 package feeds should by now be updated to the new naming.
3621
3622.. _migration-2.5-removed-recipes:
3623
3624Removed Recipes
3625---------------
3626
3627The following recipes have been removed:
3628
3629- *``gcc``:* The version 6.4 recipes are replaced by 7.x.
3630
3631- *``gst-player``:* Renamed to ``gst-examples`` as per upstream.
3632
3633- *``hostap-utils``:* This software package is obsolete.
3634
3635- *``latencytop``:* This recipe is no longer maintained upstream. The
3636 last release was in 2009.
3637
3638- *``libpfm4``:* The only file that requires this recipe is
3639 ``oprofile``, which has been removed.
3640
3641- *``linux-yocto``:* The version 4.4, 4.9, and 4.10 recipes have been
3642 removed. Versions 4.12, 4.14, and 4.15 remain.
3643
3644- *``man``:* This recipe has been replaced by modern ``man-db``
3645
3646- *``mkelfimage``:* This tool has been removed in the upstream coreboot
3647 project, and is no longer needed with the removal of the ELF image
3648 type.
3649
3650- *``nativesdk-postinst-intercept``:* This recipe is not maintained.
3651
3652- *``neon``:* This software package is no longer maintained upstream
3653 and is no longer needed by anything in OpenEmbedded-Core.
3654
3655- *``oprofile``:* The functionality of this recipe is replaced by
3656 ``perf`` and keeping compatibility on an ongoing basis with ``musl``
3657 is difficult.
3658
3659- *``pax``:* This software package is obsolete.
3660
3661- *``stat``:* This software package is not maintained upstream.
3662 ``coreutils`` provides a modern stat binary.
3663
3664- *``zisofs-tools-native``:* This recipe is no longer needed because
3665 the compressed ISO image feature has been removed.
3666
3667.. _migration-2.5-scripts-and-tools-changes:
3668
3669Scripts and Tools Changes
3670-------------------------
3671
3672The following are changes to scripts and tools:
3673
3674- *``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer``*: The
3675 ``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer`` scripts
3676 previously shipped with poky but not in OpenEmbedded-Core have been
3677 removed. These scripts are not maintained and are outdated. In many
3678 cases, they are also limited in scope. The
3679 ``bitbake-layers create-layer`` command is a direct replacement for
3680 ``yocto-layer``. See the documentation to create a BSP or kernel
3681 recipe in the "`BSP Kernel Recipe
3682 Example <&YOCTO_DOCS_BSP_URL;#bsp-kernel-recipe-example>`__" section.
3683
3684- *``devtool finish``:* ``devtool finish`` now exits with an error if
3685 there are uncommitted changes or a rebase/am in progress in the
3686 recipe's source repository. If this error occurs, there might be
3687 uncommitted changes that will not be included in updates to the
3688 patches applied by the recipe. A -f/--force option is provided for
3689 situations that the uncommitted changes are inconsequential and you
3690 want to proceed regardless.
3691
3692- *``scripts/oe-setup-rpmrepo`` script:* The functionality of
3693 ``scripts/oe-setup-rpmrepo`` is replaced by
3694 ``bitbake package-index``.
3695
3696- *``scripts/test-dependencies.sh`` script:* The script is largely made
3697 obsolete by the recipe-specific sysroots functionality introduced in
3698 the previous release.
3699
3700.. _migration-2.5-bitbake-changes:
3701
3702BitBake Changes
3703---------------
3704
3705The following are BitBake changes:
3706
3707- The ``--runall`` option has changed. There are two different
3708 behaviors people might want:
3709
3710 - *Behavior A:* For a given target (or set of targets) look through
3711 the task graph and run task X only if it is present and will be
3712 built.
3713
3714 - *Behavior B:* For a given target (or set of targets) look through
3715 the task graph and run task X if any recipe in the taskgraph has
3716 such a target, even if it is not in the original task graph.
3717
3718 The ``--runall`` option now performs "Behavior B". Previously
3719 ``--runall`` behaved like "Behavior A". A ``--runonly`` option has
3720 been added to retain the ability to perform "Behavior A".
3721
3722- Several explicit "run this task for all recipes in the dependency
3723 tree" tasks have been removed (e.g. ``fetchall``, ``checkuriall``,
3724 and the ``*all`` tasks provided by the ``distrodata`` and
3725 ``archiver`` classes). There is a BitBake option to complete this for
3726 any arbitrary task. For example: bitbake <target> -c fetchall should
3727 now be replaced with: bitbake <target> --runall=fetch
3728
3729.. _migration-2.5-python-and-python3-changes:
3730
3731Python and Python 3 Changes
3732---------------------------
3733
3734The following are auto-packaging changes to Python and Python 3:
3735
3736The script-managed ``python-*-manifest.inc`` files that were previously
3737used to generate Python and Python 3 packages have been replaced with a
3738JSON-based file that is easier to read and maintain. A new task is
3739available for maintainers of the Python recipes to update the JSON file
3740when upgrading to new Python versions. You can now edit the file
3741directly instead of having to edit a script and run it to update the
3742file.
3743
3744One particular change to note is that the Python recipes no longer have
3745build-time provides for their packages. This assumes ``python-foo`` is
3746one of the packages provided by the Python recipe. You can no longer run
3747``bitbake python-foo`` or have a
3748```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ on ``python-foo``,
3749but doing either of the following causes the package to work as
3750expected: IMAGE_INSTALL_append = " python-foo" or RDEPENDS_${PN} =
3751"python-foo" The earlier build-time provides behavior was a quirk of the
3752way the Python manifest file was created. For more information on this
3753change please see `this
3754commit <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=8d94b9db221d1def42f091b991903faa2d1651ce>`__.
3755
3756.. _migration-2.5-miscellaneous-changes:
3757
3758Miscellaneous Changes
3759---------------------
3760
3761The following are additional changes:
3762
3763- The ``kernel`` class supports building packages for multiple kernels.
3764 If your kernel recipe or ``.bbappend`` file mentions packaging at
3765 all, you should replace references to the kernel in package names
3766 with ``${KERNEL_PACKAGE_NAME}``. For example, if you disable
3767 automatic installation of the kernel image using
3768 ``RDEPENDS_kernel-base = ""`` you can avoid warnings using
3769 ``RDEPENDS_${KERNEL_PACKAGE_NAME}-base = ""`` instead.
3770
3771- The ``buildhistory`` class commits changes to the repository by
3772 default so you no longer need to set ``BUILDHISTORY_COMMIT = "1"``.
3773 If you want to disable commits you need to set
3774 ``BUILDHISTORY_COMMIT = "0"`` in your configuration.
3775
3776- The ``beaglebone`` reference machine has been renamed to
3777 ``beaglebone-yocto``. The ``beaglebone-yocto`` BSP is a reference
3778 implementation using only mainline components available in
3779 OpenEmbedded-Core and ``meta-yocto-bsp``, whereas Texas Instruments
3780 maintains a full-featured BSP in the ``meta-ti`` layer. This rename
3781 avoids the previous name clash that existed between the two BSPs.
3782
3783- The ``update-alternatives`` class no longer works with SysV ``init``
3784 scripts because this usage has been problematic. Also, the
3785 ``sysklogd`` recipe no longer uses ``update-alternatives`` because it
3786 is incompatible with other implementations.
3787
3788- By default, the ```cmake`` <#ref-classes-cmake>`__ class uses
3789 ``ninja`` instead of ``make`` for building. This improves build
3790 performance. If a recipe is broken with ``ninja``, then the recipe
3791 can set ``OECMAKE_GENERATOR = "Unix Makefiles"`` to change back to
3792 ``make``.
3793
3794- The previously deprecated ``base_*`` functions have been removed in
3795 favor of their replacements in ``meta/lib/oe`` and
3796 ``bitbake/lib/bb``. These are typically used from recipes and
3797 classes. Any references to the old functions must be updated. The
3798 following table shows the removed functions and their replacements:
3799 *Removed* *Replacement* ============================
3800 ============================ base_path_join() oe.path.join()
3801 base_path_relative() oe.path.relative() base_path_out()
3802 oe.path.format_display() base_read_file() oe.utils.read_file()
3803 base_ifelse() oe.utils.ifelse() base_conditional()
3804 oe.utils.conditional() base_less_or_equal() oe.utils.less_or_equal()
3805 base_version_less_or_equal() oe.utils.version_less_or_equal()
3806 base_contains() bb.utils.contains() base_both_contain()
3807 oe.utils.both_contain() base_prune_suffix() oe.utils.prune_suffix()
3808 oe_filter() oe.utils.str_filter() oe_filter_out()
3809 oe.utils.str_filter_out() (or use the \_remove operator).
3810
3811- Using ``exit 1`` to explicitly defer a postinstall script until first
3812 boot is now deprecated since it is not an obvious mechanism and can
3813 mask actual errors. If you want to explicitly defer a postinstall to
3814 first boot on the target rather than at ``rootfs`` creation time, use
3815 ``pkg_postinst_ontarget()`` or call
3816 ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
3817 Any failure of a ``pkg_postinst()`` script (including ``exit 1``)
3818 will trigger a warning during ``do_rootfs``.
3819
3820 For more information, see the "`Post-Installation
3821 Scripts <&YOCTO_DOCS_DEV_URL;#new-recipe-post-installation-scripts>`__"
3822 section in the Yocto Project Development Tasks Manual.
3823
3824- The ``elf`` image type has been removed. This image type was removed
3825 because the ``mkelfimage`` tool that was required to create it is no
3826 longer provided by coreboot upstream and required updating every time
3827 ``binutils`` updated.
3828
3829- Support for .iso image compression (previously enabled through
3830 ``COMPRESSISO = "1"``) has been removed. The userspace tools
3831 (``zisofs-tools``) are unmaintained and ``squashfs`` provides better
3832 performance and compression. In order to build a live image with
3833 squashfs+lz4 compression enabled you should now set
3834 ``LIVE_ROOTFS_TYPE = "squashfs-lz4"`` and ensure that ``live`` is in
3835 ``IMAGE_FSTYPES``.
3836
3837- Recipes with an unconditional dependency on ``libpam`` are only
3838 buildable with ``pam`` in ``DISTRO_FEATURES``. If the dependency is
3839 truly optional then it is recommended that the dependency be
3840 conditional upon ``pam`` being in ``DISTRO_FEATURES``.
3841
3842- For EFI-based machines, the bootloader (``grub-efi`` by default) is
3843 installed into the image at /boot. Wic can be used to split the
3844 bootloader into separate boot and rootfs partitions if necessary.
3845
3846- Patches whose context does not match exactly (i.e. where patch
3847 reports "fuzz" when applying) will generate a warning. For an example
3848 of this see `this
3849 commit <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=cc97bc08125b63821ce3f616771830f77c456f57>`__.
3850
3851- Layers are expected to set ``LAYERSERIES_COMPAT_layername`` to match
3852 the version(s) of OpenEmbedded-Core they are compatible with. This is
3853 specified as codenames using spaces to separate multiple values (e.g.
3854 "rocko sumo"). If a layer does not set
3855 ``LAYERSERIES_COMPAT_layername``, a warning will is shown. If a layer
3856 sets a value that does not include the current version ("sumo" for
3857 the 2.5 release), then an error will be produced.
3858
3859- The ``TZ`` environment variable is set to "UTC" within the build
3860 environment in order to fix reproducibility problems in some recipes.
3861
3862Moving to the Yocto Project 2.6 Release
3863=======================================
3864
3865This section provides migration information for moving to the Yocto
3866Project 2.6 Release from the prior release.
3867
3868.. _migration-2.6-gcc-changes:
3869
3870GCC 8.2 is Now Used by Default
3871------------------------------
3872
3873The GNU Compiler Collection version 8.2 is now used by default for
3874compilation. For more information on what has changed in the GCC 8.x
3875release, see ` <https://gcc.gnu.org/gcc-8/changes.html>`__.
3876
3877If you still need to compile with version 7.x, GCC 7.3 is also provided.
3878You can select this version by setting the and can be selected by
3879setting the ```GCCVERSION`` <#var-GCCVERSION>`__ variable to "7.%" in
3880your configuration.
3881
3882.. _migration-2.6-removed-recipes:
3883
3884Removed Recipes
3885---------------
3886
3887The following recipes have been removed: *``beecrypt``:* No longer
3888needed since moving to RPM 4. *``bigreqsproto``:* Replaced by
3889``xorgproto``. *``calibrateproto``:* Removed in favor of ``xinput``.
3890*``compositeproto``:* Replaced by ``xorgproto``. *``damageproto``:*
3891Replaced by ``xorgproto``. *``dmxproto``:* Replaced by ``xorgproto``.
3892*``dri2proto``:* Replaced by ``xorgproto``. *``dri3proto``:* Replaced by
3893``xorgproto``. *``eee-acpi-scripts``:* Became obsolete.
3894*``fixesproto``:* Replaced by ``xorgproto``. *``fontsproto``:* Replaced
3895by ``xorgproto``. *``fstests``:* Became obsolete. *``gccmakedep``:* No
3896longer used. *``glproto``:* Replaced by ``xorgproto``.
3897*``gnome-desktop3``:* No longer needed. This recipe has moved to
3898``meta-oe``. *``icon-naming-utils``:* No longer used since the Sato
3899theme was removed in 2016. *``inputproto``:* Replaced by ``xorgproto``.
3900*``kbproto``:* Replaced by ``xorgproto``. *``libusb-compat``:* Became
3901obsolete. *``libuser``:* Became obsolete. *``libnfsidmap``:* No longer
3902an external requirement since ``nfs-utils`` 2.2.1. ``libnfsidmap`` is
3903now integrated. *``libxcalibrate``:* No longer needed with ``xinput``
3904*``mktemp``:* Became obsolete. The ``mktemp`` command is provided by
3905both ``busybox`` and ``coreutils``. *``ossp-uuid``:* Is not being
3906maintained and has mostly been replaced by ``uuid.h`` in ``util-linux``.
3907*``pax-utils``:* No longer needed. Previous QA tests that did use this
3908recipe are now done at build time. *``pcmciautils``:* Became obsolete.
3909*``pixz``:* No longer needed. ``xz`` now supports multi-threaded
3910compression. *``presentproto``:* Replaced by ``xorgproto``.
3911*``randrproto``:* Replaced by ``xorgproto``. *``recordproto``:* Replaced
3912by ``xorgproto``. *``renderproto``:* Replaced by ``xorgproto``.
3913*``resourceproto``:* Replaced by ``xorgproto``. *``scrnsaverproto``:*
3914Replaced by ``xorgproto``. *``trace-cmd``:* Became obsolete. ``perf``
3915replaced this recipe's functionally. *``videoproto``:* Replaced by
3916``xorgproto``. *``wireless-tools``:* Became obsolete. Superseded by
3917``iw``. *``xcmiscproto``:* Replaced by ``xorgproto``. *``xextproto``:*
3918Replaced by ``xorgproto``. *``xf86dgaproto``:* Replaced by
3919``xorgproto``. *``xf86driproto``:* Replaced by ``xorgproto``.
3920*``xf86miscproto``:* Replaced by ``xorgproto``. *``xf86-video-omapfb``:*
3921Became obsolete. Use kernel modesetting driver instead.
3922*``xf86-video-omap``:* Became obsolete. Use kernel modesetting driver
3923instead. *``xf86vidmodeproto``:* Replaced by ``xorgproto``.
3924*``xineramaproto``:* Replaced by ``xorgproto``. *``xproto``:* Replaced
3925by ``xorgproto``. *``yasm``:* No longer needed since previous usages are
3926now satisfied by ``nasm``.
3927
3928.. _migration-2.6-packaging-changes:
3929
3930Packaging Changes
3931-----------------
3932
3933The following packaging changes have been made:
3934
3935- *``cmake``:* ``cmake.m4`` and ``toolchain`` files have been moved to
3936 the main package.
3937
3938- *``iptables``:* The ``iptables`` modules have been split into
3939 separate packages.
3940
3941- *``alsa-lib``:* ``libasound`` is now in the main ``alsa-lib`` package
3942 instead of ``libasound``.
3943
3944- *``glibc``:* ``libnss-db`` is now in its own package along with a
3945 ``/var/db/makedbs.sh`` script to update databases.
3946
3947- *``python`` and ``python3``:* The main package has been removed from
3948 the recipe. You must install specific packages or ``python-modules``
3949 / ``python3-modules`` for everything.
3950
3951- *``systemtap``:* Moved ``systemtap-exporter`` into its own package.
3952
3953.. _migration-2.6-xorg-protocol-dependencies:
3954
3955XOrg Protocol dependencies
3956--------------------------
3957
3958The "*proto" upstream repositories have been combined into one
3959"xorgproto" repository. Thus, the corresponding recipes have also been
3960combined into a single ``xorgproto`` recipe. Any recipes that depend
3961upon the older ``*proto`` recipes need to be changed to depend on the
3962newer ``xorgproto`` recipe instead.
3963
3964For names of recipes removed because of this repository change, see the
3965`Removed Recipes <#migration-2.6-removed-recipes>`__ section.
3966
3967.. _migration-2.6-distutils-distutils3-fetching-dependencies:
3968
3969``distutils`` and ``distutils3`` Now Prevent Fetching Dependencies During the ``do_configure`` Task
3970---------------------------------------------------------------------------------------------------
3971
3972Previously, it was possible for Python recipes that inherited the
3973```distutils`` <#ref-classes-distutils>`__ and
3974```distutils3`` <#ref-classes-distutils3>`__ classes to fetch code
3975during the ```do_configure`` <#ref-tasks-configure>`__ task to satisfy
3976dependencies mentioned in ``setup.py`` if those dependencies were not
3977provided in the sysroot (i.e. recipes providing the dependencies were
3978missing from ```DEPENDS`` <#var-DEPENDS>`__).
3979
3980.. note::
3981
3982 This change affects classes beyond just the two mentioned (i.e.
3983 distutils
3984 and
3985 distutils3
3986 ). Any recipe that inherits
3987 distutils\*
3988 classes are affected. For example, the
3989 setuptools
3990 and
3991 setuptools3
3992 recipes are affected since they inherit the
3993 distutils\*
3994 classes.
3995
3996Fetching these types of dependencies that are not provided in the
3997sysroot negatively affects the ability to reproduce builds. This type of
3998fetching is now explicitly disabled. Consequently, any missing
3999dependencies in Python recipes that use these classes now result in an
4000error during the ``do_configure`` task.
4001
4002.. _migration-2.6-linux-yocto-configuration-audit-issues-now-correctly-reported:
4003
4004``linux-yocto`` Configuration Audit Issues Now Correctly Reported
4005-----------------------------------------------------------------
4006
4007Due to a bug, the kernel configuration audit functionality was not
4008writing out any resulting warnings during the build. This issue is now
4009corrected. You might notice these warnings now if you have a custom
4010kernel configuration with a ``linux-yocto`` style kernel recipe.
4011
4012.. _migration-2.6-image-kernel-artifact-naming-changes:
4013
4014Image/Kernel Artifact Naming Changes
4015------------------------------------
4016
4017The following changes have been made:
4018
4019- Name variables (e.g. ```IMAGE_NAME`` <#var-IMAGE_NAME>`__) use a new
4020 ``IMAGE_VERSION_SUFFIX`` variable instead of
4021 ```DATETIME`` <#var-DATETIME>`__. Using ``IMAGE_VERSION_SUFFIX``
4022 allows easier and more direct changes.
4023
4024 The ``IMAGE_VERSION_SUFFIX`` variable is set in the ``bitbake.conf``
4025 configuration file as follows: IMAGE_VERSION_SUFFIX = "-${DATETIME}"
4026
4027- Several variables have changed names for consistency: Old Variable
4028 Name New Variable Name
4029 ========================================================
4030 KERNEL_IMAGE_BASE_NAME `KERNEL_IMAGE_NAME <#var-KERNEL_IMAGE_NAME>`__
4031 KERNEL_IMAGE_SYMLINK_NAME
4032 `KERNEL_IMAGE_LINK_NAME <#var-KERNEL_IMAGE_LINK_NAME>`__
4033 MODULE_TARBALL_BASE_NAME
4034 `MODULE_TARBALL_NAME <#var-MODULE_TARBALL_NAME>`__
4035 MODULE_TARBALL_SYMLINK_NAME
4036 `MODULE_TARBALL_LINK_NAME <#var-MODULE_TARBALL_LINK_NAME>`__
4037 INITRAMFS_BASE_NAME `INITRAMFS_NAME <#var-INITRAMFS_NAME>`__
4038
4039- The ``MODULE_IMAGE_BASE_NAME`` variable has been removed. The module
4040 tarball name is now controlled directly with the
4041 ```MODULE_TARBALL_NAME`` <#var-MODULE_TARBALL_NAME>`__ variable.
4042
4043- The ```KERNEL_DTB_NAME`` <#var-KERNEL_DTB_NAME>`__ and
4044 ```KERNEL_DTB_LINK_NAME`` <#var-KERNEL_DTB_LINK_NAME>`__ variables
4045 have been introduced to control kernel Device Tree Binary (DTB)
4046 artifact names instead of mangling ``KERNEL_IMAGE_*`` variables.
4047
4048- The ```KERNEL_FIT_NAME`` <#var-KERNEL_FIT_NAME>`__ and
4049 ```KERNEL_FIT_LINK_NAME`` <#var-KERNEL_FIT_LINK_NAME>`__ variables
4050 have been introduced to specify the name of flattened image tree
4051 (FIT) kernel images similar to other deployed artifacts.
4052
4053- The ```MODULE_TARBALL_NAME`` <#var-MODULE_TARBALL_NAME>`__ and
4054 ```MODULE_TARBALL_LINK_NAME`` <#var-MODULE_TARBALL_LINK_NAME>`__
4055 variable values no longer include the "module-" prefix or ".tgz"
4056 suffix. These parts are now hardcoded so that the values are
4057 consistent with other artifact naming variables.
4058
4059- Added the ```INITRAMFS_LINK_NAME`` <#var-INITRAMFS_LINK_NAME>`__
4060 variable so that the symlink can be controlled similarly to other
4061 artifact types.
4062
4063- ```INITRAMFS_NAME`` <#var-INITRAMFS_NAME>`__ now uses
4064 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" instead
4065 of "${PV}-${PR}-${MACHINE}-${DATETIME}", which makes it consistent
4066 with other variables.
4067
4068.. _migration-2.6-serial-console-deprecated:
4069
4070``SERIAL_CONSOLE`` Deprecated
4071-----------------------------
4072
4073The ```SERIAL_CONSOLE`` <#var-SERIAL_CONSOLE>`__ variable has been
4074functionally replaced by the
4075```SERIAL_CONSOLES`` <#var-SERIAL_CONSOLES>`__ variable for some time.
4076With the Yocto Project 2.6 release, ``SERIAL_CONSOLE`` has been
4077officially deprecated.
4078
4079``SERIAL_CONSOLE`` will continue to work as before for the 2.6 release.
4080However, for the sake of future compatibility, it is recommended that
4081you replace all instances of ``SERIAL_CONSOLE`` with
4082``SERIAL_CONSOLES``.
4083
4084.. note::
4085
4086 The only difference in usage is that
4087 SERIAL_CONSOLES
4088 expects entries to be separated using semicolons as compared to
4089 SERIAL_CONSOLE
4090 , which expects spaces.
4091
4092.. _migration-2.6-poky-sets-unknown-configure-option-to-qa-error:
4093
4094Configure Script Reports Unknown Options as Errors
4095--------------------------------------------------
4096
4097If the configure script reports an unknown option, this now triggers a
4098QA error instead of a warning. Any recipes that previously got away with
4099specifying such unknown options now need to be fixed.
4100
4101.. _migration-2.6-override-changes:
4102
4103Override Changes
4104----------------
4105
4106The following changes have occurred:
4107
4108- *The ``virtclass-native`` and ``virtclass-nativesdk`` Overrides Have
4109 Been Removed:* The ``virtclass-native`` and ``virtclass-nativesdk``
4110 overrides have been deprecated since 2012 in favor of
4111 ``class-native`` and ``class-nativesdk``, respectively. Both
4112 ``virtclass-native`` and ``virtclass-nativesdk`` are now dropped.
4113
4114 .. note::
4115
4116 The
4117 virtclass-multilib-
4118 overrides for multilib are still valid.
4119
4120- *The ``forcevariable`` Override Now Has a Higher Priority Than
4121 ``libc`` Overrides:* The ``forcevariable`` override is documented to
4122 be the highest priority override. However, due to a long-standing
4123 quirk of how ```OVERRIDES`` <#var-OVERRIDES>`__ is set, the ``libc``
4124 overrides (e.g. ``libc-glibc``, ``libc-musl``, and so forth)
4125 erroneously had a higher priority. This issue is now corrected.
4126
4127 It is likely this change will not cause any problems. However, it is
4128 possible with some unusual configurations that you might see a change
4129 in behavior if you were relying on the previous behavior. Be sure to
4130 check how you use ``forcevariable`` and ``libc-*`` overrides in your
4131 custom layers and configuration files to ensure they make sense.
4132
4133- *The ``build-${BUILD_OS}`` Override Has Been Removed:* The
4134 ``build-${BUILD_OS}``, which is typically ``build-linux``, override
4135 has been removed because building on a host operating system other
4136 than a recent version of Linux is neither supported nor recommended.
4137 Dropping the override avoids giving the impression that other host
4138 operating systems might be supported.
4139
4140- The "_remove" operator now preserves whitespace. Consequently, when
4141 specifying list items to remove, be aware that leading and trailing
4142 whitespace resulting from the removal is retained.
4143
4144 See the "`Removal (Override Style
4145 Syntax) <&YOCTO_DOCS_BB_URL;#removing-override-style-syntax>`__"
4146 section in the BitBake User Manual for a detailed example.
4147
4148.. _migration-2.6-systemd-configuration-now-split-out-to-system-conf:
4149
4150``systemd`` Configuration is Now Split Into ``systemd-conf``
4151------------------------------------------------------------
4152
4153The configuration for the ``systemd`` recipe has been moved into a
4154``system-conf`` recipe. Moving this configuration to a separate recipe
4155avoids the ``systemd`` recipe from becoming machine-specific for cases
4156where machine-specific configurations need to be applied (e.g. for
4157``qemu*`` machines).
4158
4159Currently, the new recipe packages the following files:
4160${sysconfdir}/machine-id ${sysconfdir}/systemd/coredump.conf
4161${sysconfdir}/systemd/journald.conf ${sysconfdir}/systemd/logind.conf
4162${sysconfdir}/systemd/system.conf ${sysconfdir}/systemd/user.conf If you
4163previously used bbappend files to append the ``systemd`` recipe to
4164change any of the listed files, you must do so for the ``systemd-conf``
4165recipe instead.
4166
4167.. _migration-2.6-automatic-testing-changes:
4168
4169Automatic Testing Changes
4170-------------------------
4171
4172This section provides information about automatic testing changes:
4173
4174- *``TEST_IMAGE`` Variable Removed:* Prior to this release, you set the
4175 ``TEST_IMAGE`` variable to "1" to enable automatic testing for
4176 successfully built images. The ``TEST_IMAGE`` variable no longer
4177 exists and has been replaced by the
4178 ```TESTIMAGE_AUTO`` <#var-TESTIMAGE_AUTO>`__ variable.
4179
4180- *Inheriting the ``testimage`` and ``testsdk`` Classes:* Best
4181 practices now dictate that you use the
4182 ```IMAGE_CLASSES`` <#var-IMAGE_CLASSES>`__ variable rather than the
4183 ```INHERIT`` <#var-INHERIT>`__ variable when you inherit the
4184 ```testimage`` <#ref-classes-testimage*>`__ and
4185 ```testsdk`` <#ref-classes-testsdk>`__ classes used for automatic
4186 testing.
4187
4188.. _migration-2.6-openssl-changes:
4189
4190OpenSSL Changes
4191---------------
4192
4193`OpenSSL <https://www.openssl.org/>`__ has been upgraded from 1.0 to
41941.1. By default, this upgrade could cause problems for recipes that have
4195both versions in their dependency chains. The problem is that both
4196versions cannot be installed together at build time.
4197
4198.. note::
4199
4200 It is possible to have both versions of the library at runtime.
4201
4202.. _migration-2.6-bitbake-changes:
4203
4204BitBake Changes
4205---------------
4206
4207The server logfile ``bitbake-cookerdaemon.log`` is now always placed in
4208the `Build Directory <#build-directory>`__ instead of the current
4209directory.
4210
4211.. _migration-2.6-security-changes:
4212
4213Security Changes
4214----------------
4215
4216The Poky distribution now uses security compiler flags by default.
4217Inclusion of these flags could cause new failures due to stricter
4218checking for various potential security issues in code.
4219
4220.. _migration-2.6-post-installation-changes:
4221
4222Post Installation Changes
4223-------------------------
4224
4225You must explicitly mark post installs to defer to the target. If you
4226want to explicitly defer a postinstall to first boot on the target
4227rather than at rootfs creation time, use ``pkg_postinst_ontarget()`` or
4228call ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
4229Any failure of a ``pkg_postinst()`` script (including exit 1) triggers
4230an error during the ```do_rootfs`` <#ref-tasks-rootfs>`__ task.
4231
4232For more information on post-installation behavior, see the
4233"`Post-Installation
4234Scripts <&YOCTO_DOCS_DEV_URL;#new-recipe-post-installation-scripts>`__"
4235section in the Yocto Project Development Tasks Manual.
4236
4237.. _migration-2.6-python-3-profile-guided-optimizations:
4238
4239Python 3 Profile-Guided Optimization
4240------------------------------------
4241
4242The ``python3`` recipe now enables profile-guided optimization. Using
4243this optimization requires a little extra build time in exchange for
4244improved performance on the target at runtime. Additionally, the
4245optimization is only enabled if the current
4246```MACHINE`` <#var-MACHINE>`__ has support for user-mode emulation in
4247QEMU (i.e. "qemu-usermode" is in
4248```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__, which it is by
4249default).
4250
4251If you wish to disable Python profile-guided optimization regardless of
4252the value of ``MACHINE_FEATURES``, then ensure that
4253```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__ for the ``python3`` recipe
4254does not contain "pgo". You could accomplish the latter using the
4255following at the configuration level: PACKAGECONFIG_remove_pn-python3 =
4256"pgo" Alternatively, you can set ``PACKAGECONFIG`` using an append file
4257for the ``python3`` recipe.
4258
4259.. _migration-2.6-miscellaneous-changes:
4260
4261Miscellaneous Changes
4262---------------------
4263
4264The following miscellaneous changes occurred:
4265
4266- Default to using the Thumb-2 instruction set for armv7a and above. If
4267 you have any custom recipes that build software that needs to be
4268 built with the ARM instruction set, change the recipe to set the
4269 instruction set as follows: ARM_INSTRUCTION_SET = "arm"
4270
4271- ``run-postinsts`` no longer uses ``/etc/*-postinsts`` for
4272 ``dpkg/opkg`` in favor of built-in postinst support. RPM behavior
4273 remains unchanged.
4274
4275- The ``NOISO`` and ``NOHDD`` variables are no longer used. You now
4276 control building ``*.iso`` and ``*.hddimg`` image types directly by
4277 using the ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable.
4278
4279- The ``scripts/contrib/mkefidisk.sh`` has been removed in favor of
4280 Wic.
4281
4282- ``kernel-modules`` has been removed from
4283 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__ for ``qemumips`` and
4284 ``qemumips64`` machines. Removal also impacts the ``x86-base.inc``
4285 file.
4286
4287 .. note::
4288
4289 genericx86
4290 and
4291 genericx86-64
4292 retain
4293 kernel-modules
4294 as part of the
4295 RRECOMMENDS
4296 variable setting.
4297
4298- The ``LGPLv2_WHITELIST_GPL-3.0`` variable has been removed. If you
4299 are setting this variable in your configuration, set or append it to
4300 the ``WHITELIST_GPL-3.0`` variable instead.
4301
4302- ``${ASNEEDED}`` is now included in the
4303 ```TARGET_LDFLAGS`` <#var-TARGET_LDFLAGS>`__ variable directly. The
4304 remaining definitions from ``meta/conf/distro/include/as-needed.inc``
4305 have been moved to corresponding recipes.
4306
4307- Support for DSA host keys has been dropped from the OpenSSH recipes.
4308 If you are still using DSA keys, you must switch over to a more
4309 secure algorithm as recommended by OpenSSH upstream.
4310
4311- The ``dhcp`` recipe now uses the ``dhcpd6.conf`` configuration file
4312 in ``dhcpd6.service`` for IPv6 DHCP rather than re-using
4313 ``dhcpd.conf``, which is now reserved for IPv4.
4314
4315Moving to the Yocto Project 2.7 Release
4316=======================================
4317
4318This section provides migration information for moving to the Yocto
4319Project 2.7 Release from the prior release.
4320
4321.. _migration-2.7-bitbake-changes:
4322
4323BitBake Changes
4324---------------
4325
4326The following changes have been made to BitBake:
4327
4328- BitBake now checks anonymous Python functions and pure Python
4329 functions (e.g. ``def funcname:``) in the metadata for tab
4330 indentation. If found, BitBake produces a warning.
4331
4332- Bitbake now checks
4333 ```BBFILE_COLLECTIONS`` <#var-BBFILE_COLLECTIONS>`__ for duplicate
4334 entries and triggers an error if any are found.
4335
4336.. _migration-2.7-eclipse-support-dropped:
4337
4338Eclipse Support Removed
4339-----------------------
4340
4341Support for the Eclipse IDE has been removed. Support continues for
4342those releases prior to 2.7 that did include support. The 2.7 release
4343does not include the Eclipse Yocto plugin.
4344
4345.. _migration-2.7-qemu-native-splits-system-and-user-mode-parts:
4346
4347``qemu-native`` Splits the System and User-Mode Parts
4348-----------------------------------------------------
4349
4350The system and user-mode parts of ``qemu-native`` are now split.
4351``qemu-native`` provides the user-mode components and
4352``qemu-system-native`` provides the system components. If you have
4353recipes that depend on QEMU's system emulation functionality at build
4354time, they should now depend upon ``qemu-system-native`` instead of
4355``qemu-native``.
4356
4357.. _migration-2.7-upstream-tracking.inc-removed:
4358
4359The ``upstream-tracking.inc`` File Has Been Removed
4360---------------------------------------------------
4361
4362The previously deprecated ``upstream-tracking.inc`` file is now removed.
4363Any ``UPSTREAM_TRACKING*`` variables are now set in the corresponding
4364recipes instead.
4365
4366Remove any references you have to the ``upstream-tracking.inc`` file in
4367your configuration.
4368
4369.. _migration-2.7-distro-features-libc-removed:
4370
4371The ``DISTRO_FEATURES_LIBC`` Variable Has Been Removed
4372------------------------------------------------------
4373
4374The ``DISTRO_FEATURES_LIBC`` variable is no longer used. The ability to
4375configure glibc using kconfig has been removed for quite some time
4376making the ``libc-*`` features set no longer effective.
4377
4378Remove any references you have to ``DISTRO_FEATURES_LIBC`` in your own
4379layers.
4380
4381.. _migration-2.7-license-values:
4382
4383License Value Corrections
4384-------------------------
4385
4386The following corrections have been made to the
4387```LICENSE`` <#var-LICENSE>`__ values set by recipes: *socat*: Corrected
4388``LICENSE`` to be "GPLv2" rather than "GPLv2+". *libgfortran*: Set
4389license to "GPL-3.0-with-GCC-exception". *elfutils*: Removed
4390"Elfutils-Exception" and set to "GPLv2" for shared libraries
4391
4392.. _migration-2.7-packaging-changes:
4393
4394Packaging Changes
4395-----------------
4396
4397This section provides information about packaging changes.
4398
4399- ``bind``: The ``nsupdate`` binary has been moved to the
4400 ``bind-utils`` package.
4401
4402- Debug split: The default debug split has been changed to create
4403 separate source packages (i.e. package_name\ ``-dbg`` and
4404 package_name\ ``-src``). If you are currently using ``dbg-pkgs`` in
4405 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ to bring in debug
4406 symbols and you still need the sources, you must now also add
4407 ``src-pkgs`` to ``IMAGE_FEATURES``. Source packages remain in the
4408 target portion of the SDK by default, unless you have set your own
4409 value for ```SDKIMAGE_FEATURES`` <#var-SDKIMAGE_FEATURES>`__ that
4410 does not include ``src-pkgs``.
4411
4412- Mount all using ``util-linux``: ``/etc/default/mountall`` has moved
4413 into the -mount sub-package.
4414
4415- Splitting binaries using ``util-linux``: ``util-linux`` now splits
4416 each binary into its own package for fine-grained control. The main
4417 ``util-linux`` package pulls in the individual binary packages using
4418 the ```RRECOMMENDS`` <#var-RRECOMMENDS>`__ and
4419 ```RDEPENDS`` <#var-RDEPENDS>`__ variables. As a result, existing
4420 images should not see any changes assuming
4421 ```NO_RECOMMENDATIONS`` <#var-NO_RECOMMENDATIONS>`__ is not set.
4422
4423- ``netbase/base-files``: ``/etc/hosts`` has moved from ``netbase`` to
4424 ``base-files``.
4425
4426- ``tzdata``: The main package has been converted to an empty meta
4427 package that pulls in all ``tzdata`` packages by default.
4428
4429- ``lrzsz``: This package has been removed from
4430 ``packagegroup-self-hosted`` and
4431 ``packagegroup-core-tools-testapps``. The X/Y/ZModem support is less
4432 likely to be needed on modern systems. If you are relying on these
4433 packagegroups to include the ``lrzsz`` package in your image, you now
4434 need to explicitly add the package.
4435
4436.. _migration-2.7-removed-recipes:
4437
4438Removed Recipes
4439---------------
4440
4441The following recipes have been removed: *gcc*: Drop version 7.3
4442recipes. Version 8.3 now remains. *linux-yocto*: Drop versions 4.14 and
44434.18 recipes. Versions 4.19 and 5.0 remain. *go*: Drop version 1.9
4444recipes. Versions 1.11 and 1.12 remain. *xvideo-tests*: Became obsolete.
4445*libart-lgpl*: Became obsolete. *gtk-icon-utils-native*: These tools are
4446now provided by gtk+3-native *gcc-cross-initial*: No longer needed.
4447gcc-cross/gcc-crosssdk is now used instead. *gcc-crosssdk-initial*: No
4448longer needed. gcc-cross/gcc-crosssdk is now used instead.
4449*glibc-initial*: Removed because the benefits of having it for
4450site_config are currently outweighed by the cost of building the recipe.
4451
4452.. _migration-2.7-removed-classes:
4453
4454Removed Classes
4455---------------
4456
4457The following classes have been removed: *distutils-tools*: This class
4458was never used. *bugzilla.bbclass*: Became obsolete. *distrodata*: This
4459functionally has been replaced by a more modern tinfoil-based
4460implementation.
4461
4462.. _migration-2.7-miscellaneous-changes:
4463
4464Miscellaneous Changes
4465---------------------
4466
4467The following miscellaneous changes occurred:
4468
4469- The ``distro`` subdirectory of the Poky repository has been removed
4470 from the top-level ``scripts`` directory.
4471
4472- Perl now builds for the target using
4473 ```perl-cross`` <http://arsv.github.io/perl-cross/>`__ for better
4474 maintainability and improved build performance. This change should
4475 not present any problems unless you have heavily customized your Perl
4476 recipe.
4477
4478- ``arm-tunes``: Removed the "-march" option if mcpu is already added.
4479
4480- ``update-alternatives``: Convert file renames to
4481 ```PACKAGE_PREPROCESS_FUNCS`` <#var-PACKAGE_PREPROCESS_FUNCS>`__
4482
4483- ``base/pixbufcache``: Obsolete ``sstatecompletions`` code has been
4484 removed.
4485
4486- ```native`` <#ref-classes-native>`__ class:
4487 ```RDEPENDS`` <#var-RDEPENDS>`__ handling has been enabled.
4488
4489- ``inetutils``: This recipe has rsh disabled.
4490
4491Moving to the Yocto Project 3.0 Release
4492=======================================
4493
4494This section provides migration information for moving to the Yocto
4495Project 3.0 Release from the prior release.
4496
4497.. _migration-3.0-init-system-selection:
4498
4499Init System Selection
4500---------------------
4501
4502Changing the init system manager previously required setting a number of
4503different variables. You can now change the manager by setting the
4504``INIT_MANAGER`` variable and the corresponding include files (i.e.
4505``conf/distro/include/init-manager-*.conf``). Include files are provided
4506for four values: "none", "sysvinit", "systemd", and "mdev-busybox". The
4507default value, "none", for ``INIT_MANAGER`` should allow your current
4508settings to continue working. However, it is advisable to explicitly set
4509``INIT_MANAGER``.
4510
4511.. _migration-3.0-lsb-support-removed:
4512
4513LSB Support Removed
4514-------------------
4515
4516Linux Standard Base (LSB) as a standard is not current, and is not well
4517suited for embedded applications. Support can be continued in a separate
4518layer if needed. However, presently LSB support has been removed from
4519the core.
4520
4521As a result of this change, the ``poky-lsb`` derivative distribution
4522configuration that was also used for testing alternative configurations
4523has been replaced with a ``poky-altcfg`` distribution that has LSB parts
4524removed.
4525
4526.. _migration-3.0-removed-recipes:
4527
4528Removed Recipes
4529---------------
4530
4531The following recipes have been removed.
4532
4533- ``core-image-lsb-dev``: Part of removed LSB support.
4534
4535- ``core-image-lsb``: Part of removed LSB support.
4536
4537- ``core-image-lsb-sdk``: Part of removed LSB support.
4538
4539- ``cve-check-tool``: Functionally replaced by the ``cve-update-db``
4540 recipe and ``cve-check`` class.
4541
4542- ``eglinfo``: No longer maintained. ``eglinfo`` from ``mesa-demos`` is
4543 an adequate and maintained alternative.
4544
4545- ``gcc-8.3``: Version 8.3 removed. Replaced by 9.2.
4546
4547- ``gnome-themes-standard``: Only needed by gtk+ 2.x, which has been
4548 removed.
4549
4550- ``gtk+``: GTK+ 2 is obsolete and has been replaced by gtk+3.
4551
4552- ``irda-utils``: Has become obsolete. IrDA support has been removed
4553 from the Linux kernel in version 4.17 and later.
4554
4555- ``libnewt-python``: ``libnewt`` Python support merged into main
4556 ``libnewt`` recipe.
4557
4558- ``libsdl``: Replaced by newer ``libsdl2``.
4559
4560- ``libx11-diet``: Became obsolete.
4561
4562- ``libxx86dga``: Removed obsolete client library.
4563
4564- ``libxx86misc``: Removed. Library is redundant.
4565
4566- ``linux-yocto``: Version 5.0 removed, which is now redundant (5.2 /
4567 4.19 present).
4568
4569- ``lsbinitscripts``: Part of removed LSB support.
4570
4571- ``lsb``: Part of removed LSB support.
4572
4573- ``lsbtest``: Part of removed LSB support.
4574
4575- ``openssl10``: Replaced by newer ``openssl`` version 1.1.
4576
4577- ``packagegroup-core-lsb``: Part of removed LSB support.
4578
4579- ``python-nose``: Removed the Python 2.x version of the recipe.
4580
4581- ``python-numpy``: Removed the Python 2.x version of the recipe.
4582
4583- ``python-scons``: Removed the Python 2.x version of the recipe.
4584
4585- ``source-highlight``: No longer needed.
4586
4587- ``stress``: Replaced by ``stress-ng``.
4588
4589- ``vulkan``: Split into ``vulkan-loader``, ``vulkan-headers``, and
4590 ``vulkan-tools``.
4591
4592- ``weston-conf``: Functionality moved to ``weston-init``.
4593
4594.. _migration-3.0-packaging-changes:
4595
4596Packaging Changes
4597-----------------
4598
4599The following packaging changes have occurred.
4600
4601- The `Epiphany <https://en.wikipedia.org/wiki/GNOME_Web>`__ browser
4602 has been dropped from ``packagegroup-self-hosted`` as it has not been
4603 needed inside ``build-appliance-image`` for quite some time and was
4604 causing resource problems.
4605
4606- ``libcap-ng`` Python support has been moved to a separate
4607 ``libcap-ng-python`` recipe to streamline the build process when the
4608 Python bindings are not needed.
4609
4610- ``libdrm`` now packages the file ``amdgpu.ids`` into a separate
4611 ``libdrm-amdgpu`` package.
4612
4613- ``python3``: The ``runpy`` module is now in the ``python3-core``
4614 package as it is required to support the common "python3 -m" command
4615 usage.
4616
4617- ``distcc`` now provides separate ``distcc-client`` and
4618 ``distcc-server`` packages as typically one or the other are needed,
4619 rather than both.
4620
4621- ``python*-setuptools`` recipes now separately package the
4622 ``pkg_resources`` module in a ``python-pkg-resources`` /
4623 ``python3-pkg-resources`` package as the module is useful independent
4624 of the rest of the setuptools package. The main ``python-setuptools``
4625 / ``python3-setuptools`` package depends on this new package so you
4626 should only need to update dependencies unless you want to take
4627 advantage of the increased granularity.
4628
4629.. _migration-3.0-cve-checking:
4630
4631CVE Checking
4632------------
4633
4634``cve-check-tool`` has been functionally replaced by a new
4635``cve-update-db`` recipe and functionality built into the ``cve-check``
4636class. The result uses NVD JSON data feeds rather than the deprecated
4637XML feeds that ``cve-check-tool`` was using, supports CVSSv3 scoring,
4638and makes other improvements.
4639
4640Additionally, the ``CVE_CHECK_CVE_WHITELIST`` variable has been replaced
4641by ``CVE_CHECK_WHITELIST``.
4642
4643.. _migration-3.0-bitbake-changes:
4644
4645Bitbake Changes
4646---------------
4647
4648The following BitBake changes have occurred.
4649
4650- ``addtask`` statements now properly validate dependent tasks.
4651 Previously, an invalid task was silently ignored. With this change,
4652 the invalid task generates a warning.
4653
4654- Other invalid ``addtask`` and ``deltask`` usages now trigger these
4655 warnings: "multiple target tasks arguments with addtask / deltask",
4656 and "multiple before/after clauses".
4657
4658- The "multiconfig" prefix is now shortened to "mc". "multiconfig" will
4659 continue to work, however it may be removed in a future release.
4660
4661- The ``bitbake -g`` command no longer generates a
4662 ``recipe-depends.dot`` file as the contents (i.e. a reprocessed
4663 version of ``task-depends.dot``) were confusing.
4664
4665- The ``bb.build.FuncFailed`` exception, previously raised by
4666 ``bb.build.exec_func()`` when certain other exceptions have occurred,
4667 has been removed. The real underlying exceptions will be raised
4668 instead. If you have calls to ``bb.build.exec_func()`` in custom
4669 classes or ``tinfoil-using`` scripts, any references to
4670 ``bb.build.FuncFailed`` should be cleaned up.
4671
4672- Additionally, the ``bb.build.exec_func()`` no longer accepts the
4673 "pythonexception" parameter. The function now always raises
4674 exceptions. Remove this argument in any calls to
4675 ``bb.build.exec_func()`` in custom classes or scripts.
4676
4677- The
4678 ```BB_SETSCENE_VERIFY_FUNCTION2`` <&YOCTO_DOCS_BB_URL;#var-bb-BB_SETSCENE_VERIFY_FUNCTION2>`__
4679 is no longer used. In the unlikely event that you have any references
4680 to it, they should be removed.
4681
4682- The ``RunQueueExecuteScenequeue`` and ``RunQueueExecuteTasks`` events
4683 have been removed since setscene tasks are now executed as part of
4684 the normal runqueue. Any event handling code in custom classes or
4685 scripts that handles these two events need to be updated.
4686
4687- The arguments passed to functions used with
4688 ```BB_HASHCHECK_FUNCTION`` <&YOCTO_DOCS_BB_URL;#var-bb-BB_HASHCHECK_FUNCTION>`__
4689 have changed. If you are using your own custom hash check function,
4690 see
4691 ` <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725>`__
4692 for details.
4693
4694- Task specifications in ``BB_TASKDEPDATA`` and class implementations
4695 used in signature generator classes now use "<fn>:<task>" everywhere
4696 rather than the "." delimiter that was being used in some places.
4697 This change makes it consistent with all areas in the code. Custom
4698 signature generator classes and code that reads ``BB_TASKDEPDATA``
4699 need to be updated to use ':' as a separator rather than '.'.
4700
4701.. _migration-3.0-sanity-checks:
4702
4703Sanity Checks
4704-------------
4705
4706The following sanity check changes occurred.
4707
4708- ```SRC_URI`` <#var-SRC_URI>`__ is now checked for usage of two
4709 problematic items:
4710
4711 - "${PN}" prefix/suffix use - Warnings always appear if ${PN} is
4712 used. You must fix the issue regardless of whether multiconfig or
4713 anything else that would cause prefixing/suffixing to happen.
4714
4715 - Github archive tarballs - these are not guaranteed to be stable.
4716 Consequently, it is likely that the tarballs will be refreshed and
4717 thus the SRC_URI checksums will fail to apply. It is recommended
4718 that you fetch either an official release tarball or a specific
4719 revision from the actual Git repository instead.
4720
4721 Either one of these items now trigger a warning by default. If you
4722 wish to disable this check, remove ``src-uri-bad`` from
4723 ```WARN_QA`` <#var-WARN_QA>`__.
4724
4725- The ``file-rdeps`` runtime dependency check no longer expands
4726 ```RDEPENDS`` <#var-RDEPENDS>`__ recursively as there is no mechanism
4727 to ensure they can be fully computed, and thus races sometimes result
4728 in errors either showing up or not. Thus, you might now see errors
4729 for missing runtime dependencies that were previously satisfied
4730 recursively. Here is an example: package A contains a shell script
4731 starting with ``#!/bin/bash`` but has no dependency on bash. However,
4732 package A depends on package B, which does depend on bash. You need
4733 to add the missing dependency or dependencies to resolve the warning.
4734
4735- Setting ``DEPENDS_${PN}`` anywhere (i.e. typically in a recipe) now
4736 triggers an error. The error is triggered because
4737 ```DEPENDS`` <#var-DEPENDS>`__ is not a package-specific variable
4738 unlike RDEPENDS. You should set ``DEPENDS`` instead.
4739
4740- systemd currently does not work well with the musl C library because
4741 only upstream officially supports linking the library with glibc.
4742 Thus, a warning is shown when building systemd in conjunction with
4743 musl.
4744
4745.. _migration-3.0-miscellaneous-changes:
4746
4747Miscellaneous Changes
4748---------------------
4749
4750The following miscellaneous changes have occurred.
4751
4752- The ``gnome`` class has been removed because it now does very little.
4753 You should update recipes that previously inherited this class to do
4754 the following: inherit gnomebase gtk-icon-cache gconf mime
4755
4756- The ``meta/recipes-kernel/linux/linux-dtb.inc`` file has been
4757 removed. This file was previously deprecated in favor of setting
4758 ```KERNEL_DEVICETREE`` <#var-KERNEL_DEVICETREE>`__ in any kernel
4759 recipe and only produced a warning. Remove any ``include`` or
4760 ``require`` statements pointing to this file.
4761
4762- ```TARGET_CFLAGS`` <#var-TARGET_CFLAGS>`__,
4763 ```TARGET_CPPFLAGS`` <#var-TARGET_CPPFLAGS>`__,
4764 ```TARGET_CXXFLAGS`` <#var-TARGET_CXXFLAGS>`__, and
4765 ```TARGET_LDFLAGS`` <#var-TARGET_LDFLAGS>`__ are no longer exported
4766 to the external environment. This change did not require any changes
4767 to core recipes, which is a good indicator that no changes will be
4768 required. However, if for some reason the software being built by one
4769 of your recipes is expecting these variables to be set, then building
4770 the recipe will fail. In such cases, you must either export the
4771 variable or variables in the recipe or change the scripts so that
4772 exporting is not necessary.
4773
4774- You must change the host distro identifier used in
4775 ```NATIVELSBSTRING`` <#var-NATIVELSBSTRING>`__ to use all lowercase
4776 characters even if it does not contain a version number. This change
4777 is necessary only if you are not using ``uninative`` and
4778 ```SANITY_TESTED_DISTROS`` <#var-SANITY_TESTED_DISTROS>`__.
4779
4780- In the ``base-files`` recipe, writing the hostname into
4781 ``/etc/hosts`` and ``/etc/hostname`` is now done within the main
4782 ```do_install`` <#ref-tasks-install>`__ function rather than in the
4783 ``do_install_basefilesissue`` function. The reason for the change is
4784 because ``do_install_basefilesissue`` is more easily overridden
4785 without having to duplicate the hostname functionality. If you have
4786 done the latter (e.g. in a ``base-files`` bbappend), then you should
4787 remove it from your customized ``do_install_basefilesissue``
4788 function.
4789
4790- The ``wic --expand`` command now uses commas to separate "key:value"
4791 pairs rather than hyphens.
4792
4793 .. note::
4794
4795 The wic command-line help is not updated.
4796
4797 You must update any scripts or commands where you use
4798 ``wic --expand`` with multiple "key:value" pairs.
4799
4800- UEFI image variable settings have been moved from various places to a
4801 central ``conf/image-uefi.conf``. This change should not influence
4802 any existing configuration as the ``meta/conf/image-uefi.conf`` in
4803 the core metadata sets defaults that can be overridden in the same
4804 manner as before.
4805
4806- ``conf/distro/include/world-broken.inc`` has been removed. For cases
4807 where certain recipes need to be disabled when using the musl C
4808 library, these recipes now have ``COMPATIBLE_HOST_libc-musl`` set
4809 with a comment that explains why.
4810
4811Moving to the Yocto Project 3.1 Release
4812=======================================
4813
4814This section provides migration information for moving to the Yocto
4815Project 3.1 Release from the prior release.
4816
4817.. _migration-3.1-minimum-system-requirements:
4818
4819Minimum system requirements
4820---------------------------
4821
4822The following versions / requirements of build host components have been
4823updated:
4824
4825- gcc 5.0
4826
4827- python 3.5
4828
4829- tar 1.28
4830
4831- ``rpcgen`` is now required on the host (part of the ``libc-dev-bin``
4832 package on Ubuntu, Debian and related distributions, and the
4833 ``glibc`` package on RPM-based distributions).
4834
4835Additionally, the ``makeinfo`` and ``pod2man`` tools are *no longer*
4836required on the host.
4837
4838.. _migration-3.1-mpc8315e-rdb-removed:
4839
4840mpc8315e-rdb machine removed
4841----------------------------
4842
4843The MPC8315E-RDB machine is old/obsolete and unobtainable, thus given
4844the maintenance burden the ``mpc8315e-rdb`` machine configuration that
4845supported it has been removed in this release. The removal does leave a
4846gap in official PowerPC reference hardware support; this may change in
4847future if a suitable machine with accompanying support resources is
4848found.
4849
4850.. _migration-3.1-python-2-removed:
4851
4852Python 2 removed
4853----------------
4854
4855Due to the expiration of upstream support in January 2020, support for
4856Python 2 has now been removed; it is recommended that you use Python 3
4857instead. If absolutely needed there is a meta-python2 community layer
4858containing Python 2, related classes and various Python 2-based modules,
4859however it should not be considered as supported.
4860
4861.. _migration-3.1-reproducible-builds:
4862
4863Reproducible builds now enabled by default
4864------------------------------------------
4865
4866In order to avoid unnecessary differences in output files (aiding binary
4867reproducibility), the Poky distribution configuration
4868(``DISTRO = "poky"``) now inherits the ``reproducible_build`` class by
4869default.
4870
4871.. _migration-3.1-ptest-feature-impact:
4872
4873Impact of ptest feature is now more significant
4874-----------------------------------------------
4875
4876The Poky distribution configuration (``DISTRO = "poky"``) enables ptests
4877by default to enable runtime testing of various components. In this
4878release, a dependency needed to be added that has resulted in a
4879significant increase in the number of components that will be built just
4880when building a simple image such as core-image-minimal. If you do not
4881need runtime tests enabled for core components, then it is recommended
4882that you remove "ptest" from
4883```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ to save a significant
4884amount of build time e.g. by adding the following in your configuration:
4885DISTRO_FEATURES_remove = "ptest"
4886
4887.. _migration-3.1-removed-recipes:
4888
4889Removed recipes
4890---------------
4891
4892The following recipes have been removed:
4893
4894- ``chkconfig``: obsolete
4895
4896- ``console-tools``: obsolete
4897
4898- ``enchant``: replaced by ``enchant2``
4899
4900- ``foomatic-filters``: obsolete
4901
4902- ``libidn``: no longer needed, moved to meta-oe
4903
4904- ``libmodulemd``: replaced by ``libmodulemd-v1``
4905
4906- ``linux-yocto``: drop 4.19, 5.2 version recipes (5.4 now provided)
4907
4908- ``nspr``: no longer needed, moved to meta-oe
4909
4910- ``nss``: no longer needed, moved to meta-oe
4911
4912- ``python``: Python 2 removed (Python 3 preferred)
4913
4914- ``python-setuptools``: Python 2 version removed (python3-setuptools
4915 preferred)
4916
4917- ``sysprof``: no longer needed, moved to meta-oe
4918
4919- ``texi2html``: obsolete
4920
4921- ``u-boot-fw-utils``: functionally replaced by ``libubootenv``
4922
4923.. _migration-3.1-features-check:
4924
4925features_check class replaces distro_features_check
4926---------------------------------------------------
4927
4928The ``distro_features_check`` class has had its functionality expanded,
4929now supporting ``ANY_OF_MACHINE_FEATURES``,
4930``REQUIRED_MACHINE_FEATURES``, ``CONFLICT_MACHINE_FEATURES``,
4931``ANY_OF_COMBINED_FEATURES``, ``REQUIRED_COMBINED_FEATURES``,
4932``CONFLICT_COMBINED_FEATURES``. As a result the class has now been
4933renamed to ``features_check``; the ``distro_features_check`` class still
4934exists but generates a warning and redirects to the new class. In
4935preparation for a future removal of the old class it is recommended that
4936you update recipes currently inheriting ``distro_features_check`` to
4937inherit ``features_check`` instead.
4938
4939.. _migration-3.1-removed-classes:
4940
4941Removed classes
4942---------------
4943
4944The following classes have been removed:
4945
4946- ``distutils-base``: moved to meta-python2
4947
4948- ``distutils``: moved to meta-python2
4949
4950- ``libc-common``: merged into the glibc recipe as nothing else used
4951 it.
4952
4953- ``python-dir``: moved to meta-python2
4954
4955- ``pythonnative``: moved to meta-python2
4956
4957- ``setuptools``: moved to meta-python2
4958
4959- ``tinderclient``: dropped as it was obsolete.
4960
4961.. _migration-3.1-src-uri-checksums:
4962
4963SRC_URI checksum behaviour
4964--------------------------
4965
4966Previously, recipes by tradition included both SHA256 and MD5 checksums
4967for remotely fetched files in ```SRC_URI`` <#var-SRC_URI>`__, even
4968though only one is actually mandated. However, the MD5 checksum does not
4969add much given its inherent weakness; thus when a checksum fails only
4970the SHA256 sum will now be printed. The md5sum will still be verified if
4971it is specified.
4972
4973.. _migration-3.1-npm:
4974
4975npm fetcher changes
4976-------------------
4977
4978The npm fetcher has been completely reworked in this release. The npm
4979fetcher now only fetches the package source itself and no longer the
4980dependencies; there is now also an npmsw fetcher which explicitly
4981fetches the shrinkwrap file and the dependencies. This removes the
4982slightly awkward ``NPM_LOCKDOWN`` and ``NPM_SHRINKWRAP`` variables which
4983pointed to local files; the lockdown file is no longer needed at all.
4984Additionally, the package name in ``npm://`` entries in
4985```SRC_URI`` <#var-SRC_URI>`__ is now specified using a ``package``
4986parameter instead of the earlier ``name`` which overlapped with the
4987generic ``name`` parameter. All recipes using the npm fetcher will need
4988to be changed as a result.
4989
4990An example of the new scheme: SRC_URI =
4991"npm://registry.npmjs.org;package=array-flatten;version=1.1.1 \\
4992npmsw://${THISDIR}/npm-shrinkwrap.json" Another example where the
4993sources are fetched from git rather than an npm repository: SRC_URI =
4994"git://github.com/foo/bar.git;protocol=https \\
4995npmsw://${THISDIR}/npm-shrinkwrap.json"
4996
4997devtool and recipetool have also been updated to match with the npm
4998fetcher changes. Other than producing working and more complete recipes
4999for npm sources, there is also a minor change to the command line for
5000devtool: the ``--fetch-dev`` option has been renamed to ``--npm-dev`` as
5001it is npm-specific.
5002
5003.. _migration-3.1-packaging-changes:
5004
5005Packaging changes
5006-----------------
5007
5008- ``intltool`` has been removed from ``packagegroup-core-sdk`` as it is
5009 rarely needed to build modern software - gettext can do most of the
5010 things it used to be needed for. ``intltool`` has also been removed
5011 from ``packagegroup-core-self-hosted`` as it is not needed to for
5012 standard builds.
5013
5014- git: ``git-am``, ``git-difftool``, ``git-submodule``, and
5015 ``git-request-pull`` are no longer perl-based, so are now installed
5016 with the main ``git`` package instead of within ``git-perltools``.
5017
5018- The ``ldconfig`` binary built as part of glibc has now been moved to
5019 its own ``ldconfig`` package (note no ``glibc-`` prefix). This
5020 package is in the ```RRECOMMENDS`` <#var-RRECOMMENDS>`__ of the main
5021 ``glibc`` package if ``ldconfig`` is present in
5022 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__.
5023
5024- ``libevent`` now splits each shared library into its own package (as
5025 Debian does). Since these are shared libraries and will be pulled in
5026 through the normal shared library dependency handling, there should
5027 be no impact to existing configurations other than less unnecessary
5028 libraries being installed in some cases.
5029
5030- linux-firmware now has a new package for ``bcm4366c`` and includes
5031 available NVRAM config files into the ``bcm43340``, ``bcm43362``,
5032 ``bcm43430`` and ``bcm4356-pcie`` packages.
5033
5034- ``harfbuzz`` now splits the new ``libharfbuzz-subset.so`` library
5035 into its own package to reduce the main package size in cases where
5036 ``libharfbuzz-subset.so`` is not needed.
5037
5038.. _migration-3.1-package-qa-warnings:
5039
5040Additional warnings
5041-------------------
5042
5043Warnings will now be shown at ``do_package_qa`` time in the following
5044circumstances:
5045
5046- A recipe installs ``.desktop`` files containing ``MimeType`` keys but
5047 does not inherit the new ``mime-xdg`` class
5048
5049- A recipe installs ``.xml`` files into ``${datadir}/mime/packages``
5050 but does not inherit the ``mime`` class
5051
5052.. _migration-3.1-x86-live-wic:
5053
5054``wic`` image type now used instead of ``live`` by default for x86
5055------------------------------------------------------------------
5056
5057``conf/machine/include/x86-base.inc`` (inherited by most x86 machine
5058configurations) now specifies ``wic`` instead of ``live`` by default in
5059```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__. The ``live`` image type will
5060likely be removed in a future release so it is recommended that you use
5061``wic`` instead.
5062
5063.. _migration-3.1-misc:
5064
5065Miscellaneous changes
5066---------------------
5067
5068- The undocumented ``SRC_DISTRIBUTE_LICENSES`` variable has now been
5069 removed in favour of a new ``AVAILABLE_LICENSES`` variable which is
5070 dynamically set based upon license files found in
5071 ``${COMMON_LICENSE_DIR}`` and ``${LICENSE_PATH}``.
5072
5073- The tune definition for big-endian microblaze machines is now
5074 ``microblaze`` instead of ``microblazeeb``.
5075
5076- ``newlib`` no longer has built-in syscalls. ``libgloss`` should then
5077 provide the syscalls, ``crt0.o`` and other functions that are no
5078 longer part of ``newlib`` itself. If you are using
5079 ``TCLIBC = "newlib"`` this now means that you must link applications
5080 with both ``newlib`` and ``libgloss``, whereas before ``newlib``
5081 would run in many configurations by itself.
diff --git a/documentation/ref-manual/ref-classes.rst b/documentation/ref-manual/ref-classes.rst
new file mode 100644
index 0000000000..3b6f450fab
--- /dev/null
+++ b/documentation/ref-manual/ref-classes.rst
@@ -0,0 +1,2881 @@
1*******
2Classes
3*******
4
5Class files are used to abstract common functionality and share it
6amongst multiple recipe (``.bb``) files. To use a class file, you simply
7make sure the recipe inherits the class. In most cases, when a recipe
8inherits a class it is enough to enable its features. There are cases,
9however, where in the recipe you might need to set variables or override
10some default behavior.
11
12Any `Metadata <#metadata>`__ usually found in a recipe can also be
13placed in a class file. Class files are identified by the extension
14``.bbclass`` and are usually placed in a ``classes/`` directory beneath
15the ``meta*/`` directory found in the `Source
16Directory <#source-directory>`__. Class files can also be pointed to by
17```BUILDDIR`` <#var-BUILDDIR>`__ (e.g. ``build/``) in the same way as
18``.conf`` files in the ``conf`` directory. Class files are searched for
19in ```BBPATH`` <#var-BBPATH>`__ using the same method by which ``.conf``
20files are searched.
21
22This chapter discusses only the most useful and important classes. Other
23classes do exist within the ``meta/classes`` directory in the Source
24Directory. You can reference the ``.bbclass`` files directly for more
25information.
26
27.. _ref-classes-allarch:
28
29``allarch.bbclass``
30===================
31
32The ``allarch`` class is inherited by recipes that do not produce
33architecture-specific output. The class disables functionality that is
34normally needed for recipes that produce executable binaries (such as
35building the cross-compiler and a C library as pre-requisites, and
36splitting out of debug symbols during packaging).
37
38.. note::
39
40 Unlike some distro recipes (e.g. Debian), OpenEmbedded recipes that
41 produce packages that depend on tunings through use of the
42 ```RDEPENDS`` <#var-RDEPENDS>`__ and
43 ```TUNE_PKGARCH`` <#var-TUNE_PKGARCH>`__ variables, should never be
44 configured for all architectures using ``allarch``. This is the case
45 even if the recipes do not produce architecture-specific output.
46
47 Configuring such recipes for all architectures causes the
48 ```do_package_write_*`` <#ref-tasks-package_write_deb>`__ tasks to
49 have different signatures for the machines with different tunings.
50 Additionally, unnecessary rebuilds occur every time an image for a
51 different ``MACHINE`` is built even when the recipe never changes.
52
53By default, all recipes inherit the ```base`` <#ref-classes-base>`__ and
54```package`` <#ref-classes-package>`__ classes, which enable
55functionality needed for recipes that produce executable output. If your
56recipe, for example, only produces packages that contain configuration
57files, media files, or scripts (e.g. Python and Perl), then it should
58inherit the ``allarch`` class.
59
60.. _ref-classes-archiver:
61
62``archiver.bbclass``
63====================
64
65The ``archiver`` class supports releasing source code and other
66materials with the binaries.
67
68For more details on the source archiver, see the "`Maintaining Open
69Source License Compliance During Your Product's
70Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
71section in the Yocto Project Development Tasks Manual. You can also see
72the ```ARCHIVER_MODE`` <#var-ARCHIVER_MODE>`__ variable for information
73about the variable flags (varflags) that help control archive creation.
74
75.. _ref-classes-autotools:
76
77``autotools*.bbclass``
78======================
79
80The ``autotools*`` classes support Autotooled packages.
81
82The ``autoconf``, ``automake``, and ``libtool`` packages bring
83standardization. This class defines a set of tasks (e.g. ``configure``,
84``compile`` and so forth) that work for all Autotooled packages. It
85should usually be enough to define a few standard variables and then
86simply ``inherit autotools``. These classes can also work with software
87that emulates Autotools. For more information, see the "`Autotooled
88Package <&YOCTO_DOCS_DEV_URL;#new-recipe-autotooled-package>`__" section
89in the Yocto Project Development Tasks Manual.
90
91By default, the ``autotools*`` classes use out-of-tree builds (i.e.
92``autotools.bbclass`` building with ``B != S``).
93
94If the software being built by a recipe does not support using
95out-of-tree builds, you should have the recipe inherit the
96``autotools-brokensep`` class. The ``autotools-brokensep`` class behaves
97the same as the ``autotools`` class but builds with ```B`` <#var-B>`__
98== ```S`` <#var-S>`__. This method is useful when out-of-tree build
99support is either not present or is broken.
100
101.. note::
102
103 It is recommended that out-of-tree support be fixed and used if at
104 all possible.
105
106It's useful to have some idea of how the tasks defined by the
107``autotools*`` classes work and what they do behind the scenes.
108
109- ```do_configure`` <#ref-tasks-configure>`__ - Regenerates the
110 configure script (using ``autoreconf``) and then launches it with a
111 standard set of arguments used during cross-compilation. You can pass
112 additional parameters to ``configure`` through the ``EXTRA_OECONF``
113 or ```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__
114 variables.
115
116- ```do_compile`` <#ref-tasks-compile>`__ - Runs ``make`` with
117 arguments that specify the compiler and linker. You can pass
118 additional arguments through the ``EXTRA_OEMAKE`` variable.
119
120- ```do_install`` <#ref-tasks-install>`__ - Runs ``make install`` and
121 passes in ``${``\ ```D`` <#var-D>`__\ ``}`` as ``DESTDIR``.
122
123.. _ref-classes-base:
124
125``base.bbclass``
126================
127
128The ``base`` class is special in that every ``.bb`` file implicitly
129inherits the class. This class contains definitions for standard basic
130tasks such as fetching, unpacking, configuring (empty by default),
131compiling (runs any ``Makefile`` present), installing (empty by default)
132and packaging (empty by default). These classes are often overridden or
133extended by other classes such as the
134```autotools`` <#ref-classes-autotools>`__ class or the
135```package`` <#ref-classes-package>`__ class.
136
137The class also contains some commonly used functions such as
138``oe_runmake``, which runs ``make`` with the arguments specified in
139```EXTRA_OEMAKE`` <#var-EXTRA_OEMAKE>`__ variable as well as the
140arguments passed directly to ``oe_runmake``.
141
142.. _ref-classes-bash-completion:
143
144``bash-completion.bbclass``
145===========================
146
147Sets up packaging and dependencies appropriate for recipes that build
148software that includes bash-completion data.
149
150.. _ref-classes-bin-package:
151
152``bin_package.bbclass``
153=======================
154
155The ``bin_package`` class is a helper class for recipes that extract the
156contents of a binary package (e.g. an RPM) and install those contents
157rather than building the binary from source. The binary package is
158extracted and new packages in the configured output package format are
159created. Extraction and installation of proprietary binaries is a good
160example use for this class.
161
162.. note::
163
164 For RPMs and other packages that do not contain a subdirectory, you
165 should specify an appropriate fetcher parameter to point to the
166 subdirectory. For example, if BitBake is using the Git fetcher (
167 git://
168 ), the "subpath" parameter limits the checkout to a specific subpath
169 of the tree. Here is an example where
170 ${BP}
171 is used so that the files are extracted into the subdirectory
172 expected by the default value of
173 S
174 :
175 ::
176
177 SRC_URI = "git://example.com/downloads/somepackage.rpm;subpath=${BP}"
178
179
180 See the "
181 Fetchers
182 " section in the BitBake User Manual for more information on
183 supported BitBake Fetchers.
184
185.. _ref-classes-binconfig:
186
187``binconfig.bbclass``
188=====================
189
190The ``binconfig`` class helps to correct paths in shell scripts.
191
192Before ``pkg-config`` had become widespread, libraries shipped shell
193scripts to give information about the libraries and include paths needed
194to build software (usually named ``LIBNAME-config``). This class assists
195any recipe using such scripts.
196
197During staging, the OpenEmbedded build system installs such scripts into
198the ``sysroots/`` directory. Inheriting this class results in all paths
199in these scripts being changed to point into the ``sysroots/`` directory
200so that all builds that use the script use the correct directories for
201the cross compiling layout. See the
202```BINCONFIG_GLOB`` <#var-BINCONFIG_GLOB>`__ variable for more
203information.
204
205.. _ref-classes-binconfig-disabled:
206
207``binconfig-disabled.bbclass``
208==============================
209
210An alternative version of the ```binconfig`` <#ref-classes-binconfig>`__
211class, which disables binary configuration scripts by making them return
212an error in favor of using ``pkg-config`` to query the information. The
213scripts to be disabled should be specified using the
214```BINCONFIG`` <#var-BINCONFIG>`__ variable within the recipe inheriting
215the class.
216
217.. _ref-classes-blacklist:
218
219``blacklist.bbclass``
220=====================
221
222The ``blacklist`` class prevents the OpenEmbedded build system from
223building specific recipes (blacklists them). To use this class, inherit
224the class globally and set ```PNBLACKLIST`` <#var-PNBLACKLIST>`__ for
225each recipe you wish to blacklist. Specify the ```PN`` <#var-PN>`__
226value as a variable flag (varflag) and provide a reason, which is
227reported, if the package is requested to be built as the value. For
228example, if you want to blacklist a recipe called "exoticware", you add
229the following to your ``local.conf`` or distribution configuration:
230INHERIT += "blacklist" PNBLACKLIST[exoticware] = "Not supported by our
231organization."
232
233.. _ref-classes-buildhistory:
234
235``buildhistory.bbclass``
236========================
237
238The ``buildhistory`` class records a history of build output metadata,
239which can be used to detect possible regressions as well as used for
240analysis of the build output. For more information on using Build
241History, see the "`Maintaining Build Output
242Quality <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__"
243section in the Yocto Project Development Tasks Manual.
244
245.. _ref-classes-buildstats:
246
247``buildstats.bbclass``
248======================
249
250The ``buildstats`` class records performance statistics about each task
251executed during the build (e.g. elapsed time, CPU usage, and I/O usage).
252
253When you use this class, the output goes into the
254```BUILDSTATS_BASE`` <#var-BUILDSTATS_BASE>`__ directory, which defaults
255to ``${TMPDIR}/buildstats/``. You can analyze the elapsed time using
256``scripts/pybootchartgui/pybootchartgui.py``, which produces a cascading
257chart of the entire build process and can be useful for highlighting
258bottlenecks.
259
260Collecting build statistics is enabled by default through the
261```USER_CLASSES`` <#var-USER_CLASSES>`__ variable from your
262``local.conf`` file. Consequently, you do not have to do anything to
263enable the class. However, if you want to disable the class, simply
264remove "buildstats" from the ``USER_CLASSES`` list.
265
266.. _ref-classes-buildstats-summary:
267
268``buildstats-summary.bbclass``
269==============================
270
271When inherited globally, prints statistics at the end of the build on
272sstate re-use. In order to function, this class requires the
273```buildstats`` <#ref-classes-buildstats>`__ class be enabled.
274
275.. _ref-classes-ccache:
276
277``ccache.bbclass``
278==================
279
280The ``ccache`` class enables the C/C++ Compiler Cache for the build.
281This class is used to give a minor performance boost during the build.
282However, using the class can lead to unexpected side-effects. Thus, it
283is recommended that you do not use this class. See
284` <http://ccache.samba.org/>`__ for information on the C/C++ Compiler
285Cache.
286
287.. _ref-classes-chrpath:
288
289``chrpath.bbclass``
290===================
291
292The ``chrpath`` class is a wrapper around the "chrpath" utility, which
293is used during the build process for ``nativesdk``, ``cross``, and
294``cross-canadian`` recipes to change ``RPATH`` records within binaries
295in order to make them relocatable.
296
297.. _ref-classes-clutter:
298
299``clutter.bbclass``
300===================
301
302The ``clutter`` class consolidates the major and minor version naming
303and other common items used by Clutter and related recipes.
304
305.. note::
306
307 Unlike some other classes related to specific libraries, recipes
308 building other software that uses Clutter do not need to inherit this
309 class unless they use the same recipe versioning scheme that the
310 Clutter and related recipes do.
311
312.. _ref-classes-cmake:
313
314``cmake.bbclass``
315=================
316
317The ``cmake`` class allows for recipes that need to build software using
318the `CMake <https://cmake.org/overview/>`__ build system. You can use
319the ```EXTRA_OECMAKE`` <#var-EXTRA_OECMAKE>`__ variable to specify
320additional configuration options to be passed using the ``cmake``
321command line.
322
323On the occasion that you would be installing custom CMake toolchain
324files supplied by the application being built, you should install them
325to the preferred CMake Module directory: ``${D}${datadir}/cmake/``
326Modules during
327```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__.
328
329.. _ref-classes-cml1:
330
331``cml1.bbclass``
332================
333
334The ``cml1`` class provides basic support for the Linux kernel style
335build configuration system.
336
337.. _ref-classes-compress_doc:
338
339``compress_doc.bbclass``
340========================
341
342Enables compression for man pages and info pages. This class is intended
343to be inherited globally. The default compression mechanism is gz (gzip)
344but you can select an alternative mechanism by setting the
345```DOC_COMPRESS`` <#var-DOC_COMPRESS>`__ variable.
346
347.. _ref-classes-copyleft_compliance:
348
349``copyleft_compliance.bbclass``
350===============================
351
352The ``copyleft_compliance`` class preserves source code for the purposes
353of license compliance. This class is an alternative to the ``archiver``
354class and is still used by some users even though it has been deprecated
355in favor of the ```archiver`` <#ref-classes-archiver>`__ class.
356
357.. _ref-classes-copyleft_filter:
358
359``copyleft_filter.bbclass``
360===========================
361
362A class used by the ```archiver`` <#ref-classes-archiver>`__ and
363```copyleft_compliance`` <#ref-classes-copyleft_compliance>`__ classes
364for filtering licenses. The ``copyleft_filter`` class is an internal
365class and is not intended to be used directly.
366
367.. _ref-classes-core-image:
368
369``core-image.bbclass``
370======================
371
372The ``core-image`` class provides common definitions for the
373``core-image-*`` image recipes, such as support for additional
374```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__.
375
376.. _ref-classes-cpan:
377
378``cpan*.bbclass``
379=================
380
381The ``cpan*`` classes support Perl modules.
382
383Recipes for Perl modules are simple. These recipes usually only need to
384point to the source's archive and then inherit the proper class file.
385Building is split into two methods depending on which method the module
386authors used.
387
388- Modules that use old ``Makefile.PL``-based build system require
389 ``cpan.bbclass`` in their recipes.
390
391- Modules that use ``Build.PL``-based build system require using
392 ``cpan_build.bbclass`` in their recipes.
393
394Both build methods inherit the ``cpan-base`` class for basic Perl
395support.
396
397.. _ref-classes-cross:
398
399``cross.bbclass``
400=================
401
402The ``cross`` class provides support for the recipes that build the
403cross-compilation tools.
404
405.. _ref-classes-cross-canadian:
406
407``cross-canadian.bbclass``
408==========================
409
410The ``cross-canadian`` class provides support for the recipes that build
411the Canadian Cross-compilation tools for SDKs. See the
412"`Cross-Development Toolchain
413Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
414section in the Yocto Project Overview and Concepts Manual for more
415discussion on these cross-compilation tools.
416
417.. _ref-classes-crosssdk:
418
419``crosssdk.bbclass``
420====================
421
422The ``crosssdk`` class provides support for the recipes that build the
423cross-compilation tools used for building SDKs. See the
424"`Cross-Development Toolchain
425Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
426section in the Yocto Project Overview and Concepts Manual for more
427discussion on these cross-compilation tools.
428
429.. _ref-classes-debian:
430
431``debian.bbclass``
432==================
433
434The ``debian`` class renames output packages so that they follow the
435Debian naming policy (i.e. ``glibc`` becomes ``libc6`` and
436``glibc-devel`` becomes ``libc6-dev``.) Renaming includes the library
437name and version as part of the package name.
438
439If a recipe creates packages for multiple libraries (shared object files
440of ``.so`` type), use the ```LEAD_SONAME`` <#var-LEAD_SONAME>`__
441variable in the recipe to specify the library on which to apply the
442naming scheme.
443
444.. _ref-classes-deploy:
445
446``deploy.bbclass``
447==================
448
449The ``deploy`` class handles deploying files to the
450```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__ directory. The main
451function of this class is to allow the deploy step to be accelerated by
452shared state. Recipes that inherit this class should define their own
453```do_deploy`` <#ref-tasks-deploy>`__ function to copy the files to be
454deployed to ```DEPLOYDIR`` <#var-DEPLOYDIR>`__, and use ``addtask`` to
455add the task at the appropriate place, which is usually after
456```do_compile`` <#ref-tasks-compile>`__ or
457```do_install`` <#ref-tasks-install>`__. The class then takes care of
458staging the files from ``DEPLOYDIR`` to ``DEPLOY_DIR_IMAGE``.
459
460.. _ref-classes-devshell:
461
462``devshell.bbclass``
463====================
464
465The ``devshell`` class adds the ``do_devshell`` task. Distribution
466policy dictates whether to include this class. See the "`Using a
467Development Shell <&YOCTO_DOCS_DEV_URL;#platdev-appdev-devshell>`__"
468section in the Yocto Project Development Tasks Manual for more
469information about using ``devshell``.
470
471.. _ref-classes-devupstream:
472
473``devupstream.bbclass``
474=======================
475
476The ``devupstream`` class uses
477```BBCLASSEXTEND`` <#var-BBCLASSEXTEND>`__ to add a variant of the
478recipe that fetches from an alternative URI (e.g. Git) instead of a
479tarball. Following is an example: BBCLASSEXTEND = "devupstream:target"
480SRC_URI_class-devupstream = "git://git.example.com/example"
481SRCREV_class-devupstream = "abcd1234" Adding the above statements to
482your recipe creates a variant that has
483```DEFAULT_PREFERENCE`` <#var-DEFAULT_PREFERENCE>`__ set to "-1".
484Consequently, you need to select the variant of the recipe to use it.
485Any development-specific adjustments can be done by using the
486``class-devupstream`` override. Here is an example:
487DEPENDS_append_class-devupstream = " gperf-native"
488do_configure_prepend_class-devupstream() { touch ${S}/README } The class
489currently only supports creating a development variant of the target
490recipe, not ``native`` or ``nativesdk`` variants.
491
492The ``BBCLASSEXTEND`` syntax (i.e. ``devupstream:target``) provides
493support for ``native`` and ``nativesdk`` variants. Consequently, this
494functionality can be added in a future release.
495
496Support for other version control systems such as Subversion is limited
497due to BitBake's automatic fetch dependencies (e.g.
498``subversion-native``).
499
500.. _ref-classes-distro_features_check:
501
502``distro_features_check.bbclass``
503=================================
504
505The ``distro_features_check`` class allows individual recipes to check
506for required and conflicting
507```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__.
508
509This class provides support for the
510```REQUIRED_DISTRO_FEATURES`` <#var-REQUIRED_DISTRO_FEATURES>`__ and
511```CONFLICT_DISTRO_FEATURES`` <#var-CONFLICT_DISTRO_FEATURES>`__
512variables. If any conditions specified in the recipe using the above
513variables are not met, the recipe will be skipped.
514
515.. _ref-classes-distutils:
516
517``distutils*.bbclass``
518======================
519
520The ``distutils*`` classes support recipes for Python version 2.x
521extensions, which are simple. These recipes usually only need to point
522to the source's archive and then inherit the proper class. Building is
523split into two methods depending on which method the module authors
524used.
525
526- Extensions that use an Autotools-based build system require Autotools
527 and the classes based on ``distutils`` in their recipes.
528
529- Extensions that use build systems based on ``distutils`` require the
530 ``distutils`` class in their recipes.
531
532- Extensions that use build systems based on ``setuptools`` require the
533 ```setuptools`` <#ref-classes-setuptools>`__ class in their recipes.
534
535The ``distutils-common-base`` class is required by some of the
536``distutils*`` classes to provide common Python2 support.
537
538.. _ref-classes-distutils3:
539
540``distutils3*.bbclass``
541=======================
542
543The ``distutils3*`` classes support recipes for Python version 3.x
544extensions, which are simple. These recipes usually only need to point
545to the source's archive and then inherit the proper class. Building is
546split into three methods depending on which method the module authors
547used.
548
549- Extensions that use an Autotools-based build system require Autotools
550 and ``distutils``-based classes in their recipes.
551
552- Extensions that use ``distutils``-based build systems require the
553 ``distutils`` class in their recipes.
554
555- Extensions that use build systems based on ``setuptools3`` require
556 the ```setuptools3`` <#ref-classes-setuptools>`__ class in their
557 recipes.
558
559The ``distutils3*`` classes either inherit their corresponding
560``distutils*`` class or replicate them using a Python3 version instead
561(e.g. ``distutils3-base`` inherits ``distutils-common-base``, which is
562the same as ``distutils-base`` but inherits ``python3native`` instead of
563``pythonnative``).
564
565.. _ref-classes-externalsrc:
566
567``externalsrc.bbclass``
568=======================
569
570The ``externalsrc`` class supports building software from source code
571that is external to the OpenEmbedded build system. Building software
572from an external source tree means that the build system's normal fetch,
573unpack, and patch process is not used.
574
575By default, the OpenEmbedded build system uses the ```S`` <#var-S>`__
576and ```B`` <#var-B>`__ variables to locate unpacked recipe source code
577and to build it, respectively. When your recipe inherits the
578``externalsrc`` class, you use the
579```EXTERNALSRC`` <#var-EXTERNALSRC>`__ and
580```EXTERNALSRC_BUILD`` <#var-EXTERNALSRC_BUILD>`__ variables to
581ultimately define ``S`` and ``B``.
582
583By default, this class expects the source code to support recipe builds
584that use the ```B`` <#var-B>`__ variable to point to the directory in
585which the OpenEmbedded build system places the generated objects built
586from the recipes. By default, the ``B`` directory is set to the
587following, which is separate from the source directory (``S``):
588${WORKDIR}/${BPN}/{PV}/ See these variables for more information:
589```WORKDIR`` <#var-WORKDIR>`__, ```BPN`` <#var-BPN>`__, and
590```PV`` <#var-PV>`__,
591
592For more information on the ``externalsrc`` class, see the comments in
593``meta/classes/externalsrc.bbclass`` in the `Source
594Directory <#source-directory>`__. For information on how to use the
595``externalsrc`` class, see the "`Building Software from an External
596Source <&YOCTO_DOCS_DEV_URL;#building-software-from-an-external-source>`__"
597section in the Yocto Project Development Tasks Manual.
598
599.. _ref-classes-extrausers:
600
601``extrausers.bbclass``
602======================
603
604The ``extrausers`` class allows additional user and group configuration
605to be applied at the image level. Inheriting this class either globally
606or from an image recipe allows additional user and group operations to
607be performed using the
608```EXTRA_USERS_PARAMS`` <#var-EXTRA_USERS_PARAMS>`__ variable.
609
610.. note::
611
612 The user and group operations added using the
613 extrausers
614 class are not tied to a specific recipe outside of the recipe for the
615 image. Thus, the operations can be performed across the image as a
616 whole. Use the
617 useradd
618 class to add user and group configuration to a specific recipe.
619
620Here is an example that uses this class in an image recipe: inherit
621extrausers EXTRA_USERS_PARAMS = "\\ useradd -p '' tester; \\ groupadd
622developers; \\ userdel nobody; \\ groupdel -g video; \\ groupmod -g 1020
623developers; \\ usermod -s /bin/sh tester; \\ " Here is an example that
624adds two users named "tester-jim" and "tester-sue" and assigns
625passwords: inherit extrausers EXTRA_USERS_PARAMS = "\\ useradd -P
626tester01 tester-jim; \\ useradd -P tester01 tester-sue; \\ " Finally,
627here is an example that sets the root password to "1876*18": inherit
628extrausers EXTRA_USERS_PARAMS = "\\ usermod -P 1876*18 root; \\ "
629
630.. _ref-classes-fontcache:
631
632``fontcache.bbclass``
633=====================
634
635The ``fontcache`` class generates the proper post-install and
636post-remove (postinst and postrm) scriptlets for font packages. These
637scriptlets call ``fc-cache`` (part of ``Fontconfig``) to add the fonts
638to the font information cache. Since the cache files are
639architecture-specific, ``fc-cache`` runs using QEMU if the postinst
640scriptlets need to be run on the build host during image creation.
641
642If the fonts being installed are in packages other than the main
643package, set ```FONT_PACKAGES`` <#var-FONT_PACKAGES>`__ to specify the
644packages containing the fonts.
645
646.. _ref-classes-fs-uuid:
647
648``fs-uuid.bbclass``
649===================
650
651The ``fs-uuid`` class extracts UUID from
652``${``\ ```ROOTFS`` <#var-ROOTFS>`__\ ``}``, which must have been built
653by the time that this function gets called. The ``fs-uuid`` class only
654works on ``ext`` file systems and depends on ``tune2fs``.
655
656.. _ref-classes-gconf:
657
658``gconf.bbclass``
659=================
660
661The ``gconf`` class provides common functionality for recipes that need
662to install GConf schemas. The schemas will be put into a separate
663package (``${``\ ```PN`` <#var-PN>`__\ ``}-gconf``) that is created
664automatically when this class is inherited. This package uses the
665appropriate post-install and post-remove (postinst/postrm) scriptlets to
666register and unregister the schemas in the target image.
667
668.. _ref-classes-gettext:
669
670``gettext.bbclass``
671===================
672
673The ``gettext`` class provides support for building software that uses
674the GNU ``gettext`` internationalization and localization system. All
675recipes building software that use ``gettext`` should inherit this
676class.
677
678.. _ref-classes-gnomebase:
679
680``gnomebase.bbclass``
681=====================
682
683The ``gnomebase`` class is the base class for recipes that build
684software from the GNOME stack. This class sets
685```SRC_URI`` <#var-SRC_URI>`__ to download the source from the GNOME
686mirrors as well as extending ```FILES`` <#var-FILES>`__ with the typical
687GNOME installation paths.
688
689.. _ref-classes-gobject-introspection:
690
691``gobject-introspection.bbclass``
692=================================
693
694Provides support for recipes building software that supports GObject
695introspection. This functionality is only enabled if the
696"gobject-introspection-data" feature is in
697```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ as well as
698"qemu-usermode" being in
699```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__.
700
701.. note::
702
703 This functionality is backfilled by default and, if not applicable,
704 should be disabled through
705 DISTRO_FEATURES_BACKFILL_CONSIDERED
706 or
707 MACHINE_FEATURES_BACKFILL_CONSIDERED
708 , respectively.
709
710.. _ref-classes-grub-efi:
711
712``grub-efi.bbclass``
713====================
714
715The ``grub-efi`` class provides ``grub-efi``-specific functions for
716building bootable images.
717
718This class supports several variables:
719
720- ```INITRD`` <#var-INITRD>`__: Indicates list of filesystem images to
721 concatenate and use as an initial RAM disk (initrd) (optional).
722
723- ```ROOTFS`` <#var-ROOTFS>`__: Indicates a filesystem image to include
724 as the root filesystem (optional).
725
726- ```GRUB_GFXSERIAL`` <#var-GRUB_GFXSERIAL>`__: Set this to "1" to have
727 graphics and serial in the boot menu.
728
729- ```LABELS`` <#var-LABELS>`__: A list of targets for the automatic
730 configuration.
731
732- ```APPEND`` <#var-APPEND>`__: An override list of append strings for
733 each ``LABEL``.
734
735- ```GRUB_OPTS`` <#var-GRUB_OPTS>`__: Additional options to add to the
736 configuration (optional). Options are delimited using semi-colon
737 characters (``;``).
738
739- ```GRUB_TIMEOUT`` <#var-GRUB_TIMEOUT>`__: Timeout before executing
740 the default ``LABEL`` (optional).
741
742.. _ref-classes-gsettings:
743
744``gsettings.bbclass``
745=====================
746
747The ``gsettings`` class provides common functionality for recipes that
748need to install GSettings (glib) schemas. The schemas are assumed to be
749part of the main package. Appropriate post-install and post-remove
750(postinst/postrm) scriptlets are added to register and unregister the
751schemas in the target image.
752
753.. _ref-classes-gtk-doc:
754
755``gtk-doc.bbclass``
756===================
757
758The ``gtk-doc`` class is a helper class to pull in the appropriate
759``gtk-doc`` dependencies and disable ``gtk-doc``.
760
761.. _ref-classes-gtk-icon-cache:
762
763``gtk-icon-cache.bbclass``
764==========================
765
766The ``gtk-icon-cache`` class generates the proper post-install and
767post-remove (postinst/postrm) scriptlets for packages that use GTK+ and
768install icons. These scriptlets call ``gtk-update-icon-cache`` to add
769the fonts to GTK+'s icon cache. Since the cache files are
770architecture-specific, ``gtk-update-icon-cache`` is run using QEMU if
771the postinst scriptlets need to be run on the build host during image
772creation.
773
774.. _ref-classes-gtk-immodules-cache:
775
776``gtk-immodules-cache.bbclass``
777===============================
778
779The ``gtk-immodules-cache`` class generates the proper post-install and
780post-remove (postinst/postrm) scriptlets for packages that install GTK+
781input method modules for virtual keyboards. These scriptlets call
782``gtk-update-icon-cache`` to add the input method modules to the cache.
783Since the cache files are architecture-specific,
784``gtk-update-icon-cache`` is run using QEMU if the postinst scriptlets
785need to be run on the build host during image creation.
786
787If the input method modules being installed are in packages other than
788the main package, set
789```GTKIMMODULES_PACKAGES`` <#var-GTKIMMODULES_PACKAGES>`__ to specify
790the packages containing the modules.
791
792.. _ref-classes-gzipnative:
793
794``gzipnative.bbclass``
795======================
796
797The ``gzipnative`` class enables the use of different native versions of
798``gzip`` and ``pigz`` rather than the versions of these tools from the
799build host.
800
801.. _ref-classes-icecc:
802
803``icecc.bbclass``
804=================
805
806The ``icecc`` class supports
807`Icecream <https://github.com/icecc/icecream>`__, which facilitates
808taking compile jobs and distributing them among remote machines.
809
810The class stages directories with symlinks from ``gcc`` and ``g++`` to
811``icecc``, for both native and cross compilers. Depending on each
812configure or compile, the OpenEmbedded build system adds the directories
813at the head of the ``PATH`` list and then sets the ``ICECC_CXX`` and
814``ICEC_CC`` variables, which are the paths to the ``g++`` and ``gcc``
815compilers, respectively.
816
817For the cross compiler, the class creates a ``tar.gz`` file that
818contains the Yocto Project toolchain and sets ``ICECC_VERSION``, which
819is the version of the cross-compiler used in the cross-development
820toolchain, accordingly.
821
822The class handles all three different compile stages (i.e native
823,cross-kernel and target) and creates the necessary environment
824``tar.gz`` file to be used by the remote machines. The class also
825supports SDK generation.
826
827If ```ICECC_PATH`` <#var-ICECC_PATH>`__ is not set in your
828``local.conf`` file, then the class tries to locate the ``icecc`` binary
829using ``which``. If ```ICECC_ENV_EXEC`` <#var-ICECC_ENV_EXEC>`__ is set
830in your ``local.conf`` file, the variable should point to the
831``icecc-create-env`` script provided by the user. If you do not point to
832a user-provided script, the build system uses the default script
833provided by the recipe ``icecc-create-env-native.bb``.
834
835.. note::
836
837 This script is a modified version and not the one that comes with
838 icecc
839 .
840
841If you do not want the Icecream distributed compile support to apply to
842specific recipes or classes, you can effectively "blacklist" them by
843listing the recipes and classes using the
844```ICECC_USER_PACKAGE_BL`` <#var-ICECC_USER_PACKAGE_BL>`__ and
845```ICECC_USER_CLASS_BL`` <#var-ICECC_USER_CLASS_BL>`__, variables,
846respectively, in your ``local.conf`` file. Doing so causes the
847OpenEmbedded build system to handle these compilations locally.
848
849Additionally, you can list recipes using the
850```ICECC_USER_PACKAGE_WL`` <#var-ICECC_USER_PACKAGE_WL>`__ variable in
851your ``local.conf`` file to force ``icecc`` to be enabled for recipes
852using an empty ```PARALLEL_MAKE`` <#var-PARALLEL_MAKE>`__ variable.
853
854Inheriting the ``icecc`` class changes all sstate signatures.
855Consequently, if a development team has a dedicated build system that
856populates ```STATE_MIRRORS`` <#var-SSTATE_MIRRORS>`__ and they want to
857reuse sstate from ``STATE_MIRRORS``, then all developers and the build
858system need to either inherit the ``icecc`` class or nobody should.
859
860At the distribution level, you can inherit the ``icecc`` class to be
861sure that all builders start with the same sstate signatures. After
862inheriting the class, you can then disable the feature by setting the
863```ICECC_DISABLED`` <#var-ICECC_DISABLED>`__ variable to "1" as follows:
864INHERIT_DISTRO_append = " icecc" ICECC_DISABLED ??= "1" This practice
865makes sure everyone is using the same signatures but also requires
866individuals that do want to use Icecream to enable the feature
867individually as follows in your ``local.conf`` file: ICECC_DISABLED = ""
868
869.. _ref-classes-image:
870
871``image.bbclass``
872=================
873
874The ``image`` class helps support creating images in different formats.
875First, the root filesystem is created from packages using one of the
876``rootfs*.bbclass`` files (depending on the package format used) and
877then one or more image files are created.
878
879- The ``IMAGE_FSTYPES`` variable controls the types of images to
880 generate.
881
882- The ``IMAGE_INSTALL`` variable controls the list of packages to
883 install into the image.
884
885For information on customizing images, see the "`Customizing
886Images <&YOCTO_DOCS_DEV_URL;#usingpoky-extend-customimage>`__" section
887in the Yocto Project Development Tasks Manual. For information on how
888images are created, see the
889"`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__" section in the
890Yocto Project Overview and Concpets Manual.
891
892.. _ref-classes-image-buildinfo:
893
894``image-buildinfo.bbclass``
895===========================
896
897The ``image-buildinfo`` class writes information to the target
898filesystem on ``/etc/build``.
899
900.. _ref-classes-image_types:
901
902``image_types.bbclass``
903=======================
904
905The ``image_types`` class defines all of the standard image output types
906that you can enable through the
907```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable. You can use this
908class as a reference on how to add support for custom image output
909types.
910
911By default, the ```image`` <#ref-classes-image>`__ class automatically
912enables the ``image_types`` class. The ``image`` class uses the
913``IMGCLASSES`` variable as follows: IMGCLASSES =
914"rootfs_${IMAGE_PKGTYPE} image_types ${IMAGE_CLASSES}" IMGCLASSES +=
915"${@['populate_sdk_base', 'populate_sdk_ext']['linux' in
916d.getVar("SDK_OS")]}" IMGCLASSES +=
917"${@bb.utils.contains_any('IMAGE_FSTYPES', 'live iso hddimg',
918'image-live', '', d)}" IMGCLASSES +=
919"${@bb.utils.contains('IMAGE_FSTYPES', 'container', 'image-container',
920'', d)}" IMGCLASSES += "image_types_wic" IMGCLASSES +=
921"rootfs-postcommands" IMGCLASSES += "image-postinst-intercepts" inherit
922${IMGCLASSES}
923
924The ``image_types`` class also handles conversion and compression of
925images.
926
927.. note::
928
929 To build a VMware VMDK image, you need to add "wic.vmdk" to
930 IMAGE_FSTYPES
931 . This would also be similar for Virtual Box Virtual Disk Image
932 ("vdi") and QEMU Copy On Write Version 2 ("qcow2") images.
933
934.. _ref-classes-image-live:
935
936``image-live.bbclass``
937======================
938
939This class controls building "live" (i.e. HDDIMG and ISO) images. Live
940images contain syslinux for legacy booting, as well as the bootloader
941specified by ```EFI_PROVIDER`` <#var-EFI_PROVIDER>`__ if
942```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__ contains "efi".
943
944Normally, you do not use this class directly. Instead, you add "live" to
945```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__.
946
947.. _ref-classes-image-mklibs:
948
949``image-mklibs.bbclass``
950========================
951
952The ``image-mklibs`` class enables the use of the ``mklibs`` utility
953during the ```do_rootfs`` <#ref-tasks-rootfs>`__ task, which optimizes
954the size of libraries contained in the image.
955
956By default, the class is enabled in the ``local.conf.template`` using
957the ```USER_CLASSES`` <#var-USER_CLASSES>`__ variable as follows:
958USER_CLASSES ?= "buildstats image-mklibs image-prelink"
959
960.. _ref-classes-image-prelink:
961
962``image-prelink.bbclass``
963=========================
964
965The ``image-prelink`` class enables the use of the ``prelink`` utility
966during the ```do_rootfs`` <#ref-tasks-rootfs>`__ task, which optimizes
967the dynamic linking of shared libraries to reduce executable startup
968time.
969
970By default, the class is enabled in the ``local.conf.template`` using
971the ```USER_CLASSES`` <#var-USER_CLASSES>`__ variable as follows:
972USER_CLASSES ?= "buildstats image-mklibs image-prelink"
973
974.. _ref-classes-insane:
975
976``insane.bbclass``
977==================
978
979The ``insane`` class adds a step to the package generation process so
980that output quality assurance checks are generated by the OpenEmbedded
981build system. A range of checks are performed that check the build's
982output for common problems that show up during runtime. Distribution
983policy usually dictates whether to include this class.
984
985You can configure the sanity checks so that specific test failures
986either raise a warning or an error message. Typically, failures for new
987tests generate a warning. Subsequent failures for the same test would
988then generate an error message once the metadata is in a known and good
989condition. See the "`QA Error and Warning Messages <#ref-qa-checks>`__"
990Chapter for a list of all the warning and error messages you might
991encounter using a default configuration.
992
993Use the ```WARN_QA`` <#var-WARN_QA>`__ and
994```ERROR_QA`` <#var-ERROR_QA>`__ variables to control the behavior of
995these checks at the global level (i.e. in your custom distro
996configuration). However, to skip one or more checks in recipes, you
997should use ```INSANE_SKIP`` <#var-INSANE_SKIP>`__. For example, to skip
998the check for symbolic link ``.so`` files in the main package of a
999recipe, add the following to the recipe. You need to realize that the
1000package name override, in this example ``${PN}``, must be used:
1001INSANE_SKIP_${PN} += "dev-so" Please keep in mind that the QA checks
1002exist in order to detect real or potential problems in the packaged
1003output. So exercise caution when disabling these checks.
1004
1005The following list shows the tests you can list with the ``WARN_QA`` and
1006``ERROR_QA`` variables:
1007
1008- *``already-stripped:``* Checks that produced binaries have not
1009 already been stripped prior to the build system extracting debug
1010 symbols. It is common for upstream software projects to default to
1011 stripping debug symbols for output binaries. In order for debugging
1012 to work on the target using ``-dbg`` packages, this stripping must be
1013 disabled.
1014
1015- *``arch:``* Checks the Executable and Linkable Format (ELF) type, bit
1016 size, and endianness of any binaries to ensure they match the target
1017 architecture. This test fails if any binaries do not match the type
1018 since there would be an incompatibility. The test could indicate that
1019 the wrong compiler or compiler options have been used. Sometimes
1020 software, like bootloaders, might need to bypass this check.
1021
1022- *``buildpaths:``* Checks for paths to locations on the build host
1023 inside the output files. Currently, this test triggers too many false
1024 positives and thus is not normally enabled.
1025
1026- *``build-deps:``* Determines if a build-time dependency that is
1027 specified through ```DEPENDS`` <#var-DEPENDS>`__, explicit
1028 ```RDEPENDS`` <#var-RDEPENDS>`__, or task-level dependencies exists
1029 to match any runtime dependency. This determination is particularly
1030 useful to discover where runtime dependencies are detected and added
1031 during packaging. If no explicit dependency has been specified within
1032 the metadata, at the packaging stage it is too late to ensure that
1033 the dependency is built, and thus you can end up with an error when
1034 the package is installed into the image during the
1035 ```do_rootfs`` <#ref-tasks-rootfs>`__ task because the auto-detected
1036 dependency was not satisfied. An example of this would be where the
1037 ```update-rc.d`` <#ref-classes-update-rc.d>`__ class automatically
1038 adds a dependency on the ``initscripts-functions`` package to
1039 packages that install an initscript that refers to
1040 ``/etc/init.d/functions``. The recipe should really have an explicit
1041 ``RDEPENDS`` for the package in question on ``initscripts-functions``
1042 so that the OpenEmbedded build system is able to ensure that the
1043 ``initscripts`` recipe is actually built and thus the
1044 ``initscripts-functions`` package is made available.
1045
1046- *``compile-host-path:``* Checks the
1047 ```do_compile`` <#ref-tasks-compile>`__ log for indications that
1048 paths to locations on the build host were used. Using such paths
1049 might result in host contamination of the build output.
1050
1051- *``debug-deps:``* Checks that all packages except ``-dbg`` packages
1052 do not depend on ``-dbg`` packages, which would cause a packaging
1053 bug.
1054
1055- *``debug-files:``* Checks for ``.debug`` directories in anything but
1056 the ``-dbg`` package. The debug files should all be in the ``-dbg``
1057 package. Thus, anything packaged elsewhere is incorrect packaging.
1058
1059- *``dep-cmp:``* Checks for invalid version comparison statements in
1060 runtime dependency relationships between packages (i.e. in
1061 ```RDEPENDS`` <#var-RDEPENDS>`__,
1062 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__,
1063 ```RSUGGESTS`` <#var-RSUGGESTS>`__,
1064 ```RPROVIDES`` <#var-RPROVIDES>`__,
1065 ```RREPLACES`` <#var-RREPLACES>`__, and
1066 ```RCONFLICTS`` <#var-RCONFLICTS>`__ variable values). Any invalid
1067 comparisons might trigger failures or undesirable behavior when
1068 passed to the package manager.
1069
1070- *``desktop:``* Runs the ``desktop-file-validate`` program against any
1071 ``.desktop`` files to validate their contents against the
1072 specification for ``.desktop`` files.
1073
1074- *``dev-deps:``* Checks that all packages except ``-dev`` or
1075 ``-staticdev`` packages do not depend on ``-dev`` packages, which
1076 would be a packaging bug.
1077
1078- *``dev-so:``* Checks that the ``.so`` symbolic links are in the
1079 ``-dev`` package and not in any of the other packages. In general,
1080 these symlinks are only useful for development purposes. Thus, the
1081 ``-dev`` package is the correct location for them. Some very rare
1082 cases do exist for dynamically loaded modules where these symlinks
1083 are needed instead in the main package.
1084
1085- *``file-rdeps:``* Checks that file-level dependencies identified by
1086 the OpenEmbedded build system at packaging time are satisfied. For
1087 example, a shell script might start with the line ``#!/bin/bash``.
1088 This line would translate to a file dependency on ``/bin/bash``. Of
1089 the three package managers that the OpenEmbedded build system
1090 supports, only RPM directly handles file-level dependencies,
1091 resolving them automatically to packages providing the files.
1092 However, the lack of that functionality in the other two package
1093 managers does not mean the dependencies do not still need resolving.
1094 This QA check attempts to ensure that explicitly declared
1095 ```RDEPENDS`` <#var-RDEPENDS>`__ exist to handle any file-level
1096 dependency detected in packaged files.
1097
1098- *``files-invalid:``* Checks for ```FILES`` <#var-FILES>`__ variable
1099 values that contain "//", which is invalid.
1100
1101- *``host-user-contaminated:``* Checks that no package produced by the
1102 recipe contains any files outside of ``/home`` with a user or group
1103 ID that matches the user running BitBake. A match usually indicates
1104 that the files are being installed with an incorrect UID/GID, since
1105 target IDs are independent from host IDs. For additional information,
1106 see the section describing the
1107 ```do_install`` <#ref-tasks-install>`__ task.
1108
1109- *``incompatible-license:``* Report when packages are excluded from
1110 being created due to being marked with a license that is in
1111 ```INCOMPATIBLE_LICENSE`` <#var-INCOMPATIBLE_LICENSE>`__.
1112
1113- *``install-host-path:``* Checks the
1114 ```do_install`` <#ref-tasks-install>`__ log for indications that
1115 paths to locations on the build host were used. Using such paths
1116 might result in host contamination of the build output.
1117
1118- *``installed-vs-shipped:``* Reports when files have been installed
1119 within ``do_install`` but have not been included in any package by
1120 way of the ```FILES`` <#var-FILES>`__ variable. Files that do not
1121 appear in any package cannot be present in an image later on in the
1122 build process. Ideally, all installed files should be packaged or not
1123 installed at all. These files can be deleted at the end of
1124 ``do_install`` if the files are not needed in any package.
1125
1126- *``invalid-chars:``* Checks that the recipe metadata variables
1127 ```DESCRIPTION`` <#var-DESCRIPTION>`__,
1128 ```SUMMARY`` <#var-SUMMARY>`__, ```LICENSE`` <#var-LICENSE>`__, and
1129 ```SECTION`` <#var-SECTION>`__ do not contain non-UTF-8 characters.
1130 Some package managers do not support such characters.
1131
1132- *``invalid-packageconfig:``* Checks that no undefined features are
1133 being added to ```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__. For
1134 example, any name "foo" for which the following form does not exist:
1135 PACKAGECONFIG[foo] = "..."
1136
1137- *``la:``* Checks ``.la`` files for any ``TMPDIR`` paths. Any ``.la``
1138 file containing these paths is incorrect since ``libtool`` adds the
1139 correct sysroot prefix when using the files automatically itself.
1140
1141- *``ldflags:``* Ensures that the binaries were linked with the
1142 ```LDFLAGS`` <#var-LDFLAGS>`__ options provided by the build system.
1143 If this test fails, check that the ``LDFLAGS`` variable is being
1144 passed to the linker command.
1145
1146- *``libdir:``* Checks for libraries being installed into incorrect
1147 (possibly hardcoded) installation paths. For example, this test will
1148 catch recipes that install ``/lib/bar.so`` when ``${base_libdir}`` is
1149 "lib32". Another example is when recipes install
1150 ``/usr/lib64/foo.so`` when ``${libdir}`` is "/usr/lib".
1151
1152- *``libexec:``* Checks if a package contains files in
1153 ``/usr/libexec``. This check is not performed if the ``libexecdir``
1154 variable has been set explicitly to ``/usr/libexec``.
1155
1156- *``packages-list:``* Checks for the same package being listed
1157 multiple times through the ```PACKAGES`` <#var-PACKAGES>`__ variable
1158 value. Installing the package in this manner can cause errors during
1159 packaging.
1160
1161- *``perm-config:``* Reports lines in ``fs-perms.txt`` that have an
1162 invalid format.
1163
1164- *``perm-line:``* Reports lines in ``fs-perms.txt`` that have an
1165 invalid format.
1166
1167- *``perm-link:``* Reports lines in ``fs-perms.txt`` that specify
1168 'link' where the specified target already exists.
1169
1170- *``perms:``* Currently, this check is unused but reserved.
1171
1172- *``pkgconfig:``* Checks ``.pc`` files for any
1173 ```TMPDIR`` <#var-TMPDIR>`__/```WORKDIR`` <#var-WORKDIR>`__ paths.
1174 Any ``.pc`` file containing these paths is incorrect since
1175 ``pkg-config`` itself adds the correct sysroot prefix when the files
1176 are accessed.
1177
1178- *``pkgname:``* Checks that all packages in
1179 ```PACKAGES`` <#var-PACKAGES>`__ have names that do not contain
1180 invalid characters (i.e. characters other than 0-9, a-z, ., +, and
1181 -).
1182
1183- *``pkgv-undefined:``* Checks to see if the ``PKGV`` variable is
1184 undefined during ```do_package`` <#ref-tasks-package>`__.
1185
1186- *``pkgvarcheck:``* Checks through the variables
1187 ```RDEPENDS`` <#var-RDEPENDS>`__,
1188 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__,
1189 ```RSUGGESTS`` <#var-RSUGGESTS>`__,
1190 ```RCONFLICTS`` <#var-RCONFLICTS>`__,
1191 ```RPROVIDES`` <#var-RPROVIDES>`__,
1192 ```RREPLACES`` <#var-RREPLACES>`__, ```FILES`` <#var-FILES>`__,
1193 ```ALLOW_EMPTY`` <#var-ALLOW_EMPTY>`__, ``pkg_preinst``,
1194 ``pkg_postinst``, ``pkg_prerm`` and ``pkg_postrm``, and reports if
1195 there are variable sets that are not package-specific. Using these
1196 variables without a package suffix is bad practice, and might
1197 unnecessarily complicate dependencies of other packages within the
1198 same recipe or have other unintended consequences.
1199
1200- *``pn-overrides:``* Checks that a recipe does not have a name
1201 (```PN`` <#var-PN>`__) value that appears in
1202 ```OVERRIDES`` <#var-OVERRIDES>`__. If a recipe is named such that
1203 its ``PN`` value matches something already in ``OVERRIDES`` (e.g.
1204 ``PN`` happens to be the same as ```MACHINE`` <#var-MACHINE>`__ or
1205 ```DISTRO`` <#var-DISTRO>`__), it can have unexpected consequences.
1206 For example, assignments such as ``FILES_${PN} = "xyz"`` effectively
1207 turn into ``FILES = "xyz"``.
1208
1209- *``rpaths:``* Checks for rpaths in the binaries that contain build
1210 system paths such as ``TMPDIR``. If this test fails, bad ``-rpath``
1211 options are being passed to the linker commands and your binaries
1212 have potential security issues.
1213
1214- *``split-strip:``* Reports that splitting or stripping debug symbols
1215 from binaries has failed.
1216
1217- *``staticdev:``* Checks for static library files (``*.a``) in
1218 non-``staticdev`` packages.
1219
1220- *``symlink-to-sysroot:``* Checks for symlinks in packages that point
1221 into ```TMPDIR`` <#var-TMPDIR>`__ on the host. Such symlinks will
1222 work on the host, but are clearly invalid when running on the target.
1223
1224- *``textrel:``* Checks for ELF binaries that contain relocations in
1225 their ``.text`` sections, which can result in a performance impact at
1226 runtime. See the explanation for the
1227 ```ELF binary`` <#qa-issue-textrel>`__ message for more information
1228 regarding runtime performance issues.
1229
1230- *``unlisted-pkg-lics:``* Checks that all declared licenses applying
1231 for a package are also declared on the recipe level (i.e. any license
1232 in ``LICENSE_*`` should appear in ```LICENSE`` <#var-LICENSE>`__).
1233
1234- *``useless-rpaths:``* Checks for dynamic library load paths (rpaths)
1235 in the binaries that by default on a standard system are searched by
1236 the linker (e.g. ``/lib`` and ``/usr/lib``). While these paths will
1237 not cause any breakage, they do waste space and are unnecessary.
1238
1239- *``var-undefined:``* Reports when variables fundamental to packaging
1240 (i.e. ```WORKDIR`` <#var-WORKDIR>`__,
1241 ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__, ```D`` <#var-D>`__,
1242 ```PN`` <#var-PN>`__, and ```PKGD`` <#var-PKGD>`__) are undefined
1243 during ```do_package`` <#ref-tasks-package>`__.
1244
1245- *``version-going-backwards:``* If Build History is enabled, reports
1246 when a package being written out has a lower version than the
1247 previously written package under the same name. If you are placing
1248 output packages into a feed and upgrading packages on a target system
1249 using that feed, the version of a package going backwards can result
1250 in the target system not correctly upgrading to the "new" version of
1251 the package.
1252
1253 .. note::
1254
1255 If you are not using runtime package management on your target
1256 system, then you do not need to worry about this situation.
1257
1258- *``xorg-driver-abi:``* Checks that all packages containing Xorg
1259 drivers have ABI dependencies. The ``xserver-xorg`` recipe provides
1260 driver ABI names. All drivers should depend on the ABI versions that
1261 they have been built against. Driver recipes that include
1262 ``xorg-driver-input.inc`` or ``xorg-driver-video.inc`` will
1263 automatically get these versions. Consequently, you should only need
1264 to explicitly add dependencies to binary driver recipes.
1265
1266.. _ref-classes-insserv:
1267
1268``insserv.bbclass``
1269===================
1270
1271The ``insserv`` class uses the ``insserv`` utility to update the order
1272of symbolic links in ``/etc/rc?.d/`` within an image based on
1273dependencies specified by LSB headers in the ``init.d`` scripts
1274themselves.
1275
1276.. _ref-classes-kernel:
1277
1278``kernel.bbclass``
1279==================
1280
1281The ``kernel`` class handles building Linux kernels. The class contains
1282code to build all kernel trees. All needed headers are staged into the
1283``STAGING_KERNEL_DIR`` directory to allow out-of-tree module builds
1284using the ```module`` <#ref-classes-module>`__ class.
1285
1286This means that each built kernel module is packaged separately and
1287inter-module dependencies are created by parsing the ``modinfo`` output.
1288If all modules are required, then installing the ``kernel-modules``
1289package installs all packages with modules and various other kernel
1290packages such as ``kernel-vmlinux``.
1291
1292The ``kernel`` class contains logic that allows you to embed an initial
1293RAM filesystem (initramfs) image when you build the kernel image. For
1294information on how to build an initramfs, see the "`Building an Initial
1295RAM Filesystem (initramfs)
1296Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__" section in
1297the Yocto Project Development Tasks Manual.
1298
1299Various other classes are used by the ``kernel`` and ``module`` classes
1300internally including the ```kernel-arch`` <#ref-classes-kernel-arch>`__,
1301```module-base`` <#ref-classes-module-base>`__, and
1302```linux-kernel-base`` <#ref-classes-linux-kernel-base>`__ classes.
1303
1304.. _ref-classes-kernel-arch:
1305
1306``kernel-arch.bbclass``
1307=======================
1308
1309The ``kernel-arch`` class sets the ``ARCH`` environment variable for
1310Linux kernel compilation (including modules).
1311
1312.. _ref-classes-kernel-devicetree:
1313
1314``kernel-devicetree.bbclass``
1315=============================
1316
1317The ``kernel-devicetree`` class, which is inherited by the
1318```kernel`` <#ref-classes-kernel>`__ class, supports device tree
1319generation.
1320
1321.. _ref-classes-kernel-fitimage:
1322
1323``kernel-fitimage.bbclass``
1324===========================
1325
1326The ``kernel-fitimage`` class provides support to pack a kernel Image,
1327device trees and a RAM disk into a single FIT image. In theory, a FIT
1328image can support any number of kernels, RAM disks and device-trees.
1329However, ``kernel-fitimage`` currently only supports
1330limited usescases: just one kernel image, an optional RAM disk, and
1331any number of device tree.
1332
1333To create a FIT image, it is required that :term:`KERNEL_CLASSES`
1334is set to "kernel-fitimage" and :term:`KERNEL_IMAGETYPE`
1335is set to "fitImage".
1336
1337The options for the device tree compiler passed to mkimage -D feature
1338when creating the FIT image are specified using the
1339:term:`UBOOT_MKIMAGE_DTCOPTS` variable.
1340
1341Only a single kernel can be added to the FIT image created by
1342``kernel-fitimage`` and the kernel image in FIT is mandatory. The
1343address where the kernel image is to be loaded by U-boot is
1344specified by :term:`UBOOT_LOADADDRESS` and the entrypoint by
1345:term:`UBOOT_ENTRYPOINT`.
1346
1347Multiple device trees can be added to the FIT image created by
1348``kernel-fitimage`` and the device tree is optional.
1349The address where the device tree is to be loaded by U-boot is
1350specified by :term:`UBOOT_DTBO_LOADADDRESS` for device tree overlays
1351and by `:term:`UBOOT_DTB_LOADADDRESS` for device tree binaries.
1352
1353Only a single RAM disk can be added to the FIT image created by
1354``kernel-fitimage`` and the RAM disk in FIT is optional.
1355The address where the RAM disk image is to be loaded by U-boot
1356is specified by :term:`UBOOT_RD_LOADADDRESS` and the entrypoint by
1357:term:`UBOOT_RD_ENTRYPOINT`. The ramdisk is added to FIT image when
1358:term:`INITRAMFS_IMAGE` is specified.
1359
1360The FIT image generated by ``kernel-fitimage`` class is signed when the
1361variables :term:`UBOOT_SIGN_ENABLE`, :term:`UBOOT_MKIMAGE_DTCOPTS`,
1362:term:`UBOOT_SIGN_KEYDIR` and :term:`UBOOT_SIGN_KEYNAME` are set
1363appropriately. The default values used for :term:`FIT_HASH_ALG` and
1364:term:`FIT_SIGN_ALG` in ``kernel-fitimage`` are "sha256" and
1365"rsa2048" respectively.
1366
1367
1368.. _ref-classes-kernel-grub:
1369
1370``kernel-grub.bbclass``
1371=======================
1372
1373The ``kernel-grub`` class updates the boot area and the boot menu with
1374the kernel as the priority boot mechanism while installing a RPM to
1375update the kernel on a deployed target.
1376
1377.. _ref-classes-kernel-module-split:
1378
1379``kernel-module-split.bbclass``
1380===============================
1381
1382The ``kernel-module-split`` class provides common functionality for
1383splitting Linux kernel modules into separate packages.
1384
1385.. _ref-classes-kernel-uboot:
1386
1387``kernel-uboot.bbclass``
1388========================
1389
1390The ``kernel-uboot`` class provides support for building from
1391vmlinux-style kernel sources.
1392
1393.. _ref-classes-kernel-uimage:
1394
1395``kernel-uimage.bbclass``
1396=========================
1397
1398The ``kernel-uimage`` class provides support to pack uImage.
1399
1400.. _ref-classes-kernel-yocto:
1401
1402``kernel-yocto.bbclass``
1403========================
1404
1405The ``kernel-yocto`` class provides common functionality for building
1406from linux-yocto style kernel source repositories.
1407
1408.. _ref-classes-kernelsrc:
1409
1410``kernelsrc.bbclass``
1411=====================
1412
1413The ``kernelsrc`` class sets the Linux kernel source and version.
1414
1415.. _ref-classes-lib_package:
1416
1417``lib_package.bbclass``
1418=======================
1419
1420The ``lib_package`` class supports recipes that build libraries and
1421produce executable binaries, where those binaries should not be
1422installed by default along with the library. Instead, the binaries are
1423added to a separate ``${``\ ```PN`` <#var-PN>`__\ ``}-bin`` package to
1424make their installation optional.
1425
1426.. _ref-classes-libc*:
1427
1428``libc*.bbclass``
1429=================
1430
1431The ``libc*`` classes support recipes that build packages with ``libc``:
1432
1433- The ``libc-common`` class provides common support for building with
1434 ``libc``.
1435
1436- The ``libc-package`` class supports packaging up ``glibc`` and
1437 ``eglibc``.
1438
1439.. _ref-classes-license:
1440
1441``license.bbclass``
1442===================
1443
1444The ``license`` class provides license manifest creation and license
1445exclusion. This class is enabled by default using the default value for
1446the ```INHERIT_DISTRO`` <#var-INHERIT_DISTRO>`__ variable.
1447
1448.. _ref-classes-linux-kernel-base:
1449
1450``linux-kernel-base.bbclass``
1451=============================
1452
1453The ``linux-kernel-base`` class provides common functionality for
1454recipes that build out of the Linux kernel source tree. These builds
1455goes beyond the kernel itself. For example, the Perf recipe also
1456inherits this class.
1457
1458.. _ref-classes-linuxloader:
1459
1460``linuxloader.bbclass``
1461=======================
1462
1463Provides the function ``linuxloader()``, which gives the value of the
1464dynamic loader/linker provided on the platform. This value is used by a
1465number of other classes.
1466
1467.. _ref-classes-logging:
1468
1469``logging.bbclass``
1470===================
1471
1472The ``logging`` class provides the standard shell functions used to log
1473messages for various BitBake severity levels (i.e. ``bbplain``,
1474``bbnote``, ``bbwarn``, ``bberror``, ``bbfatal``, and ``bbdebug``).
1475
1476This class is enabled by default since it is inherited by the ``base``
1477class.
1478
1479.. _ref-classes-meta:
1480
1481``meta.bbclass``
1482================
1483
1484The ``meta`` class is inherited by recipes that do not build any output
1485packages themselves, but act as a "meta" target for building other
1486recipes.
1487
1488.. _ref-classes-metadata_scm:
1489
1490``metadata_scm.bbclass``
1491========================
1492
1493The ``metadata_scm`` class provides functionality for querying the
1494branch and revision of a Source Code Manager (SCM) repository.
1495
1496The ```base`` <#ref-classes-base>`__ class uses this class to print the
1497revisions of each layer before starting every build. The
1498``metadata_scm`` class is enabled by default because it is inherited by
1499the ``base`` class.
1500
1501.. _ref-classes-migrate_localcount:
1502
1503``migrate_localcount.bbclass``
1504==============================
1505
1506The ``migrate_localcount`` class verifies a recipe's localcount data and
1507increments it appropriately.
1508
1509.. _ref-classes-mime:
1510
1511``mime.bbclass``
1512================
1513
1514The ``mime`` class generates the proper post-install and post-remove
1515(postinst/postrm) scriptlets for packages that install MIME type files.
1516These scriptlets call ``update-mime-database`` to add the MIME types to
1517the shared database.
1518
1519.. _ref-classes-mirrors:
1520
1521``mirrors.bbclass``
1522===================
1523
1524The ``mirrors`` class sets up some standard
1525```MIRRORS`` <#var-MIRRORS>`__ entries for source code mirrors. These
1526mirrors provide a fall-back path in case the upstream source specified
1527in ```SRC_URI`` <#var-SRC_URI>`__ within recipes is unavailable.
1528
1529This class is enabled by default since it is inherited by the
1530```base`` <#ref-classes-base>`__ class.
1531
1532.. _ref-classes-module:
1533
1534``module.bbclass``
1535==================
1536
1537The ``module`` class provides support for building out-of-tree Linux
1538kernel modules. The class inherits the
1539```module-base`` <#ref-classes-module-base>`__ and
1540```kernel-module-split`` <#ref-classes-kernel-module-split>`__ classes,
1541and implements the ```do_compile`` <#ref-tasks-compile>`__ and
1542```do_install`` <#ref-tasks-install>`__ tasks. The class provides
1543everything needed to build and package a kernel module.
1544
1545For general information on out-of-tree Linux kernel modules, see the
1546"`Incorporating Out-of-Tree
1547Modules <&YOCTO_DOCS_KERNEL_DEV_URL;#incorporating-out-of-tree-modules>`__"
1548section in the Yocto Project Linux Kernel Development Manual.
1549
1550.. _ref-classes-module-base:
1551
1552``module-base.bbclass``
1553=======================
1554
1555The ``module-base`` class provides the base functionality for building
1556Linux kernel modules. Typically, a recipe that builds software that
1557includes one or more kernel modules and has its own means of building
1558the module inherits this class as opposed to inheriting the
1559```module`` <#ref-classes-module>`__ class.
1560
1561.. _ref-classes-multilib*:
1562
1563``multilib*.bbclass``
1564=====================
1565
1566The ``multilib*`` classes provide support for building libraries with
1567different target optimizations or target architectures and installing
1568them side-by-side in the same image.
1569
1570For more information on using the Multilib feature, see the "`Combining
1571Multiple Versions of Library Files into One
1572Image <&YOCTO_DOCS_DEV_URL;#combining-multiple-versions-library-files-into-one-image>`__"
1573section in the Yocto Project Development Tasks Manual.
1574
1575.. _ref-classes-native:
1576
1577``native.bbclass``
1578==================
1579
1580The ``native`` class provides common functionality for recipes that
1581build tools to run on the `build host <#hardware-build-system-term>`__
1582(i.e. tools that use the compiler or other tools from the build host).
1583
1584You can create a recipe that builds tools that run natively on the host
1585a couple different ways:
1586
1587- Create a myrecipe\ ``-native.bb`` recipe that inherits the ``native``
1588 class. If you use this method, you must order the inherit statement
1589 in the recipe after all other inherit statements so that the
1590 ``native`` class is inherited last.
1591
1592 .. note::
1593
1594 When creating a recipe this way, the recipe name must follow this
1595 naming convention:
1596 ::
1597
1598 myrecipe-native.bb
1599
1600
1601 Not using this naming convention can lead to subtle problems
1602 caused by existing code that depends on that naming convention.
1603
1604- Create or modify a target recipe that contains the following:
1605 ```BBCLASSEXTEND`` <#var-BBCLASSEXTEND>`__ = "native" Inside the
1606 recipe, use ``_class-native`` and ``_class-target`` overrides to
1607 specify any functionality specific to the respective native or target
1608 case.
1609
1610Although applied differently, the ``native`` class is used with both
1611methods. The advantage of the second method is that you do not need to
1612have two separate recipes (assuming you need both) for native and
1613target. All common parts of the recipe are automatically shared.
1614
1615.. _ref-classes-nativesdk:
1616
1617``nativesdk.bbclass``
1618=====================
1619
1620The ``nativesdk`` class provides common functionality for recipes that
1621wish to build tools to run as part of an SDK (i.e. tools that run on
1622```SDKMACHINE`` <#var-SDKMACHINE>`__).
1623
1624You can create a recipe that builds tools that run on the SDK machine a
1625couple different ways:
1626
1627- Create a ``nativesdk-``\ myrecipe\ ``.bb`` recipe that inherits the
1628 ``nativesdk`` class. If you use this method, you must order the
1629 inherit statement in the recipe after all other inherit statements so
1630 that the ``nativesdk`` class is inherited last.
1631
1632- Create a ``nativesdk`` variant of any recipe by adding the following:
1633 ```BBCLASSEXTEND`` <#var-BBCLASSEXTEND>`__ = "nativesdk" Inside the
1634 recipe, use ``_class-nativesdk`` and ``_class-target`` overrides to
1635 specify any functionality specific to the respective SDK machine or
1636 target case.
1637
1638.. note::
1639
1640 When creating a recipe, you must follow this naming convention:
1641 ::
1642
1643 nativesdk-myrecipe.bb
1644
1645
1646 Not doing so can lead to subtle problems because code exists that
1647 depends on the naming convention.
1648
1649Although applied differently, the ``nativesdk`` class is used with both
1650methods. The advantage of the second method is that you do not need to
1651have two separate recipes (assuming you need both) for the SDK machine
1652and the target. All common parts of the recipe are automatically shared.
1653
1654.. _ref-classes-nopackages:
1655
1656``nopackages.bbclass``
1657======================
1658
1659Disables packaging tasks for those recipes and classes where packaging
1660is not needed.
1661
1662.. _ref-classes-npm:
1663
1664``npm.bbclass``
1665===============
1666
1667Provides support for building Node.js software fetched using the `node
1668package manager (NPM) <https://en.wikipedia.org/wiki/Npm_(software)>`__.
1669
1670.. note::
1671
1672 Currently, recipes inheriting this class must use the
1673 npm://
1674 fetcher to have dependencies fetched and packaged automatically.
1675
1676For information on how to create NPM packages, see the "`Creating Node
1677Package Manager (NPM)
1678Packages <&YOCTO_DOCS_DEV_URL;#creating-node-package-manager-npm-packages>`__"
1679section in the Yocto Project Development Tasks Manual.
1680
1681.. _ref-classes-oelint:
1682
1683``oelint.bbclass``
1684==================
1685
1686The ``oelint`` class is an obsolete lint checking tool that exists in
1687``meta/classes`` in the `Source Directory <#source-directory>`__.
1688
1689A number of classes exist that could be generally useful in OE-Core but
1690are never actually used within OE-Core itself. The ``oelint`` class is
1691one such example. However, being aware of this class can reduce the
1692proliferation of different versions of similar classes across multiple
1693layers.
1694
1695.. _ref-classes-own-mirrors:
1696
1697``own-mirrors.bbclass``
1698=======================
1699
1700The ``own-mirrors`` class makes it easier to set up your own
1701```PREMIRRORS`` <#var-PREMIRRORS>`__ from which to first fetch source
1702before attempting to fetch it from the upstream specified in
1703```SRC_URI`` <#var-SRC_URI>`__ within each recipe.
1704
1705To use this class, inherit it globally and specify
1706```SOURCE_MIRROR_URL`` <#var-SOURCE_MIRROR_URL>`__. Here is an example:
1707INHERIT += "own-mirrors" SOURCE_MIRROR_URL =
1708"http://example.com/my-source-mirror" You can specify only a single URL
1709in ``SOURCE_MIRROR_URL``.
1710
1711.. _ref-classes-package:
1712
1713``package.bbclass``
1714===================
1715
1716The ``package`` class supports generating packages from a build's
1717output. The core generic functionality is in ``package.bbclass``. The
1718code specific to particular package types resides in these
1719package-specific classes:
1720```package_deb`` <#ref-classes-package_deb>`__,
1721```package_rpm`` <#ref-classes-package_rpm>`__,
1722```package_ipk`` <#ref-classes-package_ipk>`__, and
1723```package_tar`` <#ref-classes-package_tar>`__.
1724
1725.. note::
1726
1727 The
1728 package_tar
1729 class is broken and not supported. It is recommended that you do not
1730 use this class.
1731
1732You can control the list of resulting package formats by using the
1733``PACKAGE_CLASSES`` variable defined in your ``conf/local.conf``
1734configuration file, which is located in the `Build
1735Directory <#build-directory>`__. When defining the variable, you can
1736specify one or more package types. Since images are generated from
1737packages, a packaging class is needed to enable image generation. The
1738first class listed in this variable is used for image generation.
1739
1740If you take the optional step to set up a repository (package feed) on
1741the development host that can be used by DNF, you can install packages
1742from the feed while you are running the image on the target (i.e.
1743runtime installation of packages). For more information, see the "`Using
1744Runtime Package
1745Management <&YOCTO_DOCS_DEV_URL;#using-runtime-package-management>`__"
1746section in the Yocto Project Development Tasks Manual.
1747
1748The package-specific class you choose can affect build-time performance
1749and has space ramifications. In general, building a package with IPK
1750takes about thirty percent less time as compared to using RPM to build
1751the same or similar package. This comparison takes into account a
1752complete build of the package with all dependencies previously built.
1753The reason for this discrepancy is because the RPM package manager
1754creates and processes more `Metadata <#metadata>`__ than the IPK package
1755manager. Consequently, you might consider setting ``PACKAGE_CLASSES`` to
1756"package_ipk" if you are building smaller systems.
1757
1758Before making your package manager decision, however, you should
1759consider some further things about using RPM:
1760
1761- RPM starts to provide more abilities than IPK due to the fact that it
1762 processes more Metadata. For example, this information includes
1763 individual file types, file checksum generation and evaluation on
1764 install, sparse file support, conflict detection and resolution for
1765 Multilib systems, ACID style upgrade, and repackaging abilities for
1766 rollbacks.
1767
1768- For smaller systems, the extra space used for the Berkeley Database
1769 and the amount of metadata when using RPM can affect your ability to
1770 perform on-device upgrades.
1771
1772You can find additional information on the effects of the package class
1773at these two Yocto Project mailing list links:
1774
1775- `https://lists.yoctoproject.org/pipermail/poky/2011-May/006362.html <&YOCTO_LISTS_URL;/pipermail/poky/2011-May/006362.html>`__
1776
1777- `https://lists.yoctoproject.org/pipermail/poky/2011-May/006363.html <&YOCTO_LISTS_URL;/pipermail/poky/2011-May/006363.html>`__
1778
1779.. _ref-classes-package_deb:
1780
1781``package_deb.bbclass``
1782=======================
1783
1784The ``package_deb`` class provides support for creating packages that
1785use the Debian (i.e. ``.deb``) file format. The class ensures the
1786packages are written out in a ``.deb`` file format to the
1787``${``\ ```DEPLOY_DIR_DEB`` <#var-DEPLOY_DIR_DEB>`__\ ``}`` directory.
1788
1789This class inherits the ```package`` <#ref-classes-package>`__ class and
1790is enabled through the ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__
1791variable in the ``local.conf`` file.
1792
1793.. _ref-classes-package_ipk:
1794
1795``package_ipk.bbclass``
1796=======================
1797
1798The ``package_ipk`` class provides support for creating packages that
1799use the IPK (i.e. ``.ipk``) file format. The class ensures the packages
1800are written out in a ``.ipk`` file format to the
1801``${``\ ```DEPLOY_DIR_IPK`` <#var-DEPLOY_DIR_IPK>`__\ ``}`` directory.
1802
1803This class inherits the ```package`` <#ref-classes-package>`__ class and
1804is enabled through the ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__
1805variable in the ``local.conf`` file.
1806
1807.. _ref-classes-package_rpm:
1808
1809``package_rpm.bbclass``
1810=======================
1811
1812The ``package_rpm`` class provides support for creating packages that
1813use the RPM (i.e. ``.rpm``) file format. The class ensures the packages
1814are written out in a ``.rpm`` file format to the
1815``${``\ ```DEPLOY_DIR_RPM`` <#var-DEPLOY_DIR_RPM>`__\ ``}`` directory.
1816
1817This class inherits the ```package`` <#ref-classes-package>`__ class and
1818is enabled through the ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__
1819variable in the ``local.conf`` file.
1820
1821.. _ref-classes-package_tar:
1822
1823``package_tar.bbclass``
1824=======================
1825
1826The ``package_tar`` class provides support for creating tarballs. The
1827class ensures the packages are written out in a tarball format to the
1828``${``\ ```DEPLOY_DIR_TAR`` <#var-DEPLOY_DIR_TAR>`__\ ``}`` directory.
1829
1830This class inherits the ```package`` <#ref-classes-package>`__ class and
1831is enabled through the ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__
1832variable in the ``local.conf`` file.
1833
1834.. note::
1835
1836 You cannot specify the
1837 package_tar
1838 class first using the
1839 PACKAGE_CLASSES
1840 variable. You must use
1841 .deb
1842 ,
1843 .ipk
1844 , or
1845 .rpm
1846 file formats for your image or SDK.
1847
1848.. _ref-classes-packagedata:
1849
1850``packagedata.bbclass``
1851=======================
1852
1853The ``packagedata`` class provides common functionality for reading
1854``pkgdata`` files found in ```PKGDATA_DIR`` <#var-PKGDATA_DIR>`__. These
1855files contain information about each output package produced by the
1856OpenEmbedded build system.
1857
1858This class is enabled by default because it is inherited by the
1859```package`` <#ref-classes-package>`__ class.
1860
1861.. _ref-classes-packagegroup:
1862
1863``packagegroup.bbclass``
1864========================
1865
1866The ``packagegroup`` class sets default values appropriate for package
1867group recipes (e.g. ``PACKAGES``, ``PACKAGE_ARCH``, ``ALLOW_EMPTY``, and
1868so forth). It is highly recommended that all package group recipes
1869inherit this class.
1870
1871For information on how to use this class, see the "`Customizing Images
1872Using Custom Package
1873Groups <&YOCTO_DOCS_DEV_URL;#usingpoky-extend-customimage-customtasks>`__"
1874section in the Yocto Project Development Tasks Manual.
1875
1876Previously, this class was called the ``task`` class.
1877
1878.. _ref-classes-patch:
1879
1880``patch.bbclass``
1881=================
1882
1883The ``patch`` class provides all functionality for applying patches
1884during the ```do_patch`` <#ref-tasks-patch>`__ task.
1885
1886This class is enabled by default because it is inherited by the
1887```base`` <#ref-classes-base>`__ class.
1888
1889.. _ref-classes-perlnative:
1890
1891``perlnative.bbclass``
1892======================
1893
1894When inherited by a recipe, the ``perlnative`` class supports using the
1895native version of Perl built by the build system rather than using the
1896version provided by the build host.
1897
1898.. _ref-classes-pixbufcache:
1899
1900``pixbufcache.bbclass``
1901=======================
1902
1903The ``pixbufcache`` class generates the proper post-install and
1904post-remove (postinst/postrm) scriptlets for packages that install
1905pixbuf loaders, which are used with ``gdk-pixbuf``. These scriptlets
1906call ``update_pixbuf_cache`` to add the pixbuf loaders to the cache.
1907Since the cache files are architecture-specific, ``update_pixbuf_cache``
1908is run using QEMU if the postinst scriptlets need to be run on the build
1909host during image creation.
1910
1911If the pixbuf loaders being installed are in packages other than the
1912recipe's main package, set
1913```PIXBUF_PACKAGES`` <#var-PIXBUF_PACKAGES>`__ to specify the packages
1914containing the loaders.
1915
1916.. _ref-classes-pkgconfig:
1917
1918``pkgconfig.bbclass``
1919=====================
1920
1921The ``pkgconfig`` class provides a standard way to get header and
1922library information by using ``pkg-config``. This class aims to smooth
1923integration of ``pkg-config`` into libraries that use it.
1924
1925During staging, BitBake installs ``pkg-config`` data into the
1926``sysroots/`` directory. By making use of sysroot functionality within
1927``pkg-config``, the ``pkgconfig`` class no longer has to manipulate the
1928files.
1929
1930.. _ref-classes-populate-sdk:
1931
1932``populate_sdk.bbclass``
1933========================
1934
1935The ``populate_sdk`` class provides support for SDK-only recipes. For
1936information on advantages gained when building a cross-development
1937toolchain using the ```do_populate_sdk`` <#ref-tasks-populate_sdk>`__
1938task, see the "`Building an SDK
1939Installer <&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer>`__"
1940section in the Yocto Project Application Development and the Extensible
1941Software Development Kit (eSDK) manual.
1942
1943.. _ref-classes-populate-sdk-*:
1944
1945``populate_sdk_*.bbclass``
1946==========================
1947
1948The ``populate_sdk_*`` classes support SDK creation and consist of the
1949following classes:
1950
1951- *``populate_sdk_base``:* The base class supporting SDK creation under
1952 all package managers (i.e. DEB, RPM, and opkg).
1953
1954- *``populate_sdk_deb``:* Supports creation of the SDK given the Debian
1955 package manager.
1956
1957- *``populate_sdk_rpm``:* Supports creation of the SDK given the RPM
1958 package manager.
1959
1960- *``populate_sdk_ipk``:* Supports creation of the SDK given the opkg
1961 (IPK format) package manager.
1962
1963- *``populate_sdk_ext``:* Supports extensible SDK creation under all
1964 package managers.
1965
1966The ``populate_sdk_base`` class inherits the appropriate
1967``populate_sdk_*`` (i.e. ``deb``, ``rpm``, and ``ipk``) based on
1968```IMAGE_PKGTYPE`` <#var-IMAGE_PKGTYPE>`__.
1969
1970The base class ensures all source and destination directories are
1971established and then populates the SDK. After populating the SDK, the
1972``populate_sdk_base`` class constructs two sysroots:
1973``${``\ ```SDK_ARCH`` <#var-SDK_ARCH>`__\ ``}-nativesdk``, which
1974contains the cross-compiler and associated tooling, and the target,
1975which contains a target root filesystem that is configured for the SDK
1976usage. These two images reside in ```SDK_OUTPUT`` <#var-SDK_OUTPUT>`__,
1977which consists of the following:
1978${SDK_OUTPUT}/${SDK_ARCH}-nativesdk-pkgs
1979${SDK_OUTPUT}/${SDKTARGETSYSROOT}/target-pkgs
1980
1981Finally, the base populate SDK class creates the toolchain environment
1982setup script, the tarball of the SDK, and the installer.
1983
1984The respective ``populate_sdk_deb``, ``populate_sdk_rpm``, and
1985``populate_sdk_ipk`` classes each support the specific type of SDK.
1986These classes are inherited by and used with the ``populate_sdk_base``
1987class.
1988
1989For more information on the cross-development toolchain generation, see
1990the "`Cross-Development Toolchain
1991Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
1992section in the Yocto Project Overview and Concepts Manual. For
1993information on advantages gained when building a cross-development
1994toolchain using the ```do_populate_sdk`` <#ref-tasks-populate_sdk>`__
1995task, see the "`Building an SDK
1996Installer <&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer>`__"
1997section in the Yocto Project Application Development and the Extensible
1998Software Development Kit (eSDK) manual.
1999
2000.. _ref-classes-prexport:
2001
2002``prexport.bbclass``
2003====================
2004
2005The ``prexport`` class provides functionality for exporting
2006```PR`` <#var-PR>`__ values.
2007
2008.. note::
2009
2010 This class is not intended to be used directly. Rather, it is enabled
2011 when using "
2012 bitbake-prserv-tool export
2013 ".
2014
2015.. _ref-classes-primport:
2016
2017``primport.bbclass``
2018====================
2019
2020The ``primport`` class provides functionality for importing
2021```PR`` <#var-PR>`__ values.
2022
2023.. note::
2024
2025 This class is not intended to be used directly. Rather, it is enabled
2026 when using "
2027 bitbake-prserv-tool import
2028 ".
2029
2030.. _ref-classes-prserv:
2031
2032``prserv.bbclass``
2033==================
2034
2035The ``prserv`` class provides functionality for using a `PR
2036service <&YOCTO_DOCS_DEV_URL;#working-with-a-pr-service>`__ in order to
2037automatically manage the incrementing of the ```PR`` <#var-PR>`__
2038variable for each recipe.
2039
2040This class is enabled by default because it is inherited by the
2041```package`` <#ref-classes-package>`__ class. However, the OpenEmbedded
2042build system will not enable the functionality of this class unless
2043```PRSERV_HOST`` <#var-PRSERV_HOST>`__ has been set.
2044
2045.. _ref-classes-ptest:
2046
2047``ptest.bbclass``
2048=================
2049
2050The ``ptest`` class provides functionality for packaging and installing
2051runtime tests for recipes that build software that provides these tests.
2052
2053This class is intended to be inherited by individual recipes. However,
2054the class' functionality is largely disabled unless "ptest" appears in
2055```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__. See the "`Testing
2056Packages With
2057ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section in
2058the Yocto Project Development Tasks Manual for more information on
2059ptest.
2060
2061.. _ref-classes-ptest-gnome:
2062
2063``ptest-gnome.bbclass``
2064=======================
2065
2066Enables package tests (ptests) specifically for GNOME packages, which
2067have tests intended to be executed with ``gnome-desktop-testing``.
2068
2069For information on setting up and running ptests, see the "`Testing
2070Packages With
2071ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section in
2072the Yocto Project Development Tasks Manual.
2073
2074.. _ref-classes-python-dir:
2075
2076``python-dir.bbclass``
2077======================
2078
2079The ``python-dir`` class provides the base version, location, and site
2080package location for Python.
2081
2082.. _ref-classes-python3native:
2083
2084``python3native.bbclass``
2085=========================
2086
2087The ``python3native`` class supports using the native version of Python
20883 built by the build system rather than support of the version provided
2089by the build host.
2090
2091.. _ref-classes-pythonnative:
2092
2093``pythonnative.bbclass``
2094========================
2095
2096When inherited by a recipe, the ``pythonnative`` class supports using
2097the native version of Python built by the build system rather than using
2098the version provided by the build host.
2099
2100.. _ref-classes-qemu:
2101
2102``qemu.bbclass``
2103================
2104
2105The ``qemu`` class provides functionality for recipes that either need
2106QEMU or test for the existence of QEMU. Typically, this class is used to
2107run programs for a target system on the build host using QEMU's
2108application emulation mode.
2109
2110.. _ref-classes-recipe_sanity:
2111
2112``recipe_sanity.bbclass``
2113=========================
2114
2115The ``recipe_sanity`` class checks for the presence of any host system
2116recipe prerequisites that might affect the build (e.g. variables that
2117are set or software that is present).
2118
2119.. _ref-classes-relocatable:
2120
2121``relocatable.bbclass``
2122=======================
2123
2124The ``relocatable`` class enables relocation of binaries when they are
2125installed into the sysroot.
2126
2127This class makes use of the ```chrpath`` <#ref-classes-chrpath>`__ class
2128and is used by both the ```cross`` <#ref-classes-cross>`__ and
2129```native`` <#ref-classes-native>`__ classes.
2130
2131.. _ref-classes-remove-libtool:
2132
2133``remove-libtool.bbclass``
2134==========================
2135
2136The ``remove-libtool`` class adds a post function to the
2137```do_install`` <#ref-tasks-install>`__ task to remove all ``.la`` files
2138installed by ``libtool``. Removing these files results in them being
2139absent from both the sysroot and target packages.
2140
2141If a recipe needs the ``.la`` files to be installed, then the recipe can
2142override the removal by setting ``REMOVE_LIBTOOL_LA`` to "0" as follows:
2143REMOVE_LIBTOOL_LA = "0"
2144
2145.. note::
2146
2147 The
2148 remove-libtool
2149 class is not enabled by default.
2150
2151.. _ref-classes-report-error:
2152
2153``report-error.bbclass``
2154========================
2155
2156The ``report-error`` class supports enabling the `error reporting
2157tool <&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool>`__, which
2158allows you to submit build error information to a central database.
2159
2160The class collects debug information for recipe, recipe version, task,
2161machine, distro, build system, target system, host distro, branch,
2162commit, and log. From the information, report files using a JSON format
2163are created and stored in
2164``${``\ ```LOG_DIR`` <#var-LOG_DIR>`__\ ``}/error-report``.
2165
2166.. _ref-classes-rm-work:
2167
2168``rm_work.bbclass``
2169===================
2170
2171The ``rm_work`` class supports deletion of temporary workspace, which
2172can ease your hard drive demands during builds.
2173
2174The OpenEmbedded build system can use a substantial amount of disk space
2175during the build process. A portion of this space is the work files
2176under the ``${TMPDIR}/work`` directory for each recipe. Once the build
2177system generates the packages for a recipe, the work files for that
2178recipe are no longer needed. However, by default, the build system
2179preserves these files for inspection and possible debugging purposes. If
2180you would rather have these files deleted to save disk space as the
2181build progresses, you can enable ``rm_work`` by adding the following to
2182your ``local.conf`` file, which is found in the `Build
2183Directory <#build-directory>`__. INHERIT += "rm_work" If you are
2184modifying and building source code out of the work directory for a
2185recipe, enabling ``rm_work`` will potentially result in your changes to
2186the source being lost. To exclude some recipes from having their work
2187directories deleted by ``rm_work``, you can add the names of the recipe
2188or recipes you are working on to the ``RM_WORK_EXCLUDE`` variable, which
2189can also be set in your ``local.conf`` file. Here is an example:
2190RM_WORK_EXCLUDE += "busybox glibc"
2191
2192.. _ref-classes-rootfs*:
2193
2194``rootfs*.bbclass``
2195===================
2196
2197The ``rootfs*`` classes support creating the root filesystem for an
2198image and consist of the following classes:
2199
2200- The ``rootfs-postcommands`` class, which defines filesystem
2201 post-processing functions for image recipes.
2202
2203- The ``rootfs_deb`` class, which supports creation of root filesystems
2204 for images built using ``.deb`` packages.
2205
2206- The ``rootfs_rpm`` class, which supports creation of root filesystems
2207 for images built using ``.rpm`` packages.
2208
2209- The ``rootfs_ipk`` class, which supports creation of root filesystems
2210 for images built using ``.ipk`` packages.
2211
2212- The ``rootfsdebugfiles`` class, which installs additional files found
2213 on the build host directly into the root filesystem.
2214
2215The root filesystem is created from packages using one of the
2216``rootfs*.bbclass`` files as determined by the
2217```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ variable.
2218
2219For information on how root filesystem images are created, see the
2220"`Image
2221Generation <&YOCTO_DOCS_OM_URL;#image-generation-dev-environment>`__"
2222section in the Yocto Project Overview and Concepts Manual.
2223
2224.. _ref-classes-sanity:
2225
2226``sanity.bbclass``
2227==================
2228
2229The ``sanity`` class checks to see if prerequisite software is present
2230on the host system so that users can be notified of potential problems
2231that might affect their build. The class also performs basic user
2232configuration checks from the ``local.conf`` configuration file to
2233prevent common mistakes that cause build failures. Distribution policy
2234usually determines whether to include this class.
2235
2236.. _ref-classes-scons:
2237
2238``scons.bbclass``
2239=================
2240
2241The ``scons`` class supports recipes that need to build software that
2242uses the SCons build system. You can use the
2243```EXTRA_OESCONS`` <#var-EXTRA_OESCONS>`__ variable to specify
2244additional configuration options you want to pass SCons command line.
2245
2246.. _ref-classes-sdl:
2247
2248``sdl.bbclass``
2249===============
2250
2251The ``sdl`` class supports recipes that need to build software that uses
2252the Simple DirectMedia Layer (SDL) library.
2253
2254.. _ref-classes-setuptools:
2255
2256``setuptools.bbclass``
2257======================
2258
2259The ``setuptools`` class supports Python version 2.x extensions that use
2260build systems based on ``setuptools``. If your recipe uses these build
2261systems, the recipe needs to inherit the ``setuptools`` class.
2262
2263.. _ref-classes-setuptools3:
2264
2265``setuptools3.bbclass``
2266=======================
2267
2268The ``setuptools3`` class supports Python version 3.x extensions that
2269use build systems based on ``setuptools3``. If your recipe uses these
2270build systems, the recipe needs to inherit the ``setuptools3`` class.
2271
2272.. _ref-classes-sign_rpm:
2273
2274``sign_rpm.bbclass``
2275====================
2276
2277The ``sign_rpm`` class supports generating signed RPM packages.
2278
2279.. _ref-classes-sip:
2280
2281``sip.bbclass``
2282===============
2283
2284The ``sip`` class supports recipes that build or package SIP-based
2285Python bindings.
2286
2287.. _ref-classes-siteconfig:
2288
2289``siteconfig.bbclass``
2290======================
2291
2292The ``siteconfig`` class provides functionality for handling site
2293configuration. The class is used by the
2294```autotools`` <#ref-classes-autotools>`__ class to accelerate the
2295```do_configure`` <#ref-tasks-configure>`__ task.
2296
2297.. _ref-classes-siteinfo:
2298
2299``siteinfo.bbclass``
2300====================
2301
2302The ``siteinfo`` class provides information about the targets that might
2303be needed by other classes or recipes.
2304
2305As an example, consider Autotools, which can require tests that must
2306execute on the target hardware. Since this is not possible in general
2307when cross compiling, site information is used to provide cached test
2308results so these tests can be skipped over but still make the correct
2309values available. The ``meta/site directory`` contains test results
2310sorted into different categories such as architecture, endianness, and
2311the ``libc`` used. Site information provides a list of files containing
2312data relevant to the current build in the ``CONFIG_SITE`` variable that
2313Autotools automatically picks up.
2314
2315The class also provides variables like ``SITEINFO_ENDIANNESS`` and
2316``SITEINFO_BITS`` that can be used elsewhere in the metadata.
2317
2318.. _ref-classes-spdx:
2319
2320``spdx.bbclass``
2321================
2322
2323The ``spdx`` class integrates real-time license scanning, generation of
2324SPDX standard output, and verification of license information during the
2325build.
2326
2327.. note::
2328
2329 This class is currently at the prototype stage in the 1.6 release.
2330
2331.. _ref-classes-sstate:
2332
2333``sstate.bbclass``
2334==================
2335
2336The ``sstate`` class provides support for Shared State (sstate). By
2337default, the class is enabled through the
2338```INHERIT_DISTRO`` <#var-INHERIT_DISTRO>`__ variable's default value.
2339
2340For more information on sstate, see the "`Shared State
2341Cache <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__" section in the Yocto
2342Project Overview and Concepts Manual.
2343
2344.. _ref-classes-staging:
2345
2346``staging.bbclass``
2347===================
2348
2349The ``staging`` class installs files into individual recipe work
2350directories for sysroots. The class contains the following key tasks:
2351
2352- The ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task,
2353 which is responsible for handing the files that end up in the recipe
2354 sysroots.
2355
2356- The
2357 ```do_prepare_recipe_sysroot`` <#ref-tasks-prepare_recipe_sysroot>`__
2358 task (a "partner" task to the ``populate_sysroot`` task), which
2359 installs the files into the individual recipe work directories (i.e.
2360 ```WORKDIR`` <#var-WORKDIR>`__).
2361
2362The code in the ``staging`` class is complex and basically works in two
2363stages:
2364
2365- *Stage One:* The first stage addresses recipes that have files they
2366 want to share with other recipes that have dependencies on the
2367 originating recipe. Normally these dependencies are installed through
2368 the ```do_install`` <#ref-tasks-install>`__ task into
2369 ``${``\ ```D`` <#var-D>`__\ ``}``. The ``do_populate_sysroot`` task
2370 copies a subset of these files into ``${SYSROOT_DESTDIR}``. This
2371 subset of files is controlled by the
2372 ```SYSROOT_DIRS`` <#var-SYSROOT_DIRS>`__,
2373 ```SYSROOT_DIRS_NATIVE`` <#var-SYSROOT_DIRS_NATIVE>`__, and
2374 ```SYSROOT_DIRS_BLACKLIST`` <#var-SYSROOT_DIRS_BLACKLIST>`__
2375 variables.
2376
2377 .. note::
2378
2379 Additionally, a recipe can customize the files further by
2380 declaring a processing function in the
2381 SYSROOT_PREPROCESS_FUNCS
2382 variable.
2383
2384 A shared state (sstate) object is built from these files and the
2385 files are placed into a subdirectory of
2386 ```tmp/sysroots-components/`` <#structure-build-tmp-sysroots-components>`__.
2387 The files are scanned for hardcoded paths to the original
2388 installation location. If the location is found in text files, the
2389 hardcoded locations are replaced by tokens and a list of the files
2390 needing such replacements is created. These adjustments are referred
2391 to as "FIXMEs". The list of files that are scanned for paths is
2392 controlled by the ```SSTATE_SCAN_FILES`` <#var-SSTATE_SCAN_FILES>`__
2393 variable.
2394
2395- *Stage Two:* The second stage addresses recipes that want to use
2396 something from another recipe and declare a dependency on that recipe
2397 through the ```DEPENDS`` <#var-DEPENDS>`__ variable. The recipe will
2398 have a
2399 ```do_prepare_recipe_sysroot`` <#ref-tasks-prepare_recipe_sysroot>`__
2400 task and when this task executes, it creates the ``recipe-sysroot``
2401 and ``recipe-sysroot-native`` in the recipe work directory (i.e.
2402 ```WORKDIR`` <#var-WORKDIR>`__). The OpenEmbedded build system
2403 creates hard links to copies of the relevant files from
2404 ``sysroots-components`` into the recipe work directory.
2405
2406 .. note::
2407
2408 If hard links are not possible, the build system uses actual
2409 copies.
2410
2411 The build system then addresses any "FIXMEs" to paths as defined from
2412 the list created in the first stage.
2413
2414 Finally, any files in ``${bindir}`` within the sysroot that have the
2415 prefix "``postinst-``" are executed.
2416
2417 .. note::
2418
2419 Although such sysroot post installation scripts are not
2420 recommended for general use, the files do allow some issues such
2421 as user creation and module indexes to be addressed.
2422
2423 Because recipes can have other dependencies outside of ``DEPENDS``
2424 (e.g. ``do_unpack[depends] += "tar-native:do_populate_sysroot"``),
2425 the sysroot creation function ``extend_recipe_sysroot`` is also added
2426 as a pre-function for those tasks whose dependencies are not through
2427 ``DEPENDS`` but operate similarly.
2428
2429 When installing dependencies into the sysroot, the code traverses the
2430 dependency graph and processes dependencies in exactly the same way
2431 as the dependencies would or would not be when installed from sstate.
2432 This processing means, for example, a native tool would have its
2433 native dependencies added but a target library would not have its
2434 dependencies traversed or installed. The same sstate dependency code
2435 is used so that builds should be identical regardless of whether
2436 sstate was used or not. For a closer look, see the
2437 ``setscene_depvalid()`` function in the
2438 ```sstate`` <#ref-classes-sstate>`__ class.
2439
2440 The build system is careful to maintain manifests of the files it
2441 installs so that any given dependency can be installed as needed. The
2442 sstate hash of the installed item is also stored so that if it
2443 changes, the build system can reinstall it.
2444
2445.. _ref-classes-syslinux:
2446
2447``syslinux.bbclass``
2448====================
2449
2450The ``syslinux`` class provides syslinux-specific functions for building
2451bootable images.
2452
2453The class supports the following variables:
2454
2455- ```INITRD`` <#var-INITRD>`__: Indicates list of filesystem images to
2456 concatenate and use as an initial RAM disk (initrd). This variable is
2457 optional.
2458
2459- ```ROOTFS`` <#var-ROOTFS>`__: Indicates a filesystem image to include
2460 as the root filesystem. This variable is optional.
2461
2462- ```AUTO_SYSLINUXMENU`` <#var-AUTO_SYSLINUXMENU>`__: Enables creating
2463 an automatic menu when set to "1".
2464
2465- ```LABELS`` <#var-LABELS>`__: Lists targets for automatic
2466 configuration.
2467
2468- ```APPEND`` <#var-APPEND>`__: Lists append string overrides for each
2469 label.
2470
2471- ```SYSLINUX_OPTS`` <#var-SYSLINUX_OPTS>`__: Lists additional options
2472 to add to the syslinux file. Semicolon characters separate multiple
2473 options.
2474
2475- ```SYSLINUX_SPLASH`` <#var-SYSLINUX_SPLASH>`__: Lists a background
2476 for the VGA boot menu when you are using the boot menu.
2477
2478- ```SYSLINUX_DEFAULT_CONSOLE`` <#var-SYSLINUX_DEFAULT_CONSOLE>`__: Set
2479 to "console=ttyX" to change kernel boot default console.
2480
2481- ```SYSLINUX_SERIAL`` <#var-SYSLINUX_SERIAL>`__: Sets an alternate
2482 serial port. Or, turns off serial when the variable is set with an
2483 empty string.
2484
2485- ```SYSLINUX_SERIAL_TTY`` <#var-SYSLINUX_SERIAL_TTY>`__: Sets an
2486 alternate "console=tty..." kernel boot argument.
2487
2488.. _ref-classes-systemd:
2489
2490``systemd.bbclass``
2491===================
2492
2493The ``systemd`` class provides support for recipes that install systemd
2494unit files.
2495
2496The functionality for this class is disabled unless you have "systemd"
2497in ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__.
2498
2499Under this class, the recipe or Makefile (i.e. whatever the recipe is
2500calling during the ```do_install`` <#ref-tasks-install>`__ task)
2501installs unit files into
2502``${``\ ```D`` <#var-D>`__\ ``}${systemd_unitdir}/system``. If the unit
2503files being installed go into packages other than the main package, you
2504need to set ```SYSTEMD_PACKAGES`` <#var-SYSTEMD_PACKAGES>`__ in your
2505recipe to identify the packages in which the files will be installed.
2506
2507You should set ```SYSTEMD_SERVICE`` <#var-SYSTEMD_SERVICE>`__ to the
2508name of the service file. You should also use a package name override to
2509indicate the package to which the value applies. If the value applies to
2510the recipe's main package, use ``${``\ ```PN`` <#var-PN>`__\ ``}``. Here
2511is an example from the connman recipe: SYSTEMD_SERVICE_${PN} =
2512"connman.service" Services are set up to start on boot automatically
2513unless you have set
2514```SYSTEMD_AUTO_ENABLE`` <#var-SYSTEMD_AUTO_ENABLE>`__ to "disable".
2515
2516For more information on ``systemd``, see the "`Selecting an
2517Initialization
2518Manager <&YOCTO_DOCS_DEV_URL;#selecting-an-initialization-manager>`__"
2519section in the Yocto Project Development Tasks Manual.
2520
2521.. _ref-classes-systemd-boot:
2522
2523``systemd-boot.bbclass``
2524========================
2525
2526The ``systemd-boot`` class provides functions specific to the
2527systemd-boot bootloader for building bootable images. This is an
2528internal class and is not intended to be used directly.
2529
2530.. note::
2531
2532 The
2533 systemd-boot
2534 class is a result from merging the
2535 gummiboot
2536 class used in previous Yocto Project releases with the
2537 systemd
2538 project.
2539
2540Set the ```EFI_PROVIDER`` <#var-EFI_PROVIDER>`__ variable to
2541"systemd-boot" to use this class. Doing so creates a standalone EFI
2542bootloader that is not dependent on systemd.
2543
2544For information on more variables used and supported in this class, see
2545the ```SYSTEMD_BOOT_CFG`` <#var-SYSTEMD_BOOT_CFG>`__,
2546```SYSTEMD_BOOT_ENTRIES`` <#var-SYSTEMD_BOOT_ENTRIES>`__, and
2547```SYSTEMD_BOOT_TIMEOUT`` <#var-SYSTEMD_BOOT_TIMEOUT>`__ variables.
2548
2549You can also see the `Systemd-boot
2550documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__
2551for more information.
2552
2553.. _ref-classes-terminal:
2554
2555``terminal.bbclass``
2556====================
2557
2558The ``terminal`` class provides support for starting a terminal session.
2559The ```OE_TERMINAL`` <#var-OE_TERMINAL>`__ variable controls which
2560terminal emulator is used for the session.
2561
2562Other classes use the ``terminal`` class anywhere a separate terminal
2563session needs to be started. For example, the
2564```patch`` <#ref-classes-patch>`__ class assuming
2565```PATCHRESOLVE`` <#var-PATCHRESOLVE>`__ is set to "user", the
2566```cml1`` <#ref-classes-cml1>`__ class, and the
2567```devshell`` <#ref-classes-devshell>`__ class all use the ``terminal``
2568class.
2569
2570.. _ref-classes-testimage*:
2571
2572``testimage*.bbclass``
2573======================
2574
2575The ``testimage*`` classes support running automated tests against
2576images using QEMU and on actual hardware. The classes handle loading the
2577tests and starting the image. To use the classes, you need to perform
2578steps to set up the environment.
2579
2580.. note::
2581
2582 Best practices include using
2583 IMAGE_CLASSES
2584 rather than
2585 INHERIT
2586 to inherit the
2587 testimage
2588 class for automated image testing.
2589
2590The tests are commands that run on the target system over ``ssh``. Each
2591test is written in Python and makes use of the ``unittest`` module.
2592
2593The ``testimage.bbclass`` runs tests on an image when called using the
2594following: $ bitbake -c testimage image The ``testimage-auto`` class
2595runs tests on an image after the image is constructed (i.e.
2596```TESTIMAGE_AUTO`` <#var-TESTIMAGE_AUTO>`__ must be set to "1").
2597
2598For information on how to enable, run, and create new tests, see the
2599"`Performing Automated Runtime
2600Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
2601section in the Yocto Project Development Tasks Manual.
2602
2603.. _ref-classes-testsdk:
2604
2605``testsdk.bbclass``
2606===================
2607
2608This class supports running automated tests against software development
2609kits (SDKs). The ``testsdk`` class runs tests on an SDK when called
2610using the following: $ bitbake -c testsdk image
2611
2612.. note::
2613
2614 Best practices include using
2615 IMAGE_CLASSES
2616 rather than
2617 INHERIT
2618 to inherit the
2619 testsdk
2620 class for automated SDK testing.
2621
2622.. _ref-classes-texinfo:
2623
2624``texinfo.bbclass``
2625===================
2626
2627This class should be inherited by recipes whose upstream packages invoke
2628the ``texinfo`` utilities at build-time. Native and cross recipes are
2629made to use the dummy scripts provided by ``texinfo-dummy-native``, for
2630improved performance. Target architecture recipes use the genuine
2631Texinfo utilities. By default, they use the Texinfo utilities on the
2632host system.
2633
2634.. note::
2635
2636 If you want to use the Texinfo recipe shipped with the build system,
2637 you can remove "texinfo-native" from
2638 ASSUME_PROVIDED
2639 and makeinfo from
2640 SANITY_REQUIRED_UTILITIES
2641 .
2642
2643.. _ref-classes-tinderclient:
2644
2645``tinderclient.bbclass``
2646========================
2647
2648The ``tinderclient`` class submits build results to an external
2649Tinderbox instance.
2650
2651.. note::
2652
2653 This class is currently unmaintained.
2654
2655.. _ref-classes-toaster:
2656
2657``toaster.bbclass``
2658===================
2659
2660The ``toaster`` class collects information about packages and images and
2661sends them as events that the BitBake user interface can receive. The
2662class is enabled when the Toaster user interface is running.
2663
2664This class is not intended to be used directly.
2665
2666.. _ref-classes-toolchain-scripts:
2667
2668``toolchain-scripts.bbclass``
2669=============================
2670
2671The ``toolchain-scripts`` class provides the scripts used for setting up
2672the environment for installed SDKs.
2673
2674.. _ref-classes-typecheck:
2675
2676``typecheck.bbclass``
2677=====================
2678
2679The ``typecheck`` class provides support for validating the values of
2680variables set at the configuration level against their defined types.
2681The OpenEmbedded build system allows you to define the type of a
2682variable using the "type" varflag. Here is an example:
2683IMAGE_FEATURES[type] = "list"
2684
2685.. _ref-classes-uboot-config:
2686
2687``uboot-config.bbclass``
2688========================
2689
2690The ``uboot-config`` class provides support for U-Boot configuration for
2691a machine. Specify the machine in your recipe as follows: UBOOT_CONFIG
2692??= <default> UBOOT_CONFIG[foo] = "config,images" You can also specify
2693the machine using this method: UBOOT_MACHINE = "config" See the
2694```UBOOT_CONFIG`` <#var-UBOOT_CONFIG>`__ and
2695```UBOOT_MACHINE`` <#var-UBOOT_MACHINE>`__ variables for additional
2696information.
2697
2698.. _ref-classes-uninative:
2699
2700``uninative.bbclass``
2701=====================
2702
2703Attempts to isolate the build system from the host distribution's C
2704library in order to make re-use of native shared state artifacts across
2705different host distributions practical. With this class enabled, a
2706tarball containing a pre-built C library is downloaded at the start of
2707the build. In the Poky reference distribution this is enabled by default
2708through ``meta/conf/distro/include/yocto-uninative.inc``. Other
2709distributions that do not derive from poky can also
2710"``require conf/distro/include/yocto-uninative.inc``" to use this.
2711Alternatively if you prefer, you can build the uninative-tarball recipe
2712yourself, publish the resulting tarball (e.g. via HTTP) and set
2713``UNINATIVE_URL`` and ``UNINATIVE_CHECKSUM`` appropriately. For an
2714example, see the ``meta/conf/distro/include/yocto-uninative.inc``.
2715
2716The ``uninative`` class is also used unconditionally by the extensible
2717SDK. When building the extensible SDK, ``uninative-tarball`` is built
2718and the resulting tarball is included within the SDK.
2719
2720.. _ref-classes-update-alternatives:
2721
2722``update-alternatives.bbclass``
2723===============================
2724
2725The ``update-alternatives`` class helps the alternatives system when
2726multiple sources provide the same command. This situation occurs when
2727several programs that have the same or similar function are installed
2728with the same name. For example, the ``ar`` command is available from
2729the ``busybox``, ``binutils`` and ``elfutils`` packages. The
2730``update-alternatives`` class handles renaming the binaries so that
2731multiple packages can be installed without conflicts. The ``ar`` command
2732still works regardless of which packages are installed or subsequently
2733removed. The class renames the conflicting binary in each package and
2734symlinks the highest priority binary during installation or removal of
2735packages.
2736
2737To use this class, you need to define a number of variables:
2738
2739- ```ALTERNATIVE`` <#var-ALTERNATIVE>`__
2740
2741- ```ALTERNATIVE_LINK_NAME`` <#var-ALTERNATIVE_LINK_NAME>`__
2742
2743- ```ALTERNATIVE_TARGET`` <#var-ALTERNATIVE_TARGET>`__
2744
2745- ```ALTERNATIVE_PRIORITY`` <#var-ALTERNATIVE_PRIORITY>`__
2746
2747These variables list alternative commands needed by a package, provide
2748pathnames for links, default links for targets, and so forth. For
2749details on how to use this class, see the comments in the
2750```update-alternatives.bbclass`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/update-alternatives.bbclass>`__
2751file.
2752
2753.. note::
2754
2755 You can use the
2756 update-alternatives
2757 command directly in your recipes. However, this class simplifies
2758 things in most cases.
2759
2760.. _ref-classes-update-rc.d:
2761
2762``update-rc.d.bbclass``
2763=======================
2764
2765The ``update-rc.d`` class uses ``update-rc.d`` to safely install an
2766initialization script on behalf of the package. The OpenEmbedded build
2767system takes care of details such as making sure the script is stopped
2768before a package is removed and started when the package is installed.
2769
2770Three variables control this class: ``INITSCRIPT_PACKAGES``,
2771``INITSCRIPT_NAME`` and ``INITSCRIPT_PARAMS``. See the variable links
2772for details.
2773
2774.. _ref-classes-useradd:
2775
2776``useradd*.bbclass``
2777====================
2778
2779The ``useradd*`` classes support the addition of users or groups for
2780usage by the package on the target. For example, if you have packages
2781that contain system services that should be run under their own user or
2782group, you can use these classes to enable creation of the user or
2783group. The ``meta-skeleton/recipes-skeleton/useradd/useradd-example.bb``
2784recipe in the `Source Directory <#source-directory>`__ provides a simple
2785example that shows how to add three users and groups to two packages.
2786See the ``useradd-example.bb`` recipe for more information on how to use
2787these classes.
2788
2789The ``useradd_base`` class provides basic functionality for user or
2790groups settings.
2791
2792The ``useradd*`` classes support the
2793```USERADD_PACKAGES`` <#var-USERADD_PACKAGES>`__,
2794```USERADD_PARAM`` <#var-USERADD_PARAM>`__,
2795```GROUPADD_PARAM`` <#var-GROUPADD_PARAM>`__, and
2796```GROUPMEMS_PARAM`` <#var-GROUPMEMS_PARAM>`__ variables.
2797
2798The ``useradd-staticids`` class supports the addition of users or groups
2799that have static user identification (``uid``) and group identification
2800(``gid``) values.
2801
2802The default behavior of the OpenEmbedded build system for assigning
2803``uid`` and ``gid`` values when packages add users and groups during
2804package install time is to add them dynamically. This works fine for
2805programs that do not care what the values of the resulting users and
2806groups become. In these cases, the order of the installation determines
2807the final ``uid`` and ``gid`` values. However, if non-deterministic
2808``uid`` and ``gid`` values are a problem, you can override the default,
2809dynamic application of these values by setting static values. When you
2810set static values, the OpenEmbedded build system looks in
2811```BBPATH`` <#var-BBPATH>`__ for ``files/passwd`` and ``files/group``
2812files for the values.
2813
2814To use static ``uid`` and ``gid`` values, you need to set some
2815variables. See the ```USERADDEXTENSION`` <#var-USERADDEXTENSION>`__,
2816```USERADD_UID_TABLES`` <#var-USERADD_UID_TABLES>`__,
2817```USERADD_GID_TABLES`` <#var-USERADD_GID_TABLES>`__, and
2818```USERADD_ERROR_DYNAMIC`` <#var-USERADD_ERROR_DYNAMIC>`__ variables.
2819You can also see the ```useradd`` <#ref-classes-useradd>`__ class for
2820additional information.
2821
2822.. note::
2823
2824 You do not use the
2825 useradd-staticids
2826 class directly. You either enable or disable the class by setting the
2827 USERADDEXTENSION
2828 variable. If you enable or disable the class in a configured system,
2829 TMPDIR
2830 might contain incorrect
2831 uid
2832 and
2833 gid
2834 values. Deleting the
2835 TMPDIR
2836 directory will correct this condition.
2837
2838.. _ref-classes-utility-tasks:
2839
2840``utility-tasks.bbclass``
2841=========================
2842
2843The ``utility-tasks`` class provides support for various "utility" type
2844tasks that are applicable to all recipes, such as
2845```do_clean`` <#ref-tasks-clean>`__ and
2846```do_listtasks`` <#ref-tasks-listtasks>`__.
2847
2848This class is enabled by default because it is inherited by the
2849```base`` <#ref-classes-base>`__ class.
2850
2851.. _ref-classes-utils:
2852
2853``utils.bbclass``
2854=================
2855
2856The ``utils`` class provides some useful Python functions that are
2857typically used in inline Python expressions (e.g. ``${@...}``). One
2858example use is for ``bb.utils.contains()``.
2859
2860This class is enabled by default because it is inherited by the
2861```base`` <#ref-classes-base>`__ class.
2862
2863.. _ref-classes-vala:
2864
2865``vala.bbclass``
2866================
2867
2868The ``vala`` class supports recipes that need to build software written
2869using the Vala programming language.
2870
2871.. _ref-classes-waf:
2872
2873``waf.bbclass``
2874===============
2875
2876The ``waf`` class supports recipes that need to build software that uses
2877the Waf build system. You can use the
2878```EXTRA_OECONF`` <#var-EXTRA_OECONF>`__ or
2879```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__ variables
2880to specify additional configuration options to be passed on the Waf
2881command line.
diff --git a/documentation/ref-manual/ref-devtool-reference.rst b/documentation/ref-manual/ref-devtool-reference.rst
new file mode 100644
index 0000000000..c91ba5bdfa
--- /dev/null
+++ b/documentation/ref-manual/ref-devtool-reference.rst
@@ -0,0 +1,533 @@
1***************************
2``devtool`` Quick Reference
3***************************
4
5The ``devtool`` command-line tool provides a number of features that
6help you build, test, and package software. This command is available
7alongside the ``bitbake`` command. Additionally, the ``devtool`` command
8is a key part of the extensible SDK.
9
10This chapter provides a Quick Reference for the ``devtool`` command. For
11more information on how to apply the command when using the extensible
12SDK, see the "`Using the Extensible
13SDK <&YOCTO_DOCS_SDK_URL;#sdk-extensible>`__" chapter in the Yocto
14Project Application Development and the Extensible Software Development
15Kit (eSDK) manual.
16
17.. _devtool-getting-help:
18
19Getting Help
20============
21
22The ``devtool`` command line is organized similarly to Git in that it
23has a number of sub-commands for each function. You can run
24``devtool --help`` to see all the commands: $ devtool -h NOTE: Starting
25bitbake server... usage: devtool [--basepath BASEPATH] [--bbpath BBPATH]
26[-d] [-q] [--color COLOR] [-h] <subcommand> ... OpenEmbedded development
27tool options: --basepath BASEPATH Base directory of SDK / build
28directory --bbpath BBPATH Explicitly specify the BBPATH, rather than
29getting it from the metadata -d, --debug Enable debug output -q, --quiet
30Print only errors --color COLOR Colorize output (where COLOR is auto,
31always, never) -h, --help show this help message and exit subcommands:
32Beginning work on a recipe: add Add a new recipe modify Modify the
33source for an existing recipe upgrade Upgrade an existing recipe Getting
34information: status Show workspace status search Search available
35recipes latest-version Report the latest version of an existing recipe
36check-upgrade-status Report upgradability for multiple (or all) recipes
37Working on a recipe in the workspace: build Build a recipe rename Rename
38a recipe file in the workspace edit-recipe Edit a recipe file
39find-recipe Find a recipe file configure-help Get help on configure
40script options update-recipe Apply changes from external source tree to
41recipe reset Remove a recipe from your workspace finish Finish working
42on a recipe in your workspace Testing changes on target: deploy-target
43Deploy recipe output files to live target machine undeploy-target
44Undeploy recipe output files in live target machine build-image Build
45image including workspace recipe packages Advanced: create-workspace Set
46up workspace in an alternative location export Export workspace into a
47tar archive import Import exported tar archive into workspace extract
48Extract the source for an existing recipe sync Synchronize the source
49tree for an existing recipe Use devtool <subcommand> --help to get help
50on a specific command As directed in the general help output, you can
51get more syntax on a specific command by providing the command name and
52using "--help": $ devtool add --help NOTE: Starting bitbake server...
53usage: devtool add [-h] [--same-dir \| --no-same-dir] [--fetch URI]
54[--fetch-dev] [--version VERSION] [--no-git] [--srcrev SRCREV \|
55--autorev] [--srcbranch SRCBRANCH] [--binary] [--also-native]
56[--src-subdir SUBDIR] [--mirrors] [--provides PROVIDES] [recipename]
57[srctree] [fetchuri] Adds a new recipe to the workspace to build a
58specified source tree. Can optionally fetch a remote URI and unpack it
59to create the source tree. arguments: recipename Name for new recipe to
60add (just name - no version, path or extension). If not specified, will
61attempt to auto-detect it. srctree Path to external source tree. If not
62specified, a subdirectory of /home/scottrif/poky/build/workspace/sources
63will be used. fetchuri Fetch the specified URI and extract it to create
64the source tree options: -h, --help show this help message and exit
65--same-dir, -s Build in same directory as source --no-same-dir Force
66build in a separate build directory --fetch URI, -f URI Fetch the
67specified URI and extract it to create the source tree (deprecated -
68pass as positional argument instead) --fetch-dev For npm, also fetch
69devDependencies --version VERSION, -V VERSION Version to use within
70recipe (PV) --no-git, -g If fetching source, do not set up source tree
71as a git repository --srcrev SRCREV, -S SRCREV Source revision to fetch
72if fetching from an SCM such as git (default latest) --autorev, -a When
73fetching from a git repository, set SRCREV in the recipe to a floating
74revision instead of fixed --srcbranch SRCBRANCH, -B SRCBRANCH Branch in
75source repository if fetching from an SCM such as git (default master)
76--binary, -b Treat the source tree as something that should be installed
77verbatim (no compilation, same directory structure). Useful with binary
78packages e.g. RPMs. --also-native Also add native variant (i.e. support
79building recipe for the build host as well as the target machine)
80--src-subdir SUBDIR Specify subdirectory within source tree to use
81--mirrors Enable PREMIRRORS and MIRRORS for source tree fetching
82(disable by default). --provides PROVIDES, -p PROVIDES Specify an alias
83for the item provided by the recipe. E.g. virtual/libgl
84
85.. _devtool-the-workspace-layer-structure:
86
87The Workspace Layer Structure
88=============================
89
90``devtool`` uses a "Workspace" layer in which to accomplish builds. This
91layer is not specific to any single ``devtool`` command but is rather a
92common working area used across the tool.
93
94The following figure shows the workspace structure:
95
96attic - A directory created if devtool believes it must preserve
97anything when you run "devtool reset". For example, if you run "devtool
98add", make changes to the recipe, and then run "devtool reset", devtool
99takes notice that the file has been changed and moves it into the attic
100should you still want the recipe. README - Provides information on what
101is in workspace layer and how to manage it. .devtool_md5 - A checksum
102file used by devtool. appends - A directory that contains \*.bbappend
103files, which point to external source. conf - A configuration directory
104that contains the layer.conf file. recipes - A directory containing
105recipes. This directory contains a folder for each directory added whose
106name matches that of the added recipe. devtool places the recipe.bb file
107within that sub-directory. sources - A directory containing a working
108copy of the source files used when building the recipe. This is the
109default directory used as the location of the source tree when you do
110not provide a source tree path. This directory contains a folder for
111each set of source files matched to a corresponding recipe.
112
113.. _devtool-adding-a-new-recipe-to-the-workspace:
114
115Adding a New Recipe to the Workspace Layer
116==========================================
117
118Use the ``devtool add`` command to add a new recipe to the workspace
119layer. The recipe you add should not exist - ``devtool`` creates it for
120you. The source files the recipe uses should exist in an external area.
121
122The following example creates and adds a new recipe named ``jackson`` to
123a workspace layer the tool creates. The source code built by the recipes
124resides in ``/home/user/sources/jackson``: $ devtool add jackson
125/home/user/sources/jackson
126
127If you add a recipe and the workspace layer does not exist, the command
128creates the layer and populates it as described in "`The Workspace Layer
129Structure <#devtool-the-workspace-layer-structure>`__" section.
130
131Running ``devtool add`` when the workspace layer exists causes the tool
132to add the recipe, append files, and source files into the existing
133workspace layer. The ``.bbappend`` file is created to point to the
134external source tree.
135
136.. note::
137
138 If your recipe has runtime dependencies defined, you must be sure
139 that these packages exist on the target hardware before attempting to
140 run your application. If dependent packages (e.g. libraries) do not
141 exist on the target, your application, when run, will fail to find
142 those functions. For more information, see the "
143 Deploying Your Software on the Target Machine
144 " section.
145
146By default, ``devtool add`` uses the latest revision (i.e. master) when
147unpacking files from a remote URI. In some cases, you might want to
148specify a source revision by branch, tag, or commit hash. You can
149specify these options when using the ``devtool add`` command:
150
151- To specify a source branch, use the ``--srcbranch`` option: $ devtool
152 add --srcbranch DISTRO_NAME_NO_CAP jackson /home/user/sources/jackson
153 In the previous example, you are checking out the DISTRO_NAME_NO_CAP
154 branch.
155
156- To specify a specific tag or commit hash, use the ``--srcrev``
157 option: $ devtool add --srcrev DISTRO_REL_TAG jackson
158 /home/user/sources/jackson $ devtool add --srcrev some_commit_hash
159 /home/user/sources/jackson The previous examples check out the
160 DISTRO_REL_TAG tag and the commit associated with the
161 some_commit_hash hash.
162
163.. note::
164
165 If you prefer to use the latest revision every time the recipe is
166 built, use the options
167 --autorev
168 or
169 -a
170 .
171
172.. _devtool-extracting-the-source-for-an-existing-recipe:
173
174Extracting the Source for an Existing Recipe
175============================================
176
177Use the ``devtool extract`` command to extract the source for an
178existing recipe. When you use this command, you must supply the root
179name of the recipe (i.e. no version, paths, or extensions), and you must
180supply the directory to which you want the source extracted.
181
182Additional command options let you control the name of a development
183branch into which you can checkout the source and whether or not to keep
184a temporary directory, which is useful for debugging.
185
186.. _devtool-synchronizing-a-recipes-extracted-source-tree:
187
188Synchronizing a Recipe's Extracted Source Tree
189==============================================
190
191Use the ``devtool sync`` command to synchronize a previously extracted
192source tree for an existing recipe. When you use this command, you must
193supply the root name of the recipe (i.e. no version, paths, or
194extensions), and you must supply the directory to which you want the
195source extracted.
196
197Additional command options let you control the name of a development
198branch into which you can checkout the source and whether or not to keep
199a temporary directory, which is useful for debugging.
200
201.. _devtool-modifying-a-recipe:
202
203Modifying an Existing Recipe
204============================
205
206Use the ``devtool modify`` command to begin modifying the source of an
207existing recipe. This command is very similar to the
208```add`` <#devtool-adding-a-new-recipe-to-the-workspace>`__ command
209except that it does not physically create the recipe in the workspace
210layer because the recipe already exists in an another layer.
211
212The ``devtool modify`` command extracts the source for a recipe, sets it
213up as a Git repository if the source had not already been fetched from
214Git, checks out a branch for development, and applies any patches from
215the recipe as commits on top. You can use the following command to
216checkout the source files: $ devtool modify recipe Using the above
217command form, ``devtool`` uses the existing recipe's
218```SRC_URI`` <#var-SRC_URI>`__ statement to locate the upstream source,
219extracts the source into the default sources location in the workspace.
220The default development branch used is "devtool".
221
222.. _devtool-edit-an-existing-recipe:
223
224Edit an Existing Recipe
225=======================
226
227Use the ``devtool edit-recipe`` command to run the default editor, which
228is identified using the ``EDITOR`` variable, on the specified recipe.
229
230When you use the ``devtool edit-recipe`` command, you must supply the
231root name of the recipe (i.e. no version, paths, or extensions). Also,
232the recipe file itself must reside in the workspace as a result of the
233``devtool add`` or ``devtool upgrade`` commands. However, you can
234override that requirement by using the "-a" or "--any-recipe" option.
235Using either of these options allows you to edit any recipe regardless
236of its location.
237
238.. _devtool-updating-a-recipe:
239
240Updating a Recipe
241=================
242
243Use the ``devtool update-recipe`` command to update your recipe with
244patches that reflect changes you make to the source files. For example,
245if you know you are going to work on some code, you could first use the
246```devtool modify`` <#devtool-modifying-a-recipe>`__ command to extract
247the code and set up the workspace. After which, you could modify,
248compile, and test the code.
249
250When you are satisfied with the results and you have committed your
251changes to the Git repository, you can then run the
252``devtool update-recipe`` to create the patches and update the recipe: $
253devtool update-recipe recipe If you run the ``devtool update-recipe``
254without committing your changes, the command ignores the changes.
255
256Often, you might want to apply customizations made to your software in
257your own layer rather than apply them to the original recipe. If so, you
258can use the ``-a`` or ``--append`` option with the
259``devtool update-recipe`` command. These options allow you to specify
260the layer into which to write an append file: $ devtool update-recipe
261recipe -a base-layer-directory The ``*.bbappend`` file is created at the
262appropriate path within the specified layer directory, which may or may
263not be in your ``bblayers.conf`` file. If an append file already exists,
264the command updates it appropriately.
265
266.. _devtool-checking-on-the-upgrade-status-of-a-recipe:
267
268Checking on the Upgrade Status of a Recipe
269==========================================
270
271Upstream recipes change over time. Consequently, you might find that you
272need to determine if you can upgrade a recipe to a newer version.
273
274To check on the upgrade status of a recipe, use the
275``devtool check-upgrade-status`` command. The command displays a table
276of your current recipe versions, the latest upstream versions, the email
277address of the recipe's maintainer, and any additional information such
278as commit hash strings and reasons you might not be able to upgrade a
279particular recipe.
280
281.. note::
282
283 - For the ``oe-core`` layer, recipe maintainers come from the
284 ```maintainers.inc`` <http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/distro/include/maintainers.inc>`__
285 file.
286
287 - If the recipe is using the `Git
288 fetcher <&YOCTO_DOCS_BB_URL;#git-fetcher>`__ rather than a
289 tarball, the commit hash points to the commit that matches the
290 recipe's latest version tag.
291
292As with all ``devtool`` commands, you can get help on the individual
293command: $ devtool check-upgrade-status -h NOTE: Starting bitbake
294server... usage: devtool check-upgrade-status [-h] [--all] [recipe
295[recipe ...]] Prints a table of recipes together with versions currently
296provided by recipes, and latest upstream versions, when there is a later
297version available arguments: recipe Name of the recipe to report (omit
298to report upgrade info for all recipes) options: -h, --help show this
299help message and exit --all, -a Show all recipes, not just recipes
300needing upgrade
301
302Unless you provide a specific recipe name on the command line, the
303command checks all recipes in all configured layers.
304
305Following is a partial example table that reports on all the recipes.
306Notice the reported reason for not upgrading the ``base-passwd`` recipe.
307In this example, while a new version is available upstream, you do not
308want to use it because the dependency on ``cdebconf`` is not easily
309satisfied.
310
311.. note::
312
313 When a reason for not upgrading displays, the reason is usually
314 written into the recipe using the
315 RECIPE_NO_UPDATE_REASON
316 variable. See the
317 base-passwd.bb
318 recipe for an example.
319
320$ devtool check-upgrade-status ... NOTE: acpid 2.0.30 2.0.31 Ross Burton
321<ross.burton@intel.com> NOTE: u-boot-fw-utils 2018.11 2019.01 Marek
322Vasut <marek.vasut@gmail.com> d3689267f92c5956e09cc7d1baa4700141662bff
323NOTE: u-boot-tools 2018.11 2019.01 Marek Vasut <marek.vasut@gmail.com>
324d3689267f92c5956e09cc7d1baa4700141662bff . . . NOTE: base-passwd 3.5.29
3253.5.45 Anuj Mittal <anuj.mittal@intel.com> cannot be updated due to:
326Version 3.5.38 requires cdebconf for update-passwd utility NOTE: busybox
3271.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com> NOTE: dbus-test
3281.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
329
330.. _devtool-upgrading-a-recipe:
331
332Upgrading a Recipe
333==================
334
335As software matures, upstream recipes are upgraded to newer versions. As
336a developer, you need to keep your local recipes up-to-date with the
337upstream version releases. Several methods exist by which you can
338upgrade recipes. You can read about them in the "`Upgrading
339Recipes <&YOCTO_DOCS_DEV_URL;#gs-upgrading-recipes>`__" section of the
340Yocto Project Development Tasks Manual. This section overviews the
341``devtool upgrade`` command.
342
343.. note::
344
345 Before you upgrade a recipe, you can check on its upgrade status. See
346 the "
347 Checking on the Upgrade Status of a Recipe
348 " for more information.
349
350The ``devtool upgrade`` command upgrades an existing recipe to a more
351recent version of the recipe upstream. The command puts the upgraded
352recipe file along with any associated files into a "workspace" and, if
353necessary, extracts the source tree to a specified location. During the
354upgrade, patches associated with the recipe are rebased or added as
355needed.
356
357When you use the ``devtool upgrade`` command, you must supply the root
358name of the recipe (i.e. no version, paths, or extensions), and you must
359supply the directory to which you want the source extracted. Additional
360command options let you control things such as the version number to
361which you want to upgrade (i.e. the ```PV`` <#var-PV>`__), the source
362revision to which you want to upgrade (i.e. the
363```SRCREV`` <#var-SRCREV>`__), whether or not to apply patches, and so
364forth.
365
366You can read more on the ``devtool upgrade`` workflow in the "`Use
367``devtool upgrade`` to Create a Version of the Recipe that Supports a
368Newer Version of the
369Software <&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software>`__"
370section in the Yocto Project Application Development and the Extensible
371Software Development Kit (eSDK) manual. You can also see an example of
372how to use ``devtool upgrade`` in the "`Using
373``devtool upgrade`` <&YOCTO_DOCS_DEV_URL;#gs-using-devtool-upgrade>`__"
374section in the Yocto Project Development Tasks Manual.
375
376.. _devtool-resetting-a-recipe:
377
378Resetting a Recipe
379==================
380
381Use the ``devtool reset`` command to remove a recipe and its
382configuration (e.g. the corresponding ``.bbappend`` file) from the
383workspace layer. Realize that this command deletes the recipe and the
384append file. The command does not physically move them for you.
385Consequently, you must be sure to physically relocate your updated
386recipe and the append file outside of the workspace layer before running
387the ``devtool reset`` command.
388
389If the ``devtool reset`` command detects that the recipe or the append
390files have been modified, the command preserves the modified files in a
391separate "attic" subdirectory under the workspace layer.
392
393Here is an example that resets the workspace directory that contains the
394``mtr`` recipe: $ devtool reset mtr NOTE: Cleaning sysroot for recipe
395mtr... NOTE: Leaving source tree
396/home/scottrif/poky/build/workspace/sources/mtr as-is; if you no longer
397need it then please delete it manually $
398
399.. _devtool-building-your-recipe:
400
401Building Your Recipe
402====================
403
404Use the ``devtool build`` command to build your recipe. The
405``devtool build`` command is equivalent to the
406``bitbake -c populate_sysroot`` command.
407
408When you use the ``devtool build`` command, you must supply the root
409name of the recipe (i.e. do not provide versions, paths, or extensions).
410You can use either the "-s" or the "--disable-parallel-make" options to
411disable parallel makes during the build. Here is an example: $ devtool
412build recipe
413
414.. _devtool-building-your-image:
415
416Building Your Image
417===================
418
419Use the ``devtool build-image`` command to build an image, extending it
420to include packages from recipes in the workspace. Using this command is
421useful when you want an image that ready for immediate deployment onto a
422device for testing. For proper integration into a final image, you need
423to edit your custom image recipe appropriately.
424
425When you use the ``devtool build-image`` command, you must supply the
426name of the image. This command has no command line options: $ devtool
427build-image image
428
429.. _devtool-deploying-your-software-on-the-target-machine:
430
431Deploying Your Software on the Target Machine
432=============================================
433
434Use the ``devtool deploy-target`` command to deploy the recipe's build
435output to the live target machine: $ devtool deploy-target recipe target
436The target is the address of the target machine, which must be running
437an SSH server (i.e. ``user@hostname[:destdir]``).
438
439This command deploys all files installed during the
440```do_install`` <#ref-tasks-install>`__ task. Furthermore, you do not
441need to have package management enabled within the target machine. If
442you do, the package manager is bypassed.
443
444.. note::
445
446 The ``deploy-target`` functionality is for development only. You
447 should never use it to update an image that will be used in
448 production.
449
450Some conditions exist that could prevent a deployed application from
451behaving as expected. When both of the following conditions exist, your
452application has the potential to not behave correctly when run on the
453target:
454
455- You are deploying a new application to the target and the recipe you
456 used to build the application had correctly defined runtime
457 dependencies.
458
459- The target does not physically have the packages on which the
460 application depends installed.
461
462If both of these conditions exist, your application will not behave as
463expected. The reason for this misbehavior is because the
464``devtool deploy-target`` command does not deploy the packages (e.g.
465libraries) on which your new application depends. The assumption is that
466the packages are already on the target. Consequently, when a runtime
467call is made in the application for a dependent function (e.g. a library
468call), the function cannot be found.
469
470To be sure you have all the dependencies local to the target, you need
471to be sure that the packages are pre-deployed (installed) on the target
472before attempting to run your application.
473
474.. _devtool-removing-your-software-from-the-target-machine:
475
476Removing Your Software from the Target Machine
477==============================================
478
479Use the ``devtool undeploy-target`` command to remove deployed build
480output from the target machine. For the ``devtool undeploy-target``
481command to work, you must have previously used the
482```devtool deploy-target`` <#devtool-deploying-your-software-on-the-target-machine>`__
483command. $ devtool undeploy-target recipe target The target is the
484address of the target machine, which must be running an SSH server (i.e.
485``user@hostname``).
486
487.. _devtool-creating-the-workspace:
488
489Creating the Workspace Layer in an Alternative Location
490=======================================================
491
492Use the ``devtool create-workspace`` command to create a new workspace
493layer in your `Build Directory <#build-directory>`__. When you create a
494new workspace layer, it is populated with the ``README`` file and the
495``conf`` directory only.
496
497The following example creates a new workspace layer in your current
498working and by default names the workspace layer "workspace": $ devtool
499create-workspace
500
501You can create a workspace layer anywhere by supplying a pathname with
502the command. The following command creates a new workspace layer named
503"new-workspace": $ devtool create-workspace /home/scottrif/new-workspace
504
505.. _devtool-get-the-status-of-the-recipes-in-your-workspace:
506
507Get the Status of the Recipes in Your Workspace
508===============================================
509
510Use the ``devtool status`` command to list the recipes currently in your
511workspace. Information includes the paths to their respective external
512source trees.
513
514The ``devtool status`` command has no command-line options: $ devtool
515status Following is sample output after using
516```devtool add`` <#devtool-adding-a-new-recipe-to-the-workspace>`__ to
517create and add the ``mtr_0.86.bb`` recipe to the ``workspace``
518directory: $ devtool status mtr:
519/home/scottrif/poky/build/workspace/sources/mtr
520(/home/scottrif/poky/build/workspace/recipes/mtr/mtr_0.86.bb) $
521
522.. _devtool-search-for-available-target-recipes:
523
524Search for Available Target Recipes
525===================================
526
527Use the ``devtool search`` command to search for available target
528recipes. The command matches the recipe name, package name, description,
529and installed files. The command displays the recipe name as a result of
530a match.
531
532When you use the ``devtool search`` command, you must supply a keyword.
533The command uses the keyword when searching for a match.
diff --git a/documentation/ref-manual/ref-features.rst b/documentation/ref-manual/ref-features.rst
new file mode 100644
index 0000000000..1cdf09bfdb
--- /dev/null
+++ b/documentation/ref-manual/ref-features.rst
@@ -0,0 +1,353 @@
1********
2Features
3********
4
5This chapter provides a reference of shipped machine and distro features
6you can include as part of your image, a reference on image features you
7can select, and a reference on feature backfilling.
8
9Features provide a mechanism for working out which packages should be
10included in the generated images. Distributions can select which
11features they want to support through the ``DISTRO_FEATURES`` variable,
12which is set or appended to in a distribution's configuration file such
13as ``poky.conf``, ``poky-tiny.conf``, ``poky-lsb.conf`` and so forth.
14Machine features are set in the ``MACHINE_FEATURES`` variable, which is
15set in the machine configuration file and specifies the hardware
16features for a given machine.
17
18These two variables combine to work out which kernel modules, utilities,
19and other packages to include. A given distribution can support a
20selected subset of features so some machine features might not be
21included if the distribution itself does not support them.
22
23One method you can use to determine which recipes are checking to see if
24a particular feature is contained or not is to ``grep`` through the
25`Metadata <#metadata>`__ for the feature. Here is an example that
26discovers the recipes whose build is potentially changed based on a
27given feature: $ cd poky $ git grep
28'contains.*MACHINE_FEATURES.*feature'
29
30.. _ref-features-machine:
31
32Machine Features
33================
34
35The items below are features you can use with
36```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__. Features do not have a
37one-to-one correspondence to packages, and they can go beyond simply
38controlling the installation of a package or packages. Sometimes a
39feature can influence how certain recipes are built. For example, a
40feature might determine whether a particular configure option is
41specified within the ```do_configure`` <#ref-tasks-configure>`__ task
42for a particular recipe.
43
44This feature list only represents features as shipped with the Yocto
45Project metadata:
46
47- *acpi:* Hardware has ACPI (x86/x86_64 only)
48
49- *alsa:* Hardware has ALSA audio drivers
50
51- *apm:* Hardware uses APM (or APM emulation)
52
53- *bluetooth:* Hardware has integrated BT
54
55- *efi:* Support for booting through EFI
56
57- *ext2:* Hardware HDD or Microdrive
58
59- *keyboard:* Hardware has a keyboard
60
61- *pcbios:* Support for booting through BIOS
62
63- *pci:* Hardware has a PCI bus
64
65- *pcmcia:* Hardware has PCMCIA or CompactFlash sockets
66
67- *phone:* Mobile phone (voice) support
68
69- *qvga:* Machine has a QVGA (320x240) display
70
71- *rtc:* Machine has a Real-Time Clock
72
73- *screen:* Hardware has a screen
74
75- *serial:* Hardware has serial support (usually RS232)
76
77- *touchscreen:* Hardware has a touchscreen
78
79- *usbgadget:* Hardware is USB gadget device capable
80
81- *usbhost:* Hardware is USB Host capable
82
83- *vfat:* FAT file system support
84
85- *wifi:* Hardware has integrated WiFi
86
87.. _ref-features-distro:
88
89Distro Features
90===============
91
92The items below are features you can use with
93```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ to enable features across
94your distribution. Features do not have a one-to-one correspondence to
95packages, and they can go beyond simply controlling the installation of
96a package or packages. In most cases, the presence or absence of a
97feature translates to the appropriate option supplied to the configure
98script during the ```do_configure`` <#ref-tasks-configure>`__ task for
99the recipes that optionally support the feature.
100
101Some distro features are also machine features. These select features
102make sense to be controlled both at the machine and distribution
103configuration level. See the
104```COMBINED_FEATURES`` <#var-COMBINED_FEATURES>`__ variable for more
105information.
106
107This list only represents features as shipped with the Yocto Project
108metadata:
109
110- *alsa:* Include ALSA support (OSS compatibility kernel modules
111 installed if available).
112
113- *api-documentation:* Enables generation of API documentation during
114 recipe builds. The resulting documentation is added to SDK tarballs
115 when the ``bitbake -c populate_sdk`` command is used. See the
116 "`Adding API Documentation to the Standard
117 SDK <&YOCTO_DOCS_SDK_URL;#adding-api-documentation-to-the-standard-sdk>`__"
118 section in the Yocto Project Application Development and the
119 Extensible Software Development Kit (eSDK) manual.
120
121- *bluetooth:* Include bluetooth support (integrated BT only).
122
123- *cramfs:* Include CramFS support.
124
125- *directfb:* Include DirectFB support.
126
127- *ext2:* Include tools for supporting for devices with internal
128 HDD/Microdrive for storing files (instead of Flash only devices).
129
130- *ipsec:* Include IPSec support.
131
132- *ipv6:* Include IPv6 support.
133
134- *keyboard:* Include keyboard support (e.g. keymaps will be loaded
135 during boot).
136
137- *ldconfig:* Include support for ldconfig and ``ld.so.conf`` on the
138 target.
139
140- *nfs:* Include NFS client support (for mounting NFS exports on
141 device).
142
143- *opengl:* Include the Open Graphics Library, which is a
144 cross-language, multi-platform application programming interface used
145 for rendering two and three-dimensional graphics.
146
147- *pci:* Include PCI bus support.
148
149- *pcmcia:* Include PCMCIA/CompactFlash support.
150
151- *ppp:* Include PPP dialup support.
152
153- *ptest:* Enables building the package tests where supported by
154 individual recipes. For more information on package tests, see the
155 "`Testing Packages With
156 ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section
157 in the Yocto Project Development Tasks Manual.
158
159- *smbfs:* Include SMB networks client support (for mounting
160 Samba/Microsoft Windows shares on device).
161
162- *systemd:* Include support for this ``init`` manager, which is a full
163 replacement of for ``init`` with parallel starting of services,
164 reduced shell overhead, and other features. This ``init`` manager is
165 used by many distributions.
166
167- *usbgadget:* Include USB Gadget Device support (for USB
168 networking/serial/storage).
169
170- *usbhost:* Include USB Host support (allows to connect external
171 keyboard, mouse, storage, network etc).
172
173- *usrmerge:* Merges the ``/bin``, ``/sbin``, ``/lib``, and ``/lib64``
174 directories into their respective counterparts in the ``/usr``
175 directory to provide better package and application compatibility.
176
177- *wayland:* Include the Wayland display server protocol and the
178 library that supports it.
179
180- *wifi:* Include WiFi support (integrated only).
181
182- *x11:* Include the X server and libraries.
183
184.. _ref-features-image:
185
186Image Features
187==============
188
189The contents of images generated by the OpenEmbedded build system can be
190controlled by the ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ and
191```EXTRA_IMAGE_FEATURES`` <#var-EXTRA_IMAGE_FEATURES>`__ variables that
192you typically configure in your image recipes. Through these variables,
193you can add several different predefined packages such as development
194utilities or packages with debug information needed to investigate
195application problems or profile applications.
196
197The following image features are available for all images:
198
199- *allow-empty-password:* Allows Dropbear and OpenSSH to accept root
200 logins and logins from accounts having an empty password string.
201
202- *dbg-pkgs:* Installs debug symbol packages for all packages installed
203 in a given image.
204
205- *debug-tweaks:* Makes an image suitable for development (e.g. allows
206 root logins without passwords and enables post-installation logging).
207 See the 'allow-empty-password', 'empty-root-password', and
208 'post-install-logging' features in this list for additional
209 information.
210
211- *dev-pkgs:* Installs development packages (headers and extra library
212 links) for all packages installed in a given image.
213
214- *doc-pkgs:* Installs documentation packages for all packages
215 installed in a given image.
216
217- *empty-root-password:* Sets the root password to an empty string,
218 which allows logins with a blank password.
219
220- *package-management:* Installs package management tools and preserves
221 the package manager database.
222
223- *post-install-logging:* Enables logging postinstall script runs to
224 the ``/var/log/postinstall.log`` file on first boot of the image on
225 the target system.
226
227 .. note::
228
229 To make the
230 /var/log
231 directory on the target persistent, use the
232 VOLATILE_LOG_DIR
233 variable by setting it to "no".
234
235- *ptest-pkgs:* Installs ptest packages for all ptest-enabled recipes.
236
237- *read-only-rootfs:* Creates an image whose root filesystem is
238 read-only. See the "`Creating a Read-Only Root
239 Filesystem <&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem>`__"
240 section in the Yocto Project Development Tasks Manual for more
241 information.
242
243- *splash:* Enables showing a splash screen during boot. By default,
244 this screen is provided by ``psplash``, which does allow
245 customization. If you prefer to use an alternative splash screen
246 package, you can do so by setting the ``SPLASH`` variable to a
247 different package name (or names) within the image recipe or at the
248 distro configuration level.
249
250- *staticdev-pkgs:* Installs static development packages, which are
251 static libraries (i.e. ``*.a`` files), for all packages installed in
252 a given image.
253
254Some image features are available only when you inherit the
255```core-image`` <#ref-classes-core-image>`__ class. The current list of
256these valid features is as follows:
257
258- *hwcodecs:* Installs hardware acceleration codecs.
259
260- *nfs-server:* Installs an NFS server.
261
262- *perf:* Installs profiling tools such as ``perf``, ``systemtap``, and
263 ``LTTng``. For general information on user-space tools, see the
264 `Yocto Project Application Development and the Extensible Software
265 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
266
267- *ssh-server-dropbear:* Installs the Dropbear minimal SSH server.
268
269- *ssh-server-openssh:* Installs the OpenSSH SSH server, which is more
270 full-featured than Dropbear. Note that if both the OpenSSH SSH server
271 and the Dropbear minimal SSH server are present in
272 ``IMAGE_FEATURES``, then OpenSSH will take precedence and Dropbear
273 will not be installed.
274
275- *tools-debug:* Installs debugging tools such as ``strace`` and
276 ``gdb``. For information on GDB, see the "`Debugging With the GNU
277 Project Debugger (GDB)
278 Remotely <&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug>`__" section
279 in the Yocto Project Development Tasks Manual. For information on
280 tracing and profiling, see the `Yocto Project Profiling and Tracing
281 Manual <&YOCTO_DOCS_PROF_URL;>`__.
282
283- *tools-sdk:* Installs a full SDK that runs on the device.
284
285- *tools-testapps:* Installs device testing tools (e.g. touchscreen
286 debugging).
287
288- *x11:* Installs the X server.
289
290- *x11-base:* Installs the X server with a minimal environment.
291
292- *x11-sato:* Installs the OpenedHand Sato environment.
293
294.. _ref-features-backfill:
295
296Feature Backfilling
297===================
298
299Sometimes it is necessary in the OpenEmbedded build system to extend
300```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__ or
301```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ to control functionality
302that was previously enabled and not able to be disabled. For these
303cases, we need to add an additional feature item to appear in one of
304these variables, but we do not want to force developers who have
305existing values of the variables in their configuration to add the new
306feature in order to retain the same overall level of functionality.
307Thus, the OpenEmbedded build system has a mechanism to automatically
308"backfill" these added features into existing distro or machine
309configurations. You can see the list of features for which this is done
310by finding the
311```DISTRO_FEATURES_BACKFILL`` <#var-DISTRO_FEATURES_BACKFILL>`__ and
312```MACHINE_FEATURES_BACKFILL`` <#var-MACHINE_FEATURES_BACKFILL>`__
313variables in the ``meta/conf/bitbake.conf`` file.
314
315Because such features are backfilled by default into all configurations
316as described in the previous paragraph, developers who wish to disable
317the new features need to be able to selectively prevent the backfilling
318from occurring. They can do this by adding the undesired feature or
319features to the
320```DISTRO_FEATURES_BACKFILL_CONSIDERED`` <#var-DISTRO_FEATURES_BACKFILL_CONSIDERED>`__
321or
322```MACHINE_FEATURES_BACKFILL_CONSIDERED`` <#var-MACHINE_FEATURES_BACKFILL_CONSIDERED>`__
323variables for distro features and machine features respectively.
324
325Here are two examples to help illustrate feature backfilling:
326
327- *The "pulseaudio" distro feature option*: Previously, PulseAudio
328 support was enabled within the Qt and GStreamer frameworks. Because
329 of this, the feature is backfilled and thus enabled for all distros
330 through the ``DISTRO_FEATURES_BACKFILL`` variable in the
331 ``meta/conf/bitbake.conf`` file. However, your distro needs to
332 disable the feature. You can disable the feature without affecting
333 other existing distro configurations that need PulseAudio support by
334 adding "pulseaudio" to ``DISTRO_FEATURES_BACKFILL_CONSIDERED`` in
335 your distro's ``.conf`` file. Adding the feature to this variable
336 when it also exists in the ``DISTRO_FEATURES_BACKFILL`` variable
337 prevents the build system from adding the feature to your
338 configuration's ``DISTRO_FEATURES``, effectively disabling the
339 feature for that particular distro.
340
341- *The "rtc" machine feature option*: Previously, real time clock (RTC)
342 support was enabled for all target devices. Because of this, the
343 feature is backfilled and thus enabled for all machines through the
344 ``MACHINE_FEATURES_BACKFILL`` variable in the
345 ``meta/conf/bitbake.conf`` file. However, your target device does not
346 have this capability. You can disable RTC support for your device
347 without affecting other machines that need RTC support by adding the
348 feature to your machine's ``MACHINE_FEATURES_BACKFILL_CONSIDERED``
349 list in the machine's ``.conf`` file. Adding the feature to this
350 variable when it also exists in the ``MACHINE_FEATURES_BACKFILL``
351 variable prevents the build system from adding the feature to your
352 configuration's ``MACHINE_FEATURES``, effectively disabling RTC
353 support for that particular machine.
diff --git a/documentation/ref-manual/ref-images.rst b/documentation/ref-manual/ref-images.rst
new file mode 100644
index 0000000000..62863b640a
--- /dev/null
+++ b/documentation/ref-manual/ref-images.rst
@@ -0,0 +1,137 @@
1******
2Images
3******
4
5The OpenEmbedded build system provides several example images to satisfy
6different needs. When you issue the ``bitbake`` command you provide a
7“top-level” recipe that essentially begins the build for the type of
8image you want.
9
10.. note::
11
12 Building an image without GNU General Public License Version 3
13 (GPLv3), GNU Lesser General Public License Version 3 (LGPLv3), and
14 the GNU Affero General Public License Version 3 (AGPL-3.0) components
15 is only supported for minimal and base images. Furthermore, if you
16 are going to build an image using non-GPLv3 and similarly licensed
17 components, you must make the following changes in the
18 local.conf
19 file before using the BitBake command to build the minimal or base
20 image:
21 ::
22
23 1. Comment out the EXTRA_IMAGE_FEATURES line
24 2. Set INCOMPATIBLE_LICENSE = "GPL-3.0 LGPL-3.0 AGPL-3.0"
25
26
27From within the ``poky`` Git repository, you can use the following
28command to display the list of directories within the `Source
29Directory <#source-directory>`__ that contain image recipe files: $ ls
30meta*/recipes*/images/*.bb
31
32Following is a list of supported recipes:
33
34- ``build-appliance-image``: An example virtual machine that contains
35 all the pieces required to run builds using the build system as well
36 as the build system itself. You can boot and run the image using
37 either the `VMware
38 Player <http://www.vmware.com/products/player/overview.html>`__ or
39 `VMware
40 Workstation <http://www.vmware.com/products/workstation/overview.html>`__.
41 For more information on this image, see the `Build
42 Appliance <&YOCTO_HOME_URL;/software-item/build-appliance/>`__ page
43 on the Yocto Project website.
44
45- ``core-image-base``: A console-only image that fully supports the
46 target device hardware.
47
48- ``core-image-clutter``: An image with support for the Open GL-based
49 toolkit Clutter, which enables development of rich and animated
50 graphical user interfaces.
51
52- ``core-image-full-cmdline``: A console-only image with more
53 full-featured Linux system functionality installed.
54
55- ``core-image-lsb``: An image that conforms to the Linux Standard Base
56 (LSB) specification. This image requires a distribution configuration
57 that enables LSB compliance (e.g. ``poky-lsb``). If you build
58 ``core-image-lsb`` without that configuration, the image will not be
59 LSB-compliant.
60
61- ``core-image-lsb-dev``: A ``core-image-lsb`` image that is suitable
62 for development work using the host. The image includes headers and
63 libraries you can use in a host development environment. This image
64 requires a distribution configuration that enables LSB compliance
65 (e.g. ``poky-lsb``). If you build ``core-image-lsb-dev`` without that
66 configuration, the image will not be LSB-compliant.
67
68- ``core-image-lsb-sdk``: A ``core-image-lsb`` that includes everything
69 in the cross-toolchain but also includes development headers and
70 libraries to form a complete standalone SDK. This image requires a
71 distribution configuration that enables LSB compliance (e.g.
72 ``poky-lsb``). If you build ``core-image-lsb-sdk`` without that
73 configuration, the image will not be LSB-compliant. This image is
74 suitable for development using the target.
75
76- ``core-image-minimal``: A small image just capable of allowing a
77 device to boot.
78
79- ``core-image-minimal-dev``: A ``core-image-minimal`` image suitable
80 for development work using the host. The image includes headers and
81 libraries you can use in a host development environment.
82
83- ``core-image-minimal-initramfs``: A ``core-image-minimal`` image that
84 has the Minimal RAM-based Initial Root Filesystem (initramfs) as part
85 of the kernel, which allows the system to find the first “init”
86 program more efficiently. See the
87 ```PACKAGE_INSTALL`` <#var-PACKAGE_INSTALL>`__ variable for
88 additional information helpful when working with initramfs images.
89
90- ``core-image-minimal-mtdutils``: A ``core-image-minimal`` image that
91 has support for the Minimal MTD Utilities, which let the user
92 interact with the MTD subsystem in the kernel to perform operations
93 on flash devices.
94
95- ``core-image-rt``: A ``core-image-minimal`` image plus a real-time
96 test suite and tools appropriate for real-time use.
97
98- ``core-image-rt-sdk``: A ``core-image-rt`` image that includes
99 everything in the cross-toolchain. The image also includes
100 development headers and libraries to form a complete stand-alone SDK
101 and is suitable for development using the target.
102
103- ``core-image-sato``: An image with Sato support, a mobile environment
104 and visual style that works well with mobile devices. The image
105 supports X11 with a Sato theme and applications such as a terminal,
106 editor, file manager, media player, and so forth.
107
108- ``core-image-sato-dev``: A ``core-image-sato`` image suitable for
109 development using the host. The image includes libraries needed to
110 build applications on the device itself, testing and profiling tools,
111 and debug symbols. This image was formerly ``core-image-sdk``.
112
113- ``core-image-sato-sdk``: A ``core-image-sato`` image that includes
114 everything in the cross-toolchain. The image also includes
115 development headers and libraries to form a complete standalone SDK
116 and is suitable for development using the target.
117
118- ``core-image-testmaster``: A "master" image designed to be used for
119 automated runtime testing. Provides a "known good" image that is
120 deployed to a separate partition so that you can boot into it and use
121 it to deploy a second image to be tested. You can find more
122 information about runtime testing in the "`Performing Automated
123 Runtime
124 Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
125 section in the Yocto Project Development Tasks Manual.
126
127- ``core-image-testmaster-initramfs``: A RAM-based Initial Root
128 Filesystem (initramfs) image tailored for use with the
129 ``core-image-testmaster`` image.
130
131- ``core-image-weston``: A very basic Wayland image with a terminal.
132 This image provides the Wayland protocol libraries and the reference
133 Weston compositor. For more information, see the "`Using Wayland and
134 Weston <&YOCTO_DOCS_DEV_URL;#dev-using-wayland-and-weston>`__"
135 section in the Yocto Project Development Tasks Manual.
136
137- ``core-image-x11``: A very basic X11 image with a terminal.
diff --git a/documentation/ref-manual/ref-kickstart.rst b/documentation/ref-manual/ref-kickstart.rst
new file mode 100644
index 0000000000..c019406c6b
--- /dev/null
+++ b/documentation/ref-manual/ref-kickstart.rst
@@ -0,0 +1,205 @@
1*******************************************
2OpenEmbedded Kickstart (``.wks``) Reference
3*******************************************
4
5.. _openembedded-kickstart-wks-reference:
6
7Introduction
8============
9
10The current Wic implementation supports only the basic kickstart
11partitioning commands: ``partition`` (or ``part`` for short) and
12``bootloader``.
13
14.. note::
15
16 Future updates will implement more commands and options. If you use
17 anything that is not specifically supported, results can be
18 unpredictable.
19
20This chapter provides a reference on the available kickstart commands.
21The information lists the commands, their syntax, and meanings.
22Kickstart commands are based on the Fedora kickstart versions but with
23modifications to reflect Wic capabilities. You can see the original
24documentation for those commands at the following link:
25http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html
26
27Command: part or partition
28==========================
29
30Either of these commands creates a partition on the system and uses the
31following syntax: part [mntpoint] partition [mntpoint] If you do not
32provide mntpoint, Wic creates a partition but does not mount it.
33
34The ``mntpoint`` is where the partition is mounted and must be in one of
35the following forms:
36
37- ``/path``: For example, "/", "/usr", or "/home"
38
39- ``swap``: The created partition is used as swap space
40
41Specifying a mntpoint causes the partition to automatically be mounted.
42Wic achieves this by adding entries to the filesystem table (fstab)
43during image generation. In order for Wic to generate a valid fstab, you
44must also provide one of the ``--ondrive``, ``--ondisk``, or
45``--use-uuid`` partition options as part of the command.
46
47.. note::
48
49 The mount program must understand the PARTUUID syntax you use with
50 --use-uuid
51 and non-root
52 mountpoint
53 , including swap. The busybox versions of these application are
54 currently excluded.
55
56Here is an example that uses "/" as the mountpoint. The command uses
57``--ondisk`` to force the partition onto the ``sdb`` disk: part /
58--source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024
59
60Here is a list that describes other supported options you can use with
61the ``part`` and ``partition`` commands:
62
63- *``--size``:* The minimum partition size in MBytes. Specify an
64 integer value such as 500. Do not append the number with "MB". You do
65 not need this option if you use ``--source``.
66
67- *``--fixed-size``:* The exact partition size in MBytes. You cannot
68 specify with ``--size``. An error occurs when assembling the disk
69 image if the partition data is larger than ``--fixed-size``.
70
71- *``--source``:* This option is a Wic-specific option that names the
72 source of the data that populates the partition. The most common
73 value for this option is "rootfs", but you can use any value that
74 maps to a valid source plugin. For information on the source plugins,
75 see the "`Using the Wic Plugins
76 Interface <&YOCTO_DOCS_DEV_URL;#wic-using-the-wic-plugin-interface>`__"
77 section in the Yocto Project Development Tasks Manual.
78
79 If you use ``--source rootfs``, Wic creates a partition as large as
80 needed and fills it with the contents of the root filesystem pointed
81 to by the ``-r`` command-line option or the equivalent rootfs derived
82 from the ``-e`` command-line option. The filesystem type used to
83 create the partition is driven by the value of the ``--fstype``
84 option specified for the partition. See the entry on ``--fstype``
85 that follows for more information.
86
87 If you use ``--source plugin-name``, Wic creates a partition as large
88 as needed and fills it with the contents of the partition that is
89 generated by the specified plugin name using the data pointed to by
90 the ``-r`` command-line option or the equivalent rootfs derived from
91 the ``-e`` command-line option. Exactly what those contents are and
92 filesystem type used are dependent on the given plugin
93 implementation.
94
95 If you do not use the ``--source`` option, the ``wic`` command
96 creates an empty partition. Consequently, you must use the ``--size``
97 option to specify the size of the empty partition.
98
99- *``--ondisk`` or ``--ondrive``:* Forces the partition to be created
100 on a particular disk.
101
102- *``--fstype``:* Sets the file system type for the partition. Valid
103 values are:
104
105 - ``ext4``
106
107 - ``ext3``
108
109 - ``ext2``
110
111 - ``btrfs``
112
113 - ``squashfs``
114
115 - ``swap``
116
117- *``--fsoptions``:* Specifies a free-form string of options to be used
118 when mounting the filesystem. This string is copied into the
119 ``/etc/fstab`` file of the installed system and should be enclosed in
120 quotes. If not specified, the default string is "defaults".
121
122- *``--label label``:* Specifies the label to give to the filesystem to
123 be made on the partition. If the given label is already in use by
124 another filesystem, a new label is created for the partition.
125
126- *``--active``:* Marks the partition as active.
127
128- *``--align (in KBytes)``:* This option is a Wic-specific option that
129 says to start partitions on boundaries given x KBytes.
130
131- *``--no-table``:* This option is a Wic-specific option. Using the
132 option reserves space for the partition and causes it to become
133 populated. However, the partition is not added to the partition
134 table.
135
136- *``--exclude-path``:* This option is a Wic-specific option that
137 excludes the given relative path from the resulting image. This
138 option is only effective with the rootfs source plugin.
139
140- *``--extra-space``:* This option is a Wic-specific option that adds
141 extra space after the space filled by the content of the partition.
142 The final size can exceed the size specified by the ``--size``
143 option. The default value is 10 Mbytes.
144
145- *``--overhead-factor``:* This option is a Wic-specific option that
146 multiplies the size of the partition by the option's value. You must
147 supply a value greater than or equal to "1". The default value is
148 "1.3".
149
150- *``--part-name``:* This option is a Wic-specific option that
151 specifies a name for GPT partitions.
152
153- *``--part-type``:* This option is a Wic-specific option that
154 specifies the partition type globally unique identifier (GUID) for
155 GPT partitions. You can find the list of partition type GUIDs at
156 ` <http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs>`__.
157
158- *``--use-uuid``:* This option is a Wic-specific option that causes
159 Wic to generate a random GUID for the partition. The generated
160 identifier is used in the bootloader configuration to specify the
161 root partition.
162
163- *``--uuid``:* This option is a Wic-specific option that specifies the
164 partition UUID.
165
166- *``--fsuuid``:* This option is a Wic-specific option that specifies
167 the filesystem UUID. You can generate or modify
168 ```WKS_FILE`` <#var-WKS_FILE>`__ with this option if a preconfigured
169 filesystem UUID is added to the kernel command line in the bootloader
170 configuration before you run Wic.
171
172- *``--system-id``:* This option is a Wic-specific option that
173 specifies the partition system ID, which is a one byte long,
174 hexadecimal parameter with or without the 0x prefix.
175
176- *``--mkfs-extraopts``:* This option specifies additional options to
177 pass to the ``mkfs`` utility. Some default options for certain
178 filesystems do not take effect. See Wic's help on kickstart (i.e.
179 ``wic help kickstart``).
180
181Command: bootloader
182===================
183
184This command specifies how the bootloader should be configured and
185supports the following options:
186
187.. note::
188
189 Bootloader functionality and boot partitions are implemented by the
190 various
191 --source
192 plugins that implement bootloader functionality. The bootloader
193 command essentially provides a means of modifying bootloader
194 configuration.
195
196- *``--timeout``:* Specifies the number of seconds before the
197 bootloader times out and boots the default option.
198
199- *``--append``:* Specifies kernel parameters. These parameters will be
200 added to the syslinux ``APPEND`` or ``grub`` kernel command line.
201
202- *``--configfile``:* Specifies a user-defined configuration file for
203 the bootloader. You can provide a full pathname for the file or a
204 file that exists in the ``canned-wks`` folder. This option overrides
205 all other bootloader options.
diff --git a/documentation/ref-manual/ref-manual.rst b/documentation/ref-manual/ref-manual.rst
new file mode 100644
index 0000000000..a8433f5817
--- /dev/null
+++ b/documentation/ref-manual/ref-manual.rst
@@ -0,0 +1,24 @@
1==============================
2Yocto Project Reference Manual
3==============================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 ref-system-requirements
10 ref-terms
11 ref-release-process
12 migration
13 ref-structure
14 ref-classes
15 ref-tasks
16 ref-devtool-reference
17 ref-kickstart
18 ref-qa-checks
19 ref-images
20 ref-features
21 ref-variables
22 ref-varlocality
23 faq
24 resources
diff --git a/documentation/ref-manual/ref-qa-checks.rst b/documentation/ref-manual/ref-qa-checks.rst
new file mode 100644
index 0000000000..a8b9ef60e4
--- /dev/null
+++ b/documentation/ref-manual/ref-qa-checks.rst
@@ -0,0 +1,524 @@
1*****************************
2QA Error and Warning Messages
3*****************************
4
5.. _qa-introduction:
6
7Introduction
8============
9
10When building a recipe, the OpenEmbedded build system performs various
11QA checks on the output to ensure that common issues are detected and
12reported. Sometimes when you create a new recipe to build new software,
13it will build with no problems. When this is not the case, or when you
14have QA issues building any software, it could take a little time to
15resolve them.
16
17While it is tempting to ignore a QA message or even to disable QA
18checks, it is best to try and resolve any reported QA issues. This
19chapter provides a list of the QA messages and brief explanations of the
20issues you could encounter so that you can properly resolve problems.
21
22The next section provides a list of all QA error and warning messages
23based on a default configuration. Each entry provides the message or
24error form along with an explanation.
25
26.. note::
27
28 - At the end of each message, the name of the associated QA test (as
29 listed in the "```insane.bbclass`` <#ref-classes-insane>`__"
30 section) appears within square brackets.
31
32 - As mentioned, this list of error and warning messages is for QA
33 checks only. The list does not cover all possible build errors or
34 warnings you could encounter.
35
36 - Because some QA checks are disabled by default, this list does not
37 include all possible QA check errors and warnings.
38
39.. _qa-errors-and-warnings:
40
41Errors and Warnings
42===================
43
44- ``<packagename>: <path> is using libexec please relocate to <libexecdir> [libexec]``
45
46 The specified package contains files in ``/usr/libexec`` when the
47 distro configuration uses a different path for ``<libexecdir>`` By
48 default, ``<libexecdir>`` is ``$prefix/libexec``. However, this
49 default can be changed (e.g. ``${libdir}``).
50
51  
52
53- ``package <packagename> contains bad RPATH <rpath> in file <file> [rpaths]``
54
55 The specified binary produced by the recipe contains dynamic library
56 load paths (rpaths) that contain build system paths such as
57 ```TMPDIR`` <#var-TMPDIR>`__, which are incorrect for the target and
58 could potentially be a security issue. Check for bad ``-rpath``
59 options being passed to the linker in your
60 ```do_compile`` <#ref-tasks-compile>`__ log. Depending on the build
61 system used by the software being built, there might be a configure
62 option to disable rpath usage completely within the build of the
63 software.
64
65  
66
67- ``<packagename>: <file> contains probably-redundant RPATH <rpath> [useless-rpaths]``
68
69 The specified binary produced by the recipe contains dynamic library
70 load paths (rpaths) that on a standard system are searched by default
71 by the linker (e.g. ``/lib`` and ``/usr/lib``). While these paths
72 will not cause any breakage, they do waste space and are unnecessary.
73 Depending on the build system used by the software being built, there
74 might be a configure option to disable rpath usage completely within
75 the build of the software.
76
77  
78
79- ``<packagename> requires <files>, but no providers in its RDEPENDS [file-rdeps]``
80
81 A file-level dependency has been identified from the specified
82 package on the specified files, but there is no explicit
83 corresponding entry in ```RDEPENDS`` <#var-RDEPENDS>`__. If
84 particular files are required at runtime then ``RDEPENDS`` should be
85 declared in the recipe to ensure the packages providing them are
86 built.
87
88  
89
90- ``<packagename1> rdepends on <packagename2>, but it isn't a build dependency? [build-deps]``
91
92 A runtime dependency exists between the two specified packages, but
93 there is nothing explicit within the recipe to enable the
94 OpenEmbedded build system to ensure that dependency is satisfied.
95 This condition is usually triggered by an
96 ```RDEPENDS`` <#var-RDEPENDS>`__ value being added at the packaging
97 stage rather than up front, which is usually automatic based on the
98 contents of the package. In most cases, you should change the recipe
99 to add an explicit ``RDEPENDS`` for the dependency.
100
101  
102
103- ``non -dev/-dbg/nativesdk- package contains symlink .so: <packagename> path '<path>' [dev-so]``
104
105 Symlink ``.so`` files are for development only, and should therefore
106 go into the ``-dev`` package. This situation might occur if you add
107 ``*.so*`` rather than ``*.so.*`` to a non-dev package. Change
108 ```FILES`` <#var-FILES>`__ (and possibly
109 ```PACKAGES`` <#var-PACKAGES>`__) such that the specified ``.so``
110 file goes into an appropriate ``-dev`` package.
111
112  
113
114- ``non -staticdev package contains static .a library: <packagename> path '<path>' [staticdev]``
115
116 Static ``.a`` library files should go into a ``-staticdev`` package.
117 Change ```FILES`` <#var-FILES>`__ (and possibly
118 ```PACKAGES`` <#var-PACKAGES>`__) such that the specified ``.a`` file
119 goes into an appropriate ``-staticdev`` package.
120
121  
122
123- ``<packagename>: found library in wrong location [libdir]``
124
125 The specified file may have been installed into an incorrect
126 (possibly hardcoded) installation path. For example, this test will
127 catch recipes that install ``/lib/bar.so`` when ``${base_libdir}`` is
128 "lib32". Another example is when recipes install
129 ``/usr/lib64/foo.so`` when ``${libdir}`` is "/usr/lib". False
130 positives occasionally exist. For these cases add "libdir" to
131 ```INSANE_SKIP`` <#var-INSANE_SKIP>`__ for the package.
132
133  
134
135- ``non debug package contains .debug directory: <packagename> path <path> [debug-files]``
136
137 The specified package contains a ``.debug`` directory, which should
138 not appear in anything but the ``-dbg`` package. This situation might
139 occur if you add a path which contains a ``.debug`` directory and do
140 not explicitly add the ``.debug`` directory to the ``-dbg`` package.
141 If this is the case, add the ``.debug`` directory explicitly to
142 ``FILES_${PN}-dbg``. See ```FILES`` <#var-FILES>`__ for additional
143 information on ``FILES``.
144
145  
146
147- ``Architecture did not match (<machine_arch> to <file_arch>) on <file> [arch]``
148
149 By default, the OpenEmbedded build system checks the Executable and
150 Linkable Format (ELF) type, bit size, and endianness of any binaries
151 to ensure they match the target architecture. This test fails if any
152 binaries do not match the type since there would be an
153 incompatibility. The test could indicate that the wrong compiler or
154 compiler options have been used. Sometimes software, like
155 bootloaders, might need to bypass this check. If the file you receive
156 the error for is firmware that is not intended to be executed within
157 the target operating system or is intended to run on a separate
158 processor within the device, you can add "arch" to
159 ```INSANE_SKIP`` <#var-INSANE_SKIP>`__ for the package. Another
160 option is to check the ```do_compile`` <#ref-tasks-compile>`__ log
161 and verify that the compiler options being used are correct.
162
163  
164
165- ``Bit size did not match (<machine_bits> to <file_bits>) <recipe> on <file> [arch]``
166
167 By default, the OpenEmbedded build system checks the Executable and
168 Linkable Format (ELF) type, bit size, and endianness of any binaries
169 to ensure they match the target architecture. This test fails if any
170 binaries do not match the type since there would be an
171 incompatibility. The test could indicate that the wrong compiler or
172 compiler options have been used. Sometimes software, like
173 bootloaders, might need to bypass this check. If the file you receive
174 the error for is firmware that is not intended to be executed within
175 the target operating system or is intended to run on a separate
176 processor within the device, you can add "arch" to
177 ```INSANE_SKIP`` <#var-INSANE_SKIP>`__ for the package. Another
178 option is to check the ```do_compile`` <#ref-tasks-compile>`__ log
179 and verify that the compiler options being used are correct.
180
181  
182
183- ``Endianness did not match (<machine_endianness> to <file_endianness>) on <file> [arch]``
184
185 By default, the OpenEmbedded build system checks the Executable and
186 Linkable Format (ELF) type, bit size, and endianness of any binaries
187 to ensure they match the target architecture. This test fails if any
188 binaries do not match the type since there would be an
189 incompatibility. The test could indicate that the wrong compiler or
190 compiler options have been used. Sometimes software, like
191 bootloaders, might need to bypass this check. If the file you receive
192 the error for is firmware that is not intended to be executed within
193 the target operating system or is intended to run on a separate
194 processor within the device, you can add "arch" to
195 ```INSANE_SKIP`` <#var-INSANE_SKIP>`__ for the package. Another
196 option is to check the ```do_compile`` <#ref-tasks-compile>`__ log
197 and verify that the compiler options being used are correct.
198
199  
200
201- ``ELF binary '<file>' has relocations in .text [textrel]``
202
203 The specified ELF binary contains relocations in its ``.text``
204 sections. This situation can result in a performance impact at
205 runtime.
206
207 Typically, the way to solve this performance issue is to add "-fPIC"
208 or "-fpic" to the compiler command-line options. For example, given
209 software that reads ```CFLAGS`` <#var-CFLAGS>`__ when you build it,
210 you could add the following to your recipe: CFLAGS_append = " -fPIC "
211
212 For more information on text relocations at runtime, see
213 ` <http://www.akkadia.org/drepper/textrelocs.html>`__.
214
215  
216
217- ``No GNU_HASH in the elf binary: '<file>' [ldflags]``
218
219 This indicates that binaries produced when building the recipe have
220 not been linked with the ```LDFLAGS`` <#var-LDFLAGS>`__ options
221 provided by the build system. Check to be sure that the ``LDFLAGS``
222 variable is being passed to the linker command. A common workaround
223 for this situation is to pass in ``LDFLAGS`` using
224 ```TARGET_CC_ARCH`` <#var-TARGET_CC_ARCH>`__ within the recipe as
225 follows: TARGET_CC_ARCH += "${LDFLAGS}"
226
227  
228
229- ``Package <packagename> contains Xorg driver (<driver>) but no xorg-abi- dependencies [xorg-driver-abi]``
230
231 The specified package contains an Xorg driver, but does not have a
232 corresponding ABI package dependency. The xserver-xorg recipe
233 provides driver ABI names. All drivers should depend on the ABI
234 versions that they have been built against. Driver recipes that
235 include ``xorg-driver-input.inc`` or ``xorg-driver-video.inc`` will
236 automatically get these versions. Consequently, you should only need
237 to explicitly add dependencies to binary driver recipes.
238
239  
240
241- ``The /usr/share/info/dir file is not meant to be shipped in a particular package. [infodir]``
242
243 The ``/usr/share/info/dir`` should not be packaged. Add the following
244 line to your ```do_install`` <#ref-tasks-install>`__ task or to your
245 ``do_install_append`` within the recipe as follows: rm
246 ${D}${infodir}/dir
247
248  
249
250- ``Symlink <path> in <packagename> points to TMPDIR [symlink-to-sysroot]``
251
252 The specified symlink points into ```TMPDIR`` <#var-TMPDIR>`__ on the
253 host. Such symlinks will work on the host. However, they are clearly
254 invalid when running on the target. You should either correct the
255 symlink to use a relative path or remove the symlink.
256
257  
258
259- ``<file> failed sanity test (workdir) in path <path> [la]``
260
261 The specified ``.la`` file contains ```TMPDIR`` <#var-TMPDIR>`__
262 paths. Any ``.la`` file containing these paths is incorrect since
263 ``libtool`` adds the correct sysroot prefix when using the files
264 automatically itself.
265
266  
267
268- ``<file> failed sanity test (tmpdir) in path <path> [pkgconfig]``
269
270 The specified ``.pc`` file contains
271 ```TMPDIR`` <#var-TMPDIR>`__\ ``/``\ ```WORKDIR`` <#var-WORKDIR>`__
272 paths. Any ``.pc`` file containing these paths is incorrect since
273 ``pkg-config`` itself adds the correct sysroot prefix when the files
274 are accessed.
275
276  
277
278- ``<packagename> rdepends on <debug_packagename> [debug-deps]``
279
280 A dependency exists between the specified non-dbg package (i.e. a
281 package whose name does not end in ``-dbg``) and a package that is a
282 ``dbg`` package. The ``dbg`` packages contain debug symbols and are
283 brought in using several different methods:
284
285 - Using the ``dbg-pkgs``
286 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ value.
287
288 - Using ```IMAGE_INSTALL`` <#var-IMAGE_INSTALL>`__.
289
290 - As a dependency of another ``dbg`` package that was brought in
291 using one of the above methods.
292
293 The dependency might have been automatically added because the
294 ``dbg`` package erroneously contains files that it should not contain
295 (e.g. a non-symlink ``.so`` file) or it might have been added
296 manually (e.g. by adding to ```RDEPENDS`` <#var-RDEPENDS>`__).
297
298  
299
300- ``<packagename> rdepends on <dev_packagename> [dev-deps]``
301
302 A dependency exists between the specified non-dev package (a package
303 whose name does not end in ``-dev``) and a package that is a ``dev``
304 package. The ``dev`` packages contain development headers and are
305 usually brought in using several different methods:
306
307 - Using the ``dev-pkgs``
308 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ value.
309
310 - Using ```IMAGE_INSTALL`` <#var-IMAGE_INSTALL>`__.
311
312 - As a dependency of another ``dev`` package that was brought in
313 using one of the above methods.
314
315 The dependency might have been automatically added (because the
316 ``dev`` package erroneously contains files that it should not have
317 (e.g. a non-symlink ``.so`` file) or it might have been added
318 manually (e.g. by adding to ```RDEPENDS`` <#var-RDEPENDS>`__).
319
320  
321
322- ``<var>_<packagename> is invalid: <comparison> (<value>) only comparisons <, =, >, <=, and >= are allowed [dep-cmp]``
323
324 If you are adding a versioned dependency relationship to one of the
325 dependency variables (```RDEPENDS`` <#var-RDEPENDS>`__,
326 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__,
327 ```RSUGGESTS`` <#var-RSUGGESTS>`__,
328 ```RPROVIDES`` <#var-RPROVIDES>`__,
329 ```RREPLACES`` <#var-RREPLACES>`__, or
330 ```RCONFLICTS`` <#var-RCONFLICTS>`__), you must only use the named
331 comparison operators. Change the versioned dependency values you are
332 adding to match those listed in the message.
333
334  
335
336- ``<recipename>: The compile log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [compile-host-path]``
337
338 The log for the ```do_compile`` <#ref-tasks-compile>`__ task
339 indicates that paths on the host were searched for files, which is
340 not appropriate when cross-compiling. Look for "is unsafe for
341 cross-compilation" or "CROSS COMPILE Badness" in the specified log
342 file.
343
344  
345
346- ``<recipename>: The install log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [install-host-path]``
347
348 The log for the ```do_install`` <#ref-tasks-install>`__ task
349 indicates that paths on the host were searched for files, which is
350 not appropriate when cross-compiling. Look for "is unsafe for
351 cross-compilation" or "CROSS COMPILE Badness" in the specified log
352 file.
353
354  
355
356- ``This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. Rerun configure task after fixing this. The path was '<path>'``
357
358 The log for the ```do_configure`` <#ref-tasks-configure>`__ task
359 indicates that paths on the host were searched for files, which is
360 not appropriate when cross-compiling. Look for "is unsafe for
361 cross-compilation" or "CROSS COMPILE Badness" in the specified log
362 file.
363
364  
365
366- ``<packagename> doesn't match the [a-z0-9.+-]+ regex [pkgname]``
367
368 The convention within the OpenEmbedded build system (sometimes
369 enforced by the package manager itself) is to require that package
370 names are all lower case and to allow a restricted set of characters.
371 If your recipe name does not match this, or you add packages to
372 ```PACKAGES`` <#var-PACKAGES>`__ that do not conform to the
373 convention, then you will receive this error. Rename your recipe. Or,
374 if you have added a non-conforming package name to ``PACKAGES``,
375 change the package name appropriately.
376
377  
378
379- ``<recipe>: configure was passed unrecognized options: <options> [unknown-configure-option]``
380
381 The configure script is reporting that the specified options are
382 unrecognized. This situation could be because the options were
383 previously valid but have been removed from the configure script. Or,
384 there was a mistake when the options were added and there is another
385 option that should be used instead. If you are unsure, consult the
386 upstream build documentation, the ``./configure --help`` output, and
387 the upstream change log or release notes. Once you have worked out
388 what the appropriate change is, you can update
389 ```EXTRA_OECONF`` <#var-EXTRA_OECONF>`__,
390 ```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__, or the
391 individual ```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__ option values
392 accordingly.
393
394  
395
396- ``Recipe <recipefile> has PN of "<recipename>" which is in OVERRIDES, this can result in unexpected behavior. [pn-overrides]``
397
398 The specified recipe has a name (```PN`` <#var-PN>`__) value that
399 appears in ```OVERRIDES`` <#var-OVERRIDES>`__. If a recipe is named
400 such that its ``PN`` value matches something already in ``OVERRIDES``
401 (e.g. ``PN`` happens to be the same as ```MACHINE`` <#var-MACHINE>`__
402 or ```DISTRO`` <#var-DISTRO>`__), it can have unexpected
403 consequences. For example, assignments such as
404 ``FILES_${PN} = "xyz"`` effectively turn into ``FILES = "xyz"``.
405 Rename your recipe (or if ``PN`` is being set explicitly, change the
406 ``PN`` value) so that the conflict does not occur. See
407 ```FILES`` <#var-FILES>`__ for additional information.
408
409  
410
411- ``<recipefile>: Variable <variable> is set as not being package specific, please fix this. [pkgvarcheck]``
412
413 Certain variables (```RDEPENDS`` <#var-RDEPENDS>`__,
414 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__,
415 ```RSUGGESTS`` <#var-RSUGGESTS>`__,
416 ```RCONFLICTS`` <#var-RCONFLICTS>`__,
417 ```RPROVIDES`` <#var-RPROVIDES>`__,
418 ```RREPLACES`` <#var-RREPLACES>`__, ```FILES`` <#var-FILES>`__,
419 ``pkg_preinst``, ``pkg_postinst``, ``pkg_prerm``, ``pkg_postrm``, and
420 ```ALLOW_EMPTY`` <#var-ALLOW_EMPTY>`__) should always be set specific
421 to a package (i.e. they should be set with a package name override
422 such as ``RDEPENDS_${PN} = "value"`` rather than
423 ``RDEPENDS = "value"``). If you receive this error, correct any
424 assignments to these variables within your recipe.
425
426  
427
428- ``File '<file>' from <recipename> was already stripped, this will prevent future debugging! [already-stripped]``
429
430 Produced binaries have already been stripped prior to the build
431 system extracting debug symbols. It is common for upstream software
432 projects to default to stripping debug symbols for output binaries.
433 In order for debugging to work on the target using ``-dbg`` packages,
434 this stripping must be disabled.
435
436 Depending on the build system used by the software being built,
437 disabling this stripping could be as easy as specifying an additional
438 configure option. If not, disabling stripping might involve patching
439 the build scripts. In the latter case, look for references to "strip"
440 or "STRIP", or the "-s" or "-S" command-line options being specified
441 on the linker command line (possibly through the compiler command
442 line if preceded with "-Wl,").
443
444 .. note::
445
446 Disabling stripping here does not mean that the final packaged
447 binaries will be unstripped. Once the OpenEmbedded build system
448 splits out debug symbols to the
449 -dbg
450 package, it will then strip the symbols from the binaries.
451
452  
453
454- ``<packagename> is listed in PACKAGES multiple times, this leads to packaging errors. [packages-list]``
455
456 Package names must appear only once in the
457 ```PACKAGES`` <#var-PACKAGES>`__ variable. You might receive this
458 error if you are attempting to add a package to ``PACKAGES`` that is
459 already in the variable's value.
460
461  
462
463- ``FILES variable for package <packagename> contains '//' which is invalid. Attempting to fix this but you should correct the metadata. [files-invalid]``
464
465 The string "//" is invalid in a Unix path. Correct all occurrences
466 where this string appears in a ```FILES`` <#var-FILES>`__ variable so
467 that there is only a single "/".
468
469  
470
471- ``<recipename>: Files/directories were installed but not shipped in any package [installed-vs-shipped]``
472
473 Files have been installed within the
474 ```do_install`` <#ref-tasks-install>`__ task but have not been
475 included in any package by way of the ```FILES`` <#var-FILES>`__
476 variable. Files that do not appear in any package cannot be present
477 in an image later on in the build process. You need to do one of the
478 following:
479
480 - Add the files to ``FILES`` for the package you want them to appear
481 in (e.g. ``FILES_${``\ ```PN`` <#var-PN>`__\ ``}`` for the main
482 package).
483
484 - Delete the files at the end of the ``do_install`` task if the
485 files are not needed in any package.
486
487  
488
489- ``<oldpackage>-<oldpkgversion> was registered as shlib provider for <library>, changing it to <newpackage>-<newpkgversion> because it was built later``
490
491 This message means that both ``<oldpackage>`` and ``<newpackage>``
492 provide the specified shared library. You can expect this message
493 when a recipe has been renamed. However, if that is not the case, the
494 message might indicate that a private version of a library is being
495 erroneously picked up as the provider for a common library. If that
496 is the case, you should add the library's ``.so`` file name to
497 ```PRIVATE_LIBS`` <#var-PRIVATE_LIBS>`__ in the recipe that provides
498 the private version of the library.
499
500- ``LICENSE_<packagename> includes licenses (<licenses>) that are not listed in LICENSE [unlisted-pkg-lics]``
501
502 The ```LICENSE`` <#var-LICENSE>`__ of the recipe should be a superset
503 of all the licenses of all packages produced by this recipe. In other
504 words, any license in ``LICENSE_*`` should also appear in
505 ```LICENSE`` <#var-LICENSE>`__.
506
507  
508
509Configuring and Disabling QA Checks
510===================================
511
512You can configure the QA checks globally so that specific check failures
513either raise a warning or an error message, using the
514```WARN_QA`` <#var-WARN_QA>`__ and ```ERROR_QA`` <#var-ERROR_QA>`__
515variables, respectively. You can also disable checks within a particular
516recipe using ```INSANE_SKIP`` <#var-INSANE_SKIP>`__. For information on
517how to work with the QA checks, see the
518"```insane.bbclass`` <#ref-classes-insane>`__" section.
519
520.. note::
521
522 Please keep in mind that the QA checks exist in order to detect real
523 or potential problems in the packaged output. So exercise caution
524 when disabling these checks.
diff --git a/documentation/ref-manual/ref-release-process.rst b/documentation/ref-manual/ref-release-process.rst
new file mode 100644
index 0000000000..1c97500d2b
--- /dev/null
+++ b/documentation/ref-manual/ref-release-process.rst
@@ -0,0 +1,182 @@
1*****************************************************
2Yocto Project Releases and the Stable Release Process
3*****************************************************
4
5The Yocto Project release process is predictable and consists of both
6major and minor (point) releases. This brief chapter provides
7information on how releases are named, their life cycle, and their
8stability.
9
10Major and Minor Release Cadence
11===============================
12
13The Yocto Project delivers major releases (e.g. DISTRO) using a six
14month cadence roughly timed each April and October of the year.
15Following are examples of some major YP releases with their codenames
16also shown. See the "`Major Release
17Codenames <#major-release-codenames>`__" section for information on
18codenames used with major releases. 2.2 (Morty) 2.1 (Krogoth) 2.0
19(Jethro) While the cadence is never perfect, this timescale facilitates
20regular releases that have strong QA cycles while not overwhelming users
21with too many new releases. The cadence is predictable and avoids many
22major holidays in various geographies.
23
24The Yocto project delivers minor (point) releases on an unscheduled
25basis and are usually driven by the accumulation of enough significant
26fixes or enhancements to the associated major release. Following are
27some example past point releases: 2.1.1 2.1.2 2.2.1 The point release
28indicates a point in the major release branch where a full QA cycle and
29release process validates the content of the new branch.
30
31.. note::
32
33 Realize that there can be patches merged onto the stable release
34 branches as and when they become available.
35
36Major Release Codenames
37=======================
38
39Each major release receives a codename that identifies the release in
40the `Yocto Project Source
41Repositories <&YOCTO_DOCS_OM_URL;#yocto-project-repositories>`__. The
42concept is that branches of `Metadata <#metadata>`__ with the same
43codename are likely to be compatible and thus work together.
44
45.. note::
46
47 Codenames are associated with major releases because a Yocto Project
48 release number (e.g. DISTRO) could conflict with a given layer or
49 company versioning scheme. Codenames are unique, interesting, and
50 easily identifiable.
51
52Releases are given a nominal release version as well but the codename is
53used in repositories for this reason. You can find information on Yocto
54Project releases and codenames at
55` <https://wiki.yoctoproject.org/wiki/Releases>`__.
56
57Stable Release Process
58======================
59
60Once released, the release enters the stable release process at which
61time a person is assigned as the maintainer for that stable release.
62This maintainer monitors activity for the release by investigating and
63handling nominated patches and backport activity. Only fixes and
64enhancements that have first been applied on the "master" branch (i.e.
65the current, in-development branch) are considered for backporting to a
66stable release.
67
68.. note::
69
70 The current Yocto Project policy regarding backporting is to consider
71 bug fixes and security fixes only. Policy dictates that features are
72 not backported to a stable release. This policy means generic recipe
73 version upgrades are unlikely to be accepted for backporting. The
74 exception to this policy occurs when a strong reason exists such as
75 the fix happens to also be the preferred upstream approach.
76
77Stable release branches have strong maintenance for about a year after
78their initial release. Should significant issues be found for any
79release regardless of its age, fixes could be backported to older
80releases. For issues that are not backported given an older release,
81Community LTS trees and branches exist where community members share
82patches for older releases. However, these types of patches do not go
83through the same release process as do point releases. You can find more
84information about stable branch maintenance at
85` <https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance>`__.
86
87Testing and Quality Assurance
88=============================
89
90Part of the Yocto Project development and release process is quality
91assurance through the execution of test strategies. Test strategies
92provide the Yocto Project team a way to ensure a release is validated.
93Additionally, because the test strategies are visible to you as a
94developer, you can validate your projects. This section overviews the
95available test infrastructure used in the Yocto Project. For information
96on how to run available tests on your projects, see the "`Performing
97Automated Runtime
98Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
99section in the Yocto Project Development Tasks Manual.
100
101The QA/testing infrastructure is woven into the project to the point
102where core developers take some of it for granted. The infrastructure
103consists of the following pieces:
104
105- ``bitbake-selftest``: A standalone command that runs unit tests on
106 key pieces of BitBake and its fetchers.
107
108- ```sanity.bbclass`` <#ref-classes-sanity>`__: This automatically
109 included class checks the build environment for missing tools (e.g.
110 ``gcc``) or common misconfigurations such as
111 ```MACHINE`` <#var-MACHINE>`__ set incorrectly.
112
113- ```insane.bbclass`` <#ref-classes-insane>`__: This class checks the
114 generated output from builds for sanity. For example, if building for
115 an ARM target, did the build produce ARM binaries. If, for example,
116 the build produced PPC binaries then there is a problem.
117
118- ```testimage.bbclass`` <#ref-classes-testimage*>`__: This class
119 performs runtime testing of images after they are built. The tests
120 are usually used with `QEMU <&YOCTO_DOCS_DEV_URL;#dev-manual-qemu>`__
121 to boot the images and check the combined runtime result boot
122 operation and functions. However, the test can also use the IP
123 address of a machine to test.
124
125- ```ptest`` <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__:
126 Runs tests against packages produced during the build for a given
127 piece of software. The test allows the packages to be be run within a
128 target image.
129
130- ``oe-selftest``: Tests combination BitBake invocations. These tests
131 operate outside the OpenEmbedded build system itself. The
132 ``oe-selftest`` can run all tests by default or can run selected
133 tests or test suites.
134
135 .. note::
136
137 Running
138 oe-selftest
139 requires host packages beyond the "Essential" grouping. See the "
140 Required Packages for the Build Host
141 " section for more information.
142
143Originally, much of this testing was done manually. However, significant
144effort has been made to automate the tests so that more people can use
145them and the Yocto Project development team can run them faster and more
146efficiently.
147
148The Yocto Project's main Autobuilder (``autobuilder.yoctoproject.org``)
149publicly tests each Yocto Project release's code in the
150`OE-Core <#oe-core>`__, Poky, and BitBake repositories. The testing
151occurs for both the current state of the "master" branch and also for
152submitted patches. Testing for submitted patches usually occurs in the
153"ross/mut" branch in the ``poky-contrib`` repository (i.e. the
154master-under-test branch) or in the "master-next" branch in the ``poky``
155repository.
156
157.. note::
158
159 You can find all these branches in the Yocto Project
160 Source Repositories
161 .
162
163Testing within these public branches ensures in a publicly visible way
164that all of the main supposed architectures and recipes in OE-Core
165successfully build and behave properly.
166
167Various features such as ``multilib``, sub architectures (e.g. ``x32``,
168``poky-tiny``, ``musl``, ``no-x11`` and and so forth),
169``bitbake-selftest``, and ``oe-selftest`` are tested as part of the QA
170process of a release. Complete testing and validation for a release
171takes the Autobuilder workers several hours.
172
173.. note::
174
175 The Autobuilder workers are non-homogeneous, which means regular
176 testing across a variety of Linux distributions occurs. The
177 Autobuilder is limited to only testing QEMU-based setups and not real
178 hardware.
179
180Finally, in addition to the Autobuilder's tests, the Yocto Project QA
181team also performs testing on a variety of platforms, which includes
182actual hardware, to ensure expected results.
diff --git a/documentation/ref-manual/ref-structure.rst b/documentation/ref-manual/ref-structure.rst
new file mode 100644
index 0000000000..59d8c0d6c4
--- /dev/null
+++ b/documentation/ref-manual/ref-structure.rst
@@ -0,0 +1,871 @@
1**************************
2Source Directory Structure
3**************************
4
5The `Source Directory <#source-directory>`__ consists of numerous files,
6directories and subdirectories; understanding their locations and
7contents is key to using the Yocto Project effectively. This chapter
8describes the Source Directory and gives information about those files
9and directories.
10
11For information on how to establish a local Source Directory on your
12development system, see the "`Locating Yocto Project Source
13Files <&YOCTO_DOCS_DEV_URL;#locating-yocto-project-source-files>`__"
14section in the Yocto Project Development Tasks Manual.
15
16.. note::
17
18 The OpenEmbedded build system does not support file or directory
19 names that contain spaces. Be sure that the Source Directory you use
20 does not contain these types of names.
21
22.. _structure-core:
23
24Top-Level Core Components
25=========================
26
27This section describes the top-level components of the `Source
28Directory <#source-directory>`__.
29
30.. _structure-core-bitbake:
31
32``bitbake/``
33------------
34
35This directory includes a copy of BitBake for ease of use. The copy
36usually matches the current stable BitBake release from the BitBake
37project. BitBake, a `Metadata <#metadata>`__ interpreter, reads the
38Yocto Project Metadata and runs the tasks defined by that data. Failures
39are usually caused by errors in your Metadata and not from BitBake
40itself; consequently, most users do not need to worry about BitBake.
41
42When you run the ``bitbake`` command, the main BitBake executable (which
43resides in the ``bitbake/bin/`` directory) starts. Sourcing the
44environment setup script (i.e. ````` <#structure-core-script>`__) places
45the ``scripts/`` and ``bitbake/bin/`` directories (in that order) into
46the shell's ``PATH`` environment variable.
47
48For more information on BitBake, see the `BitBake User
49Manual <&YOCTO_DOCS_BB_URL;>`__.
50
51.. _structure-core-build:
52
53``build/``
54----------
55
56This directory contains user configuration files and the output
57generated by the OpenEmbedded build system in its standard configuration
58where the source tree is combined with the output. The `Build
59Directory <#build-directory>`__ is created initially when you ``source``
60the OpenEmbedded build environment setup script (i.e.
61````` <#structure-core-script>`__).
62
63It is also possible to place output and configuration files in a
64directory separate from the `Source Directory <#source-directory>`__ by
65providing a directory name when you ``source`` the setup script. For
66information on separating output from your local Source Directory files
67(commonly described as an "out of tree" build), see the
68"````` <#structure-core-script>`__" section.
69
70.. _handbook:
71
72``documentation/``
73------------------
74
75This directory holds the source for the Yocto Project documentation as
76well as templates and tools that allow you to generate PDF and HTML
77versions of the manuals. Each manual is contained in its own sub-folder;
78for example, the files for this reference manual reside in the
79``ref-manual/`` directory.
80
81.. _structure-core-meta:
82
83``meta/``
84---------
85
86This directory contains the minimal, underlying OpenEmbedded-Core
87metadata. The directory holds recipes, common classes, and machine
88configuration for strictly emulated targets (``qemux86``, ``qemuarm``,
89and so forth.)
90
91.. _structure-core-meta-poky:
92
93``meta-poky/``
94--------------
95
96Designed above the ``meta/`` content, this directory adds just enough
97metadata to define the Poky reference distribution.
98
99.. _structure-core-meta-yocto-bsp:
100
101``meta-yocto-bsp/``
102-------------------
103
104This directory contains the Yocto Project reference hardware Board
105Support Packages (BSPs). For more information on BSPs, see the `Yocto
106Project Board Support Package (BSP) Developer's
107Guide <&YOCTO_DOCS_BSP_URL;>`__.
108
109.. _structure-meta-selftest:
110
111``meta-selftest/``
112------------------
113
114This directory adds additional recipes and append files used by the
115OpenEmbedded selftests to verify the behavior of the build system. You
116do not have to add this layer to your ``bblayers.conf`` file unless you
117want to run the selftests.
118
119.. _structure-meta-skeleton:
120
121``meta-skeleton/``
122------------------
123
124This directory contains template recipes for BSP and kernel development.
125
126.. _structure-core-scripts:
127
128``scripts/``
129------------
130
131This directory contains various integration scripts that implement extra
132functionality in the Yocto Project environment (e.g. QEMU scripts). The
133````` <#structure-core-script>`__ script prepends this directory to the
134shell's ``PATH`` environment variable.
135
136The ``scripts`` directory has useful scripts that assist in contributing
137back to the Yocto Project, such as ``create-pull-request`` and
138``send-pull-request``.
139
140.. _structure-core-script:
141
142````
143----
144
145This script sets up the OpenEmbedded build environment. Running this
146script with the ``source`` command in a shell makes changes to ``PATH``
147and sets other core BitBake variables based on the current working
148directory. You need to run an environment setup script before running
149BitBake commands. The script uses other scripts within the ``scripts``
150directory to do the bulk of the work.
151
152When you run this script, your Yocto Project environment is set up, a
153`Build Directory <#build-directory>`__ is created, your working
154directory becomes the Build Directory, and you are presented with some
155simple suggestions as to what to do next, including a list of some
156possible targets to build. Here is an example: $ source
157oe-init-build-env ### Shell environment set up for builds. ### You can
158now run 'bitbake <target>' Common targets are: core-image-minimal
159core-image-sato meta-toolchain meta-ide-support You can also run
160generated qemu images with a command like 'runqemu qemux86-64' The
161default output of the ``oe-init-build-env`` script is from the
162``conf-notes.txt`` file, which is found in the ``meta-poky`` directory
163within the `Source Directory <#source-directory>`__. If you design a
164custom distribution, you can include your own version of this
165configuration file to mention the targets defined by your distribution.
166See the "`Creating a Custom Template Configuration
167Directory <&YOCTO_DOCS_DEV_URL;#creating-a-custom-template-configuration-directory>`__"
168section in the Yocto Project Development Tasks Manual for more
169information.
170
171By default, running this script without a Build Directory argument
172creates the ``build/`` directory in your current working directory. If
173you provide a Build Directory argument when you ``source`` the script,
174you direct the OpenEmbedded build system to create a Build Directory of
175your choice. For example, the following command creates a Build
176Directory named ``mybuilds/`` that is outside of the `Source
177Directory <#source-directory>`__: $ source OE_INIT_FILE ~/mybuilds The
178OpenEmbedded build system uses the template configuration files, which
179are found by default in the ``meta-poky/conf/`` directory in the Source
180Directory. See the "`Creating a Custom Template Configuration
181Directory <&YOCTO_DOCS_DEV_URL;#creating-a-custom-template-configuration-directory>`__"
182section in the Yocto Project Development Tasks Manual for more
183information.
184
185.. note::
186
187 The OpenEmbedded build system does not support file or directory
188 names that contain spaces. If you attempt to run the
189 OE_INIT_FILE
190 script from a Source Directory that contains spaces in either the
191 filenames or directory names, the script returns an error indicating
192 no such file or directory. Be sure to use a Source Directory free of
193 names containing spaces.
194
195.. _structure-basic-top-level:
196
197``LICENSE, README, and README.hardware``
198----------------------------------------
199
200These files are standard top-level files.
201
202.. _structure-build:
203
204The Build Directory - ``build/``
205================================
206
207The OpenEmbedded build system creates the `Build
208Directory <#build-directory>`__ when you run the build environment setup
209script ````` <#structure-core-script>`__. If you do not give the Build
210Directory a specific name when you run the setup script, the name
211defaults to ``build/``.
212
213For subsequent parsing and processing, the name of the Build directory
214is available via the ```TOPDIR`` <#var-TOPDIR>`__ variable.
215
216.. _structure-build-buildhistory:
217
218``build/buildhistory/``
219-----------------------
220
221The OpenEmbedded build system creates this directory when you enable
222build history via the ``buildhistory`` class file. The directory
223organizes build information into image, packages, and SDK
224subdirectories. For information on the build history feature, see the
225"`Maintaining Build Output
226Quality <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__"
227section in the Yocto Project Development Tasks Manual.
228
229.. _structure-build-conf-local.conf:
230
231``build/conf/local.conf``
232-------------------------
233
234This configuration file contains all the local user configurations for
235your build environment. The ``local.conf`` file contains documentation
236on the various configuration options. Any variable set here overrides
237any variable set elsewhere within the environment unless that variable
238is hard-coded within a file (e.g. by using '=' instead of '?='). Some
239variables are hard-coded for various reasons but such variables are
240relatively rare.
241
242At a minimum, you would normally edit this file to select the target
243``MACHINE``, which package types you wish to use
244(```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__), and the location from
245which you want to access downloaded files (``DL_DIR``).
246
247If ``local.conf`` is not present when you start the build, the
248OpenEmbedded build system creates it from ``local.conf.sample`` when you
249``source`` the top-level build environment setup script
250````` <#structure-core-script>`__.
251
252The source ``local.conf.sample`` file used depends on the
253``$TEMPLATECONF`` script variable, which defaults to ``meta-poky/conf/``
254when you are building from the Yocto Project development environment,
255and to ``meta/conf/`` when you are building from the OpenEmbedded-Core
256environment. Because the script variable points to the source of the
257``local.conf.sample`` file, this implies that you can configure your
258build environment from any layer by setting the variable in the
259top-level build environment setup script as follows:
260TEMPLATECONF=your_layer/conf Once the build process gets the sample
261file, it uses ``sed`` to substitute final
262``${``\ ```OEROOT`` <#var-OEROOT>`__\ ``}`` values for all
263``##OEROOT##`` values.
264
265.. note::
266
267 You can see how the
268 TEMPLATECONF
269 variable is used by looking at the
270 scripts/oe-setup-builddir
271 script in the
272 Source Directory
273 . You can find the Yocto Project version of the
274 local.conf.sample
275 file in the
276 meta-poky/conf
277 directory.
278
279.. _structure-build-conf-bblayers.conf:
280
281``build/conf/bblayers.conf``
282----------------------------
283
284This configuration file defines
285`layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__,
286which are directory trees, traversed (or walked) by BitBake. The
287``bblayers.conf`` file uses the ```BBLAYERS`` <#var-BBLAYERS>`__
288variable to list the layers BitBake tries to find.
289
290If ``bblayers.conf`` is not present when you start the build, the
291OpenEmbedded build system creates it from ``bblayers.conf.sample`` when
292you ``source`` the top-level build environment setup script (i.e.
293````` <#structure-core-script>`__).
294
295As with the ``local.conf`` file, the source ``bblayers.conf.sample``
296file used depends on the ``$TEMPLATECONF`` script variable, which
297defaults to ``meta-poky/conf/`` when you are building from the Yocto
298Project development environment, and to ``meta/conf/`` when you are
299building from the OpenEmbedded-Core environment. Because the script
300variable points to the source of the ``bblayers.conf.sample`` file, this
301implies that you can base your build from any layer by setting the
302variable in the top-level build environment setup script as follows:
303TEMPLATECONF=your_layer/conf Once the build process gets the sample
304file, it uses ``sed`` to substitute final
305``${``\ ```OEROOT`` <#var-OEROOT>`__\ ``}`` values for all
306``##OEROOT##`` values.
307
308.. note::
309
310 You can see how the
311 TEMPLATECONF
312 variable
313 scripts/oe-setup-builddir
314 script in the
315 Source Directory
316 . You can find the Yocto Project version of the
317 bblayers.conf.sample
318 file in the
319 meta-poky/conf/
320 directory.
321
322.. _structure-build-conf-sanity_info:
323
324``build/cache/sanity_info``
325---------------------------
326
327This file indicates the state of the sanity checks and is created during
328the build.
329
330.. _structure-build-downloads:
331
332``build/downloads/``
333--------------------
334
335This directory contains downloaded upstream source tarballs. You can
336reuse the directory for multiple builds or move the directory to another
337location. You can control the location of this directory through the
338``DL_DIR`` variable.
339
340.. _structure-build-sstate-cache:
341
342``build/sstate-cache/``
343-----------------------
344
345This directory contains the shared state cache. You can reuse the
346directory for multiple builds or move the directory to another location.
347You can control the location of this directory through the
348``SSTATE_DIR`` variable.
349
350.. _structure-build-tmp:
351
352``build/tmp/``
353--------------
354
355The OpenEmbedded build system creates and uses this directory for all
356the build system's output. The ```TMPDIR`` <#var-TMPDIR>`__ variable
357points to this directory.
358
359BitBake creates this directory if it does not exist. As a last resort,
360to clean up a build and start it from scratch (other than the
361downloads), you can remove everything in the ``tmp`` directory or get
362rid of the directory completely. If you do, you should also completely
363remove the ``build/sstate-cache`` directory.
364
365.. _structure-build-tmp-buildstats:
366
367``build/tmp/buildstats/``
368-------------------------
369
370This directory stores the build statistics.
371
372.. _structure-build-tmp-cache:
373
374``build/tmp/cache/``
375--------------------
376
377When BitBake parses the metadata (recipes and configuration files), it
378caches the results in ``build/tmp/cache/`` to speed up future builds.
379The results are stored on a per-machine basis.
380
381During subsequent builds, BitBake checks each recipe (together with, for
382example, any files included or appended to it) to see if they have been
383modified. Changes can be detected, for example, through file
384modification time (mtime) changes and hashing of file contents. If no
385changes to the file are detected, then the parsed result stored in the
386cache is reused. If the file has changed, it is reparsed.
387
388.. _structure-build-tmp-deploy:
389
390``build/tmp/deploy/``
391---------------------
392
393This directory contains any "end result" output from the OpenEmbedded
394build process. The ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__ variable points
395to this directory. For more detail on the contents of the ``deploy``
396directory, see the
397"`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__" and
398"`Application Development
399SDK <&YOCTO_DOCS_OM_URL;#sdk-dev-environment>`__" sections in the Yocto
400Project Overview and Concepts Manual.
401
402.. _structure-build-tmp-deploy-deb:
403
404``build/tmp/deploy/deb/``
405-------------------------
406
407This directory receives any ``.deb`` packages produced by the build
408process. The packages are sorted into feeds for different architecture
409types.
410
411.. _structure-build-tmp-deploy-rpm:
412
413``build/tmp/deploy/rpm/``
414-------------------------
415
416This directory receives any ``.rpm`` packages produced by the build
417process. The packages are sorted into feeds for different architecture
418types.
419
420.. _structure-build-tmp-deploy-ipk:
421
422``build/tmp/deploy/ipk/``
423-------------------------
424
425This directory receives ``.ipk`` packages produced by the build process.
426
427.. _structure-build-tmp-deploy-licenses:
428
429``build/tmp/deploy/licenses/``
430------------------------------
431
432This directory receives package licensing information. For example, the
433directory contains sub-directories for ``bash``, ``busybox``, and
434``glibc`` (among others) that in turn contain appropriate ``COPYING``
435license files with other licensing information. For information on
436licensing, see the "`Maintaining Open Source License Compliance During
437Your Product's
438Lifecycle <&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle>`__"
439section in the Yocto Project Development Tasks Manual.
440
441.. _structure-build-tmp-deploy-images:
442
443``build/tmp/deploy/images/``
444----------------------------
445
446This directory is populated with the basic output objects of the build
447(think of them as the "generated artifacts" of the build process),
448including things like the boot loader image, kernel, root filesystem and
449more. If you want to flash the resulting image from a build onto a
450device, look here for the necessary components.
451
452Be careful when deleting files in this directory. You can safely delete
453old images from this directory (e.g. ``core-image-*``). However, the
454kernel (``*zImage*``, ``*uImage*``, etc.), bootloader and other
455supplementary files might be deployed here prior to building an image.
456Because these files are not directly produced from the image, if you
457delete them they will not be automatically re-created when you build the
458image again.
459
460If you do accidentally delete files here, you will need to force them to
461be re-created. In order to do that, you will need to know the target
462that produced them. For example, these commands rebuild and re-create
463the kernel files: $ bitbake -c clean virtual/kernel $ bitbake
464virtual/kernel
465
466.. _structure-build-tmp-deploy-sdk:
467
468``build/tmp/deploy/sdk/``
469-------------------------
470
471The OpenEmbedded build system creates this directory to hold toolchain
472installer scripts which, when executed, install the sysroot that matches
473your target hardware. You can find out more about these installers in
474the "`Building an SDK
475Installer <&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer>`__"
476section in the Yocto Project Application Development and the Extensible
477Software Development Kit (eSDK) manual.
478
479.. _structure-build-tmp-sstate-control:
480
481``build/tmp/sstate-control/``
482-----------------------------
483
484The OpenEmbedded build system uses this directory for the shared state
485manifest files. The shared state code uses these files to record the
486files installed by each sstate task so that the files can be removed
487when cleaning the recipe or when a newer version is about to be
488installed. The build system also uses the manifests to detect and
489produce a warning when files from one task are overwriting those from
490another.
491
492.. _structure-build-tmp-sysroots-components:
493
494``build/tmp/sysroots-components/``
495----------------------------------
496
497This directory is the location of the sysroot contents that the task
498```do_prepare_recipe_sysroot`` <#ref-tasks-prepare_recipe_sysroot>`__
499links or copies into the recipe-specific sysroot for each recipe listed
500in ```DEPENDS`` <#var-DEPENDS>`__. Population of this directory is
501handled through shared state, while the path is specified by the
502```COMPONENTS_DIR`` <#var-COMPONENTS_DIR>`__ variable. Apart from a few
503unusual circumstances, handling of the ``sysroots-components`` directory
504should be automatic, and recipes should not directly reference
505``build/tmp/sysroots-components``.
506
507.. _structure-build-tmp-sysroots:
508
509``build/tmp/sysroots/``
510-----------------------
511
512Previous versions of the OpenEmbedded build system used to create a
513global shared sysroot per machine along with a native sysroot. Beginning
514with the DISTRO version of the Yocto Project, sysroots exist in
515recipe-specific ```WORKDIR`` <#var-WORKDIR>`__ directories. Thus, the
516``build/tmp/sysroots/`` directory is unused.
517
518.. note::
519
520 The
521 build/tmp/sysroots/
522 directory can still be populated using the
523 bitbake build-sysroots
524 command and can be used for compatibility in some cases. However, in
525 general it is not recommended to populate this directory. Individual
526 recipe-specific sysroots should be used.
527
528.. _structure-build-tmp-stamps:
529
530``build/tmp/stamps/``
531---------------------
532
533This directory holds information that BitBake uses for accounting
534purposes to track what tasks have run and when they have run. The
535directory is sub-divided by architecture, package name, and version.
536Following is an example:
537stamps/all-poky-linux/distcc-config/1.0-r0.do_build-2fdd....2do Although
538the files in the directory are empty of data, BitBake uses the filenames
539and timestamps for tracking purposes.
540
541For information on how BitBake uses stamp files to determine if a task
542should be rerun, see the "`Stamp Files and the Rerunning of
543Tasks <&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks>`__"
544section in the Yocto Project Overview and Concepts Manual.
545
546.. _structure-build-tmp-log:
547
548``build/tmp/log/``
549------------------
550
551This directory contains general logs that are not otherwise placed using
552the package's ``WORKDIR``. Examples of logs are the output from the
553``do_check_pkg`` or ``do_distro_check`` tasks. Running a build does not
554necessarily mean this directory is created.
555
556.. _structure-build-tmp-work:
557
558``build/tmp/work/``
559-------------------
560
561This directory contains architecture-specific work sub-directories for
562packages built by BitBake. All tasks execute from the appropriate work
563directory. For example, the source for a particular package is unpacked,
564patched, configured and compiled all within its own work directory.
565Within the work directory, organization is based on the package group
566and version for which the source is being compiled as defined by the
567```WORKDIR`` <#var-WORKDIR>`__.
568
569It is worth considering the structure of a typical work directory. As an
570example, consider ``linux-yocto-kernel-3.0`` on the machine ``qemux86``
571built within the Yocto Project. For this package, a work directory of
572``tmp/work/qemux86-poky-linux/linux-yocto/3.0+git1+<.....>``, referred
573to as the ``WORKDIR``, is created. Within this directory, the source is
574unpacked to ``linux-qemux86-standard-build`` and then patched by Quilt.
575(See the "`Using Quilt in Your
576Workflow <&YOCTO_DOCS_DEV_URL;#using-a-quilt-workflow>`__" section in
577the Yocto Project Development Tasks Manual for more information.) Within
578the ``linux-qemux86-standard-build`` directory, standard Quilt
579directories ``linux-3.0/patches`` and ``linux-3.0/.pc`` are created, and
580standard Quilt commands can be used.
581
582There are other directories generated within ``WORKDIR``. The most
583important directory is ``WORKDIR/temp/``, which has log files for each
584task (``log.do_*.pid``) and contains the scripts BitBake runs for each
585task (``run.do_*.pid``). The ``WORKDIR/image/`` directory is where "make
586install" places its output that is then split into sub-packages within
587``WORKDIR/packages-split/``.
588
589.. _structure-build-tmp-work-tunearch-recipename-version:
590
591``build/tmp/work/tunearch/recipename/version/``
592-----------------------------------------------
593
594The recipe work directory - ``${WORKDIR}``.
595
596As described earlier in the
597"```build/tmp/sysroots/`` <#structure-build-tmp-sysroots>`__" section,
598beginning with the DISTRO release of the Yocto Project, the OpenEmbedded
599build system builds each recipe in its own work directory (i.e.
600```WORKDIR`` <#var-WORKDIR>`__). The path to the work directory is
601constructed using the architecture of the given build (e.g.
602```TUNE_PKGARCH`` <#var-TUNE_PKGARCH>`__,
603```MACHINE_ARCH`` <#var-MACHINE_ARCH>`__, or "allarch"), the recipe
604name, and the version of the recipe (i.e.
605```PE`` <#var-PE>`__\ ``:``\ ```PV`` <#var-PV>`__\ ``-``\ ```PR`` <#var-PR>`__).
606
607A number of key subdirectories exist within each recipe work directory:
608
609- ``${WORKDIR}/temp``: Contains the log files of each task executed for
610 this recipe, the "run" files for each executed task, which contain
611 the code run, and a ``log.task_order`` file, which lists the order in
612 which tasks were executed.
613
614- ``${WORKDIR}/image``: Contains the output of the
615 ```do_install`` <#ref-tasks-install>`__ task, which corresponds to
616 the ``${``\ ```D`` <#var-D>`__\ ``}`` variable in that task.
617
618- ``${WORKDIR}/pseudo``: Contains the pseudo database and log for any
619 tasks executed under pseudo for the recipe.
620
621- ``${WORKDIR}/sysroot-destdir``: Contains the output of the
622 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task.
623
624- ``${WORKDIR}/package``: Contains the output of the
625 ```do_package`` <#ref-tasks-package>`__ task before the output is
626 split into individual packages.
627
628- ``${WORKDIR}/packages-split``: Contains the output of the
629 ``do_package`` task after the output has been split into individual
630 packages. Subdirectories exist for each individual package created by
631 the recipe.
632
633- ``${WORKDIR}/recipe-sysroot``: A directory populated with the target
634 dependencies of the recipe. This directory looks like the target
635 filesystem and contains libraries that the recipe might need to link
636 against (e.g. the C library).
637
638- ``${WORKDIR}/recipe-sysroot-native``: A directory populated with the
639 native dependencies of the recipe. This directory contains the tools
640 the recipe needs to build (e.g. the compiler, Autoconf, libtool, and
641 so forth).
642
643- ``${WORKDIR}/build``: This subdirectory applies only to recipes that
644 support builds where the source is separate from the build artifacts.
645 The OpenEmbedded build system uses this directory as a separate build
646 directory (i.e. ``${``\ ```B`` <#var-B>`__\ ``}``).
647
648.. _structure-build-work-shared:
649
650``build/tmp/work-shared/``
651--------------------------
652
653For efficiency, the OpenEmbedded build system creates and uses this
654directory to hold recipes that share a work directory with other
655recipes. In practice, this is only used for ``gcc`` and its variants
656(e.g. ``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth).
657
658.. _structure-meta:
659
660The Metadata - ``meta/``
661========================
662
663As mentioned previously, `Metadata <#metadata>`__ is the core of the
664Yocto Project. Metadata has several important subdivisions:
665
666.. _structure-meta-classes:
667
668``meta/classes/``
669-----------------
670
671This directory contains the ``*.bbclass`` files. Class files are used to
672abstract common code so it can be reused by multiple packages. Every
673package inherits the ``base.bbclass`` file. Examples of other important
674classes are ``autotools.bbclass``, which in theory allows any
675Autotool-enabled package to work with the Yocto Project with minimal
676effort. Another example is ``kernel.bbclass`` that contains common code
677and functions for working with the Linux kernel. Functions like image
678generation or packaging also have their specific class files such as
679``image.bbclass``, ``rootfs_*.bbclass`` and ``package*.bbclass``.
680
681For reference information on classes, see the
682"`Classes <#ref-classes>`__" chapter.
683
684.. _structure-meta-conf:
685
686``meta/conf/``
687--------------
688
689This directory contains the core set of configuration files that start
690from ``bitbake.conf`` and from which all other configuration files are
691included. See the include statements at the end of the ``bitbake.conf``
692file and you will note that even ``local.conf`` is loaded from there.
693While ``bitbake.conf`` sets up the defaults, you can often override
694these by using the (``local.conf``) file, machine file or the
695distribution configuration file.
696
697.. _structure-meta-conf-machine:
698
699``meta/conf/machine/``
700----------------------
701
702This directory contains all the machine configuration files. If you set
703``MACHINE = "qemux86"``, the OpenEmbedded build system looks for a
704``qemux86.conf`` file in this directory. The ``include`` directory
705contains various data common to multiple machines. If you want to add
706support for a new machine to the Yocto Project, look in this directory.
707
708.. _structure-meta-conf-distro:
709
710``meta/conf/distro/``
711---------------------
712
713The contents of this directory controls any distribution-specific
714configurations. For the Yocto Project, the ``defaultsetup.conf`` is the
715main file here. This directory includes the versions and the ``SRCDATE``
716definitions for applications that are configured here. An example of an
717alternative configuration might be ``poky-bleeding.conf``. Although this
718file mainly inherits its configuration from Poky.
719
720.. _structure-meta-conf-machine-sdk:
721
722``meta/conf/machine-sdk/``
723--------------------------
724
725The OpenEmbedded build system searches this directory for configuration
726files that correspond to the value of
727```SDKMACHINE`` <#var-SDKMACHINE>`__. By default, 32-bit and 64-bit x86
728files ship with the Yocto Project that support some SDK hosts. However,
729it is possible to extend that support to other SDK hosts by adding
730additional configuration files in this subdirectory within another
731layer.
732
733.. _structure-meta-files:
734
735``meta/files/``
736---------------
737
738This directory contains common license files and several text files used
739by the build system. The text files contain minimal device information
740and lists of files and directories with known permissions.
741
742.. _structure-meta-lib:
743
744``meta/lib/``
745-------------
746
747This directory contains OpenEmbedded Python library code used during the
748build process.
749
750.. _structure-meta-recipes-bsp:
751
752``meta/recipes-bsp/``
753---------------------
754
755This directory contains anything linking to specific hardware or
756hardware configuration information such as "u-boot" and "grub".
757
758.. _structure-meta-recipes-connectivity:
759
760``meta/recipes-connectivity/``
761------------------------------
762
763This directory contains libraries and applications related to
764communication with other devices.
765
766.. _structure-meta-recipes-core:
767
768``meta/recipes-core/``
769----------------------
770
771This directory contains what is needed to build a basic working Linux
772image including commonly used dependencies.
773
774.. _structure-meta-recipes-devtools:
775
776``meta/recipes-devtools/``
777--------------------------
778
779This directory contains tools that are primarily used by the build
780system. The tools, however, can also be used on targets.
781
782.. _structure-meta-recipes-extended:
783
784``meta/recipes-extended/``
785--------------------------
786
787This directory contains non-essential applications that add features
788compared to the alternatives in core. You might need this directory for
789full tool functionality or for Linux Standard Base (LSB) compliance.
790
791.. _structure-meta-recipes-gnome:
792
793``meta/recipes-gnome/``
794-----------------------
795
796This directory contains all things related to the GTK+ application
797framework.
798
799.. _structure-meta-recipes-graphics:
800
801``meta/recipes-graphics/``
802--------------------------
803
804This directory contains X and other graphically related system
805libraries.
806
807.. _structure-meta-recipes-kernel:
808
809``meta/recipes-kernel/``
810------------------------
811
812This directory contains the kernel and generic applications and
813libraries that have strong kernel dependencies.
814
815.. _structure-meta-recipes-lsb4:
816
817``meta/recipes-lsb4/``
818----------------------
819
820This directory contains recipes specifically added to support the Linux
821Standard Base (LSB) version 4.x.
822
823.. _structure-meta-recipes-multimedia:
824
825``meta/recipes-multimedia/``
826----------------------------
827
828This directory contains codecs and support utilities for audio, images
829and video.
830
831.. _structure-meta-recipes-rt:
832
833``meta/recipes-rt/``
834--------------------
835
836This directory contains package and image recipes for using and testing
837the ``PREEMPT_RT`` kernel.
838
839.. _structure-meta-recipes-sato:
840
841``meta/recipes-sato/``
842----------------------
843
844This directory contains the Sato demo/reference UI/UX and its associated
845applications and configuration data.
846
847.. _structure-meta-recipes-support:
848
849``meta/recipes-support/``
850-------------------------
851
852This directory contains recipes used by other recipes, but that are not
853directly included in images (i.e. dependencies of other recipes).
854
855.. _structure-meta-site:
856
857``meta/site/``
858--------------
859
860This directory contains a list of cached results for various
861architectures. Because certain "autoconf" test results cannot be
862determined when cross-compiling due to the tests not able to run on a
863live system, the information in this directory is passed to "autoconf"
864for the various architectures.
865
866.. _structure-meta-recipes-txt:
867
868``meta/recipes.txt``
869--------------------
870
871This file is a description of the contents of ``recipes-*``.
diff --git a/documentation/ref-manual/ref-system-requirements.rst b/documentation/ref-manual/ref-system-requirements.rst
new file mode 100644
index 0000000000..ca2744c311
--- /dev/null
+++ b/documentation/ref-manual/ref-system-requirements.rst
@@ -0,0 +1,378 @@
1*******************
2System Requirements
3*******************
4
5Welcome to the Yocto Project Reference Manual! This manual provides
6reference information for the current release of the Yocto Project, and
7is most effectively used after you have an understanding of the basics
8of the Yocto Project. The manual is neither meant to be read as a
9starting point to the Yocto Project, nor read from start to finish.
10Rather, use this manual to find variable definitions, class
11descriptions, and so forth as needed during the course of using the
12Yocto Project.
13
14For introductory information on the Yocto Project, see the `Yocto
15Project Website <&YOCTO_HOME_URL;>`__ and the "`Yocto Project
16Development
17Environment <&YOCTO_DOCS_OM_URL;#overview-development-environment>`__"
18chapter in the Yocto Project Overview and Concepts Manual.
19
20If you want to use the Yocto Project to quickly build an image without
21having to understand concepts, work through the `Yocto Project Quick
22Build <&YOCTO_DOCS_BRIEF_URL;>`__ document. You can find "how-to"
23information in the `Yocto Project Development Tasks
24Manual <&YOCTO_DOCS_DEV_URL;>`__. You can find Yocto Project overview
25and conceptual information in the `Yocto Project Overview and Concepts
26Manual <&YOCTO_DOCS_OM_URL;>`__.
27
28.. note::
29
30 For more information about the Yocto Project Documentation set, see
31 the "
32 Links and Related Documentation
33 " section.
34
35.. _detailed-supported-distros:
36
37Supported Linux Distributions
38=============================
39
40Currently, the Yocto Project is supported on the following
41distributions:
42
43.. note::
44
45 - Yocto Project releases are tested against the stable Linux
46 distributions in the following list. The Yocto Project should work
47 on other distributions but validation is not performed against
48 them.
49
50 - In particular, the Yocto Project does not support and currently
51 has no plans to support rolling-releases or development
52 distributions due to their constantly changing nature. We welcome
53 patches and bug reports, but keep in mind that our priority is on
54 the supported platforms listed below.
55
56 - You may use Windows Subsystem For Linux v2 to set up a build host
57 using Windows 10, but validation is not performed against build
58 hosts using WSLv2.
59
60 .. note::
61
62 The Yocto Project is not compatible with WSLv1, it is
63 compatible but not officially supported nor validated with
64 WSLv2, if you still decide to use WSL please upgrade to WSLv2.
65
66 - If you encounter problems, please go to `Yocto Project
67 Bugzilla <&YOCTO_BUGZILLA_URL;>`__ and submit a bug. We are
68 interested in hearing about your experience. For information on
69 how to submit a bug, see the Yocto Project `Bugzilla wiki
70 page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__
71 and the "`Submitting a Defect Against the Yocto
72 Project <&YOCTO_DOCS_DEV_URL;#submitting-a-defect-against-the-yocto-project>`__"
73 section in the Yocto Project Development Tasks Manual.
74
75- Ubuntu 16.04 (LTS)
76
77- Ubuntu 18.04 (LTS)
78
79- Ubuntu 20.04
80
81- Fedora 30
82
83- Fedora 31
84
85- Fedora 32
86
87- CentOS 7.x
88
89- CentOS 8.x
90
91- Debian GNU/Linux 8.x (Jessie)
92
93- Debian GNU/Linux 9.x (Stretch)
94
95- Debian GNU/Linux 10.x (Buster)
96
97- OpenSUSE Leap 15.1
98
99.. note::
100
101 While the Yocto Project Team attempts to ensure all Yocto Project
102 releases are one hundred percent compatible with each officially
103 supported Linux distribution, instances might exist where you
104 encounter a problem while using the Yocto Project on a specific
105 distribution.
106
107Required Packages for the Build Host
108====================================
109
110The list of packages you need on the host development system can be
111large when covering all build scenarios using the Yocto Project. This
112section describes required packages according to Linux distribution and
113function.
114
115.. _ubuntu-packages:
116
117Ubuntu and Debian
118-----------------
119
120The following list shows the required packages by function given a
121supported Ubuntu or Debian Linux distribution:
122
123.. note::
124
125 - If your build system has the ``oss4-dev`` package installed, you
126 might experience QEMU build failures due to the package installing
127 its own custom ``/usr/include/linux/soundcard.h`` on the Debian
128 system. If you run into this situation, either of the following
129 solutions exist: $ sudo apt-get build-dep qemu $ sudo apt-get
130 remove oss4-dev
131
132 - For Debian-8, ``python3-git`` and ``pylint3`` are no longer
133 available via ``apt-get``. $ sudo pip3 install GitPython
134 pylint==1.9.5
135
136- *Essentials:* Packages needed to build an image on a headless system:
137 $ sudo apt-get install UBUNTU_HOST_PACKAGES_ESSENTIAL
138
139- *Documentation:* Packages needed if you are going to build out the
140 Yocto Project documentation manuals: $ sudo apt-get install make
141 xsltproc docbook-utils fop dblatex xmlto
142
143Fedora Packages
144---------------
145
146The following list shows the required packages by function given a
147supported Fedora Linux distribution:
148
149- *Essentials:* Packages needed to build an image for a headless
150 system: $ sudo dnf install FEDORA_HOST_PACKAGES_ESSENTIAL
151
152- *Documentation:* Packages needed if you are going to build out the
153 Yocto Project documentation manuals: $ sudo dnf install
154 docbook-style-dsssl docbook-style-xsl \\ docbook-dtds docbook-utils
155 fop libxslt dblatex xmlto
156
157openSUSE Packages
158-----------------
159
160The following list shows the required packages by function given a
161supported openSUSE Linux distribution:
162
163- *Essentials:* Packages needed to build an image for a headless
164 system: $ sudo zypper install OPENSUSE_HOST_PACKAGES_ESSENTIAL
165
166- *Documentation:* Packages needed if you are going to build out the
167 Yocto Project documentation manuals: $ sudo zypper install dblatex
168 xmlto
169
170CentOS-7 Packages
171-----------------
172
173The following list shows the required packages by function given a
174supported CentOS-7 Linux distribution:
175
176- *Essentials:* Packages needed to build an image for a headless
177 system: $ sudo yum install CENTOS7_HOST_PACKAGES_ESSENTIAL
178
179 .. note::
180
181 - Extra Packages for Enterprise Linux (i.e. ``epel-release``) is
182 a collection of packages from Fedora built on RHEL/CentOS for
183 easy installation of packages not included in enterprise Linux
184 by default. You need to install these packages separately.
185
186 - The ``makecache`` command consumes additional Metadata from
187 ``epel-release``.
188
189- *Documentation:* Packages needed if you are going to build out the
190 Yocto Project documentation manuals: $ sudo yum install
191 docbook-style-dsssl docbook-style-xsl \\ docbook-dtds docbook-utils
192 fop libxslt dblatex xmlto
193
194CentOS-8 Packages
195-----------------
196
197The following list shows the required packages by function given a
198supported CentOS-8 Linux distribution:
199
200- *Essentials:* Packages needed to build an image for a headless
201 system: $ sudo dnf install CENTOS8_HOST_PACKAGES_ESSENTIAL
202
203 .. note::
204
205 - Extra Packages for Enterprise Linux (i.e. ``epel-release``) is
206 a collection of packages from Fedora built on RHEL/CentOS for
207 easy installation of packages not included in enterprise Linux
208 by default. You need to install these packages separately.
209
210 - The ``PowerTools`` repo provides additional packages such as
211 ``rpcgen`` and ``texinfo``.
212
213 - The ``makecache`` command consumes additional Metadata from
214 ``epel-release``.
215
216- *Documentation:* Packages needed if you are going to build out the
217 Yocto Project documentation manuals: $ sudo dnf install
218 docbook-style-dsssl docbook-style-xsl \\ docbook-dtds docbook-utils
219 fop libxslt dblatex xmlto
220
221Required Git, tar, Python and gcc Versions
222==========================================
223
224In order to use the build system, your host development system must meet
225the following version requirements for Git, tar, and Python:
226
227- Git 1.8.3.1 or greater
228
229- tar 1.28 or greater
230
231- Python 3.5.0 or greater
232
233If your host development system does not meet all these requirements,
234you can resolve this by installing a ``buildtools`` tarball that
235contains these tools. You can get the tarball one of two ways: download
236a pre-built tarball or use BitBake to build the tarball.
237
238In addition, your host development system must meet the following
239version requirement for gcc:
240
241- gcc 5.0 or greater
242
243If your host development system does not meet this requirement, you can
244resolve this by installing a ``buildtools-extended`` tarball that
245contains additional tools, the equivalent of ``buildtools-essential``.
246
247Installing a Pre-Built ``buildtools`` Tarball with ``install-buildtools`` script
248--------------------------------------------------------------------------------
249
250The ``install-buildtools`` script is the easiest of the three methods by
251which you can get these tools. It downloads a pre-built buildtools
252installer and automatically installs the tools for you:
253
2541. Execute the ``install-buildtools`` script. Here is an example: $ cd
255 poky $ scripts/install-buildtools --without-extended-buildtools \\
256 --base-url YOCTO_DL_URL/releases/yocto \\ --release yocto-DISTRO \\
257 --installer-version DISTRO
258
259 During execution, the buildtools tarball will be downloaded, the
260 checksum of the download will be verified, the installer will be run
261 for you, and some basic checks will be run to to make sure the
262 installation is functional.
263
264 To avoid the need of ``sudo`` privileges, the ``install-buildtools``
265 script will by default tell the installer to install in:
266 /path/to/poky/buildtools
267
268 If your host development system needs the additional tools provided
269 in the ``buildtools-extended`` tarball, you can instead execute the
270 ``install-buildtools`` script with the default parameters: $ cd poky
271 $ scripts/install-buildtools
272
2732. Source the tools environment setup script by using a command like the
274 following: $ source
275 /path/to/poky/buildtools/environment-setup-x86_64-pokysdk-linux Of
276 course, you need to supply your installation directory and be sure to
277 use the right file (i.e. i586 or x86_64).
278
279 After you have sourced the setup script, the tools are added to
280 ``PATH`` and any other environment variables required to run the
281 tools are initialized. The results are working versions versions of
282 Git, tar, Python and ``chrpath``. And in the case of the
283 ``buildtools-extended`` tarball, additional working versions of tools
284 including ``gcc``, ``make`` and the other tools included in
285 ``packagegroup-core-buildessential``.
286
287Downloading a Pre-Built ``buildtools`` Tarball
288----------------------------------------------
289
290Downloading and running a pre-built buildtools installer is the easiest
291of the two methods by which you can get these tools:
292
2931. Locate and download the ``*.sh`` at
294 ` <&YOCTO_RELEASE_DL_URL;/buildtools/>`__.
295
2962. Execute the installation script. Here is an example for the
297 traditional installer: $ sh
298 ~/Downloads/x86_64-buildtools-nativesdk-standalone-DISTRO.sh Here is
299 an example for the extended installer: $ sh
300 ~/Downloads/x86_64-buildtools-extended-nativesdk-standalone-DISTRO.sh
301 During execution, a prompt appears that allows you to choose the
302 installation directory. For example, you could choose the following:
303 /home/your-username/buildtools
304
3053. Source the tools environment setup script by using a command like the
306 following: $ source
307 /home/your_username/buildtools/environment-setup-i586-poky-linux Of
308 course, you need to supply your installation directory and be sure to
309 use the right file (i.e. i585 or x86-64).
310
311 After you have sourced the setup script, the tools are added to
312 ``PATH`` and any other environment variables required to run the
313 tools are initialized. The results are working versions versions of
314 Git, tar, Python and ``chrpath``. And in the case of the
315 ``buildtools-extended`` tarball, additional working versions of tools
316 including ``gcc``, ``make`` and the other tools included in
317 ``packagegroup-core-buildessential``.
318
319Building Your Own ``buildtools`` Tarball
320----------------------------------------
321
322Building and running your own buildtools installer applies only when you
323have a build host that can already run BitBake. In this case, you use
324that machine to build the ``.sh`` file and then take steps to transfer
325and run it on a machine that does not meet the minimal Git, tar, and
326Python (or gcc) requirements.
327
328Here are the steps to take to build and run your own buildtools
329installer:
330
3311. On the machine that is able to run BitBake, be sure you have set up
332 your build environment with the setup script
333 (````` <#structure-core-script>`__).
334
3352. Run the BitBake command to build the tarball: $ bitbake
336 buildtools-tarball or run the BitBake command to build the extended
337 tarball: $ bitbake buildtools-extended-tarball
338
339 .. note::
340
341 The
342 SDKMACHINE
343 variable in your
344 local.conf
345 file determines whether you build tools for a 32-bit or 64-bit
346 system.
347
348 Once the build completes, you can find the ``.sh`` file that installs
349 the tools in the ``tmp/deploy/sdk`` subdirectory of the `Build
350 Directory <#build-directory>`__. The installer file has the string
351 "buildtools" (or "buildtools-extended") in the name.
352
3533. Transfer the ``.sh`` file from the build host to the machine that
354 does not meet the Git, tar, or Python (or gcc) requirements.
355
3564. On the machine that does not meet the requirements, run the ``.sh``
357 file to install the tools. Here is an example for the traditional
358 installer: $ sh
359 ~/Downloads/x86_64-buildtools-nativesdk-standalone-DISTRO.sh Here is
360 an example for the extended installer: $ sh
361 ~/Downloads/x86_64-buildtools-extended-nativesdk-standalone-DISTRO.sh
362 During execution, a prompt appears that allows you to choose the
363 installation directory. For example, you could choose the following:
364 /home/your_username/buildtools
365
3665. Source the tools environment setup script by using a command like the
367 following: $ source
368 /home/your_username/buildtools/environment-setup-x86_64-poky-linux Of
369 course, you need to supply your installation directory and be sure to
370 use the right file (i.e. i586 or x86_64).
371
372 After you have sourced the setup script, the tools are added to
373 ``PATH`` and any other environment variables required to run the
374 tools are initialized. The results are working versions versions of
375 Git, tar, Python and ``chrpath``. And in the case of the
376 ``buildtools-extended`` tarball, additional working versions of tools
377 including ``gcc``, ``make`` and the other tools included in
378 ``packagegroup-core-buildessential``.
diff --git a/documentation/ref-manual/ref-tasks.rst b/documentation/ref-manual/ref-tasks.rst
new file mode 100644
index 0000000000..be7db8d4fc
--- /dev/null
+++ b/documentation/ref-manual/ref-tasks.rst
@@ -0,0 +1,834 @@
1*****
2Tasks
3*****
4
5Tasks are units of execution for BitBake. Recipes (``.bb`` files) use
6tasks to complete configuring, compiling, and packaging software. This
7chapter provides a reference of the tasks defined in the OpenEmbedded
8build system.
9
10Normal Recipe Build Tasks
11=========================
12
13The following sections describe normal tasks associated with building a
14recipe. For more information on tasks and dependencies, see the
15"`Tasks <&YOCTO_DOCS_BB_URL;#tasks>`__" and
16"`Dependencies <&YOCTO_DOCS_BB_URL;#dependencies>`__" sections in the
17BitBake User Manual.
18
19.. _ref-tasks-build:
20
21``do_build``
22------------
23
24The default task for all recipes. This task depends on all other normal
25tasks required to build a recipe.
26
27.. _ref-tasks-compile:
28
29``do_compile``
30--------------
31
32Compiles the source code. This task runs with the current working
33directory set to ``${``\ ```B`` <#var-B>`__\ ``}``.
34
35The default behavior of this task is to run the ``oe_runmake`` function
36if a makefile (``Makefile``, ``makefile``, or ``GNUmakefile``) is found.
37If no such file is found, the ``do_compile`` task does nothing.
38
39.. _ref-tasks-compile_ptest_base:
40
41``do_compile_ptest_base``
42-------------------------
43
44Compiles the runtime test suite included in the software being built.
45
46.. _ref-tasks-configure:
47
48``do_configure``
49----------------
50
51Configures the source by enabling and disabling any build-time and
52configuration options for the software being built. The task runs with
53the current working directory set to ``${``\ ```B`` <#var-B>`__\ ``}``.
54
55The default behavior of this task is to run ``oe_runmake clean`` if a
56makefile (``Makefile``, ``makefile``, or ``GNUmakefile``) is found and
57```CLEANBROKEN`` <#var-CLEANBROKEN>`__ is not set to "1". If no such
58file is found or the ``CLEANBROKEN`` variable is set to "1", the
59``do_configure`` task does nothing.
60
61.. _ref-tasks-configure_ptest_base:
62
63``do_configure_ptest_base``
64---------------------------
65
66Configures the runtime test suite included in the software being built.
67
68.. _ref-tasks-deploy:
69
70``do_deploy``
71-------------
72
73Writes output files that are to be deployed to
74``${``\ ```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__\ ``}``. The
75task runs with the current working directory set to
76``${``\ ```B`` <#var-B>`__\ ``}``.
77
78Recipes implementing this task should inherit the
79```deploy`` <#ref-classes-deploy>`__ class and should write the output
80to ``${``\ ```DEPLOYDIR`` <#var-DEPLOYDIR>`__\ ``}``, which is not to be
81confused with ``${DEPLOY_DIR}``. The ``deploy`` class sets up
82``do_deploy`` as a shared state (sstate) task that can be accelerated
83through sstate use. The sstate mechanism takes care of copying the
84output from ``${DEPLOYDIR}`` to ``${DEPLOY_DIR_IMAGE}``.
85
86.. note::
87
88 Do not write the output directly to
89 ${DEPLOY_DIR_IMAGE}
90 , as this causes the sstate mechanism to malfunction.
91
92The ``do_deploy`` task is not added as a task by default and
93consequently needs to be added manually. If you want the task to run
94after ```do_compile`` <#ref-tasks-compile>`__, you can add it by doing
95the following: addtask deploy after do_compile Adding ``do_deploy``
96after other tasks works the same way.
97
98.. note::
99
100 You do not need to add
101 before do_build
102 to the
103 addtask
104 command (though it is harmless), because the
105 base
106 class contains the following:
107 ::
108
109 do_build[recrdeptask] += "do_deploy"
110
111
112 See the "
113 Dependencies
114 " section in the BitBake User Manual for more information.
115
116If the ``do_deploy`` task re-executes, any previous output is removed
117(i.e. "cleaned").
118
119.. _ref-tasks-fetch:
120
121``do_fetch``
122------------
123
124Fetches the source code. This task uses the
125```SRC_URI`` <#var-SRC_URI>`__ variable and the argument's prefix to
126determine the correct `fetcher <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__
127module.
128
129.. _ref-tasks-image:
130
131``do_image``
132------------
133
134Starts the image generation process. The ``do_image`` task runs after
135the OpenEmbedded build system has run the
136```do_rootfs`` <#ref-tasks-rootfs>`__ task during which packages are
137identified for installation into the image and the root filesystem is
138created, complete with post-processing.
139
140The ``do_image`` task performs pre-processing on the image through the
141```IMAGE_PREPROCESS_COMMAND`` <#var-IMAGE_PREPROCESS_COMMAND>`__ and
142dynamically generates supporting ``do_image_*`` tasks as needed.
143
144For more information on image creation, see the "`Image
145Generation <&YOCTO_DOCS_OM_URL;#image-generation-dev-environment>`__"
146section in the Yocto Project Overview and Concepts Manual.
147
148.. _ref-tasks-image-complete:
149
150``do_image_complete``
151---------------------
152
153Completes the image generation process. The ``do_image_complete`` task
154runs after the OpenEmbedded build system has run the
155```do_image`` <#ref-tasks-image>`__ task during which image
156pre-processing occurs and through dynamically generated ``do_image_*``
157tasks the image is constructed.
158
159The ``do_image_complete`` task performs post-processing on the image
160through the
161```IMAGE_POSTPROCESS_COMMAND`` <#var-IMAGE_POSTPROCESS_COMMAND>`__.
162
163For more information on image creation, see the "`Image
164Generation <&YOCTO_DOCS_OM_URL;#image-generation-dev-environment>`__"
165section in the Yocto Project Overview and Concepts Manual.
166
167.. _ref-tasks-install:
168
169``do_install``
170--------------
171
172Copies files that are to be packaged into the holding area
173``${``\ ```D`` <#var-D>`__\ ``}``. This task runs with the current
174working directory set to ``${``\ ```B`` <#var-B>`__\ ``}``, which is the
175compilation directory. The ``do_install`` task, as well as other tasks
176that either directly or indirectly depend on the installed files (e.g.
177```do_package`` <#ref-tasks-package>`__,
178```do_package_write_*`` <#ref-tasks-package_write_deb>`__, and
179```do_rootfs`` <#ref-tasks-rootfs>`__), run under
180`fakeroot <&YOCTO_DOCS_OM_URL;#fakeroot-and-pseudo>`__.
181
182.. note::
183
184 When installing files, be careful not to set the owner and group IDs
185 of the installed files to unintended values. Some methods of copying
186 files, notably when using the recursive ``cp`` command, can preserve
187 the UID and/or GID of the original file, which is usually not what
188 you want. The
189 ```host-user-contaminated`` <#insane-host-user-contaminated>`__ QA
190 check checks for files that probably have the wrong ownership.
191
192 Safe methods for installing files include the following:
193
194 - The ``install`` utility. This utility is the preferred method.
195
196 - The ``cp`` command with the "--no-preserve=ownership" option.
197
198 - The ``tar`` command with the "--no-same-owner" option. See the
199 ``bin_package.bbclass`` file in the ``meta/classes`` directory of
200 the `Source Directory <#source-directory>`__ for an example.
201
202.. _ref-tasks-install_ptest_base:
203
204``do_install_ptest_base``
205-------------------------
206
207Copies the runtime test suite files from the compilation directory to a
208holding area.
209
210.. _ref-tasks-package:
211
212``do_package``
213--------------
214
215Analyzes the content of the holding area
216``${``\ ```D`` <#var-D>`__\ ``}`` and splits the content into subsets
217based on available packages and files. This task makes use of the
218```PACKAGES`` <#var-PACKAGES>`__ and ```FILES`` <#var-FILES>`__
219variables.
220
221The ``do_package`` task, in conjunction with the
222```do_packagedata`` <#ref-tasks-packagedata>`__ task, also saves some
223important package metadata. For additional information, see the
224```PKGDESTWORK`` <#var-PKGDESTWORK>`__ variable and the "`Automatically
225Added Runtime
226Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
227section in the Yocto Project Overview and Concepts Manual.
228
229.. _ref-tasks-package_qa:
230
231``do_package_qa``
232-----------------
233
234Runs QA checks on packaged files. For more information on these checks,
235see the ```insane`` <#ref-classes-insane>`__ class.
236
237.. _ref-tasks-package_write_deb:
238
239``do_package_write_deb``
240------------------------
241
242Creates Debian packages (i.e. ``*.deb`` files) and places them in the
243``${``\ ```DEPLOY_DIR_DEB`` <#var-DEPLOY_DIR_DEB>`__\ ``}`` directory in
244the package feeds area. For more information, see the "`Package
245Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section in
246the Yocto Project Overview and Concepts Manual.
247
248.. _ref-tasks-package_write_ipk:
249
250``do_package_write_ipk``
251------------------------
252
253Creates IPK packages (i.e. ``*.ipk`` files) and places them in the
254``${``\ ```DEPLOY_DIR_IPK`` <#var-DEPLOY_DIR_IPK>`__\ ``}`` directory in
255the package feeds area. For more information, see the "`Package
256Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section in
257the Yocto Project Overview and Concepts Manual.
258
259.. _ref-tasks-package_write_rpm:
260
261``do_package_write_rpm``
262------------------------
263
264Creates RPM packages (i.e. ``*.rpm`` files) and places them in the
265``${``\ ```DEPLOY_DIR_RPM`` <#var-DEPLOY_DIR_RPM>`__\ ``}`` directory in
266the package feeds area. For more information, see the "`Package
267Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section in
268the Yocto Project Overview and Concepts Manual.
269
270.. _ref-tasks-package_write_tar:
271
272``do_package_write_tar``
273------------------------
274
275Creates tarballs and places them in the
276``${``\ ```DEPLOY_DIR_TAR`` <#var-DEPLOY_DIR_TAR>`__\ ``}`` directory in
277the package feeds area. For more information, see the "`Package
278Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section in
279the Yocto Project Overview and Concepts Manual.
280
281.. _ref-tasks-packagedata:
282
283``do_packagedata``
284------------------
285
286Saves package metadata generated by the
287```do_package`` <#ref-tasks-package>`__ task in
288```PKGDATA_DIR`` <#var-PKGDATA_DIR>`__ to make it available globally.
289
290.. _ref-tasks-patch:
291
292``do_patch``
293------------
294
295Locates patch files and applies them to the source code.
296
297After fetching and unpacking source files, the build system uses the
298recipe's ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements
299to locate and apply patch files to the source code.
300
301.. note::
302
303 The build system uses the
304 FILESPATH
305 variable to determine the default set of directories when searching
306 for patches.
307
308Patch files, by default, are ``*.patch`` and ``*.diff`` files created
309and kept in a subdirectory of the directory holding the recipe file. For
310example, consider the
311```bluez5`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/recipes-connectivity/bluez5>`__
312recipe from the OE-Core layer (i.e. ``poky/meta``):
313poky/meta/recipes-connectivity/bluez5 This recipe has two patch files
314located here: poky/meta/recipes-connectivity/bluez5/bluez5
315
316In the ``bluez5`` recipe, the ``SRC_URI`` statements point to the source
317and patch files needed to build the package.
318
319.. note::
320
321 In the case for the
322 bluez5_5.48.bb
323 recipe, the
324 SRC_URI
325 statements are from an include file
326 bluez5.inc
327 .
328
329As mentioned earlier, the build system treats files whose file types are
330``.patch`` and ``.diff`` as patch files. However, you can use the
331"apply=yes" parameter with the ``SRC_URI`` statement to indicate any
332file as a patch file: SRC_URI = " \\ git://path_to_repo/some_package \\
333file://file;apply=yes \\ "
334
335Conversely, if you have a directory full of patch files and you want to
336exclude some so that the ``do_patch`` task does not apply them during
337the patch phase, you can use the "apply=no" parameter with the
338``SRC_URI`` statement: SRC_URI = " \\ git://path_to_repo/some_package \\
339file://path_to_lots_of_patch_files \\
340file://path_to_lots_of_patch_files/patch_file5;apply=no \\ " In the
341previous example, assuming all the files in the directory holding the
342patch files end with either ``.patch`` or ``.diff``, every file would be
343applied as a patch by default except for the patch_file5 patch.
344
345You can find out more about the patching process in the
346"`Patching <&YOCTO_DOCS_OM_URL;#patching-dev-environment>`__" section in
347the Yocto Project Overview and Concepts Manual and the "`Patching
348Code <&YOCTO_DOCS_DEV_URL;#new-recipe-patching-code>`__" section in the
349Yocto Project Development Tasks Manual.
350
351.. _ref-tasks-populate_lic:
352
353``do_populate_lic``
354-------------------
355
356Writes license information for the recipe that is collected later when
357the image is constructed.
358
359.. _ref-tasks-populate_sdk:
360
361``do_populate_sdk``
362-------------------
363
364Creates the file and directory structure for an installable SDK. See the
365"`SDK
366Generation <&YOCTO_DOCS_OM_URL;#sdk-generation-dev-environment>`__"
367section in the Yocto Project Overview and Concepts Manual for more
368information.
369
370.. _ref-tasks-populate_sysroot:
371
372``do_populate_sysroot``
373-----------------------
374
375Stages (copies) a subset of the files installed by the
376```do_install`` <#ref-tasks-install>`__ task into the appropriate
377sysroot. For information on how to access these files from other
378recipes, see the ```STAGING_DIR*`` <#var-STAGING_DIR_HOST>`__ variables.
379Directories that would typically not be needed by other recipes at build
380time (e.g. ``/etc``) are not copied by default.
381
382For information on what directories are copied by default, see the
383```SYSROOT_DIRS*`` <#var-SYSROOT_DIRS>`__ variables. You can change
384these variables inside your recipe if you need to make additional (or
385fewer) directories available to other recipes at build time.
386
387The ``do_populate_sysroot`` task is a shared state (sstate) task, which
388means that the task can be accelerated through sstate use. Realize also
389that if the task is re-executed, any previous output is removed (i.e.
390"cleaned").
391
392.. _ref-tasks-prepare_recipe_sysroot:
393
394``do_prepare_recipe_sysroot``
395-----------------------------
396
397Installs the files into the individual recipe specific sysroots (i.e.
398``recipe-sysroot`` and ``recipe-sysroot-native`` under
399``${``\ ```WORKDIR`` <#var-WORKDIR>`__\ ``}`` based upon the
400dependencies specified by ```DEPENDS`` <#var-DEPENDS>`__). See the
401"```staging`` <#ref-classes-staging>`__" class for more information.
402
403.. _ref-tasks-rm_work:
404
405``do_rm_work``
406--------------
407
408Removes work files after the OpenEmbedded build system has finished with
409them. You can learn more by looking at the
410"```rm_work.bbclass`` <#ref-classes-rm-work>`__" section.
411
412.. _ref-tasks-unpack:
413
414``do_unpack``
415-------------
416
417Unpacks the source code into a working directory pointed to by
418``${``\ ```WORKDIR`` <#var-WORKDIR>`__\ ``}``. The ```S`` <#var-S>`__
419variable also plays a role in where unpacked source files ultimately
420reside. For more information on how source files are unpacked, see the
421"`Source
422Fetching <&YOCTO_DOCS_OM_URL;#source-fetching-dev-environment>`__"
423section in the Yocto Project Overview and Concepts Manual and also see
424the ``WORKDIR`` and ``S`` variable descriptions.
425
426Manually Called Tasks
427=====================
428
429These tasks are typically manually triggered (e.g. by using the
430``bitbake -c`` command-line option):
431
432.. _ref-tasks-checkpkg:
433
434``do_checkpkg``
435---------------
436
437Provides information about the recipe including its upstream version and
438status. The upstream version and status reveals whether or not a version
439of the recipe exists upstream and a status of not updated, updated, or
440unknown.
441
442To check the upstream version and status of a recipe, use the following
443devtool commands: $ devtool latest-version $ devtool
444check-upgrade-status See the "```devtool`` Quick
445Reference <#ref-devtool-reference>`__" chapter for more information on
446``devtool``. See the "`Checking on the Upgrade Status of a
447Recipe <&YOCTO_DOCS_REF_URL;#devtool-checking-on-the-upgrade-status-of-a-recipe>`__"
448section for information on checking the upgrade status of a recipe.
449
450To build the ``checkpkg`` task, use the ``bitbake`` command with the
451"-c" option and task name: $ bitbake core-image-minimal -c checkpkg By
452default, the results are stored in ```$LOG_DIR`` <#var-LOG_DIR>`__ (e.g.
453``$BUILD_DIR/tmp/log``).
454
455.. _ref-tasks-checkuri:
456
457``do_checkuri``
458---------------
459
460Validates the ```SRC_URI`` <#var-SRC_URI>`__ value.
461
462.. _ref-tasks-clean:
463
464``do_clean``
465------------
466
467Removes all output files for a target from the
468```do_unpack`` <#ref-tasks-unpack>`__ task forward (i.e. ``do_unpack``,
469```do_configure`` <#ref-tasks-configure>`__,
470```do_compile`` <#ref-tasks-compile>`__,
471```do_install`` <#ref-tasks-install>`__, and
472```do_package`` <#ref-tasks-package>`__).
473
474You can run this task using BitBake as follows: $ bitbake -c clean
475recipe
476
477Running this task does not remove the
478`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__ cache files.
479Consequently, if no changes have been made and the recipe is rebuilt
480after cleaning, output files are simply restored from the sstate cache.
481If you want to remove the sstate cache files for the recipe, you need to
482use the ```do_cleansstate`` <#ref-tasks-cleansstate>`__ task instead
483(i.e. ``bitbake -c cleansstate`` recipe).
484
485.. _ref-tasks-cleanall:
486
487``do_cleanall``
488---------------
489
490Removes all output files, shared state
491(`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__) cache, and
492downloaded source files for a target (i.e. the contents of
493```DL_DIR`` <#var-DL_DIR>`__). Essentially, the ``do_cleanall`` task is
494identical to the ```do_cleansstate`` <#ref-tasks-cleansstate>`__ task
495with the added removal of downloaded source files.
496
497You can run this task using BitBake as follows: $ bitbake -c cleanall
498recipe
499
500Typically, you would not normally use the ``cleanall`` task. Do so only
501if you want to start fresh with the ```do_fetch`` <#ref-tasks-fetch>`__
502task.
503
504.. _ref-tasks-cleansstate:
505
506``do_cleansstate``
507------------------
508
509Removes all output files and shared state
510(`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__) cache for a
511target. Essentially, the ``do_cleansstate`` task is identical to the
512```do_clean`` <#ref-tasks-clean>`__ task with the added removal of
513shared state (`sstate <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__)
514cache.
515
516You can run this task using BitBake as follows: $ bitbake -c cleansstate
517recipe
518
519When you run the ``do_cleansstate`` task, the OpenEmbedded build system
520no longer uses any sstate. Consequently, building the recipe from
521scratch is guaranteed.
522
523.. note::
524
525 The
526 do_cleansstate
527 task cannot remove sstate from a remote sstate mirror. If you need to
528 build a target from scratch using remote mirrors, use the "-f" option
529 as follows:
530 ::
531
532 $ bitbake -f -c do_cleansstate target
533
534
535.. _ref-tasks-devpyshell:
536
537``do_devpyshell``
538-----------------
539
540Starts a shell in which an interactive Python interpreter allows you to
541interact with the BitBake build environment. From within this shell, you
542can directly examine and set bits from the data store and execute
543functions as if within the BitBake environment. See the "`Using a
544Development Python
545Shell <&YOCTO_DOCS_DEV_URL;#platdev-appdev-devpyshell>`__" section in
546the Yocto Project Development Tasks Manual for more information about
547using ``devpyshell``.
548
549.. _ref-tasks-devshell:
550
551``do_devshell``
552---------------
553
554Starts a shell whose environment is set up for development, debugging,
555or both. See the "`Using a Development
556Shell <&YOCTO_DOCS_DEV_URL;#platdev-appdev-devshell>`__" section in the
557Yocto Project Development Tasks Manual for more information about using
558``devshell``.
559
560.. _ref-tasks-listtasks:
561
562``do_listtasks``
563----------------
564
565Lists all defined tasks for a target.
566
567.. _ref-tasks-package_index:
568
569``do_package_index``
570--------------------
571
572Creates or updates the index in the `Package
573Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__ area.
574
575.. note::
576
577 This task is not triggered with the
578 bitbake -c
579 command-line option as are the other tasks in this section. Because
580 this task is specifically for the
581 package-index
582 recipe, you run it using
583 bitbake package-index
584 .
585
586Image-Related Tasks
587===================
588
589The following tasks are applicable to image recipes.
590
591.. _ref-tasks-bootimg:
592
593``do_bootimg``
594--------------
595
596Creates a bootable live image. See the
597```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable for additional
598information on live image types.
599
600.. _ref-tasks-bundle_initramfs:
601
602``do_bundle_initramfs``
603-----------------------
604
605Combines an initial RAM disk (initramfs) image and kernel together to
606form a single image. The
607```CONFIG_INITRAMFS_SOURCE`` <#var-CONFIG_INITRAMFS_SOURCE>`__ variable
608has some more information about these types of images.
609
610.. _ref-tasks-rootfs:
611
612``do_rootfs``
613-------------
614
615Creates the root filesystem (file and directory structure) for an image.
616See the "`Image
617Generation <&YOCTO_DOCS_OM_URL;#image-generation-dev-environment>`__"
618section in the Yocto Project Overview and Concepts Manual for more
619information on how the root filesystem is created.
620
621.. _ref-tasks-testimage:
622
623``do_testimage``
624----------------
625
626Boots an image and performs runtime tests within the image. For
627information on automatically testing images, see the "`Performing
628Automated Runtime
629Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
630section in the Yocto Project Development Tasks Manual.
631
632.. _ref-tasks-testimage_auto:
633
634``do_testimage_auto``
635---------------------
636
637Boots an image and performs runtime tests within the image immediately
638after it has been built. This task is enabled when you set
639```TESTIMAGE_AUTO`` <#var-TESTIMAGE_AUTO>`__ equal to "1".
640
641For information on automatically testing images, see the "`Performing
642Automated Runtime
643Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
644section in the Yocto Project Development Tasks Manual.
645
646Kernel-Related Tasks
647====================
648
649The following tasks are applicable to kernel recipes. Some of these
650tasks (e.g. the ```do_menuconfig`` <#ref-tasks-menuconfig>`__ task) are
651also applicable to recipes that use Linux kernel style configuration
652such as the BusyBox recipe.
653
654.. _ref-tasks-compile_kernelmodules:
655
656``do_compile_kernelmodules``
657----------------------------
658
659Runs the step that builds the kernel modules (if needed). Building a
660kernel consists of two steps: 1) the kernel (``vmlinux``) is built, and
6612) the modules are built (i.e. ``make modules``).
662
663.. _ref-tasks-diffconfig:
664
665``do_diffconfig``
666-----------------
667
668When invoked by the user, this task creates a file containing the
669differences between the original config as produced by
670```do_kernel_configme`` <#ref-tasks-kernel_configme>`__ task and the
671changes made by the user with other methods (i.e. using
672(```do_kernel_menuconfig`` <#ref-tasks-kernel_menuconfig>`__). Once the
673file of differences is created, it can be used to create a config
674fragment that only contains the differences. You can invoke this task
675from the command line as follows: $ bitbake linux-yocto -c diffconfig
676For more information, see the "`Creating Configuration
677Fragments <&YOCTO_DOCS_KERNEL_DEV_URL;#creating-config-fragments>`__"
678section in the Yocto Project Linux Kernel Development Manual.
679
680.. _ref-tasks-kernel_checkout:
681
682``do_kernel_checkout``
683----------------------
684
685Converts the newly unpacked kernel source into a form with which the
686OpenEmbedded build system can work. Because the kernel source can be
687fetched in several different ways, the ``do_kernel_checkout`` task makes
688sure that subsequent tasks are given a clean working tree copy of the
689kernel with the correct branches checked out.
690
691.. _ref-tasks-kernel_configcheck:
692
693``do_kernel_configcheck``
694-------------------------
695
696Validates the configuration produced by the
697```do_kernel_menuconfig`` <#ref-tasks-kernel_menuconfig>`__ task. The
698``do_kernel_configcheck`` task produces warnings when a requested
699configuration does not appear in the final ``.config`` file or when you
700override a policy configuration in a hardware configuration fragment.
701You can run this task explicitly and view the output by using the
702following command: $ bitbake linux-yocto -c kernel_configcheck -f For
703more information, see the "`Validating
704Configuration <&YOCTO_DOCS_KERNEL_DEV_URL;#validating-configuration>`__"
705section in the Yocto Project Linux Kernel Development Manual.
706
707.. _ref-tasks-kernel_configme:
708
709``do_kernel_configme``
710----------------------
711
712After the kernel is patched by the ```do_patch`` <#ref-tasks-patch>`__
713task, the ``do_kernel_configme`` task assembles and merges all the
714kernel config fragments into a merged configuration that can then be
715passed to the kernel configuration phase proper. This is also the time
716during which user-specified defconfigs are applied if present, and where
717configuration modes such as ``--allnoconfig`` are applied.
718
719.. _ref-tasks-kernel_menuconfig:
720
721``do_kernel_menuconfig``
722------------------------
723
724Invoked by the user to manipulate the ``.config`` file used to build a
725linux-yocto recipe. This task starts the Linux kernel configuration
726tool, which you then use to modify the kernel configuration.
727
728.. note::
729
730 You can also invoke this tool from the command line as follows:
731 ::
732
733 $ bitbake linux-yocto -c menuconfig
734
735
736See the "`Using
737``menuconfig`` <&YOCTO_DOCS_KERNEL_DEV_URL;#using-menuconfig>`__"
738section in the Yocto Project Linux Kernel Development Manual for more
739information on this configuration tool.
740
741.. _ref-tasks-kernel_metadata:
742
743``do_kernel_metadata``
744----------------------
745
746Collects all the features required for a given kernel build, whether the
747features come from ```SRC_URI`` <#var-SRC_URI>`__ or from Git
748repositories. After collection, the ``do_kernel_metadata`` task
749processes the features into a series of config fragments and patches,
750which can then be applied by subsequent tasks such as
751```do_patch`` <#ref-tasks-patch>`__ and
752```do_kernel_configme`` <#ref-tasks-kernel_configme>`__.
753
754.. _ref-tasks-menuconfig:
755
756``do_menuconfig``
757-----------------
758
759Runs ``make menuconfig`` for the kernel. For information on
760``menuconfig``, see the
761"`Using  ``menuconfig`` <&YOCTO_DOCS_KERNEL_DEV_URL;#using-menuconfig>`__"
762section in the Yocto Project Linux Kernel Development Manual.
763
764.. _ref-tasks-savedefconfig:
765
766``do_savedefconfig``
767--------------------
768
769When invoked by the user, creates a defconfig file that can be used
770instead of the default defconfig. The saved defconfig contains the
771differences between the default defconfig and the changes made by the
772user using other methods (i.e. the
773```do_kernel_menuconfig`` <#ref-tasks-kernel_menuconfig>`__ task. You
774can invoke the task using the following command: $ bitbake linux-yocto
775-c savedefconfig
776
777.. _ref-tasks-shared_workdir:
778
779``do_shared_workdir``
780---------------------
781
782After the kernel has been compiled but before the kernel modules have
783been compiled, this task copies files required for module builds and
784which are generated from the kernel build into the shared work
785directory. With these copies successfully copied, the
786```do_compile_kernelmodules`` <#ref-tasks-compile_kernelmodules>`__ task
787can successfully build the kernel modules in the next step of the build.
788
789.. _ref-tasks-sizecheck:
790
791``do_sizecheck``
792----------------
793
794After the kernel has been built, this task checks the size of the
795stripped kernel image against
796```KERNEL_IMAGE_MAXSIZE`` <#var-KERNEL_IMAGE_MAXSIZE>`__. If that
797variable was set and the size of the stripped kernel exceeds that size,
798the kernel build produces a warning to that effect.
799
800.. _ref-tasks-strip:
801
802``do_strip``
803------------
804
805If ``KERNEL_IMAGE_STRIP_EXTRA_SECTIONS`` is defined, this task strips
806the sections named in that variable from ``vmlinux``. This stripping is
807typically used to remove nonessential sections such as ``.comment``
808sections from a size-sensitive configuration.
809
810.. _ref-tasks-validate_branches:
811
812``do_validate_branches``
813------------------------
814
815After the kernel is unpacked but before it is patched, this task makes
816sure that the machine and metadata branches as specified by the
817```SRCREV`` <#var-SRCREV>`__ variables actually exist on the specified
818branches. If these branches do not exist and
819```AUTOREV`` <#var-AUTOREV>`__ is not being used, the
820``do_validate_branches`` task fails during the build.
821
822Miscellaneous Tasks
823===================
824
825The following sections describe miscellaneous tasks.
826
827.. _ref-tasks-spdx:
828
829``do_spdx``
830-----------
831
832A build stage that takes the source code and scans it on a remote
833FOSSOLOGY server in order to produce an SPDX document. This task applies
834only to the ```spdx`` <#ref-classes-spdx>`__ class.
diff --git a/documentation/ref-manual/ref-terms.rst b/documentation/ref-manual/ref-terms.rst
new file mode 100644
index 0000000000..16e0aa7568
--- /dev/null
+++ b/documentation/ref-manual/ref-terms.rst
@@ -0,0 +1,369 @@
1*******************
2Yocto Project Terms
3*******************
4
5Following is a list of terms and definitions users new to the Yocto
6Project development environment might find helpful. While some of these
7terms are universal, the list includes them just in case:
8
9- *Append Files:* Files that append build information to a recipe file.
10 Append files are known as BitBake append files and ``.bbappend``
11 files. The OpenEmbedded build system expects every append file to
12 have a corresponding recipe (``.bb``) file. Furthermore, the append
13 file and corresponding recipe file must use the same root filename.
14 The filenames can differ only in the file type suffix used (e.g.
15 ``formfactor_0.0.bb`` and ``formfactor_0.0.bbappend``).
16
17 Information in append files extends or overrides the information in
18 the similarly-named recipe file. For an example of an append file in
19 use, see the "`Using .bbappend Files in Your
20 Layer <&YOCTO_DOCS_DEV_URL;#using-bbappend-files>`__" section in the
21 Yocto Project Development Tasks Manual.
22
23 When you name an append file, you can use the "``%``" wildcard
24 character to allow for matching recipe names. For example, suppose
25 you have an append file named as follows: busybox_1.21.%.bbappend
26 That append file would match any ``busybox_1.21.``\ x\ ``.bb``
27 version of the recipe. So, the append file would match any of the
28 following recipe names: busybox_1.21.1.bb busybox_1.21.2.bb
29 busybox_1.21.3.bb busybox_1.21.10.bb busybox_1.21.25.bb
30
31 .. note::
32
33 The use of the "
34 %
35 " character is limited in that it only works directly in front of
36 the
37 .bbappend
38 portion of the append file's name. You cannot use the wildcard
39 character in any other location of the name.
40
41- *BitBake:* The task executor and scheduler used by the OpenEmbedded
42 build system to build images. For more information on BitBake, see
43 the `BitBake User Manual <&YOCTO_DOCS_BB_URL;>`__.
44
45- *Board Support Package (BSP):* A group of drivers, definitions, and
46 other components that provide support for a specific hardware
47 configuration. For more information on BSPs, see the `Yocto Project
48 Board Support Package (BSP) Developer's
49 Guide <&YOCTO_DOCS_BSP_URL;>`__.
50
51- *Build Directory:* This term refers to the area used by the
52 OpenEmbedded build system for builds. The area is created when you
53 ``source`` the setup environment script that is found in the Source
54 Directory (i.e. ````` <#structure-core-script>`__). The
55 ```TOPDIR`` <#var-TOPDIR>`__ variable points to the Build Directory.
56
57 You have a lot of flexibility when creating the Build Directory.
58 Following are some examples that show how to create the directory.
59 The examples assume your `Source Directory <#source-directory>`__ is
60 named ``poky``:
61
62 - Create the Build Directory inside your Source Directory and let
63 the name of the Build Directory default to ``build``: $ cd
64 $HOME/poky $ source OE_INIT_FILE
65
66 - Create the Build Directory inside your home directory and
67 specifically name it ``test-builds``: $ cd $HOME $ source
68 poky/OE_INIT_FILE test-builds
69
70 - Provide a directory path and specifically name the Build
71 Directory. Any intermediate folders in the pathname must exist.
72 This next example creates a Build Directory named
73 ``YP-POKYVERSION`` in your home directory within the existing
74 directory ``mybuilds``: $ cd $HOME $ source
75 $HOME/poky/OE_INIT_FILE $HOME/mybuilds/YP-POKYVERSION
76
77 .. note::
78
79 By default, the Build Directory contains
80 TMPDIR
81 , which is a temporary directory the build system uses for its
82 work.
83 TMPDIR
84 cannot be under NFS. Thus, by default, the Build Directory cannot
85 be under NFS. However, if you need the Build Directory to be under
86 NFS, you can set this up by setting
87 TMPDIR
88 in your
89 local.conf
90 file to use a local drive. Doing so effectively separates
91 TMPDIR
92 from
93 TOPDIR
94 , which is the Build Directory.
95
96- *Build Host:* The system used to build images in a Yocto Project
97 Development environment. The build system is sometimes referred to as
98 the development host.
99
100- *Classes:* Files that provide for logic encapsulation and inheritance
101 so that commonly used patterns can be defined once and then easily
102 used in multiple recipes. For reference information on the Yocto
103 Project classes, see the "`Classes <#ref-classes>`__" chapter. Class
104 files end with the ``.bbclass`` filename extension.
105
106- *Configuration File:* Files that hold global definitions of
107 variables, user-defined variables, and hardware configuration
108 information. These files tell the OpenEmbedded build system what to
109 build and what to put into the image to support a particular
110 platform.
111
112 Configuration files end with a ``.conf`` filename extension. The
113 ``conf/local.conf`` configuration file in the `Build
114 Directory <#build-directory>`__ contains user-defined variables that
115 affect every build. The ``meta-poky/conf/distro/poky.conf``
116 configuration file defines Yocto "distro" configuration variables
117 used only when building with this policy. Machine configuration
118 files, which are located throughout the `Source
119 Directory <#source-directory>`__, define variables for specific
120 hardware and are only used when building for that target (e.g. the
121 ``machine/beaglebone.conf`` configuration file defines variables for
122 the Texas Instruments ARM Cortex-A8 development board).
123
124- *Container Layer:* Layers that hold other layers. An example of a
125 container layer is OpenEmbedded's
126 ```meta-openembedded`` <https://github.com/openembedded/meta-openembedded>`__
127 layer. The ``meta-openembedded`` layer contains many ``meta-*``
128 layers.
129
130- *Cross-Development Toolchain:* In general, a cross-development
131 toolchain is a collection of software development tools and utilities
132 that run on one architecture and allow you to develop software for a
133 different, or targeted, architecture. These toolchains contain
134 cross-compilers, linkers, and debuggers that are specific to the
135 target architecture.
136
137 The Yocto Project supports two different cross-development
138 toolchains:
139
140 - A toolchain only used by and within BitBake when building an image
141 for a target architecture.
142
143 - A relocatable toolchain used outside of BitBake by developers when
144 developing applications that will run on a targeted device.
145
146 Creation of these toolchains is simple and automated. For information
147 on toolchain concepts as they apply to the Yocto Project, see the
148 "`Cross-Development Toolchain
149 Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
150 section in the Yocto Project Overview and Concepts Manual. You can
151 also find more information on using the relocatable toolchain in the
152 `Yocto Project Application Development and the Extensible Software
153 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
154
155- *Extensible Software Development Kit (eSDK):* A custom SDK for
156 application developers. This eSDK allows developers to incorporate
157 their library and programming changes back into the image to make
158 their code available to other application developers.
159
160 For information on the eSDK, see the `Yocto Project Application
161 Development and the Extensible Software Development Kit
162 (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
163
164- *Image:* An image is an artifact of the BitBake build process given a
165 collection of recipes and related Metadata. Images are the binary
166 output that run on specific hardware or QEMU and are used for
167 specific use-cases. For a list of the supported image types that the
168 Yocto Project provides, see the "`Images <#ref-images>`__" chapter.
169
170- *Layer:* A collection of related recipes. Layers allow you to
171 consolidate related metadata to customize your build. Layers also
172 isolate information used when building for multiple architectures.
173 Layers are hierarchical in their ability to override previous
174 specifications. You can include any number of available layers from
175 the Yocto Project and customize the build by adding your layers after
176 them. You can search the Layer Index for layers used within Yocto
177 Project.
178
179 For introductory information on layers, see the "`The Yocto Project
180 Layer Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__"
181 section in the Yocto Project Overview and Concepts Manual. For more
182 detailed information on layers, see the "`Understanding and Creating
183 Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
184 section in the Yocto Project Development Tasks Manual. For a
185 discussion specifically on BSP Layers, see the "`BSP
186 Layers <&YOCTO_DOCS_BSP_URL;#bsp-layers>`__" section in the Yocto
187 Project Board Support Packages (BSP) Developer's Guide.
188
189- *Metadata:* A key element of the Yocto Project is the Metadata that
190 is used to construct a Linux distribution and is contained in the
191 files that the `OpenEmbedded build system <#build-system-term>`__
192 parses when building an image. In general, Metadata includes recipes,
193 configuration files, and other information that refers to the build
194 instructions themselves, as well as the data used to control what
195 things get built and the effects of the build. Metadata also includes
196 commands and data used to indicate what versions of software are
197 used, from where they are obtained, and changes or additions to the
198 software itself (patches or auxiliary files) that are used to fix
199 bugs or customize the software for use in a particular situation.
200 OpenEmbedded-Core is an important set of validated metadata.
201
202 In the context of the kernel ("kernel Metadata"), the term refers to
203 the kernel config fragments and features contained in the
204 ```yocto-kernel-cache`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/yocto-kernel-cache>`__
205 Git repository.
206
207- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
208 foundational recipes, classes, and associated files that are meant to
209 be common among many different OpenEmbedded-derived systems,
210 including the Yocto Project. OE-Core is a curated subset of an
211 original repository developed by the OpenEmbedded community that has
212 been pared down into a smaller, core set of continuously validated
213 recipes. The result is a tightly controlled and an quality-assured
214 core set of recipes.
215
216 You can see the Metadata in the ``meta`` directory of the Yocto
217 Project `Source
218 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
219
220- *OpenEmbedded Build System:* The build system specific to the Yocto
221 Project. The OpenEmbedded build system is based on another project
222 known as "Poky", which uses `BitBake <#bitbake-term>`__ as the task
223 executor. Throughout the Yocto Project documentation set, the
224 OpenEmbedded build system is sometimes referred to simply as "the
225 build system". If other build systems, such as a host or target build
226 system are referenced, the documentation clearly states the
227 difference.
228
229 .. note::
230
231 For some historical information about Poky, see the
232 Poky
233 term.
234
235- *Package:* In the context of the Yocto Project, this term refers to a
236 recipe's packaged output produced by BitBake (i.e. a "baked recipe").
237 A package is generally the compiled binaries produced from the
238 recipe's sources. You "bake" something by running it through BitBake.
239
240 It is worth noting that the term "package" can, in general, have
241 subtle meanings. For example, the packages referred to in the
242 "`Required Packages for the Build
243 Host <#required-packages-for-the-build-host>`__" section are compiled
244 binaries that, when installed, add functionality to your Linux
245 distribution.
246
247 Another point worth noting is that historically within the Yocto
248 Project, recipes were referred to as packages - thus, the existence
249 of several BitBake variables that are seemingly mis-named, (e.g.
250 ```PR`` <#var-PR>`__, ```PV`` <#var-PV>`__, and
251 ```PE`` <#var-PE>`__).
252
253- *Package Groups:* Arbitrary groups of software Recipes. You use
254 package groups to hold recipes that, when built, usually accomplish a
255 single task. For example, a package group could contain the recipes
256 for a company’s proprietary or value-add software. Or, the package
257 group could contain the recipes that enable graphics. A package group
258 is really just another recipe. Because package group files are
259 recipes, they end with the ``.bb`` filename extension.
260
261- *Poky:* Poky, which is pronounced *Pock*-ee, is a reference embedded
262 distribution and a reference test configuration. Poky provides the
263 following:
264
265 - A base-level functional distro used to illustrate how to customize
266 a distribution.
267
268 - A means by which to test the Yocto Project components (i.e. Poky
269 is used to validate the Yocto Project).
270
271 - A vehicle through which you can download the Yocto Project.
272
273 Poky is not a product level distro. Rather, it is a good starting
274 point for customization.
275
276 .. note::
277
278 Poky began as an open-source project initially developed by
279 OpenedHand. OpenedHand developed Poky from the existing
280 OpenEmbedded build system to create a commercially supportable
281 build system for embedded Linux. After Intel Corporation acquired
282 OpenedHand, the poky project became the basis for the Yocto
283 Project's build system.
284
285- *Recipe:* A set of instructions for building packages. A recipe
286 describes where you get source code, which patches to apply, how to
287 configure the source, how to compile it and so on. Recipes also
288 describe dependencies for libraries or for other recipes. Recipes
289 represent the logical unit of execution, the software to build, the
290 images to build, and use the ``.bb`` file extension.
291
292- *Reference Kit:* A working example of a system, which includes a
293 `BSP <#board-support-package-bsp-term>`__ as well as a `build
294 host <#hardware-build-system-term>`__ and other components, that can
295 work on specific hardware.
296
297- *Source Directory:* This term refers to the directory structure
298 created as a result of creating a local copy of the ``poky`` Git
299 repository ``git://git.yoctoproject.org/poky`` or expanding a
300 released ``poky`` tarball.
301
302 .. note::
303
304 Creating a local copy of the
305 poky
306 Git repository is the recommended method for setting up your
307 Source Directory.
308
309 Sometimes you might hear the term "poky directory" used to refer to
310 this directory structure.
311
312 .. note::
313
314 The OpenEmbedded build system does not support file or directory
315 names that contain spaces. Be sure that the Source Directory you
316 use does not contain these types of names.
317
318 The Source Directory contains BitBake, Documentation, Metadata and
319 other files that all support the Yocto Project. Consequently, you
320 must have the Source Directory in place on your development system in
321 order to do any development using the Yocto Project.
322
323 When you create a local copy of the Git repository, you can name the
324 repository anything you like. Throughout much of the documentation,
325 "poky" is used as the name of the top-level folder of the local copy
326 of the poky Git repository. So, for example, cloning the ``poky`` Git
327 repository results in a local Git repository whose top-level folder
328 is also named "poky".
329
330 While it is not recommended that you use tarball expansion to set up
331 the Source Directory, if you do, the top-level directory name of the
332 Source Directory is derived from the Yocto Project release tarball.
333 For example, downloading and unpacking ```` results in a Source
334 Directory whose root folder is named ````.
335
336 It is important to understand the differences between the Source
337 Directory created by unpacking a released tarball as compared to
338 cloning ``git://git.yoctoproject.org/poky``. When you unpack a
339 tarball, you have an exact copy of the files based on the time of
340 release - a fixed release point. Any changes you make to your local
341 files in the Source Directory are on top of the release and will
342 remain local only. On the other hand, when you clone the ``poky`` Git
343 repository, you have an active development repository with access to
344 the upstream repository's branches and tags. In this case, any local
345 changes you make to the local Source Directory can be later applied
346 to active development branches of the upstream ``poky`` Git
347 repository.
348
349 For more information on concepts related to Git repositories,
350 branches, and tags, see the "`Repositories, Tags, and
351 Branches <&YOCTO_DOCS_OM_URL;#repositories-tags-and-branches>`__"
352 section in the Yocto Project Overview and Concepts Manual.
353
354- *Task:* A unit of execution for BitBake (e.g.
355 ```do_compile`` <#ref-tasks-compile>`__,
356 ```do_fetch`` <#ref-tasks-fetch>`__,
357 ```do_patch`` <#ref-tasks-patch>`__, and so forth).
358
359- *Toaster:* A web interface to the Yocto Project's `OpenEmbedded Build
360 System <#build-system-term>`__. The interface enables you to
361 configure and run your builds. Information about builds is collected
362 and stored in a database. For information on Toaster, see the
363 `Toaster User Manual <&YOCTO_DOCS_TOAST_URL;>`__.
364
365- *Upstream:* A reference to source code or repositories that are not
366 local to the development system but located in a master area that is
367 controlled by the maintainer of the source code. For example, in
368 order for a developer to work on a particular piece of code, they
369 need to first get a copy of it from an "upstream" source.
diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
new file mode 100644
index 0000000000..3fa1b6ccaa
--- /dev/null
+++ b/documentation/ref-manual/ref-variables.rst
@@ -0,0 +1,7924 @@
1******************
2Variables Glossary
3******************
4
5This chapter lists common variables used in the OpenEmbedded build
6system and gives an overview of their function and contents.
7
8`A <#var-ABIEXTENSION>`__ `B <#var-B>`__ `C <#var-CACHE>`__
9`D <#var-D>`__ `E <#var-EFI_PROVIDER>`__ `F <#var-FEATURE_PACKAGES>`__
10`G <#var-GCCPIE>`__ `H <#var-HOMEPAGE>`__ `I <#var-ICECC_DISABLED>`__
11`K <#var-KARCH>`__ `L <#var-LABELS>`__ `M <#var-MACHINE>`__
12`N <#var-NATIVELSBSTRING>`__ `O <#var-OBJCOPY>`__ `P <#var-P>`__
13`R <#var-RANLIB>`__ `S <#var-S>`__ `T <#var-T>`__
14`U <#var-UBOOT_CONFIG>`__ `V <#var-VOLATILE_LOG_DIR>`__
15`W <#var-WARN_QA>`__ `X <#var-XSERVER>`__
16
17ABIEXTENSION
18 Extension to the Application Binary Interface (ABI) field of the GNU
19 canonical architecture name (e.g. "eabi").
20
21 ABI extensions are set in the machine include files. For example, the
22 ``meta/conf/machine/include/arm/arch-arm.inc`` file sets the
23 following extension: ABIEXTENSION = "eabi"
24
25ALLOW_EMPTY
26 Specifies whether to produce an output package even if it is empty.
27 By default, BitBake does not produce empty packages. This default
28 behavior can cause issues when there is an
29 ```RDEPENDS`` <#var-RDEPENDS>`__ or some other hard runtime
30 requirement on the existence of the package.
31
32 Like all package-controlling variables, you must always use them in
33 conjunction with a package name override, as in: ALLOW_EMPTY_${PN} =
34 "1" ALLOW_EMPTY_${PN}-dev = "1" ALLOW_EMPTY_${PN}-staticdev = "1"
35
36ALTERNATIVE
37 Lists commands in a package that need an alternative binary naming
38 scheme. Sometimes the same command is provided in multiple packages.
39 When this occurs, the OpenEmbedded build system needs to use the
40 alternatives system to create a different binary naming scheme so the
41 commands can co-exist.
42
43 To use the variable, list out the package's commands that also exist
44 as part of another package. For example, if the ``busybox`` package
45 has four commands that also exist as part of another package, you
46 identify them as follows: ALTERNATIVE_busybox = "sh sed test bracket"
47 For more information on the alternatives system, see the
48 "```update-alternatives.bbclass`` <#ref-classes-update-alternatives>`__"
49 section.
50
51ALTERNATIVE_LINK_NAME
52 Used by the alternatives system to map duplicated commands to actual
53 locations. For example, if the ``bracket`` command provided by the
54 ``busybox`` package is duplicated through another package, you must
55 use the ``ALTERNATIVE_LINK_NAME`` variable to specify the actual
56 location: ALTERNATIVE_LINK_NAME[bracket] = "/usr/bin/["
57
58 In this example, the binary for the ``bracket`` command (i.e. ``[``)
59 from the ``busybox`` package resides in ``/usr/bin/``.
60
61 .. note::
62
63 If
64 ALTERNATIVE_LINK_NAME
65 is not defined, it defaults to
66 ${bindir}/
67 name
68 .
69
70 For more information on the alternatives system, see the
71 "```update-alternatives.bbclass`` <#ref-classes-update-alternatives>`__"
72 section.
73
74ALTERNATIVE_PRIORITY
75 Used by the alternatives system to create default priorities for
76 duplicated commands. You can use the variable to create a single
77 default regardless of the command name or package, a default for
78 specific duplicated commands regardless of the package, or a default
79 for specific commands tied to particular packages. Here are the
80 available syntax forms: ALTERNATIVE_PRIORITY = "priority"
81 ALTERNATIVE_PRIORITY[name] = "priority"
82 ALTERNATIVE_PRIORITY_pkg[name] = "priority"
83
84 For more information on the alternatives system, see the
85 "```update-alternatives.bbclass`` <#ref-classes-update-alternatives>`__"
86 section.
87
88ALTERNATIVE_TARGET
89 Used by the alternatives system to create default link locations for
90 duplicated commands. You can use the variable to create a single
91 default location for all duplicated commands regardless of the
92 command name or package, a default for specific duplicated commands
93 regardless of the package, or a default for specific commands tied to
94 particular packages. Here are the available syntax forms:
95 ALTERNATIVE_TARGET = "target" ALTERNATIVE_TARGET[name] = "target"
96 ALTERNATIVE_TARGET_pkg[name] = "target"
97
98 .. note::
99
100 If ``ALTERNATIVE_TARGET`` is not defined, it inherits the value
101 from the
102 ```ALTERNATIVE_LINK_NAME`` <#var-ALTERNATIVE_LINK_NAME>`__
103 variable.
104
105 If ``ALTERNATIVE_LINK_NAME`` and ``ALTERNATIVE_TARGET`` are the
106 same, the target for ``ALTERNATIVE_TARGET`` has "``.{BPN}``"
107 appended to it.
108
109 Finally, if the file referenced has not been renamed, the
110 alternatives system will rename it to avoid the need to rename
111 alternative files in the ```do_install`` <#ref-tasks-install>`__
112 task while retaining support for the command if necessary.
113
114 For more information on the alternatives system, see the
115 "```update-alternatives.bbclass`` <#ref-classes-update-alternatives>`__"
116 section.
117
118APPEND
119 An override list of append strings for each target specified with
120 ```LABELS`` <#var-LABELS>`__.
121
122 See the ```grub-efi`` <#ref-classes-grub-efi>`__ class for more
123 information on how this variable is used.
124
125AR
126 The minimal command and arguments used to run ``ar``.
127
128ARCHIVER_MODE
129 When used with the ```archiver`` <#ref-classes-archiver>`__ class,
130 determines the type of information used to create a released archive.
131 You can use this variable to create archives of patched source,
132 original source, configured source, and so forth by employing the
133 following variable flags (varflags): ARCHIVER_MODE[src] = "original"
134 # Uses original (unpacked) source # files. ARCHIVER_MODE[src] =
135 "patched" # Uses patched source files. This is # the default.
136 ARCHIVER_MODE[src] = "configured" # Uses configured source files.
137 ARCHIVER_MODE[diff] = "1" # Uses patches between do_unpack and #
138 do_patch. ARCHIVER_MODE[diff-exclude] ?= "file file ..." # Lists
139 files and directories to # exclude from diff. ARCHIVER_MODE[dumpdata]
140 = "1" # Uses environment data. ARCHIVER_MODE[recipe] = "1" # Uses
141 recipe and include files. ARCHIVER_MODE[srpm] = "1" # Uses RPM
142 package files. For information on how the variable works, see the
143 ``meta/classes/archiver.bbclass`` file in the `Source
144 Directory <#source-directory>`__.
145
146AS
147 Minimal command and arguments needed to run the assembler.
148
149ASSUME_PROVIDED
150 Lists recipe names (```PN`` <#var-PN>`__ values) BitBake does not
151 attempt to build. Instead, BitBake assumes these recipes have already
152 been built.
153
154 In OpenEmbedded-Core, ``ASSUME_PROVIDED`` mostly specifies native
155 tools that should not be built. An example is ``git-native``, which
156 when specified, allows for the Git binary from the host to be used
157 rather than building ``git-native``.
158
159ASSUME_SHLIBS
160 Provides additional ``shlibs`` provider mapping information, which
161 adds to or overwrites the information provided automatically by the
162 system. Separate multiple entries using spaces.
163
164 As an example, use the following form to add an ``shlib`` provider of
165 shlibname in packagename with the optional version:
166 shlibname:packagename[_version]
167
168 Here is an example that adds a shared library named ``libEGL.so.1``
169 as being provided by the ``libegl-implementation`` package:
170 ASSUME_SHLIBS = "libEGL.so.1:libegl-implementation"
171
172AUTHOR
173 The email address used to contact the original author or authors in
174 order to send patches and forward bugs.
175
176AUTO_LIBNAME_PKGS
177 When the ```debian`` <#ref-classes-debian>`__ class is inherited,
178 which is the default behavior, ``AUTO_LIBNAME_PKGS`` specifies which
179 packages should be checked for libraries and renamed according to
180 Debian library package naming.
181
182 The default value is "${PACKAGES}", which causes the debian class to
183 act on all packages that are explicitly generated by the recipe.
184
185AUTO_SYSLINUXMENU
186 Enables creating an automatic menu for the syslinux bootloader. You
187 must set this variable in your recipe. The
188 ```syslinux`` <#ref-classes-syslinux>`__ class checks this variable.
189
190AUTOREV
191 When ``SRCREV`` is set to the value of this variable, it specifies to
192 use the latest source revision in the repository. Here is an example:
193 SRCREV = "${AUTOREV}"
194
195 If you use the previous statement to retrieve the latest version of
196 software, you need to be sure ```PV`` <#var-PV>`__ contains
197 ``${``\ ```SRCPV`` <#var-SRCPV>`__\ ``}``. For example, suppose you
198 have a kernel recipe that inherits the
199 `kernel <#ref-classes-kernel>`__ class and you use the previous
200 statement. In this example, ``${SRCPV}`` does not automatically get
201 into ``PV``. Consequently, you need to change ``PV`` in your recipe
202 so that it does contain ``${SRCPV}``.
203
204 For more information see the "`Automatically Incrementing a Binary
205 Package Revision
206 Number <&YOCTO_DOCS_DEV_URL;#automatically-incrementing-a-binary-package-revision-number>`__"
207 section in the Yocto Project Development Tasks Manual.
208
209AVAILABLE_LICENSES
210 List of licenses found in the directories specified by
211 ```COMMON_LICENSE_DIR`` <#var-COMMON_LICENSE_DIR>`__ and
212 ```LICENSE_PATH`` <#var-LICENSE_PATH>`__.
213
214 .. note::
215
216 It is assumed that all changes to
217 COMMON_LICENSE_DIR
218 and
219 LICENSE_PATH
220 have been done before
221 AVAILABLE_LICENSES
222 is defined (in
223 license.bbclass
224 ).
225
226AVAILTUNES
227 The list of defined CPU and Application Binary Interface (ABI)
228 tunings (i.e. "tunes") available for use by the OpenEmbedded build
229 system.
230
231 The list simply presents the tunes that are available. Not all tunes
232 may be compatible with a particular machine configuration, or with
233 each other in a
234 `Multilib <&YOCTO_DOCS_DEV_URL;#combining-multiple-versions-library-files-into-one-image>`__
235 configuration.
236
237 To add a tune to the list, be sure to append it with spaces using the
238 "+=" BitBake operator. Do not simply replace the list by using the
239 "=" operator. See the "`Basic
240 Syntax <&YOCTO_DOCS_BB_URL;#basic-syntax>`__" section in the BitBake
241 User Manual for more information.
242
243B
244 The directory within the `Build Directory <#build-directory>`__ in
245 which the OpenEmbedded build system places generated objects during a
246 recipe's build process. By default, this directory is the same as the
247 ```S`` <#var-S>`__ directory, which is defined as: S =
248 "${WORKDIR}/${BP}"
249
250 You can separate the (``S``) directory and the directory pointed to
251 by the ``B`` variable. Most Autotools-based recipes support
252 separating these directories. The build system defaults to using
253 separate directories for ``gcc`` and some kernel recipes.
254
255BAD_RECOMMENDATIONS
256 Lists "recommended-only" packages to not install. Recommended-only
257 packages are packages installed only through the
258 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__ variable. You can prevent any
259 of these "recommended" packages from being installed by listing them
260 with the ``BAD_RECOMMENDATIONS`` variable: BAD_RECOMMENDATIONS =
261 "package_name package_name package_name ..."
262
263 You can set this variable globally in your ``local.conf`` file or you
264 can attach it to a specific image recipe by using the recipe name
265 override: BAD_RECOMMENDATIONS_pn-target_image = "package_name"
266
267 It is important to realize that if you choose to not install packages
268 using this variable and some other packages are dependent on them
269 (i.e. listed in a recipe's ```RDEPENDS`` <#var-RDEPENDS>`__
270 variable), the OpenEmbedded build system ignores your request and
271 will install the packages to avoid dependency errors.
272
273 Support for this variable exists only when using the IPK and RPM
274 packaging backend. Support does not exist for DEB.
275
276 See the ```NO_RECOMMENDATIONS`` <#var-NO_RECOMMENDATIONS>`__ and the
277 ```PACKAGE_EXCLUDE`` <#var-PACKAGE_EXCLUDE>`__ variables for related
278 information.
279
280BASE_LIB
281 The library directory name for the CPU or Application Binary
282 Interface (ABI) tune. The ``BASE_LIB`` applies only in the Multilib
283 context. See the "`Combining Multiple Versions of Library Files into
284 One
285 Image <&YOCTO_DOCS_DEV_URL;#combining-multiple-versions-library-files-into-one-image>`__"
286 section in the Yocto Project Development Tasks Manual for information
287 on Multilib.
288
289 The ``BASE_LIB`` variable is defined in the machine include files in
290 the `Source Directory <#source-directory>`__. If Multilib is not
291 being used, the value defaults to "lib".
292
293BASE_WORKDIR
294 Points to the base of the work directory for all recipes. The default
295 value is "${TMPDIR}/work".
296
297BB_ALLOWED_NETWORKS
298 Specifies a space-delimited list of hosts that the fetcher is allowed
299 to use to obtain the required source code. Following are
300 considerations surrounding this variable:
301
302 - This host list is only used if ``BB_NO_NETWORK`` is either not set
303 or set to "0".
304
305 - Limited support for wildcard matching against the beginning of
306 host names exists. For example, the following setting matches
307 ``git.gnu.org``, ``ftp.gnu.org``, and ``foo.git.gnu.org``.
308 BB_ALLOWED_NETWORKS = "*.gnu.org"
309
310 .. note::
311
312 The use of the "``*``" character only works at the beginning of
313 a host name and it must be isolated from the remainder of the
314 host name. You cannot use the wildcard character in any other
315 location of the name or combined with the front part of the
316 name.
317
318 For example, ``*.foo.bar`` is supported, while ``*aa.foo.bar``
319 is not.
320
321 - Mirrors not in the host list are skipped and logged in debug.
322
323 - Attempts to access networks not in the host list cause a failure.
324
325 Using ``BB_ALLOWED_NETWORKS`` in conjunction with
326 ```PREMIRRORS`` <#var-PREMIRRORS>`__ is very useful. Adding the host
327 you want to use to ``PREMIRRORS`` results in the source code being
328 fetched from an allowed location and avoids raising an error when a
329 host that is not allowed is in a ```SRC_URI`` <#var-SRC_URI>`__
330 statement. This is because the fetcher does not attempt to use the
331 host listed in ``SRC_URI`` after a successful fetch from the
332 ``PREMIRRORS`` occurs.
333
334BB_DANGLINGAPPENDS_WARNONLY
335 Defines how BitBake handles situations where an append file
336 (``.bbappend``) has no corresponding recipe file (``.bb``). This
337 condition often occurs when layers get out of sync (e.g. ``oe-core``
338 bumps a recipe version and the old recipe no longer exists and the
339 other layer has not been updated to the new version of the recipe
340 yet).
341
342 The default fatal behavior is safest because it is the sane reaction
343 given something is out of sync. It is important to realize when your
344 changes are no longer being applied.
345
346 You can change the default behavior by setting this variable to "1",
347 "yes", or "true" in your ``local.conf`` file, which is located in the
348 `Build Directory <#build-directory>`__: Here is an example:
349 BB_DANGLINGAPPENDS_WARNONLY = "1"
350
351BB_DISKMON_DIRS
352 Monitors disk space and available inodes during the build and allows
353 you to control the build based on these parameters.
354
355 Disk space monitoring is disabled by default. To enable monitoring,
356 add the ``BB_DISKMON_DIRS`` variable to your ``conf/local.conf`` file
357 found in the `Build Directory <#build-directory>`__. Use the
358 following form: BB_DISKMON_DIRS = "action,dir,threshold [...]" where:
359 action is: ABORT: Immediately abort the build when a threshold is
360 broken. STOPTASKS: Stop the build after the currently executing tasks
361 have finished when a threshold is broken. WARN: Issue a warning but
362 continue the build when a threshold is broken. Subsequent warnings
363 are issued as defined by the BB_DISKMON_WARNINTERVAL variable, which
364 must be defined in the conf/local.conf file. dir is: Any directory
365 you choose. You can specify one or more directories to monitor by
366 separating the groupings with a space. If two directories are on the
367 same device, only the first directory is monitored. threshold is:
368 Either the minimum available disk space, the minimum number of free
369 inodes, or both. You must specify at least one. To omit one or the
370 other, simply omit the value. Specify the threshold using G, M, K for
371 Gbytes, Mbytes, and Kbytes, respectively. If you do not specify G, M,
372 or K, Kbytes is assumed by default. Do not use GB, MB, or KB.
373
374 Here are some examples: BB_DISKMON_DIRS = "ABORT,${TMPDIR},1G,100K
375 WARN,${SSTATE_DIR},1G,100K" BB_DISKMON_DIRS =
376 "STOPTASKS,${TMPDIR},1G" BB_DISKMON_DIRS = "ABORT,${TMPDIR},,100K"
377 The first example works only if you also provide the
378 ```BB_DISKMON_WARNINTERVAL`` <#var-BB_DISKMON_WARNINTERVAL>`__
379 variable in the ``conf/local.conf``. This example causes the build
380 system to immediately abort when either the disk space in
381 ``${TMPDIR}`` drops below 1 Gbyte or the available free inodes drops
382 below 100 Kbytes. Because two directories are provided with the
383 variable, the build system also issue a warning when the disk space
384 in the ``${SSTATE_DIR}`` directory drops below 1 Gbyte or the number
385 of free inodes drops below 100 Kbytes. Subsequent warnings are issued
386 during intervals as defined by the ``BB_DISKMON_WARNINTERVAL``
387 variable.
388
389 The second example stops the build after all currently executing
390 tasks complete when the minimum disk space in the ``${TMPDIR}``
391 directory drops below 1 Gbyte. No disk monitoring occurs for the free
392 inodes in this case.
393
394 The final example immediately aborts the build when the number of
395 free inodes in the ``${TMPDIR}`` directory drops below 100 Kbytes. No
396 disk space monitoring for the directory itself occurs in this case.
397
398BB_DISKMON_WARNINTERVAL
399 Defines the disk space and free inode warning intervals. To set these
400 intervals, define the variable in your ``conf/local.conf`` file in
401 the `Build Directory <#build-directory>`__.
402
403 If you are going to use the ``BB_DISKMON_WARNINTERVAL`` variable, you
404 must also use the ```BB_DISKMON_DIRS`` <#var-BB_DISKMON_DIRS>`__
405 variable and define its action as "WARN". During the build,
406 subsequent warnings are issued each time disk space or number of free
407 inodes further reduces by the respective interval.
408
409 If you do not provide a ``BB_DISKMON_WARNINTERVAL`` variable and you
410 do use ``BB_DISKMON_DIRS`` with the "WARN" action, the disk
411 monitoring interval defaults to the following:
412 BB_DISKMON_WARNINTERVAL = "50M,5K"
413
414 When specifying the variable in your configuration file, use the
415 following form: BB_DISKMON_WARNINTERVAL =
416 "disk_space_interval,disk_inode_interval" where: disk_space_interval
417 is: An interval of memory expressed in either G, M, or K for Gbytes,
418 Mbytes, or Kbytes, respectively. You cannot use GB, MB, or KB.
419 disk_inode_interval is: An interval of free inodes expressed in
420 either G, M, or K for Gbytes, Mbytes, or Kbytes, respectively. You
421 cannot use GB, MB, or KB.
422
423 Here is an example: BB_DISKMON_DIRS = "WARN,${SSTATE_DIR},1G,100K"
424 BB_DISKMON_WARNINTERVAL = "50M,5K" These variables cause the
425 OpenEmbedded build system to issue subsequent warnings each time the
426 available disk space further reduces by 50 Mbytes or the number of
427 free inodes further reduces by 5 Kbytes in the ``${SSTATE_DIR}``
428 directory. Subsequent warnings based on the interval occur each time
429 a respective interval is reached beyond the initial warning (i.e. 1
430 Gbytes and 100 Kbytes).
431
432BB_GENERATE_MIRROR_TARBALLS
433 Causes tarballs of the source control repositories (e.g. Git
434 repositories), including metadata, to be placed in the
435 ```DL_DIR`` <#var-DL_DIR>`__ directory.
436
437 For performance reasons, creating and placing tarballs of these
438 repositories is not the default action by the OpenEmbedded build
439 system. BB_GENERATE_MIRROR_TARBALLS = "1" Set this variable in your
440 ``local.conf`` file in the `Build Directory <#build-directory>`__.
441
442 Once you have the tarballs containing your source files, you can
443 clean up your ``DL_DIR`` directory by deleting any Git or other
444 source control work directories.
445
446BB_NUMBER_THREADS
447 The maximum number of tasks BitBake should run in parallel at any one
448 time. The OpenEmbedded build system automatically configures this
449 variable to be equal to the number of cores on the build system. For
450 example, a system with a dual core processor that also uses
451 hyper-threading causes the ``BB_NUMBER_THREADS`` variable to default
452 to "4".
453
454 For single socket systems (i.e. one CPU), you should not have to
455 override this variable to gain optimal parallelism during builds.
456 However, if you have very large systems that employ multiple physical
457 CPUs, you might want to make sure the ``BB_NUMBER_THREADS`` variable
458 is not set higher than "20".
459
460 For more information on speeding up builds, see the "`Speeding Up a
461 Build <&YOCTO_DOCS_DEV_URL;#speeding-up-a-build>`__" section in the
462 Yocto Project Development Tasks Manual.
463
464BB_SERVER_TIMEOUT
465 Specifies the time (in seconds) after which to unload the BitBake
466 server due to inactivity. Set ``BB_SERVER_TIMEOUT`` to determine how
467 long the BitBake server stays resident between invocations.
468
469 For example, the following statement in your ``local.conf`` file
470 instructs the server to be unloaded after 20 seconds of inactivity:
471 BB_SERVER_TIMEOUT = "20" If you want the server to never be unloaded,
472 set ``BB_SERVER_TIMEOUT`` to "-1".
473
474BBCLASSEXTEND
475 Allows you to extend a recipe so that it builds variants of the
476 software. Common variants for recipes exist such as "natives" like
477 ``quilt-native``, which is a copy of Quilt built to run on the build
478 system; "crosses" such as ``gcc-cross``, which is a compiler built to
479 run on the build machine but produces binaries that run on the target
480 ```MACHINE`` <#var-MACHINE>`__; "nativesdk", which targets the SDK
481 machine instead of ``MACHINE``; and "mulitlibs" in the form
482 "``multilib:``\ multilib_name".
483
484 To build a different variant of the recipe with a minimal amount of
485 code, it usually is as simple as adding the following to your recipe:
486 BBCLASSEXTEND =+ "native nativesdk" BBCLASSEXTEND =+
487 "multilib:multilib_name"
488
489 .. note::
490
491 Internally, the ``BBCLASSEXTEND`` mechanism generates recipe
492 variants by rewriting variable values and applying overrides such
493 as ``_class-native``. For example, to generate a native version of
494 a recipe, a ```DEPENDS`` <#var-DEPENDS>`__ on "foo" is rewritten
495 to a ``DEPENDS`` on "foo-native".
496
497 Even when using ``BBCLASSEXTEND``, the recipe is only parsed once.
498 Parsing once adds some limitations. For example, it is not
499 possible to include a different file depending on the variant,
500 since ``include`` statements are processed when the recipe is
501 parsed.
502
503BBFILE_COLLECTIONS
504 Lists the names of configured layers. These names are used to find
505 the other ``BBFILE_*`` variables. Typically, each layer will append
506 its name to this variable in its ``conf/layer.conf`` file.
507
508BBFILE_PATTERN
509 Variable that expands to match files from
510 ```BBFILES`` <#var-BBFILES>`__ in a particular layer. This variable
511 is used in the ``conf/layer.conf`` file and must be suffixed with the
512 name of the specific layer (e.g. ``BBFILE_PATTERN_emenlow``).
513
514BBFILE_PRIORITY
515 Assigns the priority for recipe files in each layer.
516
517 This variable is useful in situations where the same recipe appears
518 in more than one layer. Setting this variable allows you to
519 prioritize a layer against other layers that contain the same recipe
520 - effectively letting you control the precedence for the multiple
521 layers. The precedence established through this variable stands
522 regardless of a recipe's version (```PV`` <#var-PV>`__ variable). For
523 example, a layer that has a recipe with a higher ``PV`` value but for
524 which the ``BBFILE_PRIORITY`` is set to have a lower precedence still
525 has a lower precedence.
526
527 A larger value for the ``BBFILE_PRIORITY`` variable results in a
528 higher precedence. For example, the value 6 has a higher precedence
529 than the value 5. If not specified, the ``BBFILE_PRIORITY`` variable
530 is set based on layer dependencies (see the ``LAYERDEPENDS`` variable
531 for more information. The default priority, if unspecified for a
532 layer with no dependencies, is the lowest defined priority + 1 (or 1
533 if no priorities are defined).
534
535 .. tip::
536
537 You can use the command
538 bitbake-layers show-layers
539 to list all configured layers along with their priorities.
540
541BBFILES
542 A space-separated list of recipe files BitBake uses to build
543 software.
544
545 When specifying recipe files, you can pattern match using Python's
546 ```glob`` <https://docs.python.org/3/library/glob.html>`__ syntax.
547 For details on the syntax, see the documentation by following the
548 previous link.
549
550BBFILES_DYNAMIC
551 Activates content when identified layers are present. You identify
552 the layers by the collections that the layers define.
553
554 Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files
555 whose corresponding ``.bb`` file is in a layer that attempts to
556 modify other layers through ``.bbappend`` but does not want to
557 introduce a hard dependency on those other layers.
558
559 Use the following form for ``BBFILES_DYNAMIC``:
560 collection_name:filename_pattern The following example identifies two
561 collection names and two filename patterns: BBFILES_DYNAMIC += " \\
562 clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \\
563 core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \\ "
564 This next example shows an error message that occurs because invalid
565 entries are found, which cause parsing to abort: ERROR:
566 BBFILES_DYNAMIC entries must be of the form <collection
567 name>:<filename pattern>, not:
568 /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend
569 /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend
570
571BBINCLUDELOGS
572 Variable that controls how BitBake displays logs on build failure.
573
574BBINCLUDELOGS_LINES
575 If ```BBINCLUDELOGS`` <#var-BBINCLUDELOGS>`__ is set, specifies the
576 maximum number of lines from the task log file to print when
577 reporting a failed task. If you do not set ``BBINCLUDELOGS_LINES``,
578 the entire log is printed.
579
580BBLAYERS
581 Lists the layers to enable during the build. This variable is defined
582 in the ``bblayers.conf`` configuration file in the `Build
583 Directory <#build-directory>`__. Here is an example: BBLAYERS = " \\
584 /home/scottrif/poky/meta \\ /home/scottrif/poky/meta-poky \\
585 /home/scottrif/poky/meta-yocto-bsp \\
586 /home/scottrif/poky/meta-mykernel \\ "
587
588 This example enables four layers, one of which is a custom,
589 user-defined layer named ``meta-mykernel``.
590
591BBMASK
592 Prevents BitBake from processing recipes and recipe append files.
593
594 You can use the ``BBMASK`` variable to "hide" these ``.bb`` and
595 ``.bbappend`` files. BitBake ignores any recipe or recipe append
596 files that match any of the expressions. It is as if BitBake does not
597 see them at all. Consequently, matching files are not parsed or
598 otherwise used by BitBake.
599
600 The values you provide are passed to Python's regular expression
601 compiler. Consequently, the syntax follows Python's Regular
602 Expression (re) syntax. The expressions are compared against the full
603 paths to the files. For complete syntax information, see Python's
604 documentation at ` <http://docs.python.org/3/library/re.html#re>`__.
605
606 The following example uses a complete regular expression to tell
607 BitBake to ignore all recipe and recipe append files in the
608 ``meta-ti/recipes-misc/`` directory: BBMASK = "meta-ti/recipes-misc/"
609 If you want to mask out multiple directories or recipes, you can
610 specify multiple regular expression fragments. This next example
611 masks out multiple directories and individual recipes: BBMASK +=
612 "/meta-ti/recipes-misc/ meta-ti/recipes-ti/packagegroup/" BBMASK +=
613 "/meta-oe/recipes-support/" BBMASK += "/meta-foo/.*/openldap" BBMASK
614 += "opencv.*\.bbappend" BBMASK += "lzma"
615
616 .. note::
617
618 When specifying a directory name, use the trailing slash character
619 to ensure you match just that directory name.
620
621BBMULTICONFIG
622 Specifies each additional separate configuration when you are
623 building targets with multiple configurations. Use this variable in
624 your ``conf/local.conf`` configuration file. Specify a
625 multiconfigname for each configuration file you are using. For
626 example, the following line specifies three configuration files:
627 BBMULTICONFIG = "configA configB configC" Each configuration file you
628 use must reside in the `Build Directory <#build-directory>`__
629 ``conf/multiconfig`` directory (e.g.
630 build_directory\ ``/conf/multiconfig/configA.conf``).
631
632 For information on how to use ``BBMULTICONFIG`` in an environment
633 that supports building targets with multiple configurations, see the
634 "`Building Images for Multiple Targets Using Multiple
635 Configurations <&YOCTO_DOCS_DEV_URL;#dev-building-images-for-multiple-targets-using-multiple-configurations>`__"
636 section in the Yocto Project Development Tasks Manual.
637
638BBPATH
639 Used by BitBake to locate ``.bbclass`` and configuration files. This
640 variable is analogous to the ``PATH`` variable.
641
642 .. note::
643
644 If you run BitBake from a directory outside of the
645 Build Directory
646 , you must be sure to set
647 BBPATH
648 to point to the Build Directory. Set the variable as you would any
649 environment variable and then run BitBake:
650 ::
651
652 $ BBPATH = "build_directory"
653 $ export BBPATH
654 $ bitbake target
655
656
657BBSERVER
658 If defined in the BitBake environment, ``BBSERVER`` points to the
659 BitBake remote server.
660
661 Use the following format to export the variable to the BitBake
662 environment: export BBSERVER=localhost:$port
663
664 By default, ``BBSERVER`` also appears in
665 ```BB_HASHBASE_WHITELIST`` <&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST>`__.
666 Consequently, ``BBSERVER`` is excluded from checksum and dependency
667 data.
668
669BINCONFIG
670 When inheriting the
671 ```binconfig-disabled`` <#ref-classes-binconfig-disabled>`__ class,
672 this variable specifies binary configuration scripts to disable in
673 favor of using ``pkg-config`` to query the information. The
674 ``binconfig-disabled`` class will modify the specified scripts to
675 return an error so that calls to them can be easily found and
676 replaced.
677
678 To add multiple scripts, separate them by spaces. Here is an example
679 from the ``libpng`` recipe: BINCONFIG = "${bindir}/libpng-config
680 ${bindir}/libpng16-config"
681
682BINCONFIG_GLOB
683 When inheriting the ```binconfig`` <#ref-classes-binconfig>`__ class,
684 this variable specifies a wildcard for configuration scripts that
685 need editing. The scripts are edited to correct any paths that have
686 been set up during compilation so that they are correct for use when
687 installed into the sysroot and called by the build processes of other
688 recipes.
689
690 .. note::
691
692 The
693 BINCONFIG_GLOB
694 variable uses
695 shell globbing
696 , which is recognition and expansion of wildcards during pattern
697 matching. Shell globbing is very similar to
698 fnmatch
699 and
700 glob
701 .
702
703 For more information on how this variable works, see
704 ``meta/classes/binconfig.bbclass`` in the `Source
705 Directory <#source-directory>`__. You can also find general
706 information on the class in the
707 "```binconfig.bbclass`` <#ref-classes-binconfig>`__" section.
708
709BP
710 The base recipe name and version but without any special recipe name
711 suffix (i.e. ``-native``, ``lib64-``, and so forth). ``BP`` is
712 comprised of the following: ${BPN}-${PV}
713
714BPN
715 This variable is a version of the ```PN`` <#var-PN>`__ variable with
716 common prefixes and suffixes removed, such as ``nativesdk-``,
717 ``-cross``, ``-native``, and multilib's ``lib64-`` and ``lib32-``.
718 The exact lists of prefixes and suffixes removed are specified by the
719 ```MLPREFIX`` <#var-MLPREFIX>`__ and
720 ```SPECIAL_PKGSUFFIX`` <#var-SPECIAL_PKGSUFFIX>`__ variables,
721 respectively.
722
723BUGTRACKER
724 Specifies a URL for an upstream bug tracking website for a recipe.
725 The OpenEmbedded build system does not use this variable. Rather, the
726 variable is a useful pointer in case a bug in the software being
727 built needs to be manually reported.
728
729BUILD_ARCH
730 Specifies the architecture of the build host (e.g. ``i686``). The
731 OpenEmbedded build system sets the value of ``BUILD_ARCH`` from the
732 machine name reported by the ``uname`` command.
733
734BUILD_AS_ARCH
735 Specifies the architecture-specific assembler flags for the build
736 host. By default, the value of ``BUILD_AS_ARCH`` is empty.
737
738BUILD_CC_ARCH
739 Specifies the architecture-specific C compiler flags for the build
740 host. By default, the value of ``BUILD_CC_ARCH`` is empty.
741
742BUILD_CCLD
743 Specifies the linker command to be used for the build host when the C
744 compiler is being used as the linker. By default, ``BUILD_CCLD``
745 points to GCC and passes as arguments the value of
746 ```BUILD_CC_ARCH`` <#var-BUILD_CC_ARCH>`__, assuming
747 ``BUILD_CC_ARCH`` is set.
748
749BUILD_CFLAGS
750 Specifies the flags to pass to the C compiler when building for the
751 build host. When building in the ``-native`` context,
752 ```CFLAGS`` <#var-CFLAGS>`__ is set to the value of this variable by
753 default.
754
755BUILD_CPPFLAGS
756 Specifies the flags to pass to the C preprocessor (i.e. to both the C
757 and the C++ compilers) when building for the build host. When
758 building in the ``-native`` context, ```CPPFLAGS`` <#var-CPPFLAGS>`__
759 is set to the value of this variable by default.
760
761BUILD_CXXFLAGS
762 Specifies the flags to pass to the C++ compiler when building for the
763 build host. When building in the ``-native`` context,
764 ```CXXFLAGS`` <#var-CXXFLAGS>`__ is set to the value of this variable
765 by default.
766
767BUILD_FC
768 Specifies the Fortran compiler command for the build host. By
769 default, ``BUILD_FC`` points to Gfortran and passes as arguments the
770 value of ```BUILD_CC_ARCH`` <#var-BUILD_CC_ARCH>`__, assuming
771 ``BUILD_CC_ARCH`` is set.
772
773BUILD_LD
774 Specifies the linker command for the build host. By default,
775 ``BUILD_LD`` points to the GNU linker (ld) and passes as arguments
776 the value of ```BUILD_LD_ARCH`` <#var-BUILD_LD_ARCH>`__, assuming
777 ``BUILD_LD_ARCH`` is set.
778
779BUILD_LD_ARCH
780 Specifies architecture-specific linker flags for the build host. By
781 default, the value of ``BUILD_LD_ARCH`` is empty.
782
783BUILD_LDFLAGS
784 Specifies the flags to pass to the linker when building for the build
785 host. When building in the ``-native`` context,
786 ```LDFLAGS`` <#var-LDFLAGS>`__ is set to the value of this variable
787 by default.
788
789BUILD_OPTIMIZATION
790 Specifies the optimization flags passed to the C compiler when
791 building for the build host or the SDK. The flags are passed through
792 the ```BUILD_CFLAGS`` <#var-BUILD_CFLAGS>`__ and
793 ```BUILDSDK_CFLAGS`` <#var-BUILDSDK_CFLAGS>`__ default values.
794
795 The default value of the ``BUILD_OPTIMIZATION`` variable is "-O2
796 -pipe".
797
798BUILD_OS
799 Specifies the operating system in use on the build host (e.g.
800 "linux"). The OpenEmbedded build system sets the value of
801 ``BUILD_OS`` from the OS reported by the ``uname`` command - the
802 first word, converted to lower-case characters.
803
804BUILD_PREFIX
805 The toolchain binary prefix used for native recipes. The OpenEmbedded
806 build system uses the ``BUILD_PREFIX`` value to set the
807 ```TARGET_PREFIX`` <#var-TARGET_PREFIX>`__ when building for
808 ``native`` recipes.
809
810BUILD_STRIP
811 Specifies the command to be used to strip debugging symbols from
812 binaries produced for the build host. By default, ``BUILD_STRIP``
813 points to
814 ``${``\ ```BUILD_PREFIX`` <#var-BUILD_PREFIX>`__\ ``}strip``.
815
816BUILD_SYS
817 Specifies the system, including the architecture and the operating
818 system, to use when building for the build host (i.e. when building
819 ``native`` recipes).
820
821 The OpenEmbedded build system automatically sets this variable based
822 on ```BUILD_ARCH`` <#var-BUILD_ARCH>`__,
823 ```BUILD_VENDOR`` <#var-BUILD_VENDOR>`__, and
824 ```BUILD_OS`` <#var-BUILD_OS>`__. You do not need to set the
825 ``BUILD_SYS`` variable yourself.
826
827BUILD_VENDOR
828 Specifies the vendor name to use when building for the build host.
829 The default value is an empty string ("").
830
831BUILDDIR
832 Points to the location of the `Build Directory <#build-directory>`__.
833 You can define this directory indirectly through the
834 ````` <#structure-core-script>`__ script by passing in a Build
835 Directory path when you run the script. If you run the script and do
836 not provide a Build Directory path, the ``BUILDDIR`` defaults to
837 ``build`` in the current directory.
838
839BUILDHISTORY_COMMIT
840 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
841 class, this variable specifies whether or not to commit the build
842 history output in a local Git repository. If set to "1", this local
843 repository will be maintained automatically by the ``buildhistory``
844 class and a commit will be created on every build for changes to each
845 top-level subdirectory of the build history output (images, packages,
846 and sdk). If you want to track changes to build history over time,
847 you should set this value to "1".
848
849 By default, the ``buildhistory`` class does not commit the build
850 history output in a local Git repository: BUILDHISTORY_COMMIT ?= "0"
851
852BUILDHISTORY_COMMIT_AUTHOR
853 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
854 class, this variable specifies the author to use for each Git commit.
855 In order for the ``BUILDHISTORY_COMMIT_AUTHOR`` variable to work, the
856 ```BUILDHISTORY_COMMIT`` <#var-BUILDHISTORY_COMMIT>`__ variable must
857 be set to "1".
858
859 Git requires that the value you provide for the
860 ``BUILDHISTORY_COMMIT_AUTHOR`` variable takes the form of "name
861 email@host". Providing an email address or host that is not valid
862 does not produce an error.
863
864 By default, the ``buildhistory`` class sets the variable as follows:
865 BUILDHISTORY_COMMIT_AUTHOR ?= "buildhistory <buildhistory@${DISTRO}>"
866
867BUILDHISTORY_DIR
868 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
869 class, this variable specifies the directory in which build history
870 information is kept. For more information on how the variable works,
871 see the ``buildhistory.class``.
872
873 By default, the ``buildhistory`` class sets the directory as follows:
874 BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
875
876BUILDHISTORY_FEATURES
877 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
878 class, this variable specifies the build history features to be
879 enabled. For more information on how build history works, see the
880 "`Maintaining Build Output
881 Quality <&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality>`__"
882 section in the Yocto Project Development Tasks Manual.
883
884 You can specify these features in the form of a space-separated list:
885
886 - *image:* Analysis of the contents of images, which includes the
887 list of installed packages among other things.
888
889 - *package:* Analysis of the contents of individual packages.
890
891 - *sdk:* Analysis of the contents of the software development kit
892 (SDK).
893
894 - *task:* Save output file signatures for `shared
895 state <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__ (sstate) tasks.
896 This saves one file per task and lists the SHA-256 checksums for
897 each file staged (i.e. the output of the task).
898
899 By default, the ``buildhistory`` class enables the following
900 features: BUILDHISTORY_FEATURES ?= "image package sdk"
901
902BUILDHISTORY_IMAGE_FILES
903 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
904 class, this variable specifies a list of paths to files copied from
905 the image contents into the build history directory under an
906 "image-files" directory in the directory for the image, so that you
907 can track the contents of each file. The default is to copy
908 ``/etc/passwd`` and ``/etc/group``, which allows you to monitor for
909 changes in user and group entries. You can modify the list to include
910 any file. Specifying an invalid path does not produce an error.
911 Consequently, you can include files that might not always be present.
912
913 By default, the ``buildhistory`` class provides paths to the
914 following files: BUILDHISTORY_IMAGE_FILES ?= "/etc/passwd /etc/group"
915
916BUILDHISTORY_PUSH_REPO
917 When inheriting the ```buildhistory`` <#ref-classes-buildhistory>`__
918 class, this variable optionally specifies a remote repository to
919 which build history pushes Git changes. In order for
920 ``BUILDHISTORY_PUSH_REPO`` to work,
921 ```BUILDHISTORY_COMMIT`` <#var-BUILDHISTORY_COMMIT>`__ must be set to
922 "1".
923
924 The repository should correspond to a remote address that specifies a
925 repository as understood by Git, or alternatively to a remote name
926 that you have set up manually using ``git remote`` within the local
927 repository.
928
929 By default, the ``buildhistory`` class sets the variable as follows:
930 BUILDHISTORY_PUSH_REPO ?= ""
931
932BUILDSDK_CFLAGS
933 Specifies the flags to pass to the C compiler when building for the
934 SDK. When building in the ``nativesdk-`` context,
935 ```CFLAGS`` <#var-CFLAGS>`__ is set to the value of this variable by
936 default.
937
938BUILDSDK_CPPFLAGS
939 Specifies the flags to pass to the C pre-processor (i.e. to both the
940 C and the C++ compilers) when building for the SDK. When building in
941 the ``nativesdk-`` context, ```CPPFLAGS`` <#var-CPPFLAGS>`__ is set
942 to the value of this variable by default.
943
944BUILDSDK_CXXFLAGS
945 Specifies the flags to pass to the C++ compiler when building for the
946 SDK. When building in the ``nativesdk-`` context,
947 ```CXXFLAGS`` <#var-CXXFLAGS>`__ is set to the value of this variable
948 by default.
949
950BUILDSDK_LDFLAGS
951 Specifies the flags to pass to the linker when building for the SDK.
952 When building in the ``nativesdk-`` context,
953 ```LDFLAGS`` <#var-LDFLAGS>`__ is set to the value of this variable
954 by default.
955
956BUILDSTATS_BASE
957 Points to the location of the directory that holds build statistics
958 when you use and enable the
959 ```buildstats`` <#ref-classes-buildstats>`__ class. The
960 ``BUILDSTATS_BASE`` directory defaults to
961 ``${``\ ```TMPDIR`` <#var-TMPDIR>`__\ ``}/buildstats/``.
962
963BUSYBOX_SPLIT_SUID
964 For the BusyBox recipe, specifies whether to split the output
965 executable file into two parts: one for features that require
966 ``setuid root``, and one for the remaining features (i.e. those that
967 do not require ``setuid root``).
968
969 The ``BUSYBOX_SPLIT_SUID`` variable defaults to "1", which results in
970 splitting the output executable file. Set the variable to "0" to get
971 a single output executable file.
972
973CACHE
974 Specifies the directory BitBake uses to store a cache of the
975 `Metadata <#metadata>`__ so it does not need to be parsed every time
976 BitBake is started.
977
978CC
979 The minimal command and arguments used to run the C compiler.
980
981CFLAGS
982 Specifies the flags to pass to the C compiler. This variable is
983 exported to an environment variable and thus made visible to the
984 software being built during the compilation step.
985
986 Default initialization for ``CFLAGS`` varies depending on what is
987 being built:
988
989 - ```TARGET_CFLAGS`` <#var-TARGET_CFLAGS>`__ when building for the
990 target
991
992 - ```BUILD_CFLAGS`` <#var-BUILD_CFLAGS>`__ when building for the
993 build host (i.e. ``-native``)
994
995 - ```BUILDSDK_CFLAGS`` <#var-BUILDSDK_CFLAGS>`__ when building for
996 an SDK (i.e. ``nativesdk-``)
997
998CLASSOVERRIDE
999 An internal variable specifying the special class override that
1000 should currently apply (e.g. "class-target", "class-native", and so
1001 forth). The classes that use this variable (e.g.
1002 ```native`` <#ref-classes-native>`__,
1003 ```nativesdk`` <#ref-classes-nativesdk>`__, and so forth) set the
1004 variable to appropriate values.
1005
1006 .. note::
1007
1008 CLASSOVERRIDE
1009 gets its default "class-target" value from the
1010 bitbake.conf
1011 file.
1012
1013 As an example, the following override allows you to install extra
1014 files, but only when building for the target:
1015 do_install_append_class-target() { install my-extra-file
1016 ${D}${sysconfdir} } Here is an example where ``FOO`` is set to
1017 "native" when building for the build host, and to "other" when not
1018 building for the build host: FOO_class-native = "native" FOO =
1019 "other" The underlying mechanism behind ``CLASSOVERRIDE`` is simply
1020 that it is included in the default value of
1021 ```OVERRIDES`` <#var-OVERRIDES>`__.
1022
1023CLEANBROKEN
1024 If set to "1" within a recipe, ``CLEANBROKEN`` specifies that the
1025 ``make clean`` command does not work for the software being built.
1026 Consequently, the OpenEmbedded build system will not try to run
1027 ``make clean`` during the ```do_configure`` <#ref-tasks-configure>`__
1028 task, which is the default behavior.
1029
1030COMBINED_FEATURES
1031 Provides a list of hardware features that are enabled in both
1032 ```MACHINE_FEATURES`` <#var-MACHINE_FEATURES>`__ and
1033 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__. This select list of
1034 features contains features that make sense to be controlled both at
1035 the machine and distribution configuration level. For example, the
1036 "bluetooth" feature requires hardware support but should also be
1037 optional at the distribution level, in case the hardware supports
1038 Bluetooth but you do not ever intend to use it.
1039
1040COMMON_LICENSE_DIR
1041 Points to ``meta/files/common-licenses`` in the `Source
1042 Directory <#source-directory>`__, which is where generic license
1043 files reside.
1044
1045COMPATIBLE_HOST
1046 A regular expression that resolves to one or more hosts (when the
1047 recipe is native) or one or more targets (when the recipe is
1048 non-native) with which a recipe is compatible. The regular expression
1049 is matched against ```HOST_SYS`` <#var-HOST_SYS>`__. You can use the
1050 variable to stop recipes from being built for classes of systems with
1051 which the recipes are not compatible. Stopping these builds is
1052 particularly useful with kernels. The variable also helps to increase
1053 parsing speed since the build system skips parsing recipes not
1054 compatible with the current system.
1055
1056COMPATIBLE_MACHINE
1057 A regular expression that resolves to one or more target machines
1058 with which a recipe is compatible. The regular expression is matched
1059 against ```MACHINEOVERRIDES`` <#var-MACHINEOVERRIDES>`__. You can use
1060 the variable to stop recipes from being built for machines with which
1061 the recipes are not compatible. Stopping these builds is particularly
1062 useful with kernels. The variable also helps to increase parsing
1063 speed since the build system skips parsing recipes not compatible
1064 with the current machine.
1065
1066COMPLEMENTARY_GLOB
1067 Defines wildcards to match when installing a list of complementary
1068 packages for all the packages explicitly (or implicitly) installed in
1069 an image.
1070
1071 .. note::
1072
1073 The
1074 COMPLEMENTARY_GLOB
1075 variable uses Unix filename pattern matching (
1076 fnmatch
1077 ), which is similar to the Unix style pathname pattern expansion (
1078 glob
1079 ).
1080
1081 The resulting list of complementary packages is associated with an
1082 item that can be added to
1083 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__. An example usage of
1084 this is the "dev-pkgs" item that when added to ``IMAGE_FEATURES``
1085 will install -dev packages (containing headers and other development
1086 files) for every package in the image.
1087
1088 To add a new feature item pointing to a wildcard, use a variable flag
1089 to specify the feature item name and use the value to specify the
1090 wildcard. Here is an example: COMPLEMENTARY_GLOB[dev-pkgs] = '*-dev'
1091
1092COMPONENTS_DIR
1093 Stores sysroot components for each recipe. The OpenEmbedded build
1094 system uses ``COMPONENTS_DIR`` when constructing recipe-specific
1095 sysroots for other recipes.
1096
1097 The default is
1098 "``${``\ ```STAGING_DIR`` <#var-STAGING_DIR>`__\ ``}-components``."
1099 (i.e.
1100 "``${``\ ```TMPDIR`` <#var-TMPDIR>`__\ ``}/sysroots-components``").
1101
1102CONF_VERSION
1103 Tracks the version of the local configuration file (i.e.
1104 ``local.conf``). The value for ``CONF_VERSION`` increments each time
1105 ``build/conf/`` compatibility changes.
1106
1107CONFFILES
1108 Identifies editable or configurable files that are part of a package.
1109 If the Package Management System (PMS) is being used to update
1110 packages on the target system, it is possible that configuration
1111 files you have changed after the original installation and that you
1112 now want to remain unchanged are overwritten. In other words,
1113 editable files might exist in the package that you do not want reset
1114 as part of the package update process. You can use the ``CONFFILES``
1115 variable to list the files in the package that you wish to prevent
1116 the PMS from overwriting during this update process.
1117
1118 To use the ``CONFFILES`` variable, provide a package name override
1119 that identifies the resulting package. Then, provide a
1120 space-separated list of files. Here is an example: CONFFILES_${PN} +=
1121 "${sysconfdir}/file1 \\ ${sysconfdir}/file2 ${sysconfdir}/file3"
1122
1123 A relationship exists between the ``CONFFILES`` and ``FILES``
1124 variables. The files listed within ``CONFFILES`` must be a subset of
1125 the files listed within ``FILES``. Because the configuration files
1126 you provide with ``CONFFILES`` are simply being identified so that
1127 the PMS will not overwrite them, it makes sense that the files must
1128 already be included as part of the package through the ``FILES``
1129 variable.
1130
1131 .. note::
1132
1133 When specifying paths as part of the
1134 CONFFILES
1135 variable, it is good practice to use appropriate path variables.
1136 For example,
1137 ${sysconfdir}
1138 rather than
1139 /etc
1140 or
1141 ${bindir}
1142 rather than
1143 /usr/bin
1144 . You can find a list of these variables at the top of the
1145 meta/conf/bitbake.conf
1146 file in the
1147 Source Directory
1148 .
1149
1150CONFIG_INITRAMFS_SOURCE
1151 Identifies the initial RAM filesystem (initramfs) source files. The
1152 OpenEmbedded build system receives and uses this kernel Kconfig
1153 variable as an environment variable. By default, the variable is set
1154 to null ("").
1155
1156 The ``CONFIG_INITRAMFS_SOURCE`` can be either a single cpio archive
1157 with a ``.cpio`` suffix or a space-separated list of directories and
1158 files for building the initramfs image. A cpio archive should contain
1159 a filesystem archive to be used as an initramfs image. Directories
1160 should contain a filesystem layout to be included in the initramfs
1161 image. Files should contain entries according to the format described
1162 by the ``usr/gen_init_cpio`` program in the kernel tree.
1163
1164 If you specify multiple directories and files, the initramfs image
1165 will be the aggregate of all of them.
1166
1167 For information on creating an initramfs, see the "`Building an
1168 Initial RAM Filesystem (initramfs)
1169 Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__" section
1170 in the Yocto Project Development Tasks Manual.
1171
1172CONFIG_SITE
1173 A list of files that contains ``autoconf`` test results relevant to
1174 the current build. This variable is used by the Autotools utilities
1175 when running ``configure``.
1176
1177CONFIGURE_FLAGS
1178 The minimal arguments for GNU configure.
1179
1180CONFLICT_DISTRO_FEATURES
1181 When inheriting the
1182 ```distro_features_check`` <#ref-classes-distro_features_check>`__
1183 class, this variable identifies distribution features that would be
1184 in conflict should the recipe be built. In other words, if the
1185 ``CONFLICT_DISTRO_FEATURES`` variable lists a feature that also
1186 appears in ``DISTRO_FEATURES`` within the current configuration, an
1187 error occurs and the build stops.
1188
1189COPYLEFT_LICENSE_EXCLUDE
1190 A space-separated list of licenses to exclude from the source
1191 archived by the ```archiver`` <#ref-classes-archiver>`__ class. In
1192 other words, if a license in a recipe's
1193 ```LICENSE`` <#var-LICENSE>`__ value is in the value of
1194 ``COPYLEFT_LICENSE_EXCLUDE``, then its source is not archived by the
1195 class.
1196
1197 .. note::
1198
1199 The
1200 COPYLEFT_LICENSE_EXCLUDE
1201 variable takes precedence over the
1202 COPYLEFT_LICENSE_INCLUDE
1203 variable.
1204
1205 The default value, which is "CLOSED Proprietary", for
1206 ``COPYLEFT_LICENSE_EXCLUDE`` is set by the
1207 ```copyleft_filter`` <#ref-classes-copyleft_filter>`__ class, which
1208 is inherited by the ``archiver`` class.
1209
1210COPYLEFT_LICENSE_INCLUDE
1211 A space-separated list of licenses to include in the source archived
1212 by the ```archiver`` <#ref-classes-archiver>`__ class. In other
1213 words, if a license in a recipe's ```LICENSE`` <#var-LICENSE>`__
1214 value is in the value of ``COPYLEFT_LICENSE_INCLUDE``, then its
1215 source is archived by the class.
1216
1217 The default value is set by the
1218 ```copyleft_filter`` <#ref-classes-copyleft_filter>`__ class, which
1219 is inherited by the ``archiver`` class. The default value includes
1220 "GPL*", "LGPL*", and "AGPL*".
1221
1222COPYLEFT_PN_EXCLUDE
1223 A list of recipes to exclude in the source archived by the
1224 ```archiver`` <#ref-classes-archiver>`__ class. The
1225 ``COPYLEFT_PN_EXCLUDE`` variable overrides the license inclusion and
1226 exclusion caused through the
1227 ```COPYLEFT_LICENSE_INCLUDE`` <#var-COPYLEFT_LICENSE_INCLUDE>`__ and
1228 ```COPYLEFT_LICENSE_EXCLUDE`` <#var-COPYLEFT_LICENSE_EXCLUDE>`__
1229 variables, respectively.
1230
1231 The default value, which is "" indicating to not explicitly exclude
1232 any recipes by name, for ``COPYLEFT_PN_EXCLUDE`` is set by the
1233 ```copyleft_filter`` <#ref-classes-copyleft_filter>`__ class, which
1234 is inherited by the ``archiver`` class.
1235
1236COPYLEFT_PN_INCLUDE
1237 A list of recipes to include in the source archived by the
1238 ```archiver`` <#ref-classes-archiver>`__ class. The
1239 ``COPYLEFT_PN_INCLUDE`` variable overrides the license inclusion and
1240 exclusion caused through the
1241 ```COPYLEFT_LICENSE_INCLUDE`` <#var-COPYLEFT_LICENSE_INCLUDE>`__ and
1242 ```COPYLEFT_LICENSE_EXCLUDE`` <#var-COPYLEFT_LICENSE_EXCLUDE>`__
1243 variables, respectively.
1244
1245 The default value, which is "" indicating to not explicitly include
1246 any recipes by name, for ``COPYLEFT_PN_INCLUDE`` is set by the
1247 ```copyleft_filter`` <#ref-classes-copyleft_filter>`__ class, which
1248 is inherited by the ``archiver`` class.
1249
1250COPYLEFT_RECIPE_TYPES
1251 A space-separated list of recipe types to include in the source
1252 archived by the ```archiver`` <#ref-classes-archiver>`__ class.
1253 Recipe types are ``target``, ``native``, ``nativesdk``, ``cross``,
1254 ``crosssdk``, and ``cross-canadian``.
1255
1256 The default value, which is "target*", for ``COPYLEFT_RECIPE_TYPES``
1257 is set by the ```copyleft_filter`` <#ref-classes-copyleft_filter>`__
1258 class, which is inherited by the ``archiver`` class.
1259
1260COPY_LIC_DIRS
1261 If set to "1" along with the
1262 ```COPY_LIC_MANIFEST`` <#var-COPY_LIC_MANIFEST>`__ variable, the
1263 OpenEmbedded build system copies into the image the license files,
1264 which are located in ``/usr/share/common-licenses``, for each
1265 package. The license files are placed in directories within the image
1266 itself during build time.
1267
1268 .. note::
1269
1270 The
1271 COPY_LIC_DIRS
1272 does not offer a path for adding licenses for newly installed
1273 packages to an image, which might be most suitable for read-only
1274 filesystems that cannot be upgraded. See the
1275 LICENSE_CREATE_PACKAGE
1276 variable for additional information. You can also reference the "
1277 Providing License Text
1278 " section in the Yocto Project Development Tasks Manual for
1279 information on providing license text.
1280
1281COPY_LIC_MANIFEST
1282 If set to "1", the OpenEmbedded build system copies the license
1283 manifest for the image to
1284 ``/usr/share/common-licenses/license.manifest`` within the image
1285 itself during build time.
1286
1287 .. note::
1288
1289 The
1290 COPY_LIC_MANIFEST
1291 does not offer a path for adding licenses for newly installed
1292 packages to an image, which might be most suitable for read-only
1293 filesystems that cannot be upgraded. See the
1294 LICENSE_CREATE_PACKAGE
1295 variable for additional information. You can also reference the "
1296 Providing License Text
1297 " section in the Yocto Project Development Tasks Manual for
1298 information on providing license text.
1299
1300CORE_IMAGE_EXTRA_INSTALL
1301 Specifies the list of packages to be added to the image. You should
1302 only set this variable in the ``local.conf`` configuration file found
1303 in the `Build Directory <#build-directory>`__.
1304
1305 This variable replaces ``POKY_EXTRA_INSTALL``, which is no longer
1306 supported.
1307
1308COREBASE
1309 Specifies the parent directory of the OpenEmbedded-Core Metadata
1310 layer (i.e. ``meta``).
1311
1312 It is an important distinction that ``COREBASE`` points to the parent
1313 of this layer and not the layer itself. Consider an example where you
1314 have cloned the Poky Git repository and retained the ``poky`` name
1315 for your local copy of the repository. In this case, ``COREBASE``
1316 points to the ``poky`` folder because it is the parent directory of
1317 the ``poky/meta`` layer.
1318
1319COREBASE_FILES
1320 Lists files from the ```COREBASE`` <#var-COREBASE>`__ directory that
1321 should be copied other than the layers listed in the
1322 ``bblayers.conf`` file. The ``COREBASE_FILES`` variable exists for
1323 the purpose of copying metadata from the OpenEmbedded build system
1324 into the extensible SDK.
1325
1326 Explicitly listing files in ``COREBASE`` is needed because it
1327 typically contains build directories and other files that should not
1328 normally be copied into the extensible SDK. Consequently, the value
1329 of ``COREBASE_FILES`` is used in order to only copy the files that
1330 are actually needed.
1331
1332CPP
1333 The minimal command and arguments used to run the C preprocessor.
1334
1335CPPFLAGS
1336 Specifies the flags to pass to the C pre-processor (i.e. to both the
1337 C and the C++ compilers). This variable is exported to an environment
1338 variable and thus made visible to the software being built during the
1339 compilation step.
1340
1341 Default initialization for ``CPPFLAGS`` varies depending on what is
1342 being built:
1343
1344 - ```TARGET_CPPFLAGS`` <#var-TARGET_CPPFLAGS>`__ when building for
1345 the target
1346
1347 - ```BUILD_CPPFLAGS`` <#var-BUILD_CPPFLAGS>`__ when building for the
1348 build host (i.e. ``-native``)
1349
1350 - ```BUILDSDK_CPPFLAGS`` <#var-BUILDSDK_CPPFLAGS>`__ when building
1351 for an SDK (i.e. ``nativesdk-``)
1352
1353CROSS_COMPILE
1354 The toolchain binary prefix for the target tools. The
1355 ``CROSS_COMPILE`` variable is the same as the
1356 ```TARGET_PREFIX`` <#var-TARGET_PREFIX>`__ variable.
1357
1358 .. note::
1359
1360 The OpenEmbedded build system sets the
1361 CROSS_COMPILE
1362 variable only in certain contexts (e.g. when building for kernel
1363 and kernel module recipes).
1364
1365CVSDIR
1366 The directory in which files checked out under the CVS system are
1367 stored.
1368
1369CXX
1370 The minimal command and arguments used to run the C++ compiler.
1371
1372CXXFLAGS
1373 Specifies the flags to pass to the C++ compiler. This variable is
1374 exported to an environment variable and thus made visible to the
1375 software being built during the compilation step.
1376
1377 Default initialization for ``CXXFLAGS`` varies depending on what is
1378 being built:
1379
1380 - ```TARGET_CXXFLAGS`` <#var-TARGET_CXXFLAGS>`__ when building for
1381 the target
1382
1383 - ```BUILD_CXXFLAGS`` <#var-BUILD_CXXFLAGS>`__ when building for the
1384 build host (i.e. ``-native``)
1385
1386 - ```BUILDSDK_CXXFLAGS`` <#var-BUILDSDK_CXXFLAGS>`__ when building
1387 for an SDK (i.e. ``nativesdk-``)
1388
1389D
1390 The destination directory. The location in the `Build
1391 Directory <#build-directory>`__ where components are installed by the
1392 ```do_install`` <#ref-tasks-install>`__ task. This location defaults
1393 to: ${WORKDIR}/image
1394
1395 .. note::
1396
1397 Tasks that read from or write to this directory should run under
1398 fakeroot
1399 .
1400
1401DATE
1402 The date the build was started. Dates appear using the year, month,
1403 and day (YMD) format (e.g. "20150209" for February 9th, 2015).
1404
1405DATETIME
1406 The date and time on which the current build started. The format is
1407 suitable for timestamps.
1408
1409DEBIAN_NOAUTONAME
1410 When the ```debian`` <#ref-classes-debian>`__ class is inherited,
1411 which is the default behavior, ``DEBIAN_NOAUTONAME`` specifies a
1412 particular package should not be renamed according to Debian library
1413 package naming. You must use the package name as an override when you
1414 set this variable. Here is an example from the ``fontconfig`` recipe:
1415 DEBIAN_NOAUTONAME_fontconfig-utils = "1"
1416
1417DEBIANNAME
1418 When the ```debian`` <#ref-classes-debian>`__ class is inherited,
1419 which is the default behavior, ``DEBIANNAME`` allows you to override
1420 the library name for an individual package. Overriding the library
1421 name in these cases is rare. You must use the package name as an
1422 override when you set this variable. Here is an example from the
1423 ``dbus`` recipe: DEBIANNAME_${PN} = "dbus-1"
1424
1425DEBUG_BUILD
1426 Specifies to build packages with debugging information. This
1427 influences the value of the ``SELECTED_OPTIMIZATION`` variable.
1428
1429DEBUG_OPTIMIZATION
1430 The options to pass in ``TARGET_CFLAGS`` and ``CFLAGS`` when
1431 compiling a system for debugging. This variable defaults to "-O
1432 -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe".
1433
1434DEFAULT_PREFERENCE
1435 Specifies a weak bias for recipe selection priority.
1436
1437 The most common usage of this is variable is to set it to "-1" within
1438 a recipe for a development version of a piece of software. Using the
1439 variable in this way causes the stable version of the recipe to build
1440 by default in the absence of ``PREFERRED_VERSION`` being used to
1441 build the development version.
1442
1443 .. note::
1444
1445 The bias provided by
1446 DEFAULT_PREFERENCE
1447 is weak and is overridden by
1448 BBFILE_PRIORITY
1449 if that variable is different between two layers that contain
1450 different versions of the same recipe.
1451
1452DEFAULTTUNE
1453 The default CPU and Application Binary Interface (ABI) tunings (i.e.
1454 the "tune") used by the OpenEmbedded build system. The
1455 ``DEFAULTTUNE`` helps define
1456 ```TUNE_FEATURES`` <#var-TUNE_FEATURES>`__.
1457
1458 The default tune is either implicitly or explicitly set by the
1459 machine (```MACHINE`` <#var-MACHINE>`__). However, you can override
1460 the setting using available tunes as defined with
1461 ```AVAILTUNES`` <#var-AVAILTUNES>`__.
1462
1463DEPENDS
1464 Lists a recipe's build-time dependencies. These are dependencies on
1465 other recipes whose contents (e.g. headers and shared libraries) are
1466 needed by the recipe at build time.
1467
1468 As an example, consider a recipe ``foo`` that contains the following
1469 assignment: DEPENDS = "bar" The practical effect of the previous
1470 assignment is that all files installed by bar will be available in
1471 the appropriate staging sysroot, given by the
1472 ```STAGING_DIR*`` <#var-STAGING_DIR>`__ variables, by the time the
1473 ```do_configure`` <#ref-tasks-configure>`__ task for ``foo`` runs.
1474 This mechanism is implemented by having ``do_configure`` depend on
1475 the ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task of
1476 each recipe listed in ``DEPENDS``, through a
1477 ``[``\ ```deptask`` <&YOCTO_DOCS_BB_URL;#variable-flags>`__\ ``]``
1478 declaration in the ```base`` <#ref-classes-base>`__ class.
1479
1480 .. note::
1481
1482 It seldom is necessary to reference, for example,
1483 STAGING_DIR_HOST
1484 explicitly. The standard classes and build-related variables are
1485 configured to automatically use the appropriate staging sysroots.
1486
1487 As another example, ``DEPENDS`` can also be used to add utilities
1488 that run on the build machine during the build. For example, a recipe
1489 that makes use of a code generator built by the recipe ``codegen``
1490 might have the following: DEPENDS = "codegen-native" For more
1491 information, see the ```native`` <#ref-classes-native>`__ class and
1492 the ```EXTRANATIVEPATH`` <#var-EXTRANATIVEPATH>`__ variable.
1493
1494 .. note::
1495
1496 - ``DEPENDS`` is a list of recipe names. Or, to be more precise,
1497 it is a list of ```PROVIDES`` <#var-PROVIDES>`__ names, which
1498 usually match recipe names. Putting a package name such as
1499 "foo-dev" in ``DEPENDS`` does not make sense. Use "foo"
1500 instead, as this will put files from all the packages that make
1501 up ``foo``, which includes those from ``foo-dev``, into the
1502 sysroot.
1503
1504 - One recipe having another recipe in ``DEPENDS`` does not by
1505 itself add any runtime dependencies between the packages
1506 produced by the two recipes. However, as explained in the
1507 "`Automatically Added Runtime
1508 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
1509 section in the Yocto Project Overview and Concepts Manual,
1510 runtime dependencies will often be added automatically, meaning
1511 ``DEPENDS`` alone is sufficient for most recipes.
1512
1513 - Counterintuitively, ``DEPENDS`` is often necessary even for
1514 recipes that install precompiled components. For example, if
1515 ``libfoo`` is a precompiled library that links against
1516 ``libbar``, then linking against ``libfoo`` requires both
1517 ``libfoo`` and ``libbar`` to be available in the sysroot.
1518 Without a ``DEPENDS`` from the recipe that installs ``libfoo``
1519 to the recipe that installs ``libbar``, other recipes might
1520 fail to link against ``libfoo``.
1521
1522 For information on runtime dependencies, see the
1523 ```RDEPENDS`` <#var-RDEPENDS>`__ variable. You can also see the
1524 "`Tasks <&YOCTO_DOCS_BB_URL;#tasks>`__" and
1525 "`Dependencies <&YOCTO_DOCS_BB_URL;#dependencies>`__" sections in the
1526 BitBake User Manual for additional information on tasks and
1527 dependencies.
1528
1529DEPLOY_DIR
1530 Points to the general area that the OpenEmbedded build system uses to
1531 place images, packages, SDKs, and other output files that are ready
1532 to be used outside of the build system. By default, this directory
1533 resides within the `Build Directory <#build-directory>`__ as
1534 ``${TMPDIR}/deploy``.
1535
1536 For more information on the structure of the Build Directory, see
1537 "`The Build Directory - ``build/`` <#structure-build>`__" section.
1538 For more detail on the contents of the ``deploy`` directory, see the
1539 "`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__", "`Package
1540 Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__", and
1541 "`Application Development
1542 SDK <&YOCTO_DOCS_OM_URL;#sdk-dev-environment>`__" sections all in the
1543 Yocto Project Overview and Concepts Manual.
1544
1545DEPLOY_DIR_DEB
1546 Points to the area that the OpenEmbedded build system uses to place
1547 Debian packages that are ready to be used outside of the build
1548 system. This variable applies only when
1549 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ contains
1550 "package_deb".
1551
1552 The BitBake configuration file initially defines the
1553 ``DEPLOY_DIR_DEB`` variable as a sub-folder of
1554 ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__: DEPLOY_DIR_DEB =
1555 "${DEPLOY_DIR}/deb"
1556
1557 The ```package_deb`` <#ref-classes-package_deb>`__ class uses the
1558 ``DEPLOY_DIR_DEB`` variable to make sure the
1559 ```do_package_write_deb`` <#ref-tasks-package_write_deb>`__ task
1560 writes Debian packages into the appropriate folder. For more
1561 information on how packaging works, see the "`Package
1562 Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section
1563 in the Yocto Project Overview and Concepts Manual.
1564
1565DEPLOY_DIR_IMAGE
1566 Points to the area that the OpenEmbedded build system uses to place
1567 images and other associated output files that are ready to be
1568 deployed onto the target machine. The directory is machine-specific
1569 as it contains the ``${MACHINE}`` name. By default, this directory
1570 resides within the `Build Directory <#build-directory>`__ as
1571 ``${DEPLOY_DIR}/images/${MACHINE}/``.
1572
1573 For more information on the structure of the Build Directory, see
1574 "`The Build Directory - ``build/`` <#structure-build>`__" section.
1575 For more detail on the contents of the ``deploy`` directory, see the
1576 "`Images <&YOCTO_DOCS_OM_URL;#images-dev-environment>`__" and
1577 "`Application Development
1578 SDK <&YOCTO_DOCS_OM_URL;#sdk-dev-environment>`__" sections both in
1579 the Yocto Project Overview and Concepts Manual.
1580
1581DEPLOY_DIR_IPK
1582 Points to the area that the OpenEmbedded build system uses to place
1583 IPK packages that are ready to be used outside of the build system.
1584 This variable applies only when
1585 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ contains
1586 "package_ipk".
1587
1588 The BitBake configuration file initially defines this variable as a
1589 sub-folder of ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__: DEPLOY_DIR_IPK =
1590 "${DEPLOY_DIR}/ipk"
1591
1592 The ```package_ipk`` <#ref-classes-package_ipk>`__ class uses the
1593 ``DEPLOY_DIR_IPK`` variable to make sure the
1594 ```do_package_write_ipk`` <#ref-tasks-package_write_ipk>`__ task
1595 writes IPK packages into the appropriate folder. For more information
1596 on how packaging works, see the "`Package
1597 Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section
1598 in the Yocto Project Overview and Concepts Manual.
1599
1600DEPLOY_DIR_RPM
1601 Points to the area that the OpenEmbedded build system uses to place
1602 RPM packages that are ready to be used outside of the build system.
1603 This variable applies only when
1604 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ contains
1605 "package_rpm".
1606
1607 The BitBake configuration file initially defines this variable as a
1608 sub-folder of ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__: DEPLOY_DIR_RPM =
1609 "${DEPLOY_DIR}/rpm"
1610
1611 The ```package_rpm`` <#ref-classes-package_rpm>`__ class uses the
1612 ``DEPLOY_DIR_RPM`` variable to make sure the
1613 ```do_package_write_rpm`` <#ref-tasks-package_write_rpm>`__ task
1614 writes RPM packages into the appropriate folder. For more information
1615 on how packaging works, see the "`Package
1616 Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section
1617 in the Yocto Project Overview and Concepts Manual.
1618
1619DEPLOY_DIR_TAR
1620 Points to the area that the OpenEmbedded build system uses to place
1621 tarballs that are ready to be used outside of the build system. This
1622 variable applies only when
1623 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ contains
1624 "package_tar".
1625
1626 The BitBake configuration file initially defines this variable as a
1627 sub-folder of ```DEPLOY_DIR`` <#var-DEPLOY_DIR>`__: DEPLOY_DIR_TAR =
1628 "${DEPLOY_DIR}/tar"
1629
1630 The ```package_tar`` <#ref-classes-package_tar>`__ class uses the
1631 ``DEPLOY_DIR_TAR`` variable to make sure the
1632 ```do_package_write_tar`` <#ref-tasks-package_write_tar>`__ task
1633 writes TAR packages into the appropriate folder. For more information
1634 on how packaging works, see the "`Package
1635 Feeds <&YOCTO_DOCS_OM_URL;#package-feeds-dev-environment>`__" section
1636 in the Yocto Project Overview and Concepts Manual.
1637
1638DEPLOYDIR
1639 When inheriting the ```deploy`` <#ref-classes-deploy>`__ class, the
1640 ``DEPLOYDIR`` points to a temporary work area for deployed files that
1641 is set in the ``deploy`` class as follows: DEPLOYDIR =
1642 "${WORKDIR}/deploy-${```PN`` <#var-PN>`__}"
1643
1644 Recipes inheriting the ``deploy`` class should copy files to be
1645 deployed into ``DEPLOYDIR``, and the class will take care of copying
1646 them into ```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__
1647 afterwards.
1648
1649DESCRIPTION
1650 The package description used by package managers. If not set,
1651 ``DESCRIPTION`` takes the value of the ```SUMMARY`` <#var-SUMMARY>`__
1652 variable.
1653
1654DISTRO
1655 The short name of the distribution. For information on the long name
1656 of the distribution, see the ```DISTRO_NAME`` <#var-DISTRO_NAME>`__
1657 variable.
1658
1659 The ``DISTRO`` variable corresponds to a distribution configuration
1660 file whose root name is the same as the variable's argument and whose
1661 filename extension is ``.conf``. For example, the distribution
1662 configuration file for the Poky distribution is named ``poky.conf``
1663 and resides in the ``meta-poky/conf/distro`` directory of the `Source
1664 Directory <#source-directory>`__.
1665
1666 Within that ``poky.conf`` file, the ``DISTRO`` variable is set as
1667 follows: DISTRO = "poky"
1668
1669 Distribution configuration files are located in a ``conf/distro``
1670 directory within the `Metadata <#metadata>`__ that contains the
1671 distribution configuration. The value for ``DISTRO`` must not contain
1672 spaces, and is typically all lower-case.
1673
1674 .. note::
1675
1676 If the
1677 DISTRO
1678 variable is blank, a set of default configurations are used, which
1679 are specified within
1680 meta/conf/distro/defaultsetup.conf
1681 also in the Source Directory.
1682
1683DISTRO_CODENAME
1684 Specifies a codename for the distribution being built.
1685
1686DISTRO_EXTRA_RDEPENDS
1687 Specifies a list of distro-specific packages to add to all images.
1688 This variable takes affect through ``packagegroup-base`` so the
1689 variable only really applies to the more full-featured images that
1690 include ``packagegroup-base``. You can use this variable to keep
1691 distro policy out of generic images. As with all other distro
1692 variables, you set this variable in the distro ``.conf`` file.
1693
1694DISTRO_EXTRA_RRECOMMENDS
1695 Specifies a list of distro-specific packages to add to all images if
1696 the packages exist. The packages might not exist or be empty (e.g.
1697 kernel modules). The list of packages are automatically installed but
1698 you can remove them.
1699
1700DISTRO_FEATURES
1701 The software support you want in your distribution for various
1702 features. You define your distribution features in the distribution
1703 configuration file.
1704
1705 In most cases, the presence or absence of a feature in
1706 ``DISTRO_FEATURES`` is translated to the appropriate option supplied
1707 to the configure script during the
1708 ```do_configure`` <#ref-tasks-configure>`__ task for recipes that
1709 optionally support the feature. For example, specifying "x11" in
1710 ``DISTRO_FEATURES``, causes every piece of software built for the
1711 target that can optionally support X11 to have its X11 support
1712 enabled.
1713
1714 Two more examples are Bluetooth and NFS support. For a more complete
1715 list of features that ships with the Yocto Project and that you can
1716 provide with this variable, see the "`Distro
1717 Features <#ref-features-distro>`__" section.
1718
1719DISTRO_FEATURES_BACKFILL
1720 Features to be added to ``DISTRO_FEATURES`` if not also present in
1721 ``DISTRO_FEATURES_BACKFILL_CONSIDERED``.
1722
1723 This variable is set in the ``meta/conf/bitbake.conf`` file. It is
1724 not intended to be user-configurable. It is best to just reference
1725 the variable to see which distro features are being backfilled for
1726 all distro configurations. See the "`Feature
1727 Backfilling <#ref-features-backfill>`__" section for more
1728 information.
1729
1730DISTRO_FEATURES_BACKFILL_CONSIDERED
1731 Features from ``DISTRO_FEATURES_BACKFILL`` that should not be
1732 backfilled (i.e. added to ``DISTRO_FEATURES``) during the build. See
1733 the "`Feature Backfilling <#ref-features-backfill>`__" section for
1734 more information.
1735
1736DISTRO_FEATURES_DEFAULT
1737 A convenience variable that gives you the default list of distro
1738 features with the exception of any features specific to the C library
1739 (``libc``).
1740
1741 When creating a custom distribution, you might find it useful to be
1742 able to reuse the default
1743 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ options without the
1744 need to write out the full set. Here is an example that uses
1745 ``DISTRO_FEATURES_DEFAULT`` from a custom distro configuration file:
1746 DISTRO_FEATURES ?= "${DISTRO_FEATURES_DEFAULT} myfeature"
1747
1748DISTRO_FEATURES_FILTER_NATIVE
1749 Specifies a list of features that if present in the target
1750 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ value should be
1751 included in ``DISTRO_FEATURES`` when building native recipes. This
1752 variable is used in addition to the features filtered using the
1753 ```DISTRO_FEATURES_NATIVE`` <#var-DISTRO_FEATURES_NATIVE>`__
1754 variable.
1755
1756DISTRO_FEATURES_FILTER_NATIVESDK
1757 Specifies a list of features that if present in the target
1758 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ value should be
1759 included in ``DISTRO_FEATURES`` when building nativesdk recipes. This
1760 variable is used in addition to the features filtered using the
1761 ```DISTRO_FEATURES_NATIVESDK`` <#var-DISTRO_FEATURES_NATIVESDK>`__
1762 variable.
1763
1764DISTRO_FEATURES_NATIVE
1765 Specifies a list of features that should be included in
1766 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ when building native
1767 recipes. This variable is used in addition to the features filtered
1768 using the
1769 ```DISTRO_FEATURES_FILTER_NATIVE`` <#var-DISTRO_FEATURES_FILTER_NATIVE>`__
1770 variable.
1771
1772DISTRO_FEATURES_NATIVESDK
1773 Specifies a list of features that should be included in
1774 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__ when building
1775 nativesdk recipes. This variable is used in addition to the features
1776 filtered using the
1777 ```DISTRO_FEATURES_FILTER_NATIVESDK`` <#var-DISTRO_FEATURES_FILTER_NATIVESDK>`__
1778 variable.
1779
1780DISTRO_NAME
1781 The long name of the distribution. For information on the short name
1782 of the distribution, see the ```DISTRO`` <#var-DISTRO>`__ variable.
1783
1784 The ``DISTRO_NAME`` variable corresponds to a distribution
1785 configuration file whose root name is the same as the variable's
1786 argument and whose filename extension is ``.conf``. For example, the
1787 distribution configuration file for the Poky distribution is named
1788 ``poky.conf`` and resides in the ``meta-poky/conf/distro`` directory
1789 of the `Source Directory <#source-directory>`__.
1790
1791 Within that ``poky.conf`` file, the ``DISTRO_NAME`` variable is set
1792 as follows: DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
1793
1794 Distribution configuration files are located in a ``conf/distro``
1795 directory within the `Metadata <#metadata>`__ that contains the
1796 distribution configuration.
1797
1798 .. note::
1799
1800 If the
1801 DISTRO_NAME
1802 variable is blank, a set of default configurations are used, which
1803 are specified within
1804 meta/conf/distro/defaultsetup.conf
1805 also in the Source Directory.
1806
1807DISTRO_VERSION
1808 The version of the distribution.
1809
1810DISTROOVERRIDES
1811 A colon-separated list of overrides specific to the current
1812 distribution. By default, this list includes the value of
1813 ```DISTRO`` <#var-DISTRO>`__.
1814
1815 You can extend ``DISTROOVERRIDES`` to add extra overrides that should
1816 apply to the distribution.
1817
1818 The underlying mechanism behind ``DISTROOVERRIDES`` is simply that it
1819 is included in the default value of
1820 ```OVERRIDES`` <#var-OVERRIDES>`__.
1821
1822DL_DIR
1823 The central download directory used by the build process to store
1824 downloads. By default, ``DL_DIR`` gets files suitable for mirroring
1825 for everything except Git repositories. If you want tarballs of Git
1826 repositories, use the
1827 ```BB_GENERATE_MIRROR_TARBALLS`` <#var-BB_GENERATE_MIRROR_TARBALLS>`__
1828 variable.
1829
1830 You can set this directory by defining the ``DL_DIR`` variable in the
1831 ``conf/local.conf`` file. This directory is self-maintaining and you
1832 should not have to touch it. By default, the directory is
1833 ``downloads`` in the `Build Directory <#build-directory>`__. #DL_DIR
1834 ?= "${TOPDIR}/downloads" To specify a different download directory,
1835 simply remove the comment from the line and provide your directory.
1836
1837 During a first build, the system downloads many different source code
1838 tarballs from various upstream projects. Downloading can take a
1839 while, particularly if your network connection is slow. Tarballs are
1840 all stored in the directory defined by ``DL_DIR`` and the build
1841 system looks there first to find source tarballs.
1842
1843 .. note::
1844
1845 When wiping and rebuilding, you can preserve this directory to
1846 speed up this part of subsequent builds.
1847
1848 You can safely share this directory between multiple builds on the
1849 same development machine. For additional information on how the build
1850 process gets source files when working behind a firewall or proxy
1851 server, see this specific question in the
1852 "`FAQ <#how-does-the-yocto-project-obtain-source-code-and-will-it-work-behind-my-firewall-or-proxy-server>`__"
1853 chapter. You can also refer to the "`Working Behind a Network
1854 Proxy <&YOCTO_WIKI_URL;/wiki/Working_Behind_a_Network_Proxy>`__" Wiki
1855 page.
1856
1857DOC_COMPRESS
1858 When inheriting the ```compress_doc`` <#ref-classes-compress_doc>`__
1859 class, this variable sets the compression policy used when the
1860 OpenEmbedded build system compresses man pages and info pages. By
1861 default, the compression method used is gz (gzip). Other policies
1862 available are xz and bz2.
1863
1864 For information on policies and on how to use this variable, see the
1865 comments in the ``meta/classes/compress_doc.bbclass`` file.
1866
1867EFI_PROVIDER
1868 When building bootable images (i.e. where ``hddimg``, ``iso``, or
1869 ``wic.vmdk`` is in ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__), the
1870 ``EFI_PROVIDER`` variable specifies the EFI bootloader to use. The
1871 default is "grub-efi", but "systemd-boot" can be used instead.
1872
1873 See the ```systemd-boot`` <#ref-classes-systemd-boot>`__ and
1874 ```image-live`` <#ref-classes-image-live>`__ classes for more
1875 information.
1876
1877ENABLE_BINARY_LOCALE_GENERATION
1878 Variable that controls which locales for ``glibc`` are generated
1879 during the build (useful if the target device has 64Mbytes of RAM or
1880 less).
1881
1882ERR_REPORT_DIR
1883 When used with the ```report-error`` <#ref-classes-report-error>`__
1884 class, specifies the path used for storing the debug files created by
1885 the `error reporting
1886 tool <&YOCTO_DOCS_DEV_URL;#using-the-error-reporting-tool>`__, which
1887 allows you to submit build errors you encounter to a central
1888 database. By default, the value of this variable is
1889 ``${``\ ```LOG_DIR`` <#var-LOG_DIR>`__\ ``}/error-report``.
1890
1891 You can set ``ERR_REPORT_DIR`` to the path you want the error
1892 reporting tool to store the debug files as follows in your
1893 ``local.conf`` file: ERR_REPORT_DIR = "path"
1894
1895ERROR_QA
1896 Specifies the quality assurance checks whose failures are reported as
1897 errors by the OpenEmbedded build system. You set this variable in
1898 your distribution configuration file. For a list of the checks you
1899 can control with this variable, see the
1900 "```insane.bbclass`` <#ref-classes-insane>`__" section.
1901
1902EXCLUDE_FROM_SHLIBS
1903 Triggers the OpenEmbedded build system's shared libraries resolver to
1904 exclude an entire package when scanning for shared libraries.
1905
1906 .. note::
1907
1908 The shared libraries resolver's functionality results in part from
1909 the internal function
1910 package_do_shlibs
1911 , which is part of the
1912 do_package
1913 task. You should be aware that the shared libraries resolver might
1914 implicitly define some dependencies between packages.
1915
1916 The ``EXCLUDE_FROM_SHLIBS`` variable is similar to the
1917 ```PRIVATE_LIBS`` <#var-PRIVATE_LIBS>`__ variable, which excludes a
1918 package's particular libraries only and not the whole package.
1919
1920 Use the ``EXCLUDE_FROM_SHLIBS`` variable by setting it to "1" for a
1921 particular package: EXCLUDE_FROM_SHLIBS = "1"
1922
1923EXCLUDE_FROM_WORLD
1924 Directs BitBake to exclude a recipe from world builds (i.e.
1925 ``bitbake world``). During world builds, BitBake locates, parses and
1926 builds all recipes found in every layer exposed in the
1927 ``bblayers.conf`` configuration file.
1928
1929 To exclude a recipe from a world build using this variable, set the
1930 variable to "1" in the recipe.
1931
1932 .. note::
1933
1934 Recipes added to
1935 EXCLUDE_FROM_WORLD
1936 may still be built during a world build in order to satisfy
1937 dependencies of other recipes. Adding a recipe to
1938 EXCLUDE_FROM_WORLD
1939 only ensures that the recipe is not explicitly added to the list
1940 of build targets in a world build.
1941
1942EXTENDPE
1943 Used with file and pathnames to create a prefix for a recipe's
1944 version based on the recipe's ```PE`` <#var-PE>`__ value. If ``PE``
1945 is set and greater than zero for a recipe, ``EXTENDPE`` becomes that
1946 value (e.g if ``PE`` is equal to "1" then ``EXTENDPE`` becomes "1_").
1947 If a recipe's ``PE`` is not set (the default) or is equal to zero,
1948 ``EXTENDPE`` becomes "".
1949
1950 See the ```STAMP`` <#var-STAMP>`__ variable for an example.
1951
1952EXTENDPKGV
1953 The full package version specification as it appears on the final
1954 packages produced by a recipe. The variable's value is normally used
1955 to fix a runtime dependency to the exact same version of another
1956 package in the same recipe: RDEPENDS_${PN}-additional-module = "${PN}
1957 (= ${EXTENDPKGV})"
1958
1959 The dependency relationships are intended to force the package
1960 manager to upgrade these types of packages in lock-step.
1961
1962EXTERNAL_KERNEL_TOOLS
1963 When set, the ``EXTERNAL_KERNEL_TOOLS`` variable indicates that these
1964 tools are not in the source tree.
1965
1966 When kernel tools are available in the tree, they are preferred over
1967 any externally installed tools. Setting the ``EXTERNAL_KERNEL_TOOLS``
1968 variable tells the OpenEmbedded build system to prefer the installed
1969 external tools. See the
1970 ```kernel-yocto`` <#ref-classes-kernel-yocto>`__ class in
1971 ``meta/classes`` to see how the variable is used.
1972
1973EXTERNALSRC
1974 When inheriting the ```externalsrc`` <#ref-classes-externalsrc>`__
1975 class, this variable points to the source tree, which is outside of
1976 the OpenEmbedded build system. When set, this variable sets the
1977 ```S`` <#var-S>`__ variable, which is what the OpenEmbedded build
1978 system uses to locate unpacked recipe source code.
1979
1980 For more information on ``externalsrc.bbclass``, see the
1981 "```externalsrc.bbclass`` <#ref-classes-externalsrc>`__" section. You
1982 can also find information on how to use this variable in the
1983 "`Building Software from an External
1984 Source <&YOCTO_DOCS_DEV_URL;#building-software-from-an-external-source>`__"
1985 section in the Yocto Project Development Tasks Manual.
1986
1987EXTERNALSRC_BUILD
1988 When inheriting the ```externalsrc`` <#ref-classes-externalsrc>`__
1989 class, this variable points to the directory in which the recipe's
1990 source code is built, which is outside of the OpenEmbedded build
1991 system. When set, this variable sets the ```B`` <#var-B>`__ variable,
1992 which is what the OpenEmbedded build system uses to locate the Build
1993 Directory.
1994
1995 For more information on ``externalsrc.bbclass``, see the
1996 "```externalsrc.bbclass`` <#ref-classes-externalsrc>`__" section. You
1997 can also find information on how to use this variable in the
1998 "`Building Software from an External
1999 Source <&YOCTO_DOCS_DEV_URL;#building-software-from-an-external-source>`__"
2000 section in the Yocto Project Development Tasks Manual.
2001
2002EXTRA_AUTORECONF
2003 For recipes inheriting the ```autotools`` <#ref-classes-autotools>`__
2004 class, you can use ``EXTRA_AUTORECONF`` to specify extra options to
2005 pass to the ``autoreconf`` command that is executed during the
2006 ```do_configure`` <#ref-tasks-configure>`__ task.
2007
2008 The default value is "--exclude=autopoint".
2009
2010EXTRA_IMAGE_FEATURES
2011 A list of additional features to include in an image. When listing
2012 more than one feature, separate them with a space.
2013
2014 Typically, you configure this variable in your ``local.conf`` file,
2015 which is found in the `Build Directory <#build-directory>`__.
2016 Although you can use this variable from within a recipe, best
2017 practices dictate that you do not.
2018
2019 .. note::
2020
2021 To enable primary features from within the image recipe, use the
2022 IMAGE_FEATURES
2023 variable.
2024
2025 Here are some examples of features you can add: "dbg-pkgs" - Adds
2026 -dbg packages for all installed packages including symbol information
2027 for debugging and profiling. "debug-tweaks" - Makes an image suitable
2028 for debugging. For example, allows root logins without passwords and
2029 enables post-installation logging. See the 'allow-empty-password' and
2030 'post-install-logging' features in the "`Image
2031 Features <#ref-features-image>`__" section for more information.
2032 "dev-pkgs" - Adds -dev packages for all installed packages. This is
2033 useful if you want to develop against the libraries in the image.
2034 "read-only-rootfs" - Creates an image whose root filesystem is
2035 read-only. See the "`Creating a Read-Only Root
2036 Filesystem <&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem>`__"
2037 section in the Yocto Project Development Tasks Manual for more
2038 information "tools-debug" - Adds debugging tools such as gdb and
2039 strace. "tools-sdk" - Adds development tools such as gcc, make,
2040 pkgconfig and so forth. "tools-testapps" - Adds useful testing tools
2041 such as ts_print, aplay, arecord and so forth.
2042
2043 For a complete list of image features that ships with the Yocto
2044 Project, see the "`Image Features <#ref-features-image>`__" section.
2045
2046 For an example that shows how to customize your image by using this
2047 variable, see the "`Customizing Images Using Custom
2048 ``IMAGE_FEATURES`` and
2049 ``EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_DEV_URL;#usingpoky-extend-customimage-imagefeatures>`__"
2050 section in the Yocto Project Development Tasks Manual.
2051
2052EXTRA_IMAGECMD
2053 Specifies additional options for the image creation command that has
2054 been specified in ```IMAGE_CMD`` <#var-IMAGE_CMD>`__. When setting
2055 this variable, use an override for the associated image type. Here is
2056 an example: EXTRA_IMAGECMD_ext3 ?= "-i 4096"
2057
2058EXTRA_IMAGEDEPENDS
2059 A list of recipes to build that do not provide packages for
2060 installing into the root filesystem.
2061
2062 Sometimes a recipe is required to build the final image but is not
2063 needed in the root filesystem. You can use the ``EXTRA_IMAGEDEPENDS``
2064 variable to list these recipes and thus specify the dependencies. A
2065 typical example is a required bootloader in a machine configuration.
2066
2067 .. note::
2068
2069 To add packages to the root filesystem, see the various
2070 \*
2071 RDEPENDS
2072 and
2073 \*
2074 RRECOMMENDS
2075 variables.
2076
2077EXTRANATIVEPATH
2078 A list of subdirectories of
2079 ``${``\ ```STAGING_BINDIR_NATIVE`` <#var-STAGING_BINDIR_NATIVE>`__\ ``}``
2080 added to the beginning of the environment variable ``PATH``. As an
2081 example, the following prepends
2082 "${STAGING_BINDIR_NATIVE}/foo:${STAGING_BINDIR_NATIVE}/bar:" to
2083 ``PATH``: EXTRANATIVEPATH = "foo bar"
2084
2085EXTRA_OECMAKE
2086 Additional `CMake <https://cmake.org/overview/>`__ options. See the
2087 ```cmake`` <#ref-classes-cmake>`__ class for additional information.
2088
2089EXTRA_OECONF
2090 Additional ``configure`` script options. See
2091 ```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__ for
2092 additional information on passing configure script options.
2093
2094EXTRA_OEMAKE
2095 Additional GNU ``make`` options.
2096
2097 Because the ``EXTRA_OEMAKE`` defaults to "", you need to set the
2098 variable to specify any required GNU options.
2099
2100 ```PARALLEL_MAKE`` <#var-PARALLEL_MAKE>`__ and
2101 ```PARALLEL_MAKEINST`` <#var-PARALLEL_MAKEINST>`__ also make use of
2102 ``EXTRA_OEMAKE`` to pass the required flags.
2103
2104EXTRA_OESCONS
2105 When inheriting the ```scons`` <#ref-classes-scons>`__ class, this
2106 variable specifies additional configuration options you want to pass
2107 to the ``scons`` command line.
2108
2109EXTRA_USERS_PARAMS
2110 When inheriting the ```extrausers`` <#ref-classes-extrausers>`__
2111 class, this variable provides image level user and group operations.
2112 This is a more global method of providing user and group
2113 configuration as compared to using the
2114 ```useradd`` <#ref-classes-useradd>`__ class, which ties user and
2115 group configurations to a specific recipe.
2116
2117 The set list of commands you can configure using the
2118 ``EXTRA_USERS_PARAMS`` is shown in the ``extrausers`` class. These
2119 commands map to the normal Unix commands of the same names: #
2120 EXTRA_USERS_PARAMS = "\\ # useradd -p '' tester; \\ # groupadd
2121 developers; \\ # userdel nobody; \\ # groupdel -g video; \\ #
2122 groupmod -g 1020 developers; \\ # usermod -s /bin/sh tester; \\ # "
2123
2124FEATURE_PACKAGES
2125 Defines one or more packages to include in an image when a specific
2126 item is included in ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__.
2127 When setting the value, ``FEATURE_PACKAGES`` should have the name of
2128 the feature item as an override. Here is an example:
2129 FEATURE_PACKAGES_widget = "package1 package2"
2130
2131 In this example, if "widget" were added to ``IMAGE_FEATURES``,
2132 package1 and package2 would be included in the image.
2133
2134 .. note::
2135
2136 Packages installed by features defined through
2137 FEATURE_PACKAGES
2138 are often package groups. While similarly named, you should not
2139 confuse the
2140 FEATURE_PACKAGES
2141 variable with package groups, which are discussed elsewhere in the
2142 documentation.
2143
2144FEED_DEPLOYDIR_BASE_URI
2145 Points to the base URL of the server and location within the
2146 document-root that provides the metadata and packages required by
2147 OPKG to support runtime package management of IPK packages. You set
2148 this variable in your ``local.conf`` file.
2149
2150 Consider the following example: FEED_DEPLOYDIR_BASE_URI =
2151 "http://192.168.7.1/BOARD-dir" This example assumes you are serving
2152 your packages over HTTP and your databases are located in a directory
2153 named ``BOARD-dir``, which is underneath your HTTP server's
2154 document-root. In this case, the OpenEmbedded build system generates
2155 a set of configuration files for you in your target that work with
2156 the feed.
2157
2158FILES
2159 The list of files and directories that are placed in a package. The
2160 ```PACKAGES`` <#var-PACKAGES>`__ variable lists the packages
2161 generated by a recipe.
2162
2163 To use the ``FILES`` variable, provide a package name override that
2164 identifies the resulting package. Then, provide a space-separated
2165 list of files or paths that identify the files you want included as
2166 part of the resulting package. Here is an example: FILES_${PN} +=
2167 "${bindir}/mydir1 ${bindir}/mydir2/myfile"
2168
2169 .. note::
2170
2171 - When specifying files or paths, you can pattern match using
2172 Python's
2173 ```glob`` <https://docs.python.org/2/library/glob.html>`__
2174 syntax. For details on the syntax, see the documentation by
2175 following the previous link.
2176
2177 - When specifying paths as part of the ``FILES`` variable, it is
2178 good practice to use appropriate path variables. For example,
2179 use ``${sysconfdir}`` rather than ``/etc``, or ``${bindir}``
2180 rather than ``/usr/bin``. You can find a list of these
2181 variables at the top of the ``meta/conf/bitbake.conf`` file in
2182 the `Source Directory <#source-directory>`__. You will also
2183 find the default values of the various ``FILES_*`` variables in
2184 this file.
2185
2186 If some of the files you provide with the ``FILES`` variable are
2187 editable and you know they should not be overwritten during the
2188 package update process by the Package Management System (PMS), you
2189 can identify these files so that the PMS will not overwrite them. See
2190 the ```CONFFILES`` <#var-CONFFILES>`__ variable for information on
2191 how to identify these files to the PMS.
2192
2193FILES_SOLIBSDEV
2194 Defines the file specification to match
2195 ```SOLIBSDEV`` <#var-SOLIBSDEV>`__. In other words,
2196 ``FILES_SOLIBSDEV`` defines the full path name of the development
2197 symbolic link (symlink) for shared libraries on the target platform.
2198
2199 The following statement from the ``bitbake.conf`` shows how it is
2200 set: FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV}
2201 ${libdir}/lib*${SOLIBSDEV}"
2202
2203FILESEXTRAPATHS
2204 Extends the search path the OpenEmbedded build system uses when
2205 looking for files and patches as it processes recipes and append
2206 files. The default directories BitBake uses when it processes recipes
2207 are initially defined by the ```FILESPATH`` <#var-FILESPATH>`__
2208 variable. You can extend ``FILESPATH`` variable by using
2209 ``FILESEXTRAPATHS``.
2210
2211 Best practices dictate that you accomplish this by using
2212 ``FILESEXTRAPATHS`` from within a ``.bbappend`` file and that you
2213 prepend paths as follows: FILESEXTRAPATHS_prepend :=
2214 "${THISDIR}/${PN}:" In the above example, the build system first
2215 looks for files in a directory that has the same name as the
2216 corresponding append file.
2217
2218 .. note::
2219
2220 When extending ``FILESEXTRAPATHS``, be sure to use the immediate
2221 expansion (``:=``) operator. Immediate expansion makes sure that
2222 BitBake evaluates ```THISDIR`` <#var-THISDIR>`__ at the time the
2223 directive is encountered rather than at some later time when
2224 expansion might result in a directory that does not contain the
2225 files you need.
2226
2227 Also, include the trailing separating colon character if you are
2228 prepending. The trailing colon character is necessary because you
2229 are directing BitBake to extend the path by prepending directories
2230 to the search path.
2231
2232 Here is another common use: FILESEXTRAPATHS_prepend :=
2233 "${THISDIR}/files:" In this example, the build system extends the
2234 ``FILESPATH`` variable to include a directory named ``files`` that is
2235 in the same directory as the corresponding append file.
2236
2237 This next example specifically adds three paths:
2238 FILESEXTRAPATHS_prepend := "path_1:path_2:path_3:"
2239
2240 A final example shows how you can extend the search path and include
2241 a ```MACHINE`` <#var-MACHINE>`__-specific override, which is useful
2242 in a BSP layer: FILESEXTRAPATHS_prepend_intel-x86-common :=
2243 "${THISDIR}/${PN}:" The previous statement appears in the
2244 ``linux-yocto-dev.bbappend`` file, which is found in the Yocto
2245 Project `Source
2246 Repositories <&YOCTO_DOCS_OM_URL;#source-repositories>`__ in
2247 ``meta-intel/common/recipes-kernel/linux``. Here, the machine
2248 override is a special ```PACKAGE_ARCH`` <#var-PACKAGE_ARCH>`__
2249 definition for multiple ``meta-intel`` machines.
2250
2251 .. note::
2252
2253 For a layer that supports a single BSP, the override could just be
2254 the value of
2255 MACHINE
2256 .
2257
2258 By prepending paths in ``.bbappend`` files, you allow multiple append
2259 files that reside in different layers but are used for the same
2260 recipe to correctly extend the path.
2261
2262FILESOVERRIDES
2263 A subset of ```OVERRIDES`` <#var-OVERRIDES>`__ used by the
2264 OpenEmbedded build system for creating
2265 ```FILESPATH`` <#var-FILESPATH>`__. The ``FILESOVERRIDES`` variable
2266 uses overrides to automatically extend the
2267 ```FILESPATH`` <#var-FILESPATH>`__ variable. For an example of how
2268 that works, see the ```FILESPATH`` <#var-FILESPATH>`__ variable
2269 description. Additionally, you find more information on how overrides
2270 are handled in the "`Conditional Syntax
2271 (Overrides) <&YOCTO_DOCS_BB_URL;#conditional-syntax-overrides>`__"
2272 section of the BitBake User Manual.
2273
2274 By default, the ``FILESOVERRIDES`` variable is defined as:
2275 FILESOVERRIDES =
2276 "${TRANSLATED_TARGET_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}"
2277
2278 .. note::
2279
2280 Do not hand-edit the
2281 FILESOVERRIDES
2282 variable. The values match up with expected overrides and are used
2283 in an expected manner by the build system.
2284
2285FILESPATH
2286 The default set of directories the OpenEmbedded build system uses
2287 when searching for patches and files.
2288
2289 During the build process, BitBake searches each directory in
2290 ``FILESPATH`` in the specified order when looking for files and
2291 patches specified by each ``file://`` URI in a recipe's
2292 ```SRC_URI`` <#var-SRC_URI>`__ statements.
2293
2294 The default value for the ``FILESPATH`` variable is defined in the
2295 ``base.bbclass`` class found in ``meta/classes`` in the `Source
2296 Directory <#source-directory>`__: FILESPATH =
2297 "${@base_set_filespath(["${FILE_DIRNAME}/${BP}", \\
2298 "${FILE_DIRNAME}/${BPN}", "${FILE_DIRNAME}/files"], d)}" The
2299 ``FILESPATH`` variable is automatically extended using the overrides
2300 from the ```FILESOVERRIDES`` <#var-FILESOVERRIDES>`__ variable.
2301
2302 .. note::
2303
2304 - Do not hand-edit the ``FILESPATH`` variable. If you want the
2305 build system to look in directories other than the defaults,
2306 extend the ``FILESPATH`` variable by using the
2307 ```FILESEXTRAPATHS`` <#var-FILESEXTRAPATHS>`__ variable.
2308
2309 - Be aware that the default ``FILESPATH`` directories do not map
2310 to directories in custom layers where append files
2311 (``.bbappend``) are used. If you want the build system to find
2312 patches or files that reside with your append files, you need
2313 to extend the ``FILESPATH`` variable by using the
2314 ``FILESEXTRAPATHS`` variable.
2315
2316 You can take advantage of this searching behavior in useful ways. For
2317 example, consider a case where the following directory structure
2318 exists for general and machine-specific configurations:
2319 files/defconfig files/MACHINEA/defconfig files/MACHINEB/defconfig
2320 Also in the example, the ``SRC_URI`` statement contains
2321 "file://defconfig". Given this scenario, you can set
2322 ```MACHINE`` <#var-MACHINE>`__ to "MACHINEA" and cause the build
2323 system to use files from ``files/MACHINEA``. Set ``MACHINE`` to
2324 "MACHINEB" and the build system uses files from ``files/MACHINEB``.
2325 Finally, for any machine other than "MACHINEA" and "MACHINEB", the
2326 build system uses files from ``files/defconfig``.
2327
2328 You can find out more about the patching process in the
2329 "`Patching <&YOCTO_DOCS_OM_URL;#patching-dev-environment>`__" section
2330 in the Yocto Project Overview and Concepts Manual and the "`Patching
2331 Code <&YOCTO_DOCS_DEV_URL;#new-recipe-patching-code>`__" section in
2332 the Yocto Project Development Tasks Manual. See the
2333 ```do_patch`` <#ref-tasks-patch>`__ task as well.
2334
2335FILESYSTEM_PERMS_TABLES
2336 Allows you to define your own file permissions settings table as part
2337 of your configuration for the packaging process. For example, suppose
2338 you need a consistent set of custom permissions for a set of groups
2339 and users across an entire work project. It is best to do this in the
2340 packages themselves but this is not always possible.
2341
2342 By default, the OpenEmbedded build system uses the ``fs-perms.txt``,
2343 which is located in the ``meta/files`` folder in the `Source
2344 Directory <#source-directory>`__. If you create your own file
2345 permissions setting table, you should place it in your layer or the
2346 distro's layer.
2347
2348 You define the ``FILESYSTEM_PERMS_TABLES`` variable in the
2349 ``conf/local.conf`` file, which is found in the `Build
2350 Directory <#build-directory>`__, to point to your custom
2351 ``fs-perms.txt``. You can specify more than a single file permissions
2352 setting table. The paths you specify to these files must be defined
2353 within the ```BBPATH`` <#var-BBPATH>`__ variable.
2354
2355 For guidance on how to create your own file permissions settings
2356 table file, examine the existing ``fs-perms.txt``.
2357
2358FIT_HASH_ALG
2359 Specifies the hash algorithm used in creating the FIT Image. For e.g. sha256.
2360
2361FIT_SIGN_ALG</glossterm>
2362 Specifies the signature algorithm used in creating the FIT Image.
2363 For e.g. rsa2048.
2364
2365FONT_EXTRA_RDEPENDS
2366 When inheriting the ```fontcache`` <#ref-classes-fontcache>`__ class,
2367 this variable specifies the runtime dependencies for font packages.
2368 By default, the ``FONT_EXTRA_RDEPENDS`` is set to "fontconfig-utils".
2369
2370FONT_PACKAGES
2371 When inheriting the ```fontcache`` <#ref-classes-fontcache>`__ class,
2372 this variable identifies packages containing font files that need to
2373 be cached by Fontconfig. By default, the ``fontcache`` class assumes
2374 that fonts are in the recipe's main package (i.e.
2375 ``${``\ ```PN`` <#var-PN>`__\ ``}``). Use this variable if fonts you
2376 need are in a package other than that main package.
2377
2378FORCE_RO_REMOVE
2379 Forces the removal of the packages listed in ``ROOTFS_RO_UNNEEDED``
2380 during the generation of the root filesystem.
2381
2382 Set the variable to "1" to force the removal of these packages.
2383
2384FULL_OPTIMIZATION
2385 The options to pass in ``TARGET_CFLAGS`` and ``CFLAGS`` when
2386 compiling an optimized system. This variable defaults to "-O2 -pipe
2387 ${DEBUG_FLAGS}".
2388
2389GCCPIE
2390 Enables Position Independent Executables (PIE) within the GNU C
2391 Compiler (GCC). Enabling PIE in the GCC makes Return Oriented
2392 Programming (ROP) attacks much more difficult to execute.
2393
2394 By default the ``security_flags.inc`` file enables PIE by setting the
2395 variable as follows: GCCPIE ?= "--enable-default-pie"
2396
2397GCCVERSION
2398 Specifies the default version of the GNU C Compiler (GCC) used for
2399 compilation. By default, ``GCCVERSION`` is set to "8.x" in the
2400 ``meta/conf/distro/include/tcmode-default.inc`` include file:
2401 GCCVERSION ?= "8.%" You can override this value by setting it in a
2402 configuration file such as the ``local.conf``.
2403
2404GDB
2405 The minimal command and arguments to run the GNU Debugger.
2406
2407GITDIR
2408 The directory in which a local copy of a Git repository is stored
2409 when it is cloned.
2410
2411GLIBC_GENERATE_LOCALES
2412 Specifies the list of GLIBC locales to generate should you not wish
2413 to generate all LIBC locals, which can be time consuming.
2414
2415 .. note::
2416
2417 If you specifically remove the locale
2418 en_US.UTF-8
2419 , you must set
2420 IMAGE_LINGUAS
2421 appropriately.
2422
2423 You can set ``GLIBC_GENERATE_LOCALES`` in your ``local.conf`` file.
2424 By default, all locales are generated. GLIBC_GENERATE_LOCALES =
2425 "en_GB.UTF-8 en_US.UTF-8"
2426
2427GROUPADD_PARAM
2428 When inheriting the ```useradd`` <#ref-classes-useradd>`__ class,
2429 this variable specifies for a package what parameters should be
2430 passed to the ``groupadd`` command if you wish to add a group to the
2431 system when the package is installed.
2432
2433 Here is an example from the ``dbus`` recipe: GROUPADD_PARAM_${PN} =
2434 "-r netdev" For information on the standard Linux shell command
2435 ``groupadd``, see ` <http://linux.die.net/man/8/groupadd>`__.
2436
2437GROUPMEMS_PARAM
2438 When inheriting the ```useradd`` <#ref-classes-useradd>`__ class,
2439 this variable specifies for a package what parameters should be
2440 passed to the ``groupmems`` command if you wish to modify the members
2441 of a group when the package is installed.
2442
2443 For information on the standard Linux shell command ``groupmems``,
2444 see ` <http://linux.die.net/man/8/groupmems>`__.
2445
2446GRUB_GFXSERIAL
2447 Configures the GNU GRand Unified Bootloader (GRUB) to have graphics
2448 and serial in the boot menu. Set this variable to "1" in your
2449 ``local.conf`` or distribution configuration file to enable graphics
2450 and serial in the menu.
2451
2452 See the ```grub-efi`` <#ref-classes-grub-efi>`__ class for more
2453 information on how this variable is used.
2454
2455GRUB_OPTS
2456 Additional options to add to the GNU GRand Unified Bootloader (GRUB)
2457 configuration. Use a semi-colon character (``;``) to separate
2458 multiple options.
2459
2460 The ``GRUB_OPTS`` variable is optional. See the
2461 ```grub-efi`` <#ref-classes-grub-efi>`__ class for more information
2462 on how this variable is used.
2463
2464GRUB_TIMEOUT
2465 Specifies the timeout before executing the default ``LABEL`` in the
2466 GNU GRand Unified Bootloader (GRUB).
2467
2468 The ``GRUB_TIMEOUT`` variable is optional. See the
2469 ```grub-efi`` <#ref-classes-grub-efi>`__ class for more information
2470 on how this variable is used.
2471
2472GTKIMMODULES_PACKAGES
2473 When inheriting the
2474 ```gtk-immodules-cache`` <#ref-classes-gtk-immodules-cache>`__ class,
2475 this variable specifies the packages that contain the GTK+ input
2476 method modules being installed when the modules are in packages other
2477 than the main package.
2478
2479HOMEPAGE
2480 Website where more information about the software the recipe is
2481 building can be found.
2482
2483HOST_ARCH
2484 The name of the target architecture, which is normally the same as
2485 ```TARGET_ARCH`` <#var-TARGET_ARCH>`__. The OpenEmbedded build system
2486 supports many architectures. Here is an example list of architectures
2487 supported. This list is by no means complete as the architecture is
2488 configurable: arm i586 x86_64 powerpc powerpc64 mips mipsel
2489
2490HOST_CC_ARCH
2491 Specifies architecture-specific compiler flags that are passed to the
2492 C compiler.
2493
2494 Default initialization for ``HOST_CC_ARCH`` varies depending on what
2495 is being built:
2496
2497 - ```TARGET_CC_ARCH`` <#var-TARGET_CC_ARCH>`__ when building for the
2498 target
2499
2500 - ``BUILD_CC_ARCH`` when building for the build host (i.e.
2501 ``-native``)
2502
2503 - ``BUILDSDK_CC_ARCH`` when building for an SDK (i.e.
2504 ``nativesdk-``)
2505
2506HOST_OS
2507 Specifies the name of the target operating system, which is normally
2508 the same as the ```TARGET_OS`` <#var-TARGET_OS>`__. The variable can
2509 be set to "linux" for ``glibc``-based systems and to "linux-musl" for
2510 ``musl``. For ARM/EABI targets, there are also "linux-gnueabi" and
2511 "linux-musleabi" values possible.
2512
2513HOST_PREFIX
2514 Specifies the prefix for the cross-compile toolchain. ``HOST_PREFIX``
2515 is normally the same as ```TARGET_PREFIX`` <#var-TARGET_PREFIX>`__.
2516
2517HOST_SYS
2518 Specifies the system, including the architecture and the operating
2519 system, for which the build is occurring in the context of the
2520 current recipe.
2521
2522 The OpenEmbedded build system automatically sets this variable based
2523 on ```HOST_ARCH`` <#var-HOST_ARCH>`__,
2524 ```HOST_VENDOR`` <#var-HOST_VENDOR>`__, and
2525 ```HOST_OS`` <#var-HOST_OS>`__ variables.
2526
2527 .. note::
2528
2529 You do not need to set the variable yourself.
2530
2531 Consider these two examples:
2532
2533 - Given a native recipe on a 32-bit x86 machine running Linux, the
2534 value is "i686-linux".
2535
2536 - Given a recipe being built for a little-endian MIPS target running
2537 Linux, the value might be "mipsel-linux".
2538
2539HOSTTOOLS
2540 A space-separated list (filter) of tools on the build host that
2541 should be allowed to be called from within build tasks. Using this
2542 filter helps reduce the possibility of host contamination. If a tool
2543 specified in the value of ``HOSTTOOLS`` is not found on the build
2544 host, the OpenEmbedded build system produces an error and the build
2545 is not started.
2546
2547 For additional information, see
2548 ```HOSTTOOLS_NONFATAL`` <#var-HOSTTOOLS_NONFATAL>`__.
2549
2550HOSTTOOLS_NONFATAL
2551 A space-separated list (filter) of tools on the build host that
2552 should be allowed to be called from within build tasks. Using this
2553 filter helps reduce the possibility of host contamination. Unlike
2554 ```HOSTTOOLS`` <#var-HOSTTOOLS>`__, the OpenEmbedded build system
2555 does not produce an error if a tool specified in the value of
2556 ``HOSTTOOLS_NONFATAL`` is not found on the build host. Thus, you can
2557 use ``HOSTTOOLS_NONFATAL`` to filter optional host tools.
2558
2559HOST_VENDOR
2560 Specifies the name of the vendor. ``HOST_VENDOR`` is normally the
2561 same as ```TARGET_VENDOR`` <#var-TARGET_VENDOR>`__.
2562
2563ICECC_DISABLED
2564 Disables or enables the ``icecc`` (Icecream) function. For more
2565 information on this function and best practices for using this
2566 variable, see the "```icecc.bbclass`` <#ref-classes-icecc>`__"
2567 section.
2568
2569 Setting this variable to "1" in your ``local.conf`` disables the
2570 function: ICECC_DISABLED ??= "1" To enable the function, set the
2571 variable as follows: ICECC_DISABLED = ""
2572
2573ICECC_ENV_EXEC
2574 Points to the ``icecc-create-env`` script that you provide. This
2575 variable is used by the ```icecc`` <#ref-classes-icecc>`__ class. You
2576 set this variable in your ``local.conf`` file.
2577
2578 If you do not point to a script that you provide, the OpenEmbedded
2579 build system uses the default script provided by the
2580 ``icecc-create-env.bb`` recipe, which is a modified version and not
2581 the one that comes with ``icecc``.
2582
2583ICECC_PARALLEL_MAKE
2584 Extra options passed to the ``make`` command during the
2585 ```do_compile`` <#ref-tasks-compile>`__ task that specify parallel
2586 compilation. This variable usually takes the form of "-j x", where x
2587 represents the maximum number of parallel threads ``make`` can run.
2588
2589 .. note::
2590
2591 The options passed affect builds on all enabled machines on the
2592 network, which are machines running the
2593 iceccd
2594 daemon.
2595
2596 If your enabled machines support multiple cores, coming up with the
2597 maximum number of parallel threads that gives you the best
2598 performance could take some experimentation since machine speed,
2599 network lag, available memory, and existing machine loads can all
2600 affect build time. Consequently, unlike the
2601 ```PARALLEL_MAKE`` <#var-PARALLEL_MAKE>`__ variable, there is no
2602 rule-of-thumb for setting ``ICECC_PARALLEL_MAKE`` to achieve optimal
2603 performance.
2604
2605 If you do not set ``ICECC_PARALLEL_MAKE``, the build system does not
2606 use it (i.e. the system does not detect and assign the number of
2607 cores as is done with ``PARALLEL_MAKE``).
2608
2609ICECC_PATH
2610 The location of the ``icecc`` binary. You can set this variable in
2611 your ``local.conf`` file. If your ``local.conf`` file does not define
2612 this variable, the ```icecc`` <#ref-classes-icecc>`__ class attempts
2613 to define it by locating ``icecc`` using ``which``.
2614
2615ICECC_USER_CLASS_BL
2616 Identifies user classes that you do not want the Icecream distributed
2617 compile support to consider. This variable is used by the
2618 ```icecc`` <#ref-classes-icecc>`__ class. You set this variable in
2619 your ``local.conf`` file.
2620
2621 When you list classes using this variable, you are "blacklisting"
2622 them from distributed compilation across remote hosts. Any classes
2623 you list will be distributed and compiled locally.
2624
2625ICECC_USER_PACKAGE_BL
2626 Identifies user recipes that you do not want the Icecream distributed
2627 compile support to consider. This variable is used by the
2628 ```icecc`` <#ref-classes-icecc>`__ class. You set this variable in
2629 your ``local.conf`` file.
2630
2631 When you list packages using this variable, you are "blacklisting"
2632 them from distributed compilation across remote hosts. Any packages
2633 you list will be distributed and compiled locally.
2634
2635ICECC_USER_PACKAGE_WL
2636 Identifies user recipes that use an empty
2637 ```PARALLEL_MAKE`` <#var-PARALLEL_MAKE>`__ variable that you want to
2638 force remote distributed compilation on using the Icecream
2639 distributed compile support. This variable is used by the
2640 ```icecc`` <#ref-classes-icecc>`__ class. You set this variable in
2641 your ``local.conf`` file.
2642
2643IMAGE_BASENAME
2644 The base name of image output files. This variable defaults to the
2645 recipe name (``${``\ ```PN`` <#var-PN>`__\ ``}``).
2646
2647IMAGE_BOOT_FILES
2648 A space-separated list of files installed into the boot partition
2649 when preparing an image using the Wic tool with the
2650 ``bootimg-partition`` or ``bootimg-efi`` source plugin. By default,
2651 the files are
2652 installed under the same name as the source files. To change the
2653 installed name, separate it from the original name with a semi-colon
2654 (;). Source files need to be located in
2655 ```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__. Here are two
2656 examples: IMAGE_BOOT_FILES = "u-boot.img uImage;kernel"
2657 IMAGE_BOOT_FILES = "u-boot.${UBOOT_SUFFIX} ${KERNEL_IMAGETYPE}"
2658
2659 Alternatively, source files can be picked up using a glob pattern. In
2660 this case, the destination file must have the same name as the base
2661 name of the source file path. To install files into a directory
2662 within the target location, pass its name after a semi-colon (;).
2663 Here are two examples: IMAGE_BOOT_FILES = "bcm2835-bootfiles/*"
2664 IMAGE_BOOT_FILES = "bcm2835-bootfiles/*;boot/" The first example
2665 installs all files from ``${DEPLOY_DIR_IMAGE}/bcm2835-bootfiles``
2666 into the root of the target partition. The second example installs
2667 the same files into a ``boot`` directory within the target partition.
2668
2669 You can find information on how to use the Wic tool in the "`Creating
2670 Partitioned Images Using
2671 Wic <&YOCTO_DOCS_DEV_URL;#creating-partitioned-images-using-wic>`__"
2672 section of the Yocto Project Development Tasks Manual. Reference
2673 material for Wic is located in the "`OpenEmbedded Kickstart (.wks)
2674 Reference <&YOCTO_DOCS_REF_URL;#ref-kickstart>`__" chapter.
2675
2676IMAGE_CLASSES
2677 A list of classes that all images should inherit. You typically use
2678 this variable to specify the list of classes that register the
2679 different types of images the OpenEmbedded build system creates.
2680
2681 The default value for ``IMAGE_CLASSES`` is ``image_types``. You can
2682 set this variable in your ``local.conf`` or in a distribution
2683 configuration file.
2684
2685 For more information, see ``meta/classes/image_types.bbclass`` in the
2686 `Source Directory <#source-directory>`__.
2687
2688IMAGE_CMD
2689 Specifies the command to create the image file for a specific image
2690 type, which corresponds to the value set set in
2691 ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__, (e.g. ``ext3``,
2692 ``btrfs``, and so forth). When setting this variable, you should use
2693 an override for the associated type. Here is an example:
2694 IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} \\ --faketime
2695 --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 \\
2696 ${EXTRA_IMAGECMD}"
2697
2698 You typically do not need to set this variable unless you are adding
2699 support for a new image type. For more examples on how to set this
2700 variable, see the ```image_types`` <#ref-classes-image_types>`__
2701 class file, which is ``meta/classes/image_types.bbclass``.
2702
2703IMAGE_DEVICE_TABLES
2704 Specifies one or more files that contain custom device tables that
2705 are passed to the ``makedevs`` command as part of creating an image.
2706 These files list basic device nodes that should be created under
2707 ``/dev`` within the image. If ``IMAGE_DEVICE_TABLES`` is not set,
2708 ``files/device_table-minimal.txt`` is used, which is located by
2709 ```BBPATH`` <#var-BBPATH>`__. For details on how you should write
2710 device table files, see ``meta/files/device_table-minimal.txt`` as an
2711 example.
2712
2713IMAGE_FEATURES
2714 The primary list of features to include in an image. Typically, you
2715 configure this variable in an image recipe. Although you can use this
2716 variable from your ``local.conf`` file, which is found in the `Build
2717 Directory <#build-directory>`__, best practices dictate that you do
2718 not.
2719
2720 .. note::
2721
2722 To enable extra features from outside the image recipe, use the
2723 EXTRA_IMAGE_FEATURES
2724 variable.
2725
2726 For a list of image features that ships with the Yocto Project, see
2727 the "`Image Features <#ref-features-image>`__" section.
2728
2729 For an example that shows how to customize your image by using this
2730 variable, see the "`Customizing Images Using Custom
2731 ``IMAGE_FEATURES`` and
2732 ``EXTRA_IMAGE_FEATURES`` <&YOCTO_DOCS_DEV_URL;#usingpoky-extend-customimage-imagefeatures>`__"
2733 section in the Yocto Project Development Tasks Manual.
2734
2735IMAGE_FSTYPES
2736 Specifies the formats the OpenEmbedded build system uses during the
2737 build when creating the root filesystem. For example, setting
2738 ``IMAGE_FSTYPES`` as follows causes the build system to create root
2739 filesystems using two formats: ``.ext3`` and ``.tar.bz2``:
2740 IMAGE_FSTYPES = "ext3 tar.bz2"
2741
2742 For the complete list of supported image formats from which you can
2743 choose, see ```IMAGE_TYPES`` <#var-IMAGE_TYPES>`__.
2744
2745 .. note::
2746
2747 - If an image recipe uses the "inherit image" line and you are
2748 setting ``IMAGE_FSTYPES`` inside the recipe, you must set
2749 ``IMAGE_FSTYPES`` prior to using the "inherit image" line.
2750
2751 - Due to the way the OpenEmbedded build system processes this
2752 variable, you cannot update its contents by using ``_append``
2753 or ``_prepend``. You must use the ``+=`` operator to add one or
2754 more options to the ``IMAGE_FSTYPES`` variable.
2755
2756IMAGE_INSTALL
2757 Used by recipes to specify the packages to install into an image
2758 through the ```image`` <#ref-classes-image>`__ class. Use the
2759 ``IMAGE_INSTALL`` variable with care to avoid ordering issues.
2760
2761 Image recipes set ``IMAGE_INSTALL`` to specify the packages to
2762 install into an image through ``image.bbclass``. Additionally,
2763 "helper" classes such as the
2764 ```core-image`` <#ref-classes-core-image>`__ class exist that can
2765 take lists used with ``IMAGE_FEATURES`` and turn them into
2766 auto-generated entries in ``IMAGE_INSTALL`` in addition to its
2767 default contents.
2768
2769 When you use this variable, it is best to use it as follows:
2770 IMAGE_INSTALL_append = " package-name" Be sure to include the space
2771 between the quotation character and the start of the package name or
2772 names.
2773
2774 .. note::
2775
2776 - When working with a
2777 ```core-image-minimal-initramfs`` <#images-core-image-minimal-initramfs>`__
2778 image, do not use the ``IMAGE_INSTALL`` variable to specify
2779 packages for installation. Instead, use the
2780 ```PACKAGE_INSTALL`` <#var-PACKAGE_INSTALL>`__ variable, which
2781 allows the initial RAM filesystem (initramfs) recipe to use a
2782 fixed set of packages and not be affected by ``IMAGE_INSTALL``.
2783 For information on creating an initramfs, see the "`Building an
2784 Initial RAM Filesystem (initramfs)
2785 Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__"
2786 section in the Yocto Project Development Tasks Manual.
2787
2788 - Using ``IMAGE_INSTALL`` with the
2789 ```+=`` <&YOCTO_DOCS_BB_URL;#appending-and-prepending>`__
2790 BitBake operator within the ``/conf/local.conf`` file or from
2791 within an image recipe is not recommended. Use of this operator
2792 in these ways can cause ordering issues. Since
2793 ``core-image.bbclass`` sets ``IMAGE_INSTALL`` to a default
2794 value using the
2795 ```?=`` <&YOCTO_DOCS_BB_URL;#setting-a-default-value>`__
2796 operator, using a ``+=`` operation against ``IMAGE_INSTALL``
2797 results in unexpected behavior when used within
2798 ``conf/local.conf``. Furthermore, the same operation from
2799 within an image recipe may or may not succeed depending on the
2800 specific situation. In both these cases, the behavior is
2801 contrary to how most users expect the ``+=`` operator to work.
2802
2803IMAGE_LINGUAS
2804 Specifies the list of locales to install into the image during the
2805 root filesystem construction process. The OpenEmbedded build system
2806 automatically splits locale files, which are used for localization,
2807 into separate packages. Setting the ``IMAGE_LINGUAS`` variable
2808 ensures that any locale packages that correspond to packages already
2809 selected for installation into the image are also installed. Here is
2810 an example: IMAGE_LINGUAS = "pt-br de-de"
2811
2812 In this example, the build system ensures any Brazilian Portuguese
2813 and German locale files that correspond to packages in the image are
2814 installed (i.e. ``*-locale-pt-br`` and ``*-locale-de-de`` as well as
2815 ``*-locale-pt`` and ``*-locale-de``, since some software packages
2816 only provide locale files by language and not by country-specific
2817 language).
2818
2819 See the ```GLIBC_GENERATE_LOCALES`` <#var-GLIBC_GENERATE_LOCALES>`__
2820 variable for information on generating GLIBC locales.
2821
2822IMAGE_MANIFEST
2823 The manifest file for the image. This file lists all the installed
2824 packages that make up the image. The file contains package
2825 information on a line-per-package basis as follows: packagename
2826 packagearch version
2827
2828 The ```image`` <#ref-classes-image>`__ class defines the manifest
2829 file as follows: IMAGE_MANIFEST =
2830 "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.manifest" The location is
2831 derived using the ```DEPLOY_DIR_IMAGE`` <#var-DEPLOY_DIR_IMAGE>`__
2832 and ```IMAGE_NAME`` <#var-IMAGE_NAME>`__ variables. You can find
2833 information on how the image is created in the "`Image
2834 Generation <&YOCTO_DOCS_OM_URL;#image-generation-dev-environment>`__"
2835 section in the Yocto Project Overview and Concepts Manual.
2836
2837IMAGE_NAME
2838 The name of the output image files minus the extension. This variable
2839 is derived using the ```IMAGE_BASENAME`` <#var-IMAGE_BASENAME>`__,
2840 ```MACHINE`` <#var-MACHINE>`__, and ```DATETIME`` <#var-DATETIME>`__
2841 variables: IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
2842
2843IMAGE_OVERHEAD_FACTOR
2844 Defines a multiplier that the build system applies to the initial
2845 image size for cases when the multiplier times the returned disk
2846 usage value for the image is greater than the sum of
2847 ``IMAGE_ROOTFS_SIZE`` and ``IMAGE_ROOTFS_EXTRA_SPACE``. The result of
2848 the multiplier applied to the initial image size creates free disk
2849 space in the image as overhead. By default, the build process uses a
2850 multiplier of 1.3 for this variable. This default value results in
2851 30% free disk space added to the image when this method is used to
2852 determine the final generated image size. You should be aware that
2853 post install scripts and the package management system uses disk
2854 space inside this overhead area. Consequently, the multiplier does
2855 not produce an image with all the theoretical free disk space. See
2856 ``IMAGE_ROOTFS_SIZE`` for information on how the build system
2857 determines the overall image size.
2858
2859 The default 30% free disk space typically gives the image enough room
2860 to boot and allows for basic post installs while still leaving a
2861 small amount of free disk space. If 30% free space is inadequate, you
2862 can increase the default value. For example, the following setting
2863 gives you 50% free space added to the image: IMAGE_OVERHEAD_FACTOR =
2864 "1.5"
2865
2866 Alternatively, you can ensure a specific amount of free disk space is
2867 added to the image by using the ``IMAGE_ROOTFS_EXTRA_SPACE``
2868 variable.
2869
2870IMAGE_PKGTYPE
2871 Defines the package type (i.e. DEB, RPM, IPK, or TAR) used by the
2872 OpenEmbedded build system. The variable is defined appropriately by
2873 the ```package_deb`` <#ref-classes-package_deb>`__,
2874 ```package_rpm`` <#ref-classes-package_rpm>`__,
2875 ```package_ipk`` <#ref-classes-package_ipk>`__, or
2876 ```package_tar`` <#ref-classes-package_tar>`__ class.
2877
2878 .. note::
2879
2880 The
2881 package_tar
2882 class is broken and is not supported. It is recommended that you
2883 do not use it.
2884
2885 The ```populate_sdk_*`` <#ref-classes-populate-sdk-*>`__ and
2886 ```image`` <#ref-classes-image>`__ classes use the ``IMAGE_PKGTYPE``
2887 for packaging up images and SDKs.
2888
2889 You should not set the ``IMAGE_PKGTYPE`` manually. Rather, the
2890 variable is set indirectly through the appropriate
2891 ```package_*`` <#ref-classes-package>`__ class using the
2892 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__ variable. The
2893 OpenEmbedded build system uses the first package type (e.g. DEB, RPM,
2894 or IPK) that appears with the variable
2895
2896 .. note::
2897
2898 Files using the
2899 .tar
2900 format are never used as a substitute packaging format for DEB,
2901 RPM, and IPK formatted files for your image or SDK.
2902
2903IMAGE_POSTPROCESS_COMMAND
2904 Specifies a list of functions to call once the OpenEmbedded build
2905 system creates the final image output files. You can specify
2906 functions separated by semicolons: IMAGE_POSTPROCESS_COMMAND +=
2907 "function; ... "
2908
2909 If you need to pass the root filesystem path to a command within the
2910 function, you can use ``${IMAGE_ROOTFS}``, which points to the
2911 directory that becomes the root filesystem image. See the
2912 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
2913 information.
2914
2915IMAGE_PREPROCESS_COMMAND
2916 Specifies a list of functions to call before the OpenEmbedded build
2917 system creates the final image output files. You can specify
2918 functions separated by semicolons: IMAGE_PREPROCESS_COMMAND +=
2919 "function; ... "
2920
2921 If you need to pass the root filesystem path to a command within the
2922 function, you can use ``${IMAGE_ROOTFS}``, which points to the
2923 directory that becomes the root filesystem image. See the
2924 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
2925 information.
2926
2927IMAGE_ROOTFS
2928 The location of the root filesystem while it is under construction
2929 (i.e. during the ```do_rootfs`` <#ref-tasks-rootfs>`__ task). This
2930 variable is not configurable. Do not change it.
2931
2932IMAGE_ROOTFS_ALIGNMENT
2933 Specifies the alignment for the output image file in Kbytes. If the
2934 size of the image is not a multiple of this value, then the size is
2935 rounded up to the nearest multiple of the value. The default value is
2936 "1". See ```IMAGE_ROOTFS_SIZE`` <#var-IMAGE_ROOTFS_SIZE>`__ for
2937 additional information.
2938
2939IMAGE_ROOTFS_EXTRA_SPACE
2940 Defines additional free disk space created in the image in Kbytes. By
2941 default, this variable is set to "0". This free disk space is added
2942 to the image after the build system determines the image size as
2943 described in ``IMAGE_ROOTFS_SIZE``.
2944
2945 This variable is particularly useful when you want to ensure that a
2946 specific amount of free disk space is available on a device after an
2947 image is installed and running. For example, to be sure 5 Gbytes of
2948 free disk space is available, set the variable as follows:
2949 IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
2950
2951 For example, the Yocto Project Build Appliance specifically requests
2952 40 Gbytes of extra space with the line: IMAGE_ROOTFS_EXTRA_SPACE =
2953 "41943040"
2954
2955IMAGE_ROOTFS_SIZE
2956 Defines the size in Kbytes for the generated image. The OpenEmbedded
2957 build system determines the final size for the generated image using
2958 an algorithm that takes into account the initial disk space used for
2959 the generated image, a requested size for the image, and requested
2960 additional free disk space to be added to the image. Programatically,
2961 the build system determines the final size of the generated image as
2962 follows: if (image-du \* overhead) < rootfs-size:
2963 internal-rootfs-size = rootfs-size + xspace else:
2964 internal-rootfs-size = (image-du \* overhead) + xspace where:
2965 image-du = Returned value of the du command on the image. overhead =
2966 IMAGE_OVERHEAD_FACTOR rootfs-size = IMAGE_ROOTFS_SIZE
2967 internal-rootfs-size = Initial root filesystem size before any
2968 modifications. xspace = IMAGE_ROOTFS_EXTRA_SPACE
2969
2970 See the ```IMAGE_OVERHEAD_FACTOR`` <#var-IMAGE_OVERHEAD_FACTOR>`__
2971 and ```IMAGE_ROOTFS_EXTRA_SPACE`` <#var-IMAGE_ROOTFS_EXTRA_SPACE>`__
2972 variables for related information.
2973
2974IMAGE_TYPEDEP
2975 Specifies a dependency from one image type on another. Here is an
2976 example from the ```image-live`` <#ref-classes-image-live>`__ class:
2977 IMAGE_TYPEDEP_live = "ext3"
2978
2979 In the previous example, the variable ensures that when "live" is
2980 listed with the ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable,
2981 the OpenEmbedded build system produces an ``ext3`` image first since
2982 one of the components of the live image is an ``ext3`` formatted
2983 partition containing the root filesystem.
2984
2985IMAGE_TYPES
2986 Specifies the complete list of supported image types by default:
2987 btrfs container cpio cpio.gz cpio.lz4 cpio.lzma cpio.xz cramfs ext2
2988 ext2.bz2 ext2.gz ext2.lzma ext3 ext3.gz ext4 ext4.gz f2fs hddimg iso
2989 jffs2 jffs2.sum multiubi squashfs squashfs-lz4 squashfs-lzo
2990 squashfs-xz tar tar.bz2 tar.gz tar.lz4 tar.xz tar.zst ubi ubifs wic
2991 wic.bz2 wic.gz wic.lzma
2992
2993 For more information about these types of images, see
2994 ``meta/classes/image_types*.bbclass`` in the `Source
2995 Directory <#source-directory>`__.
2996
2997INC_PR
2998 Helps define the recipe revision for recipes that share a common
2999 ``include`` file. You can think of this variable as part of the
3000 recipe revision as set from within an include file.
3001
3002 Suppose, for example, you have a set of recipes that are used across
3003 several projects. And, within each of those recipes the revision (its
3004 ```PR`` <#var-PR>`__ value) is set accordingly. In this case, when
3005 the revision of those recipes changes, the burden is on you to find
3006 all those recipes and be sure that they get changed to reflect the
3007 updated version of the recipe. In this scenario, it can get
3008 complicated when recipes that are used in many places and provide
3009 common functionality are upgraded to a new revision.
3010
3011 A more efficient way of dealing with this situation is to set the
3012 ``INC_PR`` variable inside the ``include`` files that the recipes
3013 share and then expand the ``INC_PR`` variable within the recipes to
3014 help define the recipe revision.
3015
3016 The following provides an example that shows how to use the
3017 ``INC_PR`` variable given a common ``include`` file that defines the
3018 variable. Once the variable is defined in the ``include`` file, you
3019 can use the variable to set the ``PR`` values in each recipe. You
3020 will notice that when you set a recipe's ``PR`` you can provide more
3021 granular revisioning by appending values to the ``INC_PR`` variable:
3022 recipes-graphics/xorg-font/xorg-font-common.inc:INC_PR = "r2"
3023 recipes-graphics/xorg-font/encodings_1.0.4.bb:PR = "${INC_PR}.1"
3024 recipes-graphics/xorg-font/font-util_1.3.0.bb:PR = "${INC_PR}.0"
3025 recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3" The
3026 first line of the example establishes the baseline revision to be
3027 used for all recipes that use the ``include`` file. The remaining
3028 lines in the example are from individual recipes and show how the
3029 ``PR`` value is set.
3030
3031INCOMPATIBLE_LICENSE
3032 Specifies a space-separated list of license names (as they would
3033 appear in ```LICENSE`` <#var-LICENSE>`__) that should be excluded
3034 from the build. Recipes that provide no alternatives to listed
3035 incompatible licenses are not built. Packages that are individually
3036 licensed with the specified incompatible licenses will be deleted.
3037
3038 .. note::
3039
3040 This functionality is only regularly tested using the following
3041 setting:
3042 ::
3043
3044 INCOMPATIBLE_LICENSE = "GPL-3.0 LGPL-3.0 AGPL-3.0"
3045
3046
3047 Although you can use other settings, you might be required to
3048 remove dependencies on or provide alternatives to components that
3049 are required to produce a functional system image.
3050
3051 .. note::
3052
3053 It is possible to define a list of licenses that are allowed to be
3054 used instead of the licenses that are excluded. To do this, define
3055 a variable
3056 COMPATIBLE_LICENSES
3057 with the names of the licences that are allowed. Then define
3058 INCOMPATIBLE_LICENSE
3059 as:
3060 ::
3061
3062 INCOMPATIBLE_LICENSE = "${@' '.join(sorted(set(d.getVar('AVAILABLE_LICENSES').split()) - set(d.getVar('COMPATIBLE_LICENSES').split())))}"
3063
3064
3065 This will result in
3066 INCOMPATIBLE_LICENSE
3067 containing the names of all licences from
3068 AVAILABLE_LICENSES
3069 except the ones specified in
3070 COMPATIBLE_LICENSES
3071 , thus only allowing the latter licences to be used.
3072
3073INHERIT
3074 Causes the named class or classes to be inherited globally. Anonymous
3075 functions in the class or classes are not executed for the base
3076 configuration and in each individual recipe. The OpenEmbedded build
3077 system ignores changes to ``INHERIT`` in individual recipes.
3078
3079 For more information on ``INHERIT``, see the "```INHERIT``
3080 Configuration
3081 Directive <&YOCTO_DOCS_BB_URL;#inherit-configuration-directive>`__"
3082 section in the Bitbake User Manual.
3083
3084INHERIT_DISTRO
3085 Lists classes that will be inherited at the distribution level. It is
3086 unlikely that you want to edit this variable.
3087
3088 The default value of the variable is set as follows in the
3089 ``meta/conf/distro/defaultsetup.conf`` file: INHERIT_DISTRO ?=
3090 "debian devshell sstate license"
3091
3092INHIBIT_DEFAULT_DEPS
3093 Prevents the default dependencies, namely the C compiler and standard
3094 C library (libc), from being added to ```DEPENDS`` <#var-DEPENDS>`__.
3095 This variable is usually used within recipes that do not require any
3096 compilation using the C compiler.
3097
3098 Set the variable to "1" to prevent the default dependencies from
3099 being added.
3100
3101INHIBIT_PACKAGE_DEBUG_SPLIT
3102 Prevents the OpenEmbedded build system from splitting out debug
3103 information during packaging. By default, the build system splits out
3104 debugging information during the
3105 ```do_package`` <#ref-tasks-package>`__ task. For more information on
3106 how debug information is split out, see the
3107 ```PACKAGE_DEBUG_SPLIT_STYLE`` <#var-PACKAGE_DEBUG_SPLIT_STYLE>`__
3108 variable.
3109
3110 To prevent the build system from splitting out debug information
3111 during packaging, set the ``INHIBIT_PACKAGE_DEBUG_SPLIT`` variable as
3112 follows: INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
3113
3114INHIBIT_PACKAGE_STRIP
3115 If set to "1", causes the build to not strip binaries in resulting
3116 packages and prevents the ``-dbg`` package from containing the source
3117 files.
3118
3119 By default, the OpenEmbedded build system strips binaries and puts
3120 the debugging symbols into ``${``\ ```PN`` <#var-PN>`__\ ``}-dbg``.
3121 Consequently, you should not set ``INHIBIT_PACKAGE_STRIP`` when you
3122 plan to debug in general.
3123
3124INHIBIT_SYSROOT_STRIP
3125 If set to "1", causes the build to not strip binaries in the
3126 resulting sysroot.
3127
3128 By default, the OpenEmbedded build system strips binaries in the
3129 resulting sysroot. When you specifically set the
3130 ``INHIBIT_SYSROOT_STRIP`` variable to "1" in your recipe, you inhibit
3131 this stripping.
3132
3133 If you want to use this variable, include the
3134 ```staging`` <#ref-classes-staging>`__ class. This class uses a
3135 ``sys_strip()`` function to test for the variable and acts
3136 accordingly.
3137
3138 .. note::
3139
3140 Use of the
3141 INHIBIT_SYSROOT_STRIP
3142 variable occurs in rare and special circumstances. For example,
3143 suppose you are building bare-metal firmware by using an external
3144 GCC toolchain. Furthermore, even if the toolchain's binaries are
3145 strippable, other files exist that are needed for the build that
3146 are not strippable.
3147
3148INITRAMFS_FSTYPES
3149 Defines the format for the output image of an initial RAM filesystem
3150 (initramfs), which is used during boot. Supported formats are the
3151 same as those supported by the
3152 ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ variable.
3153
3154 The default value of this variable, which is set in the
3155 ``meta/conf/bitbake.conf`` configuration file in the `Source
3156 Directory <#source-directory>`__, is "cpio.gz". The Linux kernel's
3157 initramfs mechanism, as opposed to the initial RAM filesystem
3158 `initrd <https://en.wikipedia.org/wiki/Initrd>`__ mechanism, expects
3159 an optionally compressed cpio archive.
3160
3161INITRAMFS_IMAGE
3162 Specifies the ```PROVIDES`` <#var-PROVIDES>`__ name of an image
3163 recipe that is used to build an initial RAM filesystem (initramfs)
3164 image. In other words, the ``INITRAMFS_IMAGE`` variable causes an
3165 additional recipe to be built as a dependency to whatever root
3166 filesystem recipe you might be using (e.g. ``core-image-sato``). The
3167 initramfs image recipe you provide should set
3168 ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ to
3169 ```INITRAMFS_FSTYPES`` <#var-INITRAMFS_FSTYPES>`__.
3170
3171 An initramfs image provides a temporary root filesystem used for
3172 early system initialization (e.g. loading of modules needed to locate
3173 and mount the "real" root filesystem).
3174
3175 .. note::
3176
3177 See the
3178 meta/recipes-core/images/core-image-minimal-initramfs.bb
3179 recipe in the
3180 Source Directory
3181 for an example initramfs recipe. To select this sample recipe as
3182 the one built to provide the initramfs image, set
3183 INITRAMFS_IMAGE
3184 to "core-image-minimal-initramfs".
3185
3186 You can also find more information by referencing the
3187 ``meta-poky/conf/local.conf.sample.extended`` configuration file in
3188 the Source Directory, the ```image`` <#ref-classes-image>`__ class,
3189 and the ```kernel`` <#ref-classes-kernel>`__ class to see how to use
3190 the ``INITRAMFS_IMAGE`` variable.
3191
3192 If ``INITRAMFS_IMAGE`` is empty, which is the default, then no
3193 initramfs image is built.
3194
3195 For more information, you can also see the
3196 ```INITRAMFS_IMAGE_BUNDLE`` <#var-INITRAMFS_IMAGE_BUNDLE>`__
3197 variable, which allows the generated image to be bundled inside the
3198 kernel image. Additionally, for information on creating an initramfs
3199 image, see the "`Building an Initial RAM Filesystem (initramfs)
3200 Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__" section
3201 in the Yocto Project Development Tasks Manual.
3202
3203INITRAMFS_IMAGE_BUNDLE
3204 Controls whether or not the image recipe specified by
3205 ```INITRAMFS_IMAGE`` <#var-INITRAMFS_IMAGE>`__ is run through an
3206 extra pass
3207 (```do_bundle_initramfs`` <#ref-tasks-bundle_initramfs>`__) during
3208 kernel compilation in order to build a single binary that contains
3209 both the kernel image and the initial RAM filesystem (initramfs)
3210 image. This makes use of the
3211 ```CONFIG_INITRAMFS_SOURCE`` <#var-CONFIG_INITRAMFS_SOURCE>`__ kernel
3212 feature.
3213
3214 .. note::
3215
3216 Using an extra compilation pass to bundle the initramfs avoids a
3217 circular dependency between the kernel recipe and the initramfs
3218 recipe should the initramfs include kernel modules. Should that be
3219 the case, the initramfs recipe depends on the kernel for the
3220 kernel modules, and the kernel depends on the initramfs recipe
3221 since the initramfs is bundled inside the kernel image.
3222
3223 The combined binary is deposited into the ``tmp/deploy`` directory,
3224 which is part of the `Build Directory <#build-directory>`__.
3225
3226 Setting the variable to "1" in a configuration file causes the
3227 OpenEmbedded build system to generate a kernel image with the
3228 initramfs specified in ``INITRAMFS_IMAGE`` bundled within:
3229 INITRAMFS_IMAGE_BUNDLE = "1" By default, the
3230 ```kernel`` <#ref-classes-kernel>`__ class sets this variable to a
3231 null string as follows: INITRAMFS_IMAGE_BUNDLE ?= ""
3232
3233 .. note::
3234
3235 You must set the
3236 INITRAMFS_IMAGE_BUNDLE
3237 variable in a configuration file. You cannot set the variable in a
3238 recipe file.
3239
3240 See the
3241 ```local.conf.sample.extended`` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample.extended>`__
3242 file for additional information. Also, for information on creating an
3243 initramfs, see the "`Building an Initial RAM Filesystem (initramfs)
3244 Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__" section
3245 in the Yocto Project Development Tasks Manual.
3246
3247INITRAMFS_LINK_NAME
3248 The link name of the initial RAM filesystem image. This variable is
3249 set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3250 follows: INITRAMFS_LINK_NAME ?=
3251 "initramfs-${KERNEL_ARTIFACT_LINK_NAME}" The value of the
3252 ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3253 file, has the following value: KERNEL_ARTIFACT_LINK_NAME ?=
3254 "${MACHINE}"
3255
3256 See the ```MACHINE`` <#var-MACHINE>`__ variable for additional
3257 information.
3258
3259INITRAMFS_NAME
3260 The base name of the initial RAM filesystem image. This variable is
3261 set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3262 follows: INITRAMFS_NAME ?= "initramfs-${KERNEL_ARTIFACT_NAME}" The
3263 value of the ```KERNEL_ARTIFACT_NAME`` <#var-KERNEL_ARTIFACT_NAME>`__
3264 variable, which is set in the same file, has the following value:
3265 KERNEL_ARTIFACT_NAME ?=
3266 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3267
3268INITRD
3269 Indicates list of filesystem images to concatenate and use as an
3270 initial RAM disk (``initrd``).
3271
3272 The ``INITRD`` variable is an optional variable used with the
3273 ```image-live`` <#ref-classes-image-live>`__ class.
3274
3275INITRD_IMAGE
3276 When building a "live" bootable image (i.e. when
3277 ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ contains "live"),
3278 ``INITRD_IMAGE`` specifies the image recipe that should be built to
3279 provide the initial RAM disk image. The default value is
3280 "core-image-minimal-initramfs".
3281
3282 See the ```image-live`` <#ref-classes-image-live>`__ class for more
3283 information.
3284
3285INITSCRIPT_NAME
3286 The filename of the initialization script as installed to
3287 ``${sysconfdir}/init.d``.
3288
3289 This variable is used in recipes when using ``update-rc.d.bbclass``.
3290 The variable is mandatory.
3291
3292INITSCRIPT_PACKAGES
3293 A list of the packages that contain initscripts. If multiple packages
3294 are specified, you need to append the package name to the other
3295 ``INITSCRIPT_*`` as an override.
3296
3297 This variable is used in recipes when using ``update-rc.d.bbclass``.
3298 The variable is optional and defaults to the ```PN`` <#var-PN>`__
3299 variable.
3300
3301INITSCRIPT_PARAMS
3302 Specifies the options to pass to ``update-rc.d``. Here is an example:
3303 INITSCRIPT_PARAMS = "start 99 5 2 . stop 20 0 1 6 ."
3304
3305 In this example, the script has a runlevel of 99, starts the script
3306 in initlevels 2 and 5, and stops the script in levels 0, 1 and 6.
3307
3308 The variable's default value is "defaults", which is set in the
3309 ```update-rc.d`` <#ref-classes-update-rc.d>`__ class.
3310
3311 The value in ``INITSCRIPT_PARAMS`` is passed through to the
3312 ``update-rc.d`` command. For more information on valid parameters,
3313 please see the ``update-rc.d`` manual page at
3314 ` <http://www.tin.org/bin/man.cgi?section=8&topic=update-rc.d>`__.
3315
3316INSANE_SKIP
3317 Specifies the QA checks to skip for a specific package within a
3318 recipe. For example, to skip the check for symbolic link ``.so``
3319 files in the main package of a recipe, add the following to the
3320 recipe. The package name override must be used, which in this example
3321 is ``${PN}``: INSANE_SKIP_${PN} += "dev-so"
3322
3323 See the "```insane.bbclass`` <#ref-classes-insane>`__" section for a
3324 list of the valid QA checks you can specify using this variable.
3325
3326INSTALL_TIMEZONE_FILE
3327 By default, the ``tzdata`` recipe packages an ``/etc/timezone`` file.
3328 Set the ``INSTALL_TIMEZONE_FILE`` variable to "0" at the
3329 configuration level to disable this behavior.
3330
3331IPK_FEED_URIS
3332 When the IPK backend is in use and package management is enabled on
3333 the target, you can use this variable to set up ``opkg`` in the
3334 target image to point to package feeds on a nominated server. Once
3335 the feed is established, you can perform installations or upgrades
3336 using the package manager at runtime.
3337
3338KARCH
3339 Defines the kernel architecture used when assembling the
3340 configuration. Architectures supported for this release are: powerpc
3341 i386 x86_64 arm qemu mips
3342
3343 You define the ``KARCH`` variable in the `BSP
3344 Descriptions <&YOCTO_DOCS_KERNEL_DEV_URL;#bsp-descriptions>`__.
3345
3346KBRANCH
3347 A regular expression used by the build process to explicitly identify
3348 the kernel branch that is validated, patched, and configured during a
3349 build. You must set this variable to ensure the exact kernel branch
3350 you want is being used by the build process.
3351
3352 Values for this variable are set in the kernel's recipe file and the
3353 kernel's append file. For example, if you are using the
3354 ``linux-yocto_4.12`` kernel, the kernel recipe file is the
3355 ``meta/recipes-kernel/linux/linux-yocto_4.12.bb`` file. ``KBRANCH``
3356 is set as follows in that kernel recipe file: KBRANCH ?=
3357 "standard/base"
3358
3359 This variable is also used from the kernel's append file to identify
3360 the kernel branch specific to a particular machine or target
3361 hardware. Continuing with the previous kernel example, the kernel's
3362 append file (i.e. ``linux-yocto_4.12.bbappend``) is located in the
3363 BSP layer for a given machine. For example, the append file for the
3364 Beaglebone, EdgeRouter, and generic versions of both 32 and 64-bit IA
3365 machines (``meta-yocto-bsp``) is named
3366 ``meta-yocto-bsp/recipes-kernel/linux/linux-yocto_4.12.bbappend``.
3367 Here are the related statements from that append file:
3368 KBRANCH_genericx86 = "standard/base" KBRANCH_genericx86-64 =
3369 "standard/base" KBRANCH_edgerouter = "standard/edgerouter"
3370 KBRANCH_beaglebone = "standard/beaglebone" The ``KBRANCH`` statements
3371 identify the kernel branch to use when building for each supported
3372 BSP.
3373
3374KBUILD_DEFCONFIG
3375 When used with the ```kernel-yocto`` <#ref-classes-kernel-yocto>`__
3376 class, specifies an "in-tree" kernel configuration file for use
3377 during a kernel build.
3378
3379 Typically, when using a ``defconfig`` to configure a kernel during a
3380 build, you place the file in your layer in the same manner as you
3381 would place patch files and configuration fragment files (i.e.
3382 "out-of-tree"). However, if you want to use a ``defconfig`` file that
3383 is part of the kernel tree (i.e. "in-tree"), you can use the
3384 ``KBUILD_DEFCONFIG`` variable and append the
3385 ```KMACHINE`` <#var-KMACHINE>`__ variable to point to the
3386 ``defconfig`` file.
3387
3388 To use the variable, set it in the append file for your kernel recipe
3389 using the following form: KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file
3390 Here is an example from a "raspberrypi2" ``KMACHINE`` build that uses
3391 a ``defconfig`` file named "bcm2709_defconfig":
3392 KBUILD_DEFCONFIG_raspberrypi2 = "bcm2709_defconfig" As an
3393 alternative, you can use the following within your append file:
3394 KBUILD_DEFCONFIG_pn-linux-yocto ?= defconfig_file For more
3395 information on how to use the ``KBUILD_DEFCONFIG`` variable, see the
3396 "`Using an "In-Tree" ``defconfig``
3397 File <&YOCTO_DOCS_KERNEL_DEV_URL;#using-an-in-tree-defconfig-file>`__"
3398 section in the Yocto Project Linux Kernel Development Manual.
3399
3400KERNEL_ALT_IMAGETYPE
3401 Specifies an alternate kernel image type for creation in addition to
3402 the kernel image type specified using the
3403 ```KERNEL_IMAGETYPE`` <#var-KERNEL_IMAGETYPE>`__ variable.
3404
3405KERNEL_ARTIFACT_NAME
3406 Specifies the name of all of the build artifacts. You can change the
3407 name of the artifacts by changing the ``KERNEL_ARTIFACT_NAME``
3408 variable.
3409
3410 The value of ``KERNEL_ARTIFACT_NAME``, which is set in the
3411 ``meta/classes/kernel-artifact-names.bbclass`` file, has the
3412 following default value: KERNEL_ARTIFACT_NAME ?=
3413 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3414
3415 See the ```PKGE`` <#var-PKGE>`__, ```PKGV`` <#var-PKGV>`__,
3416 ```PKGR`` <#var-PKGR>`__, and ```MACHINE`` <#var-MACHINE>`__
3417 variables for additional information.
3418
3419 .. note::
3420
3421 The
3422 IMAGE_VERSION_SUFFIX
3423 variable is set to
3424 DATETIME
3425 .
3426
3427KERNEL_CLASSES
3428 A list of classes defining kernel image types that the
3429 ```kernel`` <#ref-classes-kernel>`__ class should inherit. You
3430 typically append this variable to enable extended image types. An
3431 example is the "kernel-fitimage", which enables fitImage support and
3432 resides in ``meta/classes/kernel-fitimage.bbclass``. You can register
3433 custom kernel image types with the ``kernel`` class using this
3434 variable.
3435
3436KERNEL_DEVICETREE
3437 Specifies the name of the generated Linux kernel device tree (i.e.
3438 the ``.dtb``) file.
3439
3440 .. note::
3441
3442 Legacy support exists for specifying the full path to the device
3443 tree. However, providing just the
3444 .dtb
3445 file is preferred.
3446
3447 In order to use this variable, the
3448 ```kernel-devicetree`` <#ref-classes-kernel-devicetree>`__ class must
3449 be inherited.
3450
3451KERNEL_DTB_LINK_NAME
3452 The link name of the kernel device tree binary (DTB). This variable
3453 is set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3454 follows: KERNEL_DTB_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" The
3455 value of the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in
3456 the same file, has the following value: KERNEL_ARTIFACT_LINK_NAME ?=
3457 "${MACHINE}"
3458
3459 See the ```MACHINE`` <#var-MACHINE>`__ variable for additional
3460 information.
3461
3462KERNEL_DTB_NAME
3463 The base name of the kernel device tree binary (DTB). This variable
3464 is set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3465 follows: KERNEL_DTB_NAME ?= "${KERNEL_ARTIFACT_NAME}" The value of
3466 the ```KERNEL_ARTIFACT_NAME`` <#var-KERNEL_ARTIFACT_NAME>`__
3467 variable, which is set in the same file, has the following value:
3468 KERNEL_ARTIFACT_NAME ?=
3469 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3470
3471KERNEL_EXTRA_ARGS
3472 Specifies additional ``make`` command-line arguments the OpenEmbedded
3473 build system passes on when compiling the kernel.
3474
3475KERNEL_FEATURES
3476 Includes additional kernel metadata. In the OpenEmbedded build
3477 system, the default Board Support Packages (BSPs)
3478 `Metadata <#metadata>`__ is provided through the
3479 ```KMACHINE`` <#var-KMACHINE>`__ and ```KBRANCH`` <#var-KBRANCH>`__
3480 variables. You can use the ``KERNEL_FEATURES`` variable from within
3481 the kernel recipe or kernel append file to further add metadata for
3482 all BSPs or specific BSPs.
3483
3484 The metadata you add through this variable includes config fragments
3485 and features descriptions, which usually includes patches as well as
3486 config fragments. You typically override the ``KERNEL_FEATURES``
3487 variable for a specific machine. In this way, you can provide
3488 validated, but optional, sets of kernel configurations and features.
3489
3490 For example, the following example from the ``linux-yocto-rt_4.12``
3491 kernel recipe adds "netfilter" and "taskstats" features to all BSPs
3492 as well as "virtio" configurations to all QEMU machines. The last two
3493 statements add specific configurations to targeted machine types:
3494 KERNEL_EXTRA_FEATURES ?= "features/netfilter/netfilter.scc
3495 features/taskstats/taskstats.scc" KERNEL_FEATURES_append = "
3496 ${KERNEL_EXTRA_FEATURES}" KERNEL_FEATURES_append_qemuall = "
3497 cfg/virtio.scc" KERNEL_FEATURES_append_qemux86 = " cfg/sound.scc
3498 cfg/paravirt_kvm.scc" KERNEL_FEATURES_append_qemux86-64 = "
3499 cfg/sound.scc"
3500
3501KERNEL_FIT_LINK_NAME
3502 The link name of the kernel flattened image tree (FIT) image. This
3503 variable is set in the ``meta/classes/kernel-artifact-names.bbclass``
3504 file as follows: KERNEL_FIT_LINK_NAME ?=
3505 "${KERNEL_ARTIFACT_LINK_NAME}" The value of the
3506 ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3507 file, has the following value: KERNEL_ARTIFACT_LINK_NAME ?=
3508 "${MACHINE}"
3509
3510 See the ```MACHINE`` <#var-MACHINE>`__ variable for additional
3511 information.
3512
3513KERNEL_FIT_NAME
3514 The base name of the kernel flattened image tree (FIT) image. This
3515 variable is set in the ``meta/classes/kernel-artifact-names.bbclass``
3516 file as follows: KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}" The
3517 value of the ```KERNEL_ARTIFACT_NAME`` <#var-KERNEL_ARTIFACT_NAME>`__
3518 variable, which is set in the same file, has the following value:
3519 KERNEL_ARTIFACT_NAME ?=
3520 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3521
3522KERNEL_IMAGE_LINK_NAME
3523 The link name for the kernel image. This variable is set in the
3524 ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
3525 KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" The value of
3526 the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3527 file, has the following value: KERNEL_ARTIFACT_LINK_NAME ?=
3528 "${MACHINE}"
3529
3530 See the ```MACHINE`` <#var-MACHINE>`__ variable for additional
3531 information.
3532
3533KERNEL_IMAGE_MAXSIZE
3534 Specifies the maximum size of the kernel image file in kilobytes. If
3535 ``KERNEL_IMAGE_MAXSIZE`` is set, the size of the kernel image file is
3536 checked against the set value during the
3537 ```do_sizecheck`` <#ref-tasks-sizecheck>`__ task. The task fails if
3538 the kernel image file is larger than the setting.
3539
3540 ``KERNEL_IMAGE_MAXSIZE`` is useful for target devices that have a
3541 limited amount of space in which the kernel image must be stored.
3542
3543 By default, this variable is not set, which means the size of the
3544 kernel image is not checked.
3545
3546KERNEL_IMAGE_NAME
3547 The base name of the kernel image. This variable is set in the
3548 ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
3549 KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}" The value of the
3550 ```KERNEL_ARTIFACT_NAME`` <#var-KERNEL_ARTIFACT_NAME>`__ variable,
3551 which is set in the same file, has the following value:
3552 KERNEL_ARTIFACT_NAME ?=
3553 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3554
3555KERNEL_IMAGETYPE
3556 The type of kernel to build for a device, usually set by the machine
3557 configuration files and defaults to "zImage". This variable is used
3558 when building the kernel and is passed to ``make`` as the target to
3559 build.
3560
3561 If you want to build an alternate kernel image type, use the
3562 ```KERNEL_ALT_IMAGETYPE`` <#var-KERNEL_ALT_IMAGETYPE>`__ variable.
3563
3564KERNEL_MODULE_AUTOLOAD
3565 Lists kernel modules that need to be auto-loaded during boot.
3566
3567 .. note::
3568
3569 This variable replaces the deprecated
3570 module_autoload
3571 variable.
3572
3573 You can use the ``KERNEL_MODULE_AUTOLOAD`` variable anywhere that it
3574 can be recognized by the kernel recipe or by an out-of-tree kernel
3575 module recipe (e.g. a machine configuration file, a distribution
3576 configuration file, an append file for the recipe, or the recipe
3577 itself).
3578
3579 Specify it as follows: KERNEL_MODULE_AUTOLOAD += "module_name1
3580 module_name2 module_name3"
3581
3582 Including ``KERNEL_MODULE_AUTOLOAD`` causes the OpenEmbedded build
3583 system to populate the ``/etc/modules-load.d/modname.conf`` file with
3584 the list of modules to be auto-loaded on boot. The modules appear
3585 one-per-line in the file. Here is an example of the most common use
3586 case: KERNEL_MODULE_AUTOLOAD += "module_name"
3587
3588 For information on how to populate the ``modname.conf`` file with
3589 ``modprobe.d`` syntax lines, see the
3590 ```KERNEL_MODULE_PROBECONF`` <#var-KERNEL_MODULE_PROBECONF>`__
3591 variable.
3592
3593KERNEL_MODULE_PROBECONF
3594 Provides a list of modules for which the OpenEmbedded build system
3595 expects to find ``module_conf_``\ modname values that specify
3596 configuration for each of the modules. For information on how to
3597 provide those module configurations, see the
3598 ```module_conf_*`` <#var-module_conf>`__ variable.
3599
3600KERNEL_PATH
3601 The location of the kernel sources. This variable is set to the value
3602 of the ```STAGING_KERNEL_DIR`` <#var-STAGING_KERNEL_DIR>`__ within
3603 the ```module`` <#ref-classes-module>`__ class. For information on
3604 how this variable is used, see the "`Incorporating Out-of-Tree
3605 Modules <&YOCTO_DOCS_KERNEL_DEV_URL;#incorporating-out-of-tree-modules>`__"
3606 section in the Yocto Project Linux Kernel Development Manual.
3607
3608 To help maximize compatibility with out-of-tree drivers used to build
3609 modules, the OpenEmbedded build system also recognizes and uses the
3610 ```KERNEL_SRC`` <#var-KERNEL_SRC>`__ variable, which is identical to
3611 the ``KERNEL_PATH`` variable. Both variables are common variables
3612 used by external Makefiles to point to the kernel source directory.
3613
3614KERNEL_SRC
3615 The location of the kernel sources. This variable is set to the value
3616 of the ```STAGING_KERNEL_DIR`` <#var-STAGING_KERNEL_DIR>`__ within
3617 the ```module`` <#ref-classes-module>`__ class. For information on
3618 how this variable is used, see the "`Incorporating Out-of-Tree
3619 Modules <&YOCTO_DOCS_KERNEL_DEV_URL;#incorporating-out-of-tree-modules>`__"
3620 section in the Yocto Project Linux Kernel Development Manual.
3621
3622 To help maximize compatibility with out-of-tree drivers used to build
3623 modules, the OpenEmbedded build system also recognizes and uses the
3624 ```KERNEL_PATH`` <#var-KERNEL_PATH>`__ variable, which is identical
3625 to the ``KERNEL_SRC`` variable. Both variables are common variables
3626 used by external Makefiles to point to the kernel source directory.
3627
3628KERNEL_VERSION
3629 Specifies the version of the kernel as extracted from ``version.h``
3630 or ``utsrelease.h`` within the kernel sources. Effects of setting
3631 this variable do not take affect until the kernel has been
3632 configured. Consequently, attempting to refer to this variable in
3633 contexts prior to configuration will not work.
3634
3635KERNELDEPMODDEPEND
3636 Specifies whether the data referenced through
3637 ```PKGDATA_DIR`` <#var-PKGDATA_DIR>`__ is needed or not. The
3638 ``KERNELDEPMODDEPEND`` does not control whether or not that data
3639 exists, but simply whether or not it is used. If you do not need to
3640 use the data, set the ``KERNELDEPMODDEPEND`` variable in your
3641 ``initramfs`` recipe. Setting the variable there when the data is not
3642 needed avoids a potential dependency loop.
3643
3644KFEATURE_DESCRIPTION
3645 Provides a short description of a configuration fragment. You use
3646 this variable in the ``.scc`` file that describes a configuration
3647 fragment file. Here is the variable used in a file named ``smp.scc``
3648 to describe SMP being enabled: define KFEATURE_DESCRIPTION "Enable
3649 SMP"
3650
3651KMACHINE
3652 The machine as known by the kernel. Sometimes the machine name used
3653 by the kernel does not match the machine name used by the
3654 OpenEmbedded build system. For example, the machine name that the
3655 OpenEmbedded build system understands as ``core2-32-intel-common``
3656 goes by a different name in the Linux Yocto kernel. The kernel
3657 understands that machine as ``intel-core2-32``. For cases like these,
3658 the ``KMACHINE`` variable maps the kernel machine name to the
3659 OpenEmbedded build system machine name.
3660
3661 These mappings between different names occur in the Yocto Linux
3662 Kernel's ``meta`` branch. As an example take a look in the
3663 ``common/recipes-kernel/linux/linux-yocto_3.19.bbappend`` file:
3664 LINUX_VERSION_core2-32-intel-common = "3.19.0"
3665 COMPATIBLE_MACHINE_core2-32-intel-common = "${MACHINE}"
3666 SRCREV_meta_core2-32-intel-common =
3667 "8897ef68b30e7426bc1d39895e71fb155d694974"
3668 SRCREV_machine_core2-32-intel-common =
3669 "43b9eced9ba8a57add36af07736344dcc383f711"
3670 KMACHINE_core2-32-intel-common = "intel-core2-32"
3671 KBRANCH_core2-32-intel-common = "standard/base"
3672 KERNEL_FEATURES_append_core2-32-intel-common =
3673 "${KERNEL_FEATURES_INTEL_COMMON}" The ``KMACHINE`` statement says
3674 that the kernel understands the machine name as "intel-core2-32".
3675 However, the OpenEmbedded build system understands the machine as
3676 "core2-32-intel-common".
3677
3678KTYPE
3679 Defines the kernel type to be used in assembling the configuration.
3680 The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
3681 kernel types. See the "`Kernel
3682 Types <&YOCTO_DOCS_KERNEL_DEV_URL;#kernel-types>`__" section in the
3683 Yocto Project Linux Kernel Development Manual for more information on
3684 kernel types.
3685
3686 You define the ``KTYPE`` variable in the `BSP
3687 Descriptions <&YOCTO_DOCS_KERNEL_DEV_URL;#bsp-descriptions>`__. The
3688 value you use must match the value used for the
3689 ```LINUX_KERNEL_TYPE`` <#var-LINUX_KERNEL_TYPE>`__ value used by the
3690 kernel recipe.
3691
3692LABELS
3693 Provides a list of targets for automatic configuration.
3694
3695 See the ```grub-efi`` <#ref-classes-grub-efi>`__ class for more
3696 information on how this variable is used.
3697
3698LAYERDEPENDS
3699 Lists the layers, separated by spaces, on which this recipe depends.
3700 Optionally, you can specify a specific layer version for a dependency
3701 by adding it to the end of the layer name. Here is an example:
3702 LAYERDEPENDS_mylayer = "anotherlayer (=3)" In this previous example,
3703 version 3 of "anotherlayer" is compared against
3704 ```LAYERVERSION`` <#var-LAYERVERSION>`__\ ``_anotherlayer``.
3705
3706 An error is produced if any dependency is missing or the version
3707 numbers (if specified) do not match exactly. This variable is used in
3708 the ``conf/layer.conf`` file and must be suffixed with the name of
3709 the specific layer (e.g. ``LAYERDEPENDS_mylayer``).
3710
3711LAYERDIR
3712 When used inside the ``layer.conf`` configuration file, this variable
3713 provides the path of the current layer. This variable is not
3714 available outside of ``layer.conf`` and references are expanded
3715 immediately when parsing of the file completes.
3716
3717LAYERRECOMMENDS
3718 Lists the layers, separated by spaces, recommended for use with this
3719 layer.
3720
3721 Optionally, you can specify a specific layer version for a
3722 recommendation by adding the version to the end of the layer name.
3723 Here is an example: LAYERRECOMMENDS_mylayer = "anotherlayer (=3)" In
3724 this previous example, version 3 of "anotherlayer" is compared
3725 against ``LAYERVERSION_anotherlayer``.
3726
3727 This variable is used in the ``conf/layer.conf`` file and must be
3728 suffixed with the name of the specific layer (e.g.
3729 ``LAYERRECOMMENDS_mylayer``).
3730
3731LAYERSERIES_COMPAT
3732 Lists the versions of the `OpenEmbedded-Core <#oe-core>`__ for which
3733 a layer is compatible. Using the ``LAYERSERIES_COMPAT`` variable
3734 allows the layer maintainer to indicate which combinations of the
3735 layer and OE-Core can be expected to work. The variable gives the
3736 system a way to detect when a layer has not been tested with new
3737 releases of OE-Core (e.g. the layer is not maintained).
3738
3739 To specify the OE-Core versions for which a layer is compatible, use
3740 this variable in your layer's ``conf/layer.conf`` configuration file.
3741 For the list, use the Yocto Project `Release
3742 Name <https://wiki.yoctoproject.org/wiki/Releases>`__ (e.g.
3743 DISTRO_NAME_NO_CAP). To specify multiple OE-Core versions for the
3744 layer, use a space-separated list: LAYERSERIES_COMPAT_layer_root_name
3745 = "DISTRO_NAME_NO_CAP DISTRO_NAME_NO_CAP_MINUS_ONE"
3746
3747 .. note::
3748
3749 Setting
3750 LAYERSERIES_COMPAT
3751 is required by the Yocto Project Compatible version 2 standard.
3752 The OpenEmbedded build system produces a warning if the variable
3753 is not set for any given layer.
3754
3755 See the "`Creating Your Own
3756 Layer <&YOCTO_DOCS_DEV_URL;#creating-your-own-layer>`__" section in
3757 the Yocto Project Development Tasks Manual.
3758
3759LAYERVERSION
3760 Optionally specifies the version of a layer as a single number. You
3761 can use this within ```LAYERDEPENDS`` <#var-LAYERDEPENDS>`__ for
3762 another layer in order to depend on a specific version of the layer.
3763 This variable is used in the ``conf/layer.conf`` file and must be
3764 suffixed with the name of the specific layer (e.g.
3765 ``LAYERVERSION_mylayer``).
3766
3767LD
3768 The minimal command and arguments used to run the linker.
3769
3770LDFLAGS
3771 Specifies the flags to pass to the linker. This variable is exported
3772 to an environment variable and thus made visible to the software
3773 being built during the compilation step.
3774
3775 Default initialization for ``LDFLAGS`` varies depending on what is
3776 being built:
3777
3778 - ```TARGET_LDFLAGS`` <#var-TARGET_LDFLAGS>`__ when building for the
3779 target
3780
3781 - ```BUILD_LDFLAGS`` <#var-BUILD_LDFLAGS>`__ when building for the
3782 build host (i.e. ``-native``)
3783
3784 - ```BUILDSDK_LDFLAGS`` <#var-BUILDSDK_LDFLAGS>`__ when building for
3785 an SDK (i.e. ``nativesdk-``)
3786
3787LEAD_SONAME
3788 Specifies the lead (or primary) compiled library file (i.e. ``.so``)
3789 that the ```debian`` <#ref-classes-debian>`__ class applies its
3790 naming policy to given a recipe that packages multiple libraries.
3791
3792 This variable works in conjunction with the ``debian`` class.
3793
3794LIC_FILES_CHKSUM
3795 Checksums of the license text in the recipe source code.
3796
3797 This variable tracks changes in license text of the source code
3798 files. If the license text is changed, it will trigger a build
3799 failure, which gives the developer an opportunity to review any
3800 license change.
3801
3802 This variable must be defined for all recipes (unless
3803 ```LICENSE`` <#var-LICENSE>`__ is set to "CLOSED").
3804
3805 For more information, see the "`Tracking License
3806 Changes <&YOCTO_DOCS_DEV_URL;#usingpoky-configuring-LIC_FILES_CHKSUM>`__"
3807 section in the Yocto Project Development Tasks Manual.
3808
3809LICENSE
3810 The list of source licenses for the recipe. Follow these rules:
3811
3812 - Do not use spaces within individual license names.
3813
3814 - Separate license names using \| (pipe) when there is a choice
3815 between licenses.
3816
3817 - Separate license names using & (ampersand) when multiple licenses
3818 exist that cover different parts of the source.
3819
3820 - You can use spaces between license names.
3821
3822 - For standard licenses, use the names of the files in
3823 ``meta/files/common-licenses/`` or the
3824 ```SPDXLICENSEMAP`` <#var-SPDXLICENSEMAP>`__ flag names defined in
3825 ``meta/conf/licenses.conf``.
3826
3827 Here are some examples: LICENSE = "LGPLv2.1 \| GPLv3" LICENSE =
3828 "MPL-1 & LGPLv2.1" LICENSE = "GPLv2+" The first example is from the
3829 recipes for Qt, which the user may choose to distribute under either
3830 the LGPL version 2.1 or GPL version 3. The second example is from
3831 Cairo where two licenses cover different parts of the source code.
3832 The final example is from ``sysstat``, which presents a single
3833 license.
3834
3835 You can also specify licenses on a per-package basis to handle
3836 situations where components of the output have different licenses.
3837 For example, a piece of software whose code is licensed under GPLv2
3838 but has accompanying documentation licensed under the GNU Free
3839 Documentation License 1.2 could be specified as follows: LICENSE =
3840 "GFDL-1.2 & GPLv2" LICENSE_${PN} = "GPLv2" LICENSE_${PN}-doc =
3841 "GFDL-1.2"
3842
3843LICENSE_CREATE_PACKAGE
3844 Setting ``LICENSE_CREATE_PACKAGE`` to "1" causes the OpenEmbedded
3845 build system to create an extra package (i.e.
3846 ``${``\ ```PN`` <#var-PN>`__\ ``}-lic``) for each recipe and to add
3847 those packages to the
3848 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__\ ``_${PN}``.
3849
3850 The ``${PN}-lic`` package installs a directory in
3851 ``/usr/share/licenses`` named ``${PN}``, which is the recipe's base
3852 name, and installs files in that directory that contain license and
3853 copyright information (i.e. copies of the appropriate license files
3854 from ``meta/common-licenses`` that match the licenses specified in
3855 the ```LICENSE`` <#var-LICENSE>`__ variable of the recipe metadata
3856 and copies of files marked in
3857 ```LIC_FILES_CHKSUM`` <#var-LIC_FILES_CHKSUM>`__ as containing
3858 license text).
3859
3860 For related information on providing license text, see the
3861 ```COPY_LIC_DIRS`` <#var-COPY_LIC_DIRS>`__ variable, the
3862 ```COPY_LIC_MANIFEST`` <#var-COPY_LIC_MANIFEST>`__ variable, and the
3863 "`Providing License
3864 Text <&YOCTO_DOCS_DEV_URL;#providing-license-text>`__" section in the
3865 Yocto Project Development Tasks Manual.
3866
3867LICENSE_FLAGS
3868 Specifies additional flags for a recipe you must whitelist through
3869 ```LICENSE_FLAGS_WHITELIST`` <#var-LICENSE_FLAGS_WHITELIST>`__ in
3870 order to allow the recipe to be built. When providing multiple flags,
3871 separate them with spaces.
3872
3873 This value is independent of ```LICENSE`` <#var-LICENSE>`__ and is
3874 typically used to mark recipes that might require additional licenses
3875 in order to be used in a commercial product. For more information,
3876 see the "`Enabling Commercially Licensed
3877 Recipes <&YOCTO_DOCS_DEV_URL;#enabling-commercially-licensed-recipes>`__"
3878 section in the Yocto Project Development Tasks Manual.
3879
3880LICENSE_FLAGS_WHITELIST
3881 Lists license flags that when specified in
3882 ```LICENSE_FLAGS`` <#var-LICENSE_FLAGS>`__ within a recipe should not
3883 prevent that recipe from being built. This practice is otherwise
3884 known as "whitelisting" license flags. For more information, see the
3885 "`Enabling Commercially Licensed
3886 Recipes <&YOCTO_DOCS_DEV_URL;#enabling-commercially-licensed-recipes>`__"
3887 section in the Yocto Project Development Tasks Manual.
3888
3889LICENSE_PATH
3890 Path to additional licenses used during the build. By default, the
3891 OpenEmbedded build system uses ``COMMON_LICENSE_DIR`` to define the
3892 directory that holds common license text used during the build. The
3893 ``LICENSE_PATH`` variable allows you to extend that location to other
3894 areas that have additional licenses: LICENSE_PATH +=
3895 "path-to-additional-common-licenses"
3896
3897LINUX_KERNEL_TYPE
3898 Defines the kernel type to be used in assembling the configuration.
3899 The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
3900 kernel types. See the "`Kernel
3901 Types <&YOCTO_DOCS_KERNEL_DEV_URL;#kernel-types>`__" section in the
3902 Yocto Project Linux Kernel Development Manual for more information on
3903 kernel types.
3904
3905 If you do not specify a ``LINUX_KERNEL_TYPE``, it defaults to
3906 "standard". Together with ```KMACHINE`` <#var-KMACHINE>`__, the
3907 ``LINUX_KERNEL_TYPE`` variable defines the search arguments used by
3908 the kernel tools to find the appropriate description within the
3909 kernel `Metadata <#metadata>`__ with which to build out the sources
3910 and configuration.
3911
3912LINUX_VERSION
3913 The Linux version from ``kernel.org`` on which the Linux kernel image
3914 being built using the OpenEmbedded build system is based. You define
3915 this variable in the kernel recipe. For example, the
3916 ``linux-yocto-3.4.bb`` kernel recipe found in
3917 ``meta/recipes-kernel/linux`` defines the variables as follows:
3918 LINUX_VERSION ?= "3.4.24"
3919
3920 The ``LINUX_VERSION`` variable is used to define ```PV`` <#var-PV>`__
3921 for the recipe: PV = "${LINUX_VERSION}+git${SRCPV}"
3922
3923LINUX_VERSION_EXTENSION
3924 A string extension compiled into the version string of the Linux
3925 kernel built with the OpenEmbedded build system. You define this
3926 variable in the kernel recipe. For example, the linux-yocto kernel
3927 recipes all define the variable as follows: LINUX_VERSION_EXTENSION
3928 ?= "-yocto-${`LINUX_KERNEL_TYPE <#var-LINUX_KERNEL_TYPE>`__}"
3929
3930 Defining this variable essentially sets the Linux kernel
3931 configuration item ``CONFIG_LOCALVERSION``, which is visible through
3932 the ``uname`` command. Here is an example that shows the extension
3933 assuming it was set as previously shown: $ uname -r 3.7.0-rc8-custom
3934
3935LOG_DIR
3936 Specifies the directory to which the OpenEmbedded build system writes
3937 overall log files. The default directory is ``${TMPDIR}/log``.
3938
3939 For the directory containing logs specific to each task, see the
3940 ```T`` <#var-T>`__ variable.
3941
3942MACHINE
3943 Specifies the target device for which the image is built. You define
3944 ``MACHINE`` in the ``local.conf`` file found in the `Build
3945 Directory <#build-directory>`__. By default, ``MACHINE`` is set to
3946 "qemux86", which is an x86-based architecture machine to be emulated
3947 using QEMU: MACHINE ?= "qemux86"
3948
3949 The variable corresponds to a machine configuration file of the same
3950 name, through which machine-specific configurations are set. Thus,
3951 when ``MACHINE`` is set to "qemux86" there exists the corresponding
3952 ``qemux86.conf`` machine configuration file, which can be found in
3953 the `Source Directory <#source-directory>`__ in
3954 ``meta/conf/machine``.
3955
3956 The list of machines supported by the Yocto Project as shipped
3957 include the following: MACHINE ?= "qemuarm" MACHINE ?= "qemuarm64"
3958 MACHINE ?= "qemumips" MACHINE ?= "qemumips64" MACHINE ?= "qemuppc"
3959 MACHINE ?= "qemux86" MACHINE ?= "qemux86-64" MACHINE ?= "genericx86"
3960 MACHINE ?= "genericx86-64" MACHINE ?= "beaglebone" MACHINE ?=
3961 "edgerouter" The last five are Yocto Project reference hardware
3962 boards, which are provided in the ``meta-yocto-bsp`` layer.
3963
3964 .. note::
3965
3966 Adding additional Board Support Package (BSP) layers to your
3967 configuration adds new possible settings for
3968 MACHINE
3969 .
3970
3971MACHINE_ARCH
3972 Specifies the name of the machine-specific architecture. This
3973 variable is set automatically from ```MACHINE`` <#var-MACHINE>`__ or
3974 ```TUNE_PKGARCH`` <#var-TUNE_PKGARCH>`__. You should not hand-edit
3975 the ``MACHINE_ARCH`` variable.
3976
3977MACHINE_ESSENTIAL_EXTRA_RDEPENDS
3978 A list of required machine-specific packages to install as part of
3979 the image being built. The build process depends on these packages
3980 being present. Furthermore, because this is a "machine-essential"
3981 variable, the list of packages are essential for the machine to boot.
3982 The impact of this variable affects images based on
3983 ``packagegroup-core-boot``, including the ``core-image-minimal``
3984 image.
3985
3986 This variable is similar to the
3987 ``MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` variable with the exception
3988 that the image being built has a build dependency on the variable's
3989 list of packages. In other words, the image will not build if a file
3990 in this list is not found.
3991
3992 As an example, suppose the machine for which you are building
3993 requires ``example-init`` to be run during boot to initialize the
3994 hardware. In this case, you would use the following in the machine's
3995 ``.conf`` configuration file: MACHINE_ESSENTIAL_EXTRA_RDEPENDS +=
3996 "example-init"
3997
3998MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
3999 A list of recommended machine-specific packages to install as part of
4000 the image being built. The build process does not depend on these
4001 packages being present. However, because this is a
4002 "machine-essential" variable, the list of packages are essential for
4003 the machine to boot. The impact of this variable affects images based
4004 on ``packagegroup-core-boot``, including the ``core-image-minimal``
4005 image.
4006
4007 This variable is similar to the ``MACHINE_ESSENTIAL_EXTRA_RDEPENDS``
4008 variable with the exception that the image being built does not have
4009 a build dependency on the variable's list of packages. In other
4010 words, the image will still build if a package in this list is not
4011 found. Typically, this variable is used to handle essential kernel
4012 modules, whose functionality may be selected to be built into the
4013 kernel rather than as a module, in which case a package will not be
4014 produced.
4015
4016 Consider an example where you have a custom kernel where a specific
4017 touchscreen driver is required for the machine to be usable. However,
4018 the driver can be built as a module or into the kernel depending on
4019 the kernel configuration. If the driver is built as a module, you
4020 want it to be installed. But, when the driver is built into the
4021 kernel, you still want the build to succeed. This variable sets up a
4022 "recommends" relationship so that in the latter case, the build will
4023 not fail due to the missing package. To accomplish this, assuming the
4024 package for the module was called ``kernel-module-ab123``, you would
4025 use the following in the machine's ``.conf`` configuration file:
4026 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-module-ab123"
4027
4028 .. note::
4029
4030 In this example, the
4031 kernel-module-ab123
4032 recipe needs to explicitly set its
4033 PACKAGES
4034 variable to ensure that BitBake does not use the kernel recipe's
4035 PACKAGES_DYNAMIC
4036 variable to satisfy the dependency.
4037
4038 Some examples of these machine essentials are flash, screen,
4039 keyboard, mouse, or touchscreen drivers (depending on the machine).
4040
4041MACHINE_EXTRA_RDEPENDS
4042 A list of machine-specific packages to install as part of the image
4043 being built that are not essential for the machine to boot. However,
4044 the build process for more fully-featured images depends on the
4045 packages being present.
4046
4047 This variable affects all images based on ``packagegroup-base``,
4048 which does not include the ``core-image-minimal`` or
4049 ``core-image-full-cmdline`` images.
4050
4051 The variable is similar to the ``MACHINE_EXTRA_RRECOMMENDS`` variable
4052 with the exception that the image being built has a build dependency
4053 on the variable's list of packages. In other words, the image will
4054 not build if a file in this list is not found.
4055
4056 An example is a machine that has WiFi capability but is not essential
4057 for the machine to boot the image. However, if you are building a
4058 more fully-featured image, you want to enable the WiFi. The package
4059 containing the firmware for the WiFi hardware is always expected to
4060 exist, so it is acceptable for the build process to depend upon
4061 finding the package. In this case, assuming the package for the
4062 firmware was called ``wifidriver-firmware``, you would use the
4063 following in the ``.conf`` file for the machine:
4064 MACHINE_EXTRA_RDEPENDS += "wifidriver-firmware"
4065
4066MACHINE_EXTRA_RRECOMMENDS
4067 A list of machine-specific packages to install as part of the image
4068 being built that are not essential for booting the machine. The image
4069 being built has no build dependency on this list of packages.
4070
4071 This variable affects only images based on ``packagegroup-base``,
4072 which does not include the ``core-image-minimal`` or
4073 ``core-image-full-cmdline`` images.
4074
4075 This variable is similar to the ``MACHINE_EXTRA_RDEPENDS`` variable
4076 with the exception that the image being built does not have a build
4077 dependency on the variable's list of packages. In other words, the
4078 image will build if a file in this list is not found.
4079
4080 An example is a machine that has WiFi capability but is not essential
4081 For the machine to boot the image. However, if you are building a
4082 more fully-featured image, you want to enable WiFi. In this case, the
4083 package containing the WiFi kernel module will not be produced if the
4084 WiFi driver is built into the kernel, in which case you still want
4085 the build to succeed instead of failing as a result of the package
4086 not being found. To accomplish this, assuming the package for the
4087 module was called ``kernel-module-examplewifi``, you would use the
4088 following in the ``.conf`` file for the machine:
4089 MACHINE_EXTRA_RRECOMMENDS += "kernel-module-examplewifi"
4090
4091MACHINE_FEATURES
4092 Specifies the list of hardware features the
4093 ```MACHINE`` <#var-MACHINE>`__ is capable of supporting. For related
4094 information on enabling features, see the
4095 ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__,
4096 ```COMBINED_FEATURES`` <#var-COMBINED_FEATURES>`__, and
4097 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ variables.
4098
4099 For a list of hardware features supported by the Yocto Project as
4100 shipped, see the "`Machine Features <#ref-features-machine>`__"
4101 section.
4102
4103MACHINE_FEATURES_BACKFILL
4104 Features to be added to ``MACHINE_FEATURES`` if not also present in
4105 ``MACHINE_FEATURES_BACKFILL_CONSIDERED``.
4106
4107 This variable is set in the ``meta/conf/bitbake.conf`` file. It is
4108 not intended to be user-configurable. It is best to just reference
4109 the variable to see which machine features are being backfilled for
4110 all machine configurations. See the "`Feature
4111 Backfilling <#ref-features-backfill>`__" section for more
4112 information.
4113
4114MACHINE_FEATURES_BACKFILL_CONSIDERED
4115 Features from ``MACHINE_FEATURES_BACKFILL`` that should not be
4116 backfilled (i.e. added to ``MACHINE_FEATURES``) during the build. See
4117 the "`Feature Backfilling <#ref-features-backfill>`__" section for
4118 more information.
4119
4120MACHINEOVERRIDES
4121 A colon-separated list of overrides that apply to the current
4122 machine. By default, this list includes the value of
4123 ```MACHINE`` <#var-MACHINE>`__.
4124
4125 You can extend ``MACHINEOVERRIDES`` to add extra overrides that
4126 should apply to a machine. For example, all machines emulated in QEMU
4127 (e.g. ``qemuarm``, ``qemux86``, and so forth) include a file named
4128 ``meta/conf/machine/include/qemu.inc`` that prepends the following
4129 override to ``MACHINEOVERRIDES``: MACHINEOVERRIDES =. "qemuall:" This
4130 override allows variables to be overriden for all machines emulated
4131 in QEMU, like in the following example from the ``connman-conf``
4132 recipe: SRC_URI_append_qemuall = "file://wired.config \\
4133 file://wired-setup \\ " The underlying mechanism behind
4134 ``MACHINEOVERRIDES`` is simply that it is included in the default
4135 value of ```OVERRIDES`` <#var-OVERRIDES>`__.
4136
4137MAINTAINER
4138 The email address of the distribution maintainer.
4139
4140MIRRORS
4141 Specifies additional paths from which the OpenEmbedded build system
4142 gets source code. When the build system searches for source code, it
4143 first tries the local download directory. If that location fails, the
4144 build system tries locations defined by
4145 ```PREMIRRORS`` <#var-PREMIRRORS>`__, the upstream source, and then
4146 locations specified by ``MIRRORS`` in that order.
4147
4148 Assuming your distribution (```DISTRO`` <#var-DISTRO>`__) is "poky",
4149 the default value for ``MIRRORS`` is defined in the
4150 ``conf/distro/poky.conf`` file in the ``meta-poky`` Git repository.
4151
4152MLPREFIX
4153 Specifies a prefix has been added to ```PN`` <#var-PN>`__ to create a
4154 special version of a recipe or package (i.e. a Multilib version). The
4155 variable is used in places where the prefix needs to be added to or
4156 removed from a the name (e.g. the ```BPN`` <#var-BPN>`__ variable).
4157 ``MLPREFIX`` gets set when a prefix has been added to ``PN``.
4158
4159 .. note::
4160
4161 The "ML" in
4162 MLPREFIX
4163 stands for "MultiLib". This representation is historical and comes
4164 from a time when
4165 nativesdk
4166 was a suffix rather than a prefix on the recipe name. When
4167 nativesdk
4168 was turned into a prefix, it made sense to set
4169 MLPREFIX
4170 for it as well.
4171
4172 To help understand when ``MLPREFIX`` might be needed, consider when
4173 ```BBCLASSEXTEND`` <#var-BBCLASSEXTEND>`__ is used to provide a
4174 ``nativesdk`` version of a recipe in addition to the target version.
4175 If that recipe declares build-time dependencies on tasks in other
4176 recipes by using ```DEPENDS`` <#var-DEPENDS>`__, then a dependency on
4177 "foo" will automatically get rewritten to a dependency on
4178 "nativesdk-foo". However, dependencies like the following will not
4179 get rewritten automatically: do_foo[depends] += "recipe:do_foo" If
4180 you want such a dependency to also get transformed, you can do the
4181 following: do_foo[depends] += "${MLPREFIX}recipe:do_foo"
4182
4183module_autoload
4184 This variable has been replaced by the ``KERNEL_MODULE_AUTOLOAD``
4185 variable. You should replace all occurrences of ``module_autoload``
4186 with additions to ``KERNEL_MODULE_AUTOLOAD``, for example:
4187 module_autoload_rfcomm = "rfcomm"
4188
4189 should now be replaced with: KERNEL_MODULE_AUTOLOAD += "rfcomm" See
4190 the ```KERNEL_MODULE_AUTOLOAD`` <#var-KERNEL_MODULE_AUTOLOAD>`__
4191 variable for more information.
4192
4193module_conf
4194 Specifies ```modprobe.d`` <http://linux.die.net/man/5/modprobe.d>`__
4195 syntax lines for inclusion in the ``/etc/modprobe.d/modname.conf``
4196 file.
4197
4198 You can use this variable anywhere that it can be recognized by the
4199 kernel recipe or out-of-tree kernel module recipe (e.g. a machine
4200 configuration file, a distribution configuration file, an append file
4201 for the recipe, or the recipe itself). If you use this variable, you
4202 must also be sure to list the module name in the
4203 ```KERNEL_MODULE_AUTOLOAD`` <#var-KERNEL_MODULE_AUTOLOAD>`__
4204 variable.
4205
4206 Here is the general syntax: module_conf_module_name =
4207 "modprobe.d-syntax" You must use the kernel module name override.
4208
4209 Run ``man modprobe.d`` in the shell to find out more information on
4210 the exact syntax you want to provide with ``module_conf``.
4211
4212 Including ``module_conf`` causes the OpenEmbedded build system to
4213 populate the ``/etc/modprobe.d/modname.conf`` file with
4214 ``modprobe.d`` syntax lines. Here is an example that adds the options
4215 ``arg1`` and ``arg2`` to a module named ``mymodule``:
4216 module_conf_mymodule = "options mymodule arg1=val1 arg2=val2"
4217
4218 For information on how to specify kernel modules to auto-load on
4219 boot, see the
4220 ```KERNEL_MODULE_AUTOLOAD`` <#var-KERNEL_MODULE_AUTOLOAD>`__
4221 variable.
4222
4223MODULE_TARBALL_DEPLOY
4224 Controls creation of the ``modules-*.tgz`` file. Set this variable to
4225 "0" to disable creation of this file, which contains all of the
4226 kernel modules resulting from a kernel build.
4227
4228MODULE_TARBALL_LINK_NAME
4229 The link name of the kernel module tarball. This variable is set in
4230 the ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
4231 MODULE_TARBALL_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}" The value
4232 of the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the
4233 same file, has the following value: KERNEL_ARTIFACT_LINK_NAME ?=
4234 "${MACHINE}"
4235
4236 See the ```MACHINE`` <#var-MACHINE>`__ variable for additional
4237 information.
4238
4239MODULE_TARBALL_NAME
4240 The base name of the kernel module tarball. This variable is set in
4241 the ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
4242 MODULE_TARBALL_NAME ?= "${KERNEL_ARTIFACT_NAME}" The value of the
4243 ```KERNEL_ARTIFACT_NAME`` <#var-KERNEL_ARTIFACT_NAME>`__ variable,
4244 which is set in the same file, has the following value:
4245 KERNEL_ARTIFACT_NAME ?=
4246 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
4247
4248MULTIMACH_TARGET_SYS
4249 Uniquely identifies the type of the target system for which packages
4250 are being built. This variable allows output for different types of
4251 target systems to be put into different subdirectories of the same
4252 output directory.
4253
4254 The default value of this variable is:
4255 ${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS} Some classes (e.g.
4256 ```cross-canadian`` <#ref-classes-cross-canadian>`__) modify the
4257 ``MULTIMACH_TARGET_SYS`` value.
4258
4259 See the ```STAMP`` <#var-STAMP>`__ variable for an example. See the
4260 ```STAGING_DIR_TARGET`` <#var-STAGING_DIR_TARGET>`__ variable for
4261 more information.
4262
4263NATIVELSBSTRING
4264 A string identifying the host distribution. Strings consist of the
4265 host distributor ID followed by the release, as reported by the
4266 ``lsb_release`` tool or as read from ``/etc/lsb-release``. For
4267 example, when running a build on Ubuntu 12.10, the value is
4268 "Ubuntu-12.10". If this information is unable to be determined, the
4269 value resolves to "Unknown".
4270
4271 This variable is used by default to isolate native shared state
4272 packages for different distributions (e.g. to avoid problems with
4273 ``glibc`` version incompatibilities). Additionally, the variable is
4274 checked against
4275 ```SANITY_TESTED_DISTROS`` <#var-SANITY_TESTED_DISTROS>`__ if that
4276 variable is set.
4277
4278NM
4279 The minimal command and arguments to run ``nm``.
4280
4281NO_GENERIC_LICENSE
4282 Avoids QA errors when you use a non-common, non-CLOSED license in a
4283 recipe. Packages exist, such as the linux-firmware package, with many
4284 licenses that are not in any way common. Also, new licenses are added
4285 occasionally to avoid introducing a lot of common license files,
4286 which are only applicable to a specific package.
4287 ``NO_GENERIC_LICENSE`` is used to allow copying a license that does
4288 not exist in common licenses.
4289
4290 The following example shows how to add ``NO_GENERIC_LICENSE`` to a
4291 recipe: NO_GENERIC_LICENSE[license_name] =
4292 "license_file_in_fetched_source" The following is an example that
4293 uses the ``LICENSE.Abilis.txt`` file as the license from the fetched
4294 source: NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
4295
4296NO_RECOMMENDATIONS
4297 Prevents installation of all "recommended-only" packages.
4298 Recommended-only packages are packages installed only through the
4299 ```RRECOMMENDS`` <#var-RRECOMMENDS>`__ variable). Setting the
4300 ``NO_RECOMMENDATIONS`` variable to "1" turns this feature on:
4301 NO_RECOMMENDATIONS = "1"
4302
4303 You can set this variable globally in your ``local.conf`` file or you
4304 can attach it to a specific image recipe by using the recipe name
4305 override: NO_RECOMMENDATIONS_pn-target_image = "1"
4306
4307 It is important to realize that if you choose to not install packages
4308 using this variable and some other packages are dependent on them
4309 (i.e. listed in a recipe's ```RDEPENDS`` <#var-RDEPENDS>`__
4310 variable), the OpenEmbedded build system ignores your request and
4311 will install the packages to avoid dependency errors.
4312
4313 .. note::
4314
4315 Some recommended packages might be required for certain system
4316 functionality, such as kernel modules. It is up to you to add
4317 packages with the
4318 IMAGE_INSTALL
4319 variable.
4320
4321 Support for this variable exists only when using the IPK and RPM
4322 packaging backend. Support does not exist for DEB.
4323
4324 See the ```BAD_RECOMMENDATIONS`` <#var-BAD_RECOMMENDATIONS>`__ and
4325 the ```PACKAGE_EXCLUDE`` <#var-PACKAGE_EXCLUDE>`__ variables for
4326 related information.
4327
4328NOAUTOPACKAGEDEBUG
4329 Disables auto package from splitting ``.debug`` files. If a recipe
4330 requires ``FILES_${PN}-dbg`` to be set manually, the
4331 ``NOAUTOPACKAGEDEBUG`` can be defined allowing you to define the
4332 content of the debug package. For example: NOAUTOPACKAGEDEBUG = "1"
4333 FILES_${PN}-dev = "${includedir}/${QT_DIR_NAME}/Qt/*" FILES_${PN}-dbg
4334 = "/usr/src/debug/" FILES_${QT_BASE_NAME}-demos-doc =
4335 "${docdir}/${QT_DIR_NAME}/qch/qt.qch"
4336
4337OBJCOPY
4338 The minimal command and arguments to run ``objcopy``.
4339
4340OBJDUMP
4341 The minimal command and arguments to run ``objdump``.
4342
4343OE_BINCONFIG_EXTRA_MANGLE
4344 When inheriting the ```binconfig`` <#ref-classes-binconfig>`__ class,
4345 this variable specifies additional arguments passed to the "sed"
4346 command. The sed command alters any paths in configuration scripts
4347 that have been set up during compilation. Inheriting this class
4348 results in all paths in these scripts being changed to point into the
4349 ``sysroots/`` directory so that all builds that use the script will
4350 use the correct directories for the cross compiling layout.
4351
4352 See the ``meta/classes/binconfig.bbclass`` in the `Source
4353 Directory <#source-directory>`__ for details on how this class
4354 applies these additional sed command arguments. For general
4355 information on the ``binconfig`` class, see the
4356 "```binconfig.bbclass`` <#ref-classes-binconfig>`__" section.
4357
4358OE_IMPORTS
4359 An internal variable used to tell the OpenEmbedded build system what
4360 Python modules to import for every Python function run by the system.
4361
4362 .. note::
4363
4364 Do not set this variable. It is for internal use only.
4365
4366OE_INIT_ENV_SCRIPT
4367 The name of the build environment setup script for the purposes of
4368 setting up the environment within the extensible SDK. The default
4369 value is "oe-init-build-env".
4370
4371 If you use a custom script to set up your build environment, set the
4372 ``OE_INIT_ENV_SCRIPT`` variable to its name.
4373
4374OE_TERMINAL
4375 Controls how the OpenEmbedded build system spawns interactive
4376 terminals on the host development system (e.g. using the BitBake
4377 command with the ``-c devshell`` command-line option). For more
4378 information, see the "`Using a Development
4379 Shell <&YOCTO_DOCS_DEV_URL;#platdev-appdev-devshell>`__" section in
4380 the Yocto Project Development Tasks Manual.
4381
4382 You can use the following values for the ``OE_TERMINAL`` variable:
4383 auto gnome xfce rxvt screen konsole none
4384
4385OEROOT
4386 The directory from which the top-level build environment setup script
4387 is sourced. The Yocto Project provides a top-level build environment
4388 setup script: ````` <#structure-core-script>`__. When you run this
4389 script, the ``OEROOT`` variable resolves to the directory that
4390 contains the script.
4391
4392 For additional information on how this variable is used, see the
4393 initialization script.
4394
4395OLDEST_KERNEL
4396 Declares the oldest version of the Linux kernel that the produced
4397 binaries must support. This variable is passed into the build of the
4398 Embedded GNU C Library (``glibc``).
4399
4400 The default for this variable comes from the
4401 ``meta/conf/bitbake.conf`` configuration file. You can override this
4402 default by setting the variable in a custom distribution
4403 configuration file.
4404
4405OVERRIDES
4406 A colon-separated list of overrides that currently apply. Overrides
4407 are a BitBake mechanism that allows variables to be selectively
4408 overridden at the end of parsing. The set of overrides in
4409 ``OVERRIDES`` represents the "state" during building, which includes
4410 the current recipe being built, the machine for which it is being
4411 built, and so forth.
4412
4413 As an example, if the string "an-override" appears as an element in
4414 the colon-separated list in ``OVERRIDES``, then the following
4415 assignment will override ``FOO`` with the value "overridden" at the
4416 end of parsing: FOO_an-override = "overridden" See the "`Conditional
4417 Syntax
4418 (Overrides) <&YOCTO_DOCS_BB_URL;#conditional-syntax-overrides>`__"
4419 section in the BitBake User Manual for more information on the
4420 overrides mechanism.
4421
4422 The default value of ``OVERRIDES`` includes the values of the
4423 ```CLASSOVERRIDE`` <#var-CLASSOVERRIDE>`__,
4424 ```MACHINEOVERRIDES`` <#var-MACHINEOVERRIDES>`__, and
4425 ```DISTROOVERRIDES`` <#var-DISTROOVERRIDES>`__ variables. Another
4426 important override included by default is ``pn-${PN}``. This override
4427 allows variables to be set for a single recipe within configuration
4428 (``.conf``) files. Here is an example: FOO_pn-myrecipe =
4429 "myrecipe-specific value"
4430
4431 .. note::
4432
4433 An easy way to see what overrides apply is to search for
4434 OVERRIDES
4435 in the output of the
4436 bitbake -e
4437 command. See the "
4438 Viewing Variable Values
4439 " section in the Yocto Project Development Tasks Manual for more
4440 information.
4441
4442P
4443 The recipe name and version. ``P`` is comprised of the following:
4444 ${PN}-${PV}
4445
4446PACKAGE_ADD_METADATA
4447 This variable defines additional metdata to add to packages.
4448
4449 You may find you need to inject additional metadata into packages.
4450 This variable allows you to do that by setting the injected data as
4451 the value. Multiple fields can be added by splitting the content with
4452 the literal separator "\n".
4453
4454 The suffixes '_IPK', '_DEB', or '_RPM' can be applied to the variable
4455 to do package type specific settings. It can also be made package
4456 specific by using the package name as a suffix.
4457
4458 You can find out more about applying this variable in the "`Adding
4459 custom metadata to
4460 packages <&YOCTO_DOCS_DEV_URL;#adding-custom-metadata-to-packages>`__"
4461 section in the Yocto Project Development Tasks Manual.
4462
4463PACKAGE_ARCH
4464 The architecture of the resulting package or packages.
4465
4466 By default, the value of this variable is set to
4467 ```TUNE_PKGARCH`` <#var-TUNE_PKGARCH>`__ when building for the
4468 target, ```BUILD_ARCH`` <#var-BUILD_ARCH>`__ when building for the
4469 build host, and "${SDK_ARCH}-${SDKPKGSUFFIX}" when building for the
4470 SDK.
4471
4472 .. note::
4473
4474 See
4475 SDK_ARCH
4476 for more information.
4477
4478 However, if your recipe's output packages are built specific to the
4479 target machine rather than generally for the architecture of the
4480 machine, you should set ``PACKAGE_ARCH`` to the value of
4481 ```MACHINE_ARCH`` <#var-MACHINE_ARCH>`__ in the recipe as follows:
4482 PACKAGE_ARCH = "${MACHINE_ARCH}"
4483
4484PACKAGE_ARCHS
4485 Specifies a list of architectures compatible with the target machine.
4486 This variable is set automatically and should not normally be
4487 hand-edited. Entries are separated using spaces and listed in order
4488 of priority. The default value for ``PACKAGE_ARCHS`` is "all any
4489 noarch ${PACKAGE_EXTRA_ARCHS} ${MACHINE_ARCH}".
4490
4491PACKAGE_BEFORE_PN
4492 Enables easily adding packages to ``PACKAGES`` before ``${PN}`` so
4493 that those added packages can pick up files that would normally be
4494 included in the default package.
4495
4496PACKAGE_CLASSES
4497 This variable, which is set in the ``local.conf`` configuration file
4498 found in the ``conf`` folder of the `Build
4499 Directory <#build-directory>`__, specifies the package manager the
4500 OpenEmbedded build system uses when packaging data.
4501
4502 You can provide one or more of the following arguments for the
4503 variable: PACKAGE_CLASSES ?= "package_rpm package_deb package_ipk
4504 package_tar"
4505
4506 .. note::
4507
4508 While it is a legal option, the
4509 package_tar
4510 class has limited functionality due to no support for package
4511 dependencies by that backend. Therefore, it is recommended that
4512 you do not use it.
4513
4514 The build system uses only the first argument in the list as the
4515 package manager when creating your image or SDK. However, packages
4516 will be created using any additional packaging classes you specify.
4517 For example, if you use the following in your ``local.conf`` file:
4518 PACKAGE_CLASSES ?= "package_ipk" The OpenEmbedded build system uses
4519 the IPK package manager to create your image or SDK.
4520
4521 For information on packaging and build performance effects as a
4522 result of the package manager in use, see the
4523 "```package.bbclass`` <#ref-classes-package>`__" section.
4524
4525PACKAGE_DEBUG_SPLIT_STYLE
4526 Determines how to split up the binary and debug information when
4527 creating ``*-dbg`` packages to be used with the GNU Project Debugger
4528 (GDB).
4529
4530 With the ``PACKAGE_DEBUG_SPLIT_STYLE`` variable, you can control
4531 where debug information, which can include or exclude source files,
4532 is stored:
4533
4534 - ".debug": Debug symbol files are placed next to the binary in a
4535 ``.debug`` directory on the target. For example, if a binary is
4536 installed into ``/bin``, the corresponding debug symbol files are
4537 installed in ``/bin/.debug``. Source files are placed in
4538 ``/usr/src/debug``.
4539
4540 - "debug-file-directory": Debug symbol files are placed under
4541 ``/usr/lib/debug`` on the target, and separated by the path from
4542 where the binary is installed. For example, if a binary is
4543 installed in ``/bin``, the corresponding debug symbols are
4544 installed in ``/usr/lib/debug/bin``. Source files are placed in
4545 ``/usr/src/debug``.
4546
4547 - "debug-without-src": The same behavior as ".debug" previously
4548 described with the exception that no source files are installed.
4549
4550 - "debug-with-srcpkg": The same behavior as ".debug" previously
4551 described with the exception that all source files are placed in a
4552 separate ``*-src`` pkg. This is the default behavior.
4553
4554 You can find out more about debugging using GDB by reading the
4555 "`Debugging With the GNU Project Debugger (GDB)
4556 Remotely <&YOCTO_DOCS_DEV_URL;#platdev-gdb-remotedebug>`__" section
4557 in the Yocto Project Development Tasks Manual.
4558
4559PACKAGE_EXCLUDE_COMPLEMENTARY
4560 Prevents specific packages from being installed when you are
4561 installing complementary packages.
4562
4563 You might find that you want to prevent installing certain packages
4564 when you are installing complementary packages. For example, if you
4565 are using ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__ to install
4566 ``dev-pkgs``, you might not want to install all packages from a
4567 particular multilib. If you find yourself in this situation, you can
4568 use the ``PACKAGE_EXCLUDE_COMPLEMENTARY`` variable to specify regular
4569 expressions to match the packages you want to exclude.
4570
4571PACKAGE_EXCLUDE
4572 Lists packages that should not be installed into an image. For
4573 example: PACKAGE_EXCLUDE = "package_name package_name package_name
4574 ..."
4575
4576 You can set this variable globally in your ``local.conf`` file or you
4577 can attach it to a specific image recipe by using the recipe name
4578 override: PACKAGE_EXCLUDE_pn-target_image = "package_name"
4579
4580 If you choose to not install a package using this variable and some
4581 other package is dependent on it (i.e. listed in a recipe's
4582 ```RDEPENDS`` <#var-RDEPENDS>`__ variable), the OpenEmbedded build
4583 system generates a fatal installation error. Because the build system
4584 halts the process with a fatal error, you can use the variable with
4585 an iterative development process to remove specific components from a
4586 system.
4587
4588 Support for this variable exists only when using the IPK and RPM
4589 packaging backend. Support does not exist for DEB.
4590
4591 See the ```NO_RECOMMENDATIONS`` <#var-NO_RECOMMENDATIONS>`__ and the
4592 ```BAD_RECOMMENDATIONS`` <#var-BAD_RECOMMENDATIONS>`__ variables for
4593 related information.
4594
4595PACKAGE_EXTRA_ARCHS
4596 Specifies the list of architectures compatible with the device CPU.
4597 This variable is useful when you build for several different devices
4598 that use miscellaneous processors such as XScale and ARM926-EJS.
4599
4600PACKAGE_FEED_ARCHS
4601 Optionally specifies the package architectures used as part of the
4602 package feed URIs during the build. When used, the
4603 ``PACKAGE_FEED_ARCHS`` variable is appended to the final package feed
4604 URI, which is constructed using the
4605 ```PACKAGE_FEED_URIS`` <#var-PACKAGE_FEED_URIS>`__ and
4606 ```PACKAGE_FEED_BASE_PATHS`` <#var-PACKAGE_FEED_BASE_PATHS>`__
4607 variables.
4608
4609 .. note::
4610
4611 You can use the
4612 PACKAGE_FEEDS_ARCHS
4613 variable to whitelist specific package architectures. If you do
4614 not need to whitelist specific architectures, which is a common
4615 case, you can omit this variable. Omitting the variable results in
4616 all available architectures for the current machine being included
4617 into remote package feeds.
4618
4619 Consider the following example where the ``PACKAGE_FEED_URIS``,
4620 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
4621 defined in your ``local.conf`` file: PACKAGE_FEED_URIS =
4622 "https://example.com/packagerepos/release \\
4623 https://example.com/packagerepos/updates" PACKAGE_FEED_BASE_PATHS =
4624 "rpm rpm-dev" PACKAGE_FEED_ARCHS = "all core2-64" Given these
4625 settings, the resulting package feeds are as follows:
4626 https://example.com/packagerepos/release/rpm/all
4627 https://example.com/packagerepos/release/rpm/core2-64
4628 https://example.com/packagerepos/release/rpm-dev/all
4629 https://example.com/packagerepos/release/rpm-dev/core2-64
4630 https://example.com/packagerepos/updates/rpm/all
4631 https://example.com/packagerepos/updates/rpm/core2-64
4632 https://example.com/packagerepos/updates/rpm-dev/all
4633 https://example.com/packagerepos/updates/rpm-dev/core2-64
4634
4635PACKAGE_FEED_BASE_PATHS
4636 Specifies the base path used when constructing package feed URIs. The
4637 ``PACKAGE_FEED_BASE_PATHS`` variable makes up the middle portion of a
4638 package feed URI used by the OpenEmbedded build system. The base path
4639 lies between the ```PACKAGE_FEED_URIS`` <#var-PACKAGE_FEED_URIS>`__
4640 and ```PACKAGE_FEED_ARCHS`` <#var-PACKAGE_FEED_ARCHS>`__ variables.
4641
4642 Consider the following example where the ``PACKAGE_FEED_URIS``,
4643 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
4644 defined in your ``local.conf`` file: PACKAGE_FEED_URIS =
4645 "https://example.com/packagerepos/release \\
4646 https://example.com/packagerepos/updates" PACKAGE_FEED_BASE_PATHS =
4647 "rpm rpm-dev" PACKAGE_FEED_ARCHS = "all core2-64" Given these
4648 settings, the resulting package feeds are as follows:
4649 https://example.com/packagerepos/release/rpm/all
4650 https://example.com/packagerepos/release/rpm/core2-64
4651 https://example.com/packagerepos/release/rpm-dev/all
4652 https://example.com/packagerepos/release/rpm-dev/core2-64
4653 https://example.com/packagerepos/updates/rpm/all
4654 https://example.com/packagerepos/updates/rpm/core2-64
4655 https://example.com/packagerepos/updates/rpm-dev/all
4656 https://example.com/packagerepos/updates/rpm-dev/core2-64
4657
4658PACKAGE_FEED_URIS
4659 Specifies the front portion of the package feed URI used by the
4660 OpenEmbedded build system. Each final package feed URI is comprised
4661 of ``PACKAGE_FEED_URIS``,
4662 ```PACKAGE_FEED_BASE_PATHS`` <#var-PACKAGE_FEED_BASE_PATHS>`__, and
4663 ```PACKAGE_FEED_ARCHS`` <#var-PACKAGE_FEED_ARCHS>`__ variables.
4664
4665 Consider the following example where the ``PACKAGE_FEED_URIS``,
4666 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
4667 defined in your ``local.conf`` file: PACKAGE_FEED_URIS =
4668 "https://example.com/packagerepos/release \\
4669 https://example.com/packagerepos/updates" PACKAGE_FEED_BASE_PATHS =
4670 "rpm rpm-dev" PACKAGE_FEED_ARCHS = "all core2-64" Given these
4671 settings, the resulting package feeds are as follows:
4672 https://example.com/packagerepos/release/rpm/all
4673 https://example.com/packagerepos/release/rpm/core2-64
4674 https://example.com/packagerepos/release/rpm-dev/all
4675 https://example.com/packagerepos/release/rpm-dev/core2-64
4676 https://example.com/packagerepos/updates/rpm/all
4677 https://example.com/packagerepos/updates/rpm/core2-64
4678 https://example.com/packagerepos/updates/rpm-dev/all
4679 https://example.com/packagerepos/updates/rpm-dev/core2-64
4680
4681PACKAGE_INSTALL
4682 The final list of packages passed to the package manager for
4683 installation into the image.
4684
4685 Because the package manager controls actual installation of all
4686 packages, the list of packages passed using ``PACKAGE_INSTALL`` is
4687 not the final list of packages that are actually installed. This
4688 variable is internal to the image construction code. Consequently, in
4689 general, you should use the
4690 ```IMAGE_INSTALL`` <#var-IMAGE_INSTALL>`__ variable to specify
4691 packages for installation. The exception to this is when working with
4692 the
4693 ```core-image-minimal-initramfs`` <#images-core-image-minimal-initramfs>`__
4694 image. When working with an initial RAM filesystem (initramfs) image,
4695 use the ``PACKAGE_INSTALL`` variable. For information on creating an
4696 initramfs, see the "`Building an Initial RAM Filesystem (initramfs)
4697 Image <&YOCTO_DOCS_DEV_URL;#building-an-initramfs-image>`__" section
4698 in the Yocto Project Development Tasks Manual.
4699
4700PACKAGE_INSTALL_ATTEMPTONLY
4701 Specifies a list of packages the OpenEmbedded build system attempts
4702 to install when creating an image. If a listed package fails to
4703 install, the build system does not generate an error. This variable
4704 is generally not user-defined.
4705
4706PACKAGE_PREPROCESS_FUNCS
4707 Specifies a list of functions run to pre-process the
4708 ```PKGD`` <#var-PKGD>`__ directory prior to splitting the files out
4709 to individual packages.
4710
4711PACKAGE_WRITE_DEPS
4712 Specifies a list of dependencies for post-installation and
4713 pre-installation scripts on native/cross tools. If your
4714 post-installation or pre-installation script can execute at rootfs
4715 creation time rather than on the target but depends on a native tool
4716 in order to execute, you need to list the tools in
4717 ``PACKAGE_WRITE_DEPS``.
4718
4719 For information on running post-installation scripts, see the
4720 "`Post-Installation
4721 Scripts <&YOCTO_DOCS_DEV_URL;#new-recipe-post-installation-scripts>`__"
4722 section in the Yocto Project Development Tasks Manual.
4723
4724PACKAGECONFIG
4725 This variable provides a means of enabling or disabling features of a
4726 recipe on a per-recipe basis. ``PACKAGECONFIG`` blocks are defined in
4727 recipes when you specify features and then arguments that define
4728 feature behaviors. Here is the basic block structure (broken over
4729 multiple lines for readability): PACKAGECONFIG ??= "f1 f2 f3 ..."
4730 PACKAGECONFIG[f1] = "\\ --with-f1, \\ --without-f1, \\
4731 build-deps-for-f1, \\ runtime-deps-for-f1, \\
4732 runtime-recommends-for-f1, \\ packageconfig-conflicts-for-f1 \\ "
4733 PACKAGECONFIG[f2] = "\\ ... and so on and so on ...
4734
4735 The ``PACKAGECONFIG`` variable itself specifies a space-separated
4736 list of the features to enable. Following the features, you can
4737 determine the behavior of each feature by providing up to six
4738 order-dependent arguments, which are separated by commas. You can
4739 omit any argument you like but must retain the separating commas. The
4740 order is important and specifies the following:
4741
4742 1. Extra arguments that should be added to the configure script
4743 argument list (```EXTRA_OECONF`` <#var-EXTRA_OECONF>`__ or
4744 ```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__) if
4745 the feature is enabled.
4746
4747 2. Extra arguments that should be added to ``EXTRA_OECONF`` or
4748 ``PACKAGECONFIG_CONFARGS`` if the feature is disabled.
4749
4750 3. Additional build dependencies (```DEPENDS`` <#var-DEPENDS>`__)
4751 that should be added if the feature is enabled.
4752
4753 4. Additional runtime dependencies (```RDEPENDS`` <#var-RDEPENDS>`__)
4754 that should be added if the feature is enabled.
4755
4756 5. Additional runtime recommendations
4757 (```RRECOMMENDS`` <#var-RRECOMMENDS>`__) that should be added if
4758 the feature is enabled.
4759
4760 6. Any conflicting (that is, mutually exclusive) ``PACKAGECONFIG``
4761 settings for this feature.
4762
4763 Consider the following ``PACKAGECONFIG`` block taken from the
4764 ``librsvg`` recipe. In this example the feature is ``gtk``, which has
4765 three arguments that determine the feature's behavior.
4766 PACKAGECONFIG[gtk] = "--with-gtk3,--without-gtk3,gtk+3" The
4767 ``--with-gtk3`` and ``gtk+3`` arguments apply only if the feature is
4768 enabled. In this case, ``--with-gtk3`` is added to the configure
4769 script argument list and ``gtk+3`` is added to ``DEPENDS``. On the
4770 other hand, if the feature is disabled say through a ``.bbappend``
4771 file in another layer, then the second argument ``--without-gtk3`` is
4772 added to the configure script instead.
4773
4774 The basic ``PACKAGECONFIG`` structure previously described holds true
4775 regardless of whether you are creating a block or changing a block.
4776 When creating a block, use the structure inside your recipe.
4777
4778 If you want to change an existing ``PACKAGECONFIG`` block, you can do
4779 so one of two ways:
4780
4781 - *Append file:* Create an append file named
4782 recipename\ ``.bbappend`` in your layer and override the value of
4783 ``PACKAGECONFIG``. You can either completely override the
4784 variable: PACKAGECONFIG = "f4 f5" Or, you can just append the
4785 variable: PACKAGECONFIG_append = " f4"
4786
4787 - *Configuration file:* This method is identical to changing the
4788 block through an append file except you edit your ``local.conf``
4789 or ``mydistro.conf`` file. As with append files previously
4790 described, you can either completely override the variable:
4791 PACKAGECONFIG_pn-recipename = "f4 f5" Or, you can just amend the
4792 variable: PACKAGECONFIG_append_pn-recipename = " f4"
4793
4794PACKAGECONFIG_CONFARGS
4795 A space-separated list of configuration options generated from the
4796 ```PACKAGECONFIG`` <#var-PACKAGECONFIG>`__ setting.
4797
4798 Classes such as ```autotools`` <#ref-classes-autotools>`__ and
4799 ```cmake`` <#ref-classes-cmake>`__ use ``PACKAGECONFIG_CONFARGS`` to
4800 pass ``PACKAGECONFIG`` options to ``configure`` and ``cmake``,
4801 respectively. If you are using ``PACKAGECONFIG`` but not a class that
4802 handles the ``do_configure`` task, then you need to use
4803 ``PACKAGECONFIG_CONFARGS`` appropriately.
4804
4805PACKAGEGROUP_DISABLE_COMPLEMENTARY
4806 For recipes inheriting the
4807 ```packagegroup`` <#ref-classes-packagegroup>`__ class, setting
4808 ``PACKAGEGROUP_DISABLE_COMPLEMENTARY`` to "1" specifies that the
4809 normal complementary packages (i.e. ``-dev``, ``-dbg``, and so forth)
4810 should not be automatically created by the ``packagegroup`` recipe,
4811 which is the default behavior.
4812
4813PACKAGES
4814 The list of packages the recipe creates. The default value is the
4815 following: ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale
4816 ${PACKAGE_BEFORE_PN} ${PN}
4817
4818 During packaging, the ```do_package`` <#ref-tasks-package>`__ task
4819 goes through ``PACKAGES`` and uses the ```FILES`` <#var-FILES>`__
4820 variable corresponding to each package to assign files to the
4821 package. If a file matches the ``FILES`` variable for more than one
4822 package in ``PACKAGES``, it will be assigned to the earliest
4823 (leftmost) package.
4824
4825 Packages in the variable's list that are empty (i.e. where none of
4826 the patterns in ``FILES_``\ pkg match any files installed by the
4827 ```do_install`` <#ref-tasks-install>`__ task) are not generated,
4828 unless generation is forced through the
4829 ```ALLOW_EMPTY`` <#var-ALLOW_EMPTY>`__ variable.
4830
4831PACKAGES_DYNAMIC
4832 A promise that your recipe satisfies runtime dependencies for
4833 optional modules that are found in other recipes.
4834 ``PACKAGES_DYNAMIC`` does not actually satisfy the dependencies, it
4835 only states that they should be satisfied. For example, if a hard,
4836 runtime dependency (```RDEPENDS`` <#var-RDEPENDS>`__) of another
4837 package is satisfied at build time through the ``PACKAGES_DYNAMIC``
4838 variable, but a package with the module name is never actually
4839 produced, then the other package will be broken. Thus, if you attempt
4840 to include that package in an image, you will get a dependency
4841 failure from the packaging system during the
4842 ```do_rootfs`` <#ref-tasks-rootfs>`__ task.
4843
4844 Typically, if there is a chance that such a situation can occur and
4845 the package that is not created is valid without the dependency being
4846 satisfied, then you should use ```RRECOMMENDS`` <#var-RRECOMMENDS>`__
4847 (a soft runtime dependency) instead of ``RDEPENDS``.
4848
4849 For an example of how to use the ``PACKAGES_DYNAMIC`` variable when
4850 you are splitting packages, see the "`Handling Optional Module
4851 Packaging <&YOCTO_DOCS_DEV_URL;#handling-optional-module-packaging>`__"
4852 section in the Yocto Project Development Tasks Manual.
4853
4854PACKAGESPLITFUNCS
4855 Specifies a list of functions run to perform additional splitting of
4856 files into individual packages. Recipes can either prepend to this
4857 variable or prepend to the ``populate_packages`` function in order to
4858 perform additional package splitting. In either case, the function
4859 should set ```PACKAGES`` <#var-PACKAGES>`__,
4860 ```FILES`` <#var-FILES>`__, ```RDEPENDS`` <#var-RDEPENDS>`__ and
4861 other packaging variables appropriately in order to perform the
4862 desired splitting.
4863
4864PARALLEL_MAKE
4865 Extra options passed to the ``make`` command during the
4866 ```do_compile`` <#ref-tasks-compile>`__ task in order to specify
4867 parallel compilation on the local build host. This variable is
4868 usually in the form "-j x", where x represents the maximum number of
4869 parallel threads ``make`` can run.
4870
4871 .. note::
4872
4873 In order for
4874 PARALLEL_MAKE
4875 to be effective,
4876 make
4877 must be called with
4878 ${
4879 EXTRA_OEMAKE
4880 }
4881 . An easy way to ensure this is to use the
4882 oe_runmake
4883 function.
4884
4885 By default, the OpenEmbedded build system automatically sets this
4886 variable to be equal to the number of cores the build system uses.
4887
4888 .. note::
4889
4890 If the software being built experiences dependency issues during
4891 the
4892 do_compile
4893 task that result in race conditions, you can clear the
4894 PARALLEL_MAKE
4895 variable within the recipe as a workaround. For information on
4896 addressing race conditions, see the "
4897 Debugging Parallel Make Races
4898 " section in the Yocto Project Development Tasks Manual.
4899
4900 For single socket systems (i.e. one CPU), you should not have to
4901 override this variable to gain optimal parallelism during builds.
4902 However, if you have very large systems that employ multiple physical
4903 CPUs, you might want to make sure the ``PARALLEL_MAKE`` variable is
4904 not set higher than "-j 20".
4905
4906 For more information on speeding up builds, see the "`Speeding Up a
4907 Build <&YOCTO_DOCS_DEV_URL;#speeding-up-a-build>`__" section in the
4908 Yocto Project Development Tasks Manual.
4909
4910PARALLEL_MAKEINST
4911 Extra options passed to the ``make install`` command during the
4912 ```do_install`` <#ref-tasks-install>`__ task in order to specify
4913 parallel installation. This variable defaults to the value of
4914 ```PARALLEL_MAKE`` <#var-PARALLEL_MAKE>`__.
4915
4916 .. note::
4917
4918 In order for ``PARALLEL_MAKEINST`` to be effective, ``make`` must
4919 be called with
4920 ``${``\ ```EXTRA_OEMAKE`` <#var-EXTRA_OEMAKE>`__\ ``}``. An easy
4921 way to ensure this is to use the ``oe_runmake`` function.
4922
4923 If the software being built experiences dependency issues during
4924 the ``do_install`` task that result in race conditions, you can
4925 clear the ``PARALLEL_MAKEINST`` variable within the recipe as a
4926 workaround. For information on addressing race conditions, see the
4927 "`Debugging Parallel Make
4928 Races <&YOCTO_DOCS_DEV_URL;#debugging-parallel-make-races>`__"
4929 section in the Yocto Project Development Tasks Manual.
4930
4931PATCHRESOLVE
4932 Determines the action to take when a patch fails. You can set this
4933 variable to one of two values: "noop" and "user".
4934
4935 The default value of "noop" causes the build to simply fail when the
4936 OpenEmbedded build system cannot successfully apply a patch. Setting
4937 the value to "user" causes the build system to launch a shell and
4938 places you in the right location so that you can manually resolve the
4939 conflicts.
4940
4941 Set this variable in your ``local.conf`` file.
4942
4943PATCHTOOL
4944 Specifies the utility used to apply patches for a recipe during the
4945 ```do_patch`` <#ref-tasks-patch>`__ task. You can specify one of
4946 three utilities: "patch", "quilt", or "git". The default utility used
4947 is "quilt" except for the quilt-native recipe itself. Because the
4948 quilt tool is not available at the time quilt-native is being
4949 patched, it uses "patch".
4950
4951 If you wish to use an alternative patching tool, set the variable in
4952 the recipe using one of the following: PATCHTOOL = "patch" PATCHTOOL
4953 = "quilt" PATCHTOOL = "git"
4954
4955PE
4956 The epoch of the recipe. By default, this variable is unset. The
4957 variable is used to make upgrades possible when the versioning scheme
4958 changes in some backwards incompatible way.
4959
4960 ``PE`` is the default value of the ```PKGE`` <#var-PKGE>`__ variable.
4961
4962PF
4963 Specifies the recipe or package name and includes all version and
4964 revision numbers (i.e. ``glibc-2.13-r20+svnr15508/`` and
4965 ``bash-4.2-r1/``). This variable is comprised of the following:
4966 ${`PN <#var-PN>`__}-${`EXTENDPE <#var-EXTENDPE>`__}${`PV <#var-PV>`__}-${`PR <#var-PR>`__}
4967
4968PIXBUF_PACKAGES
4969 When inheriting the ```pixbufcache`` <#ref-classes-pixbufcache>`__
4970 class, this variable identifies packages that contain the pixbuf
4971 loaders used with ``gdk-pixbuf``. By default, the ``pixbufcache``
4972 class assumes that the loaders are in the recipe's main package (i.e.
4973 ``${``\ ```PN`` <#var-PN>`__\ ``}``). Use this variable if the
4974 loaders you need are in a package other than that main package.
4975
4976PKG
4977 The name of the resulting package created by the OpenEmbedded build
4978 system.
4979
4980 .. note::
4981
4982 When using the
4983 PKG
4984 variable, you must use a package name override.
4985
4986 For example, when the ```debian`` <#ref-classes-debian>`__ class
4987 renames the output package, it does so by setting
4988 ``PKG_packagename``.
4989
4990PKG_CONFIG_PATH
4991 The path to ``pkg-config`` files for the current build context.
4992 ``pkg-config`` reads this variable from the environment.
4993
4994PKGD
4995 Points to the destination directory for files to be packaged before
4996 they are split into individual packages. This directory defaults to
4997 the following: ${WORKDIR}/package
4998
4999 Do not change this default.
5000
5001PKGDATA_DIR
5002 Points to a shared, global-state directory that holds data generated
5003 during the packaging process. During the packaging process, the
5004 ```do_packagedata`` <#ref-tasks-packagedata>`__ task packages data
5005 for each recipe and installs it into this temporary, shared area.
5006 This directory defaults to the following, which you should not
5007 change: ${STAGING_DIR_HOST}/pkgdata For examples of how this data is
5008 used, see the "`Automatically Added Runtime
5009 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
5010 section in the Yocto Project Overview and Concepts Manual and the
5011 "`Viewing Package Information with
5012 ``oe-pkgdata-util`` <&YOCTO_DOCS_DEV_URL;#viewing-package-information-with-oe-pkgdata-util>`__"
5013 section in the Yocto Project Development Tasks Manual. For more
5014 information on the shared, global-state directory, see
5015 ```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__.
5016
5017PKGDEST
5018 Points to the parent directory for files to be packaged after they
5019 have been split into individual packages. This directory defaults to
5020 the following: ${WORKDIR}/packages-split
5021
5022 Under this directory, the build system creates directories for each
5023 package specified in ```PACKAGES`` <#var-PACKAGES>`__. Do not change
5024 this default.
5025
5026PKGDESTWORK
5027 Points to a temporary work area where the
5028 ```do_package`` <#ref-tasks-package>`__ task saves package metadata.
5029 The ``PKGDESTWORK`` location defaults to the following:
5030 ${WORKDIR}/pkgdata Do not change this default.
5031
5032 The ```do_packagedata`` <#ref-tasks-packagedata>`__ task copies the
5033 package metadata from ``PKGDESTWORK`` to
5034 ```PKGDATA_DIR`` <#var-PKGDATA_DIR>`__ to make it available globally.
5035
5036PKGE
5037 The epoch of the package(s) built by the recipe. By default, ``PKGE``
5038 is set to ```PE`` <#var-PE>`__.
5039
5040PKGR
5041 The revision of the package(s) built by the recipe. By default,
5042 ``PKGR`` is set to ```PR`` <#var-PR>`__.
5043
5044PKGV
5045 The version of the package(s) built by the recipe. By default,
5046 ``PKGV`` is set to ```PV`` <#var-PV>`__.
5047
5048PN
5049 This variable can have two separate functions depending on the
5050 context: a recipe name or a resulting package name.
5051
5052 ``PN`` refers to a recipe name in the context of a file used by the
5053 OpenEmbedded build system as input to create a package. The name is
5054 normally extracted from the recipe file name. For example, if the
5055 recipe is named ``expat_2.0.1.bb``, then the default value of ``PN``
5056 will be "expat".
5057
5058 The variable refers to a package name in the context of a file
5059 created or produced by the OpenEmbedded build system.
5060
5061 If applicable, the ``PN`` variable also contains any special suffix
5062 or prefix. For example, using ``bash`` to build packages for the
5063 native machine, ``PN`` is ``bash-native``. Using ``bash`` to build
5064 packages for the target and for Multilib, ``PN`` would be ``bash``
5065 and ``lib64-bash``, respectively.
5066
5067PNBLACKLIST
5068 Lists recipes you do not want the OpenEmbedded build system to build.
5069 This variable works in conjunction with the
5070 ```blacklist`` <#ref-classes-blacklist>`__ class, which is inherited
5071 globally.
5072
5073 To prevent a recipe from being built, use the ``PNBLACKLIST``
5074 variable in your ``local.conf`` file. Here is an example that
5075 prevents ``myrecipe`` from being built: PNBLACKLIST[myrecipe] = "Not
5076 supported by our organization."
5077
5078POPULATE_SDK_POST_HOST_COMMAND
5079 Specifies a list of functions to call once the OpenEmbedded build
5080 system has created the host part of the SDK. You can specify
5081 functions separated by semicolons: POPULATE_SDK_POST_HOST_COMMAND +=
5082 "function; ... "
5083
5084 If you need to pass the SDK path to a command within a function, you
5085 can use ``${SDK_DIR}``, which points to the parent directory used by
5086 the OpenEmbedded build system when creating SDK output. See the
5087 ```SDK_DIR`` <#var-SDK_DIR>`__ variable for more information.
5088
5089POPULATE_SDK_POST_TARGET_COMMAND
5090 Specifies a list of functions to call once the OpenEmbedded build
5091 system has created the target part of the SDK. You can specify
5092 functions separated by semicolons: POPULATE_SDK_POST_TARGET_COMMAND
5093 += "function; ... "
5094
5095 If you need to pass the SDK path to a command within a function, you
5096 can use ``${SDK_DIR}``, which points to the parent directory used by
5097 the OpenEmbedded build system when creating SDK output. See the
5098 ```SDK_DIR`` <#var-SDK_DIR>`__ variable for more information.
5099
5100PR
5101 The revision of the recipe. The default value for this variable is
5102 "r0". Subsequent revisions of the recipe conventionally have the
5103 values "r1", "r2", and so forth. When ```PV`` <#var-PV>`__ increases,
5104 ``PR`` is conventionally reset to "r0".
5105
5106 .. note::
5107
5108 The OpenEmbedded build system does not need the aid of
5109 PR
5110 to know when to rebuild a recipe. The build system uses the task
5111 input checksums
5112 along with the
5113 stamp
5114 and
5115 shared state cache
5116 mechanisms.
5117
5118 The ``PR`` variable primarily becomes significant when a package
5119 manager dynamically installs packages on an already built image. In
5120 this case, ``PR``, which is the default value of
5121 ```PKGR`` <#var-PKGR>`__, helps the package manager distinguish which
5122 package is the most recent one in cases where many packages have the
5123 same ``PV`` (i.e. ``PKGV``). A component having many packages with
5124 the same ``PV`` usually means that the packages all install the same
5125 upstream version, but with later (``PR``) version packages including
5126 packaging fixes.
5127
5128 .. note::
5129
5130 PR
5131 does not need to be increased for changes that do not change the
5132 package contents or metadata.
5133
5134 Because manually managing ``PR`` can be cumbersome and error-prone,
5135 an automated solution exists. See the "`Working With a PR
5136 Service <&YOCTO_DOCS_DEV_URL;#working-with-a-pr-service>`__" section
5137 in the Yocto Project Development Tasks Manual for more information.
5138
5139PREFERRED_PROVIDER
5140 If multiple recipes provide the same item, this variable determines
5141 which recipe is preferred and thus provides the item (i.e. the
5142 preferred provider). You should always suffix this variable with the
5143 name of the provided item. And, you should define the variable using
5144 the preferred recipe's name (```PN`` <#var-PN>`__). Here is a common
5145 example: PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto" In the
5146 previous example, multiple recipes are providing "virtual/kernel".
5147 The ``PREFERRED_PROVIDER`` variable is set with the name (``PN``) of
5148 the recipe you prefer to provide "virtual/kernel".
5149
5150 Following are more examples: PREFERRED_PROVIDER_virtual/xserver =
5151 "xserver-xf86" PREFERRED_PROVIDER_virtual/libgl ?= "mesa" For more
5152 information, see the "`Using Virtual
5153 Providers <&YOCTO_DOCS_DEV_URL;#metadata-virtual-providers>`__"
5154 section in the Yocto Project Development Tasks Manual.
5155
5156 .. note::
5157
5158 If you use a
5159 virtual/\*
5160 item with
5161 PREFERRED_PROVIDER
5162 , then any recipe that
5163 PROVIDES
5164 that item but is not selected (defined) by
5165 PREFERRED_PROVIDER
5166 is prevented from building, which is usually desirable since this
5167 mechanism is designed to select between mutually exclusive
5168 alternative providers.
5169
5170PREFERRED_VERSION
5171 If multiple versions of recipes exist, this variable determines which
5172 version is given preference. You must always suffix the variable with
5173 the ```PN`` <#var-PN>`__ you want to select, and you should set the
5174 ```PV`` <#var-PV>`__ accordingly for precedence.
5175
5176 The ``PREFERRED_VERSION`` variable supports limited wildcard use
5177 through the "``%``" character. You can use the character to match any
5178 number of characters, which can be useful when specifying versions
5179 that contain long revision numbers that potentially change. Here are
5180 two examples: PREFERRED_VERSION_python = "3.4.0"
5181 PREFERRED_VERSION_linux-yocto = "5.0%"
5182
5183 .. note::
5184
5185 The use of the "
5186 %
5187 " character is limited in that it only works at the end of the
5188 string. You cannot use the wildcard character in any other
5189 location of the string.
5190
5191 The specified version is matched against ```PV`` <#var-PV>`__, which
5192 does not necessarily match the version part of the recipe's filename.
5193 For example, consider two recipes ``foo_1.2.bb`` and ``foo_git.bb``
5194 where ``foo_git.bb`` contains the following assignment: PV =
5195 "1.1+git${SRCPV}" In this case, the correct way to select
5196 ``foo_git.bb`` is by using an assignment such as the following:
5197 PREFERRED_VERSION_foo = "1.1+git%" Compare that previous example
5198 against the following incorrect example, which does not work:
5199 PREFERRED_VERSION_foo = "git"
5200
5201 Sometimes the ``PREFERRED_VERSION`` variable can be set by
5202 configuration files in a way that is hard to change. You can use
5203 ```OVERRIDES`` <#var-OVERRIDES>`__ to set a machine-specific
5204 override. Here is an example: PREFERRED_VERSION_linux-yocto_qemux86 =
5205 "5.0%" Although not recommended, worst case, you can also use the
5206 "forcevariable" override, which is the strongest override possible.
5207 Here is an example: PREFERRED_VERSION_linux-yocto_forcevariable =
5208 "5.0%"
5209
5210 .. note::
5211
5212 The
5213 \_forcevariable
5214 override is not handled specially. This override only works
5215 because the default value of
5216 OVERRIDES
5217 includes "forcevariable".
5218
5219PREMIRRORS
5220 Specifies additional paths from which the OpenEmbedded build system
5221 gets source code. When the build system searches for source code, it
5222 first tries the local download directory. If that location fails, the
5223 build system tries locations defined by ``PREMIRRORS``, the upstream
5224 source, and then locations specified by
5225 ```MIRRORS`` <#var-MIRRORS>`__ in that order.
5226
5227 Assuming your distribution (```DISTRO`` <#var-DISTRO>`__) is "poky",
5228 the default value for ``PREMIRRORS`` is defined in the
5229 ``conf/distro/poky.conf`` file in the ``meta-poky`` Git repository.
5230
5231 Typically, you could add a specific server for the build system to
5232 attempt before any others by adding something like the following to
5233 the ``local.conf`` configuration file in the `Build
5234 Directory <#build-directory>`__: PREMIRRORS_prepend = "\\
5235 git://.*/.\* http://www.yoctoproject.org/sources/ \\n \\ ftp://.*/.\*
5236 http://www.yoctoproject.org/sources/ \\n \\ http://.*/.\*
5237 http://www.yoctoproject.org/sources/ \\n \\ https://.*/.\*
5238 http://www.yoctoproject.org/sources/ \\n" These changes cause the
5239 build system to intercept Git, FTP, HTTP, and HTTPS requests and
5240 direct them to the ``http://`` sources mirror. You can use
5241 ``file://`` URLs to point to local directories or network shares as
5242 well.
5243
5244PRIORITY
5245 Indicates the importance of a package.
5246
5247 ``PRIORITY`` is considered to be part of the distribution policy
5248 because the importance of any given recipe depends on the purpose for
5249 which the distribution is being produced. Thus, ``PRIORITY`` is not
5250 normally set within recipes.
5251
5252 You can set ``PRIORITY`` to "required", "standard", "extra", and
5253 "optional", which is the default.
5254
5255PRIVATE_LIBS
5256 Specifies libraries installed within a recipe that should be ignored
5257 by the OpenEmbedded build system's shared library resolver. This
5258 variable is typically used when software being built by a recipe has
5259 its own private versions of a library normally provided by another
5260 recipe. In this case, you would not want the package containing the
5261 private libraries to be set as a dependency on other unrelated
5262 packages that should instead depend on the package providing the
5263 standard version of the library.
5264
5265 Libraries specified in this variable should be specified by their
5266 file name. For example, from the Firefox recipe in meta-browser:
5267 PRIVATE_LIBS = "libmozjs.so \\ libxpcom.so \\ libnspr4.so \\
5268 libxul.so \\ libmozalloc.so \\ libplc4.so \\ libplds4.so"
5269
5270 For more information, see the "`Automatically Added Runtime
5271 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
5272 section in the Yocto Project Overview and Concepts Manual.
5273
5274PROVIDES
5275 A list of aliases by which a particular recipe can be known. By
5276 default, a recipe's own ``PN`` is implicitly already in its
5277 ``PROVIDES`` list and therefore does not need to mention that it
5278 provides itself. If a recipe uses ``PROVIDES``, the additional
5279 aliases are synonyms for the recipe and can be useful for satisfying
5280 dependencies of other recipes during the build as specified by
5281 ``DEPENDS``.
5282
5283 Consider the following example ``PROVIDES`` statement from the recipe
5284 file ``eudev_3.2.9.bb``: PROVIDES = "udev" The ``PROVIDES`` statement
5285 results in the "eudev" recipe also being available as simply "udev".
5286
5287 .. note::
5288
5289 Given that a recipe's own recipe name is already implicitly in its
5290 own
5291 PROVIDES
5292 list, it is unnecessary to add aliases with the "+=" operator;
5293 using a simple assignment will be sufficient. In other words,
5294 while you could write:
5295 ::
5296
5297 PROVIDES += "udev"
5298
5299
5300 in the above, the "+=" is overkill and unnecessary.
5301
5302 In addition to providing recipes under alternate names, the
5303 ``PROVIDES`` mechanism is also used to implement virtual targets. A
5304 virtual target is a name that corresponds to some particular
5305 functionality (e.g. a Linux kernel). Recipes that provide the
5306 functionality in question list the virtual target in ``PROVIDES``.
5307 Recipes that depend on the functionality in question can include the
5308 virtual target in ``DEPENDS`` to leave the choice of provider open.
5309
5310 Conventionally, virtual targets have names on the form
5311 "virtual/function" (e.g. "virtual/kernel"). The slash is simply part
5312 of the name and has no syntactical significance.
5313
5314 The ```PREFERRED_PROVIDER`` <#var-PREFERRED_PROVIDER>`__ variable is
5315 used to select which particular recipe provides a virtual target.
5316
5317 .. note::
5318
5319 A corresponding mechanism for virtual runtime dependencies
5320 (packages) exists. However, the mechanism does not depend on any
5321 special functionality beyond ordinary variable assignments. For
5322 example, ``VIRTUAL-RUNTIME_dev_manager`` refers to the package of
5323 the component that manages the ``/dev`` directory.
5324
5325 Setting the "preferred provider" for runtime dependencies is as
5326 simple as using the following assignment in a configuration file:
5327
5328 ::
5329
5330 VIRTUAL-RUNTIME_dev_manager = "udev"
5331
5332
5333PRSERV_HOST
5334 The network based ```PR`` <#var-PR>`__ service host and port.
5335
5336 The ``conf/local.conf.sample.extended`` configuration file in the
5337 `Source Directory <#source-directory>`__ shows how the
5338 ``PRSERV_HOST`` variable is set: PRSERV_HOST = "localhost:0" You must
5339 set the variable if you want to automatically start a local `PR
5340 service <&YOCTO_DOCS_DEV_URL;#working-with-a-pr-service>`__. You can
5341 set ``PRSERV_HOST`` to other values to use a remote PR service.
5342
5343PTEST_ENABLED
5344 Specifies whether or not `Package
5345 Test <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__ (ptest)
5346 functionality is enabled when building a recipe. You should not set
5347 this variable directly. Enabling and disabling building Package Tests
5348 at build time should be done by adding "ptest" to (or removing it
5349 from) ```DISTRO_FEATURES`` <#var-DISTRO_FEATURES>`__.
5350
5351PV
5352 The version of the recipe. The version is normally extracted from the
5353 recipe filename. For example, if the recipe is named
5354 ``expat_2.0.1.bb``, then the default value of ``PV`` will be "2.0.1".
5355 ``PV`` is generally not overridden within a recipe unless it is
5356 building an unstable (i.e. development) version from a source code
5357 repository (e.g. Git or Subversion).
5358
5359 ``PV`` is the default value of the ```PKGV`` <#var-PKGV>`__ variable.
5360
5361PYTHON_ABI
5362 When used by recipes that inherit the
5363 ```distutils3`` <#ref-classes-distutils3>`__,
5364 ```setuptools3`` <#ref-classes-setuptools3>`__,
5365 ```distutils`` <#ref-classes-distutils>`__, or
5366 ```setuptools`` <#ref-classes-setuptools>`__ classes, denotes the
5367 Application Binary Interface (ABI) currently in use for Python. By
5368 default, the ABI is "m". You do not have to set this variable as the
5369 OpenEmbedded build system sets it for you.
5370
5371 The OpenEmbedded build system uses the ABI to construct directory
5372 names used when installing the Python headers and libraries in
5373 sysroot (e.g. ``.../python3.3m/...``).
5374
5375 Recipes that inherit the ``distutils`` class during cross-builds also
5376 use this variable to locate the headers and libraries of the
5377 appropriate Python that the extension is targeting.
5378
5379PYTHON_PN
5380 When used by recipes that inherit the
5381 ```distutils3`` <#ref-classes-distutils3>`__,
5382 ```setuptools3`` <#ref-classes-setuptools3>`__,
5383 ```distutils`` <#ref-classes-distutils>`__, or
5384 ```setuptools`` <#ref-classes-setuptools>`__ classes, specifies the
5385 major Python version being built. For Python 3.x, ``PYTHON_PN`` would
5386 be "python3". You do not have to set this variable as the
5387 OpenEmbedded build system automatically sets it for you.
5388
5389 The variable allows recipes to use common infrastructure such as the
5390 following: DEPENDS += "${PYTHON_PN}-native" In the previous example,
5391 the version of the dependency is ``PYTHON_PN``.
5392
5393RANLIB
5394 The minimal command and arguments to run ``ranlib``.
5395
5396RCONFLICTS
5397 The list of packages that conflict with packages. Note that packages
5398 will not be installed if conflicting packages are not first removed.
5399
5400 Like all package-controlling variables, you must always use them in
5401 conjunction with a package name override. Here is an example:
5402 RCONFLICTS_${PN} = "another_conflicting_package_name"
5403
5404 BitBake, which the OpenEmbedded build system uses, supports
5405 specifying versioned dependencies. Although the syntax varies
5406 depending on the packaging format, BitBake hides these differences
5407 from you. Here is the general syntax to specify versions with the
5408 ``RCONFLICTS`` variable: RCONFLICTS_${PN} = "package (operator
5409 version)" For ``operator``, you can specify the following: = < > <=
5410 >= For example, the following sets up a dependency on version 1.2 or
5411 greater of the package ``foo``: RCONFLICTS_${PN} = "foo (>= 1.2)"
5412
5413RDEPENDS
5414 Lists runtime dependencies of a package. These dependencies are other
5415 packages that must be installed in order for the package to function
5416 correctly. As an example, the following assignment declares that the
5417 package ``foo`` needs the packages ``bar`` and ``baz`` to be
5418 installed: RDEPENDS_foo = "bar baz" The most common types of package
5419 runtime dependencies are automatically detected and added. Therefore,
5420 most recipes do not need to set ``RDEPENDS``. For more information,
5421 see the "`Automatically Added Runtime
5422 Dependencies <&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies>`__"
5423 section in the Yocto Project Overview and Concepts Manual.
5424
5425 The practical effect of the above ``RDEPENDS`` assignment is that
5426 ``bar`` and ``baz`` will be declared as dependencies inside the
5427 package ``foo`` when it is written out by one of the
5428 ```do_package_write_*`` <#ref-tasks-package_write_deb>`__ tasks.
5429 Exactly how this is done depends on which package format is used,
5430 which is determined by
5431 ```PACKAGE_CLASSES`` <#var-PACKAGE_CLASSES>`__. When the
5432 corresponding package manager installs the package, it will know to
5433 also install the packages on which it depends.
5434
5435 To ensure that the packages ``bar`` and ``baz`` get built, the
5436 previous ``RDEPENDS`` assignment also causes a task dependency to be
5437 added. This dependency is from the recipe's
5438 ```do_build`` <#ref-tasks-build>`__ (not to be confused with
5439 ```do_compile`` <#ref-tasks-compile>`__) task to the
5440 ``do_package_write_*`` task of the recipes that build ``bar`` and
5441 ``baz``.
5442
5443 The names of the packages you list within ``RDEPENDS`` must be the
5444 names of other packages - they cannot be recipe names. Although
5445 package names and recipe names usually match, the important point
5446 here is that you are providing package names within the ``RDEPENDS``
5447 variable. For an example of the default list of packages created from
5448 a recipe, see the ```PACKAGES`` <#var-PACKAGES>`__ variable.
5449
5450 Because the ``RDEPENDS`` variable applies to packages being built,
5451 you should always use the variable in a form with an attached package
5452 name (remember that a single recipe can build multiple packages). For
5453 example, suppose you are building a development package that depends
5454 on the ``perl`` package. In this case, you would use the following
5455 ``RDEPENDS`` statement: RDEPENDS_${PN}-dev += "perl" In the example,
5456 the development package depends on the ``perl`` package. Thus, the
5457 ``RDEPENDS`` variable has the ``${PN}-dev`` package name as part of
5458 the variable.
5459
5460 .. note::
5461
5462 RDEPENDS_${PN}-dev
5463 includes
5464 ${
5465 PN
5466 }
5467 by default. This default is set in the BitBake configuration file
5468 (
5469 meta/conf/bitbake.conf
5470 ). Be careful not to accidentally remove
5471 ${PN}
5472 when modifying
5473 RDEPENDS_${PN}-dev
5474 . Use the "+=" operator rather than the "=" operator.
5475
5476 The package names you use with ``RDEPENDS`` must appear as they would
5477 in the ``PACKAGES`` variable. The ```PKG`` <#var-PKG>`__ variable
5478 allows a different name to be used for the final package (e.g. the
5479 ```debian`` <#ref-classes-debian>`__ class uses this to rename
5480 packages), but this final package name cannot be used with
5481 ``RDEPENDS``, which makes sense as ``RDEPENDS`` is meant to be
5482 independent of the package format used.
5483
5484 BitBake, which the OpenEmbedded build system uses, supports
5485 specifying versioned dependencies. Although the syntax varies
5486 depending on the packaging format, BitBake hides these differences
5487 from you. Here is the general syntax to specify versions with the
5488 ``RDEPENDS`` variable: RDEPENDS_${PN} = "package (operator version)"
5489 For operator, you can specify the following: = < > <= >= For version,
5490 provide the version number.
5491
5492 .. note::
5493
5494 You can use
5495 EXTENDPKGV
5496 to provide a full package version specification.
5497
5498 For example, the following sets up a dependency on version 1.2 or
5499 greater of the package ``foo``: RDEPENDS_${PN} = "foo (>= 1.2)"
5500
5501 For information on build-time dependencies, see the
5502 ```DEPENDS`` <#var-DEPENDS>`__ variable. You can also see the
5503 "`Tasks <&YOCTO_DOCS_BB_URL;#tasks>`__" and
5504 "`Dependencies <&YOCTO_DOCS_BB_URL;#dependencies>`__" sections in the
5505 BitBake User Manual for additional information on tasks and
5506 dependencies.
5507
5508REQUIRED_DISTRO_FEATURES
5509 When inheriting the
5510 ```distro_features_check`` <#ref-classes-distro_features_check>`__
5511 class, this variable identifies distribution features that must exist
5512 in the current configuration in order for the OpenEmbedded build
5513 system to build the recipe. In other words, if the
5514 ``REQUIRED_DISTRO_FEATURES`` variable lists a feature that does not
5515 appear in ``DISTRO_FEATURES`` within the current configuration, an
5516 error occurs and the build stops.
5517
5518RM_WORK_EXCLUDE
5519 With ``rm_work`` enabled, this variable specifies a list of recipes
5520 whose work directories should not be removed. See the
5521 "```rm_work.bbclass`` <#ref-classes-rm-work>`__" section for more
5522 details.
5523
5524ROOT_HOME
5525 Defines the root home directory. By default, this directory is set as
5526 follows in the BitBake configuration file: ROOT_HOME ??= "/home/root"
5527
5528 .. note::
5529
5530 This default value is likely used because some embedded solutions
5531 prefer to have a read-only root filesystem and prefer to keep
5532 writeable data in one place.
5533
5534 You can override the default by setting the variable in any layer or
5535 in the ``local.conf`` file. Because the default is set using a "weak"
5536 assignment (i.e. "??="), you can use either of the following forms to
5537 define your override: ROOT_HOME = "/root" ROOT_HOME ?= "/root" These
5538 override examples use ``/root``, which is probably the most commonly
5539 used override.
5540
5541ROOTFS
5542 Indicates a filesystem image to include as the root filesystem.
5543
5544 The ``ROOTFS`` variable is an optional variable used with the
5545 ```image-live`` <#ref-classes-image-live>`__ class.
5546
5547ROOTFS_POSTINSTALL_COMMAND
5548 Specifies a list of functions to call after the OpenEmbedded build
5549 system has installed packages. You can specify functions separated by
5550 semicolons: ROOTFS_POSTINSTALL_COMMAND += "function; ... "
5551
5552 If you need to pass the root filesystem path to a command within a
5553 function, you can use ``${IMAGE_ROOTFS}``, which points to the
5554 directory that becomes the root filesystem image. See the
5555 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
5556 information.
5557
5558ROOTFS_POSTPROCESS_COMMAND
5559 Specifies a list of functions to call once the OpenEmbedded build
5560 system has created the root filesystem. You can specify functions
5561 separated by semicolons: ROOTFS_POSTPROCESS_COMMAND += "function; ...
5562 "
5563
5564 If you need to pass the root filesystem path to a command within a
5565 function, you can use ``${IMAGE_ROOTFS}``, which points to the
5566 directory that becomes the root filesystem image. See the
5567 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
5568 information.
5569
5570ROOTFS_POSTUNINSTALL_COMMAND
5571 Specifies a list of functions to call after the OpenEmbedded build
5572 system has removed unnecessary packages. When runtime package
5573 management is disabled in the image, several packages are removed
5574 including ``base-passwd``, ``shadow``, and ``update-alternatives``.
5575 You can specify functions separated by semicolons:
5576 ROOTFS_POSTUNINSTALL_COMMAND += "function; ... "
5577
5578 If you need to pass the root filesystem path to a command within a
5579 function, you can use ``${IMAGE_ROOTFS}``, which points to the
5580 directory that becomes the root filesystem image. See the
5581 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
5582 information.
5583
5584ROOTFS_PREPROCESS_COMMAND
5585 Specifies a list of functions to call before the OpenEmbedded build
5586 system has created the root filesystem. You can specify functions
5587 separated by semicolons: ROOTFS_PREPROCESS_COMMAND += "function; ...
5588 "
5589
5590 If you need to pass the root filesystem path to a command within a
5591 function, you can use ``${IMAGE_ROOTFS}``, which points to the
5592 directory that becomes the root filesystem image. See the
5593 ```IMAGE_ROOTFS`` <#var-IMAGE_ROOTFS>`__ variable for more
5594 information.
5595
5596RPROVIDES
5597 A list of package name aliases that a package also provides. These
5598 aliases are useful for satisfying runtime dependencies of other
5599 packages both during the build and on the target (as specified by
5600 ``RDEPENDS``).
5601
5602 .. note::
5603
5604 A package's own name is implicitly already in its
5605 RPROVIDES
5606 list.
5607
5608 As with all package-controlling variables, you must always use the
5609 variable in conjunction with a package name override. Here is an
5610 example: RPROVIDES_${PN} = "widget-abi-2"
5611
5612RRECOMMENDS
5613 A list of packages that extends the usability of a package being
5614 built. The package being built does not depend on this list of
5615 packages in order to successfully build, but rather uses them for
5616 extended usability. To specify runtime dependencies for packages, see
5617 the ``RDEPENDS`` variable.
5618
5619 The package manager will automatically install the ``RRECOMMENDS``
5620 list of packages when installing the built package. However, you can
5621 prevent listed packages from being installed by using the
5622 ```BAD_RECOMMENDATIONS`` <#var-BAD_RECOMMENDATIONS>`__,
5623 ```NO_RECOMMENDATIONS`` <#var-NO_RECOMMENDATIONS>`__, and
5624 ```PACKAGE_EXCLUDE`` <#var-PACKAGE_EXCLUDE>`__ variables.
5625
5626 Packages specified in ``RRECOMMENDS`` need not actually be produced.
5627 However, a recipe must exist that provides each package, either
5628 through the ```PACKAGES`` <#var-PACKAGES>`__ or
5629 ```PACKAGES_DYNAMIC`` <#var-PACKAGES_DYNAMIC>`__ variables or the
5630 ```RPROVIDES`` <#var-RPROVIDES>`__ variable, or an error will occur
5631 during the build. If such a recipe does exist and the package is not
5632 produced, the build continues without error.
5633
5634 Because the ``RRECOMMENDS`` variable applies to packages being built,
5635 you should always attach an override to the variable to specify the
5636 particular package whose usability is being extended. For example,
5637 suppose you are building a development package that is extended to
5638 support wireless functionality. In this case, you would use the
5639 following: RRECOMMENDS_${PN}-dev += "wireless_package_name" In the
5640 example, the package name (``${PN}-dev``) must appear as it would in
5641 the ``PACKAGES`` namespace before any renaming of the output package
5642 by classes such as ``debian.bbclass``.
5643
5644 BitBake, which the OpenEmbedded build system uses, supports
5645 specifying versioned recommends. Although the syntax varies depending
5646 on the packaging format, BitBake hides these differences from you.
5647 Here is the general syntax to specify versions with the
5648 ``RRECOMMENDS`` variable: RRECOMMENDS_${PN} = "package (operator
5649 version)" For ``operator``, you can specify the following: = < > <=
5650 >= For example, the following sets up a recommend on version 1.2 or
5651 greater of the package ``foo``: RRECOMMENDS_${PN} = "foo (>= 1.2)"
5652
5653RREPLACES
5654 A list of packages replaced by a package. The package manager uses
5655 this variable to determine which package should be installed to
5656 replace other package(s) during an upgrade. In order to also have the
5657 other package(s) removed at the same time, you must add the name of
5658 the other package to the ``RCONFLICTS`` variable.
5659
5660 As with all package-controlling variables, you must use this variable
5661 in conjunction with a package name override. Here is an example:
5662 RREPLACES_${PN} = "other_package_being_replaced"
5663
5664 BitBake, which the OpenEmbedded build system uses, supports
5665 specifying versioned replacements. Although the syntax varies
5666 depending on the packaging format, BitBake hides these differences
5667 from you. Here is the general syntax to specify versions with the
5668 ``RREPLACES`` variable: RREPLACES_${PN} = "package (operator
5669 version)" For ``operator``, you can specify the following: = < > <=
5670 >= For example, the following sets up a replacement using version 1.2
5671 or greater of the package ``foo``: RREPLACES_${PN} = "foo (>= 1.2)"
5672
5673RSUGGESTS
5674 A list of additional packages that you can suggest for installation
5675 by the package manager at the time a package is installed. Not all
5676 package managers support this functionality.
5677
5678 As with all package-controlling variables, you must always use this
5679 variable in conjunction with a package name override. Here is an
5680 example: RSUGGESTS_${PN} = "useful_package another_package"
5681
5682S
5683 The location in the `Build Directory <#build-directory>`__ where
5684 unpacked recipe source code resides. By default, this directory is
5685 ``${``\ ```WORKDIR`` <#var-WORKDIR>`__\ ``}/${``\ ```BPN`` <#var-BPN>`__\ ``}-${``\ ```PV`` <#var-PV>`__\ ``}``,
5686 where ``${BPN}`` is the base recipe name and ``${PV}`` is the recipe
5687 version. If the source tarball extracts the code to a directory named
5688 anything other than ``${BPN}-${PV}``, or if the source code is
5689 fetched from an SCM such as Git or Subversion, then you must set
5690 ``S`` in the recipe so that the OpenEmbedded build system knows where
5691 to find the unpacked source.
5692
5693 As an example, assume a `Source Directory <#source-directory>`__
5694 top-level folder named ``poky`` and a default Build Directory at
5695 ``poky/build``. In this case, the work directory the build system
5696 uses to keep the unpacked recipe for ``db`` is the following:
5697 poky/build/tmp/work/qemux86-poky-linux/db/5.1.19-r3/db-5.1.19 The
5698 unpacked source code resides in the ``db-5.1.19`` folder.
5699
5700 This next example assumes a Git repository. By default, Git
5701 repositories are cloned to ``${WORKDIR}/git`` during
5702 ```do_fetch`` <#ref-tasks-fetch>`__. Since this path is different
5703 from the default value of ``S``, you must set it specifically so the
5704 source can be located: SRC_URI = "git://path/to/repo.git" S =
5705 "${WORKDIR}/git"
5706
5707SANITY_REQUIRED_UTILITIES
5708 Specifies a list of command-line utilities that should be checked for
5709 during the initial sanity checking process when running BitBake. If
5710 any of the utilities are not installed on the build host, then
5711 BitBake immediately exits with an error.
5712
5713SANITY_TESTED_DISTROS
5714 A list of the host distribution identifiers that the build system has
5715 been tested against. Identifiers consist of the host distributor ID
5716 followed by the release, as reported by the ``lsb_release`` tool or
5717 as read from ``/etc/lsb-release``. Separate the list items with
5718 explicit newline characters (``\n``). If ``SANITY_TESTED_DISTROS`` is
5719 not empty and the current value of
5720 ```NATIVELSBSTRING`` <#var-NATIVELSBSTRING>`__ does not appear in the
5721 list, then the build system reports a warning that indicates the
5722 current host distribution has not been tested as a build host.
5723
5724SDK_ARCH
5725 The target architecture for the SDK. Typically, you do not directly
5726 set this variable. Instead, use ```SDKMACHINE`` <#var-SDKMACHINE>`__.
5727
5728SDK_DEPLOY
5729 The directory set up and used by the
5730 ```populate_sdk_base`` <#ref-classes-populate-sdk>`__ class to which
5731 the SDK is deployed. The ``populate_sdk_base`` class defines
5732 ``SDK_DEPLOY`` as follows: SDK_DEPLOY = "${TMPDIR}/deploy/sdk"
5733
5734SDK_DIR
5735 The parent directory used by the OpenEmbedded build system when
5736 creating SDK output. The
5737 ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class defines
5738 the variable as follows: SDK_DIR = "${WORKDIR}/sdk"
5739
5740 .. note::
5741
5742 The
5743 SDK_DIR
5744 directory is a temporary directory as it is part of
5745 WORKDIR
5746 . The final output directory is
5747 SDK_DEPLOY
5748 .
5749
5750SDK_EXT_TYPE
5751 Controls whether or not shared state artifacts are copied into the
5752 extensible SDK. The default value of "full" copies all of the
5753 required shared state artifacts into the extensible SDK. The value
5754 "minimal" leaves these artifacts out of the SDK.
5755
5756 .. note::
5757
5758 If you set the variable to "minimal", you need to ensure
5759 SSTATE_MIRRORS
5760 is set in the SDK's configuration to enable the artifacts to be
5761 fetched as needed.
5762
5763SDK_HOST_MANIFEST
5764 The manifest file for the host part of the SDK. This file lists all
5765 the installed packages that make up the host part of the SDK. The
5766 file contains package information on a line-per-package basis as
5767 follows: packagename packagearch version
5768
5769 The ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class
5770 defines the manifest file as follows: SDK_HOST_MANIFEST =
5771 "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.host.manifest" The location is
5772 derived using the ```SDK_DEPLOY`` <#var-SDK_DEPLOY>`__ and
5773 ```TOOLCHAIN_OUTPUTNAME`` <#var-TOOLCHAIN_OUTPUTNAME>`__ variables.
5774
5775SDK_INCLUDE_PKGDATA
5776 When set to "1", specifies to include the packagedata for all recipes
5777 in the "world" target in the extensible SDK. Including this data
5778 allows the ``devtool search`` command to find these recipes in search
5779 results, as well as allows the ``devtool add`` command to map
5780 dependencies more effectively.
5781
5782 .. note::
5783
5784 Enabling the
5785 SDK_INCLUDE_PKGDATA
5786 variable significantly increases build time because all of world
5787 needs to be built. Enabling the variable also slightly increases
5788 the size of the extensible SDK.
5789
5790SDK_INCLUDE_TOOLCHAIN
5791 When set to "1", specifies to include the toolchain in the extensible
5792 SDK. Including the toolchain is useful particularly when
5793 ```SDK_EXT_TYPE`` <#var-SDK_EXT_TYPE>`__ is set to "minimal" to keep
5794 the SDK reasonably small but you still want to provide a usable
5795 toolchain. For example, suppose you want to use the toolchain from an
5796 IDE or from other tools and you do not want to perform additional
5797 steps to install the toolchain.
5798
5799 The ``SDK_INCLUDE_TOOLCHAIN`` variable defaults to "0" if
5800 ``SDK_EXT_TYPE`` is set to "minimal", and defaults to "1" if
5801 ``SDK_EXT_TYPE`` is set to "full".
5802
5803SDK_INHERIT_BLACKLIST
5804 A list of classes to remove from the ```INHERIT`` <#var-INHERIT>`__
5805 value globally within the extensible SDK configuration. The
5806 ```populate-sdk-ext`` <#ref-classes-populate-sdk-*>`__ class sets the
5807 default value: SDK_INHERIT_BLACKLIST ?= "buildhistory icecc"
5808
5809 Some classes are not generally applicable within the extensible SDK
5810 context. You can use this variable to disable those classes.
5811
5812 For additional information on how to customize the extensible SDK's
5813 configuration, see the "`Configuring the Extensible
5814 SDK <&YOCTO_DOCS_SDK_URL;#sdk-configuring-the-extensible-sdk>`__"
5815 section in the Yocto Project Application Development and the
5816 Extensible Software Development Kit (eSDK) manual.
5817
5818SDK_LOCAL_CONF_BLACKLIST
5819 A list of variables not allowed through from the OpenEmbedded build
5820 system configuration into the extensible SDK configuration. Usually,
5821 these are variables that are specific to the machine on which the
5822 build system is running and thus would be potentially problematic
5823 within the extensible SDK.
5824
5825 By default, ``SDK_LOCAL_CONF_BLACKLIST`` is set in the
5826 ```populate-sdk-ext`` <#ref-classes-populate-sdk-*>`__ class and
5827 excludes the following variables:
5828 `CONF_VERSION <#var-CONF_VERSION>`__
5829 `BB_NUMBER_THREADS <#var-BB_NUMBER_THREADS>`__
5830 `BB_NUMBER_PARSE_THREADS <&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS>`__
5831 `PARALLEL_MAKE <#var-PARALLEL_MAKE>`__
5832 `PRSERV_HOST <#var-PRSERV_HOST>`__
5833 `SSTATE_MIRRORS <#var-SSTATE_MIRRORS>`__ `DL_DIR <#var-DL_DIR>`__
5834 `SSTATE_DIR <#var-SSTATE_DIR>`__ `TMPDIR <#var-TMPDIR>`__
5835 `BB_SERVER_TIMEOUT <#var-BB_SERVER_TIMEOUT>`__
5836
5837 For additional information on how to customize the extensible SDK's
5838 configuration, see the "`Configuring the Extensible
5839 SDK <&YOCTO_DOCS_SDK_URL;#sdk-configuring-the-extensible-sdk>`__"
5840 section in the Yocto Project Application Development and the
5841 Extensible Software Development Kit (eSDK) manual.
5842
5843SDK_LOCAL_CONF_WHITELIST
5844 A list of variables allowed through from the OpenEmbedded build
5845 system configuration into the extensible SDK configuration. By
5846 default, the list of variables is empty and is set in the
5847 ```populate-sdk-ext`` <#ref-classes-populate-sdk-*>`__ class.
5848
5849 This list overrides the variables specified using the
5850 ```SDK_LOCAL_CONF_BLACKLIST`` <#var-SDK_LOCAL_CONF_BLACKLIST>`__
5851 variable as well as any variables identified by automatic
5852 blacklisting due to the "/" character being found at the start of the
5853 value, which is usually indicative of being a path and thus might not
5854 be valid on the system where the SDK is installed.
5855
5856 For additional information on how to customize the extensible SDK's
5857 configuration, see the "`Configuring the Extensible
5858 SDK <&YOCTO_DOCS_SDK_URL;#sdk-configuring-the-extensible-sdk>`__"
5859 section in the Yocto Project Application Development and the
5860 Extensible Software Development Kit (eSDK) manual.
5861
5862SDK_NAME
5863 The base name for SDK output files. The name is derived from the
5864 ```DISTRO`` <#var-DISTRO>`__, ```TCLIBC`` <#var-TCLIBC>`__,
5865 ```SDK_ARCH`` <#var-SDK_ARCH>`__,
5866 ```IMAGE_BASENAME`` <#var-IMAGE_BASENAME>`__, and
5867 ```TUNE_PKGARCH`` <#var-TUNE_PKGARCH>`__ variables: SDK_NAME =
5868 "${DISTRO}-${TCLIBC}-${SDK_ARCH}-${IMAGE_BASENAME}-${TUNE_PKGARCH}"
5869
5870SDK_OS
5871 Specifies the operating system for which the SDK will be built. The
5872 default value is the value of ```BUILD_OS`` <#var-BUILD_OS>`__.
5873
5874SDK_OUTPUT
5875 The location used by the OpenEmbedded build system when creating SDK
5876 output. The ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__
5877 class defines the variable as follows: SDK_DIR = "${WORKDIR}/sdk"
5878 SDK_OUTPUT = "${SDK_DIR}/image" SDK_DEPLOY = "${DEPLOY_DIR}/sdk"
5879
5880 .. note::
5881
5882 The
5883 SDK_OUTPUT
5884 directory is a temporary directory as it is part of
5885 WORKDIR
5886 by way of
5887 SDK_DIR
5888 . The final output directory is
5889 SDK_DEPLOY
5890 .
5891
5892SDK_PACKAGE_ARCHS
5893 Specifies a list of architectures compatible with the SDK machine.
5894 This variable is set automatically and should not normally be
5895 hand-edited. Entries are separated using spaces and listed in order
5896 of priority. The default value for ``SDK_PACKAGE_ARCHS`` is "all any
5897 noarch ${SDK_ARCH}-${SDKPKGSUFFIX}".
5898
5899SDK_POSTPROCESS_COMMAND
5900 Specifies a list of functions to call once the OpenEmbedded build
5901 system creates the SDK. You can specify functions separated by
5902 semicolons: SDK_POSTPROCESS_COMMAND += "function; ... "
5903
5904 If you need to pass an SDK path to a command within a function, you
5905 can use ``${SDK_DIR}``, which points to the parent directory used by
5906 the OpenEmbedded build system when creating SDK output. See the
5907 ```SDK_DIR`` <#var-SDK_DIR>`__ variable for more information.
5908
5909SDK_PREFIX
5910 The toolchain binary prefix used for ``nativesdk`` recipes. The
5911 OpenEmbedded build system uses the ``SDK_PREFIX`` value to set the
5912 ```TARGET_PREFIX`` <#var-TARGET_PREFIX>`__ when building
5913 ``nativesdk`` recipes. The default value is "${SDK_SYS}-".
5914
5915SDK_RECRDEP_TASKS
5916 A list of shared state tasks added to the extensible SDK. By default,
5917 the following tasks are added: do_populate_lic do_package_qa
5918 do_populate_sysroot do_deploy Despite the default value of "" for the
5919 ``SDK_RECRDEP_TASKS`` variable, the above four tasks are always added
5920 to the SDK. To specify tasks beyond these four, you need to use the
5921 ``SDK_RECRDEP_TASKS`` variable (e.g. you are defining additional
5922 tasks that are needed in order to build
5923 ```SDK_TARGETS`` <#var-SDK_TARGETS>`__).
5924
5925SDK_SYS
5926 Specifies the system, including the architecture and the operating
5927 system, for which the SDK will be built.
5928
5929 The OpenEmbedded build system automatically sets this variable based
5930 on ```SDK_ARCH`` <#var-SDK_ARCH>`__,
5931 ```SDK_VENDOR`` <#var-SDK_VENDOR>`__, and
5932 ```SDK_OS`` <#var-SDK_OS>`__. You do not need to set the ``SDK_SYS``
5933 variable yourself.
5934
5935SDK_TARGET_MANIFEST
5936 The manifest file for the target part of the SDK. This file lists all
5937 the installed packages that make up the target part of the SDK. The
5938 file contains package information on a line-per-package basis as
5939 follows: packagename packagearch version
5940
5941 The ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class
5942 defines the manifest file as follows: SDK_TARGET_MANIFEST =
5943 "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.target.manifest" The location
5944 is derived using the ```SDK_DEPLOY`` <#var-SDK_DEPLOY>`__ and
5945 ```TOOLCHAIN_OUTPUTNAME`` <#var-TOOLCHAIN_OUTPUTNAME>`__ variables.
5946
5947SDK_TARGETS
5948 A list of targets to install from shared state as part of the
5949 standard or extensible SDK installation. The default value is "${PN}"
5950 (i.e. the image from which the SDK is built).
5951
5952 The ``SDK_TARGETS`` variable is an internal variable and typically
5953 would not be changed.
5954
5955SDK_TITLE
5956 The title to be printed when running the SDK installer. By default,
5957 this title is based on the ```DISTRO_NAME`` <#var-DISTRO_NAME>`__ or
5958 ```DISTRO`` <#var-DISTRO>`__ variable and is set in the
5959 ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class as
5960 follows: SDK_TITLE ??= "${@d.getVar('DISTRO_NAME') or
5961 d.getVar('DISTRO')} SDK" For the default distribution "poky",
5962 ``SDK_TITLE`` is set to "Poky (Yocto Project Reference Distro)".
5963
5964 For information on how to change this default title, see the
5965 "`Changing the Extensible SDK Installer
5966 Title <&YOCTO_DOCS_SDK_URL;#sdk-changing-the-sdk-installer-title>`__"
5967 section in the Yocto Project Application Development and the
5968 Extensible Software Development Kit (eSDK) manual.
5969
5970SDK_UPDATE_URL
5971 An optional URL for an update server for the extensible SDK. If set,
5972 the value is used as the default update server when running
5973 ``devtool sdk-update`` within the extensible SDK.
5974
5975SDK_VENDOR
5976 Specifies the name of the SDK vendor.
5977
5978SDK_VERSION
5979 Specifies the version of the SDK. The distribution configuration file
5980 (e.g. ``/meta-poky/conf/distro/poky.conf``) defines the
5981 ``SDK_VERSION`` as follows: SDK_VERSION =
5982 "${@d.getVar('DISTRO_VERSION').replace('snapshot-${DATE}','snapshot')}"
5983
5984 For additional information, see the
5985 ```DISTRO_VERSION`` <#var-DISTRO_VERSION>`__ and
5986 ```DATE`` <#var-DATE>`__ variables.
5987
5988SDKEXTPATH
5989 The default installation directory for the Extensible SDK. By
5990 default, this directory is based on the ```DISTRO`` <#var-DISTRO>`__
5991 variable and is set in the
5992 ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class as
5993 follows: SDKEXTPATH ??= "~/${@d.getVar('DISTRO')}_sdk" For the
5994 default distribution "poky", the ``SDKEXTPATH`` is set to "poky_sdk".
5995
5996 For information on how to change this default directory, see the
5997 "`Changing the Default SDK Installation
5998 Directory <&YOCTO_DOCS_SDK_URL;#sdk-changing-the-default-sdk-installation-directory>`__"
5999 section in the Yocto Project Application Development and the
6000 Extensible Software Development Kit (eSDK) manual.
6001
6002SDKIMAGE_FEATURES
6003 Equivalent to ``IMAGE_FEATURES``. However, this variable applies to
6004 the SDK generated from an image using the following command: $
6005 bitbake -c populate_sdk imagename
6006
6007SDKMACHINE
6008 The machine for which the SDK is built. In other words, the SDK is
6009 built such that it runs on the target you specify with the
6010 ``SDKMACHINE`` value. The value points to a corresponding ``.conf``
6011 file under ``conf/machine-sdk/``.
6012
6013 You can use "i686" and "x86_64" as possible values for this variable.
6014 The variable defaults to "i686" and is set in the local.conf file in
6015 the Build Directory. SDKMACHINE ?= "i686"
6016
6017 .. note::
6018
6019 You cannot set the
6020 SDKMACHINE
6021 variable in your distribution configuration file. If you do, the
6022 configuration will not take affect.
6023
6024SDKPATH
6025 Defines the path offered to the user for installation of the SDK that
6026 is generated by the OpenEmbedded build system. The path appears as
6027 the default location for installing the SDK when you run the SDK's
6028 installation script. You can override the offered path when you run
6029 the script.
6030
6031SDKTARGETSYSROOT
6032 The full path to the sysroot used for cross-compilation within an SDK
6033 as it will be when installed into the default
6034 ```SDKPATH`` <#var-SDKPATH>`__.
6035
6036SECTION
6037 The section in which packages should be categorized. Package
6038 management utilities can make use of this variable.
6039
6040SELECTED_OPTIMIZATION
6041 Specifies the optimization flags passed to the C compiler when
6042 building for the target. The flags are passed through the default
6043 value of the ```TARGET_CFLAGS`` <#var-TARGET_CFLAGS>`__ variable.
6044
6045 The ``SELECTED_OPTIMIZATION`` variable takes the value of
6046 ``FULL_OPTIMIZATION`` unless ``DEBUG_BUILD`` = "1". If that is the
6047 case, the value of ``DEBUG_OPTIMIZATION`` is used.
6048
6049SERIAL_CONSOLE
6050 Defines a serial console (TTY) to enable using
6051 `getty <https://en.wikipedia.org/wiki/Getty_(Unix)>`__. Provide a
6052 value that specifies the baud rate followed by the TTY device name
6053 separated by a space. You cannot specify more than one TTY device:
6054 SERIAL_CONSOLE = "115200 ttyS0"
6055
6056 .. note::
6057
6058 The
6059 SERIAL_CONSOLE
6060 variable is deprecated. Please use the
6061 SERIAL_CONSOLES
6062 variable.
6063
6064SERIAL_CONSOLES
6065 Defines a serial console (TTY) to enable using
6066 `getty <https://en.wikipedia.org/wiki/Getty_(Unix)>`__. Provide a
6067 value that specifies the baud rate followed by the TTY device name
6068 separated by a semicolon. Use spaces to separate multiple devices:
6069 SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1"
6070
6071SERIAL_CONSOLES_CHECK
6072 Specifies serial consoles, which must be listed in
6073 ```SERIAL_CONSOLES`` <#var-SERIAL_CONSOLES>`__, to check against
6074 ``/proc/console`` before enabling them using getty. This variable
6075 allows aliasing in the format: <device>:<alias>. If a device was
6076 listed as "sclp_line0" in ``/dev/`` and "ttyS0" was listed in
6077 ``/proc/console``, you would do the following: SERIAL_CONSOLES_CHECK
6078 = "slcp_line0:ttyS0" This variable is currently only supported with
6079 SysVinit (i.e. not with systemd).
6080
6081SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS
6082 A list of recipe dependencies that should not be used to determine
6083 signatures of tasks from one recipe when they depend on tasks from
6084 another recipe. For example: SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS +=
6085 "intone->mplayer2"
6086
6087 In the previous example, ``intone`` depends on ``mplayer2``.
6088
6089 You can use the special token ``"*"`` on the left-hand side of the
6090 dependency to match all recipes except the one on the right-hand
6091 side. Here is an example: SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS +=
6092 "*->quilt-native"
6093
6094 In the previous example, all recipes except ``quilt-native`` ignore
6095 task signatures from the ``quilt-native`` recipe when determining
6096 their task signatures.
6097
6098 Use of this variable is one mechanism to remove dependencies that
6099 affect task signatures and thus force rebuilds when a recipe changes.
6100
6101 .. note::
6102
6103 If you add an inappropriate dependency for a recipe relationship,
6104 the software might break during runtime if the interface of the
6105 second recipe was changed after the first recipe had been built.
6106
6107SIGGEN_EXCLUDERECIPES_ABISAFE
6108 A list of recipes that are completely stable and will never change.
6109 The ABI for the recipes in the list are presented by output from the
6110 tasks run to build the recipe. Use of this variable is one way to
6111 remove dependencies from one recipe on another that affect task
6112 signatures and thus force rebuilds when the recipe changes.
6113
6114 .. note::
6115
6116 If you add an inappropriate variable to this list, the software
6117 might break at runtime if the interface of the recipe was changed
6118 after the other had been built.
6119
6120SITEINFO_BITS
6121 Specifies the number of bits for the target system CPU. The value
6122 should be either "32" or "64".
6123
6124SITEINFO_ENDIANNESS
6125 Specifies the endian byte order of the target system. The value
6126 should be either "le" for little-endian or "be" for big-endian.
6127
6128SKIP_FILEDEPS
6129 Enables removal of all files from the "Provides" section of an RPM
6130 package. Removal of these files is required for packages containing
6131 prebuilt binaries and libraries such as ``libstdc++`` and ``glibc``.
6132
6133 To enable file removal, set the variable to "1" in your
6134 ``conf/local.conf`` configuration file in your: `Build
6135 Directory <#build-directory>`__. SKIP_FILEDEPS = "1"
6136
6137SOC_FAMILY
6138 Groups together machines based upon the same family of SOC (System On
6139 Chip). You typically set this variable in a common ``.inc`` file that
6140 you include in the configuration files of all the machines.
6141
6142 .. note::
6143
6144 You must include
6145 conf/machine/include/soc-family.inc
6146 for this variable to appear in
6147 MACHINEOVERRIDES
6148 .
6149
6150SOLIBS
6151 Defines the suffix for shared libraries used on the target platform.
6152 By default, this suffix is ".so.*" for all Linux-based systems and is
6153 defined in the ``meta/conf/bitbake.conf`` configuration file.
6154
6155 You will see this variable referenced in the default values of
6156 ``FILES_${PN}``.
6157
6158SOLIBSDEV
6159 Defines the suffix for the development symbolic link (symlink) for
6160 shared libraries on the target platform. By default, this suffix is
6161 ".so" for Linux-based systems and is defined in the
6162 ``meta/conf/bitbake.conf`` configuration file.
6163
6164 You will see this variable referenced in the default values of
6165 ``FILES_${PN}-dev``.
6166
6167SOURCE_MIRROR_FETCH
6168 When you are fetching files to create a mirror of sources (i.e.
6169 creating a source mirror), setting ``SOURCE_MIRROR_FETCH`` to "1" in
6170 your ``local.conf`` configuration file ensures the source for all
6171 recipes are fetched regardless of whether or not a recipe is
6172 compatible with the configuration. A recipe is considered
6173 incompatible with the currently configured machine when either or
6174 both the ```COMPATIBLE_MACHINE`` <#var-COMPATIBLE_MACHINE>`__
6175 variable and ```COMPATIBLE_HOST`` <#var-COMPATIBLE_HOST>`__ variables
6176 specify compatibility with a machine other than that of the current
6177 machine or host.
6178
6179 .. note::
6180
6181 Do not set the
6182 SOURCE_MIRROR_FETCH
6183 variable unless you are creating a source mirror. In other words,
6184 do not set the variable during a normal build.
6185
6186SOURCE_MIRROR_URL
6187 Defines your own ```PREMIRRORS`` <#var-PREMIRRORS>`__ from which to
6188 first fetch source before attempting to fetch from the upstream
6189 specified in ```SRC_URI`` <#var-SRC_URI>`__.
6190
6191 To use this variable, you must globally inherit the
6192 ```own-mirrors`` <#ref-classes-own-mirrors>`__ class and then provide
6193 the URL to your mirrors. Here is the general syntax: INHERIT +=
6194 "own-mirrors" SOURCE_MIRROR_URL =
6195 "http://example.com/my_source_mirror"
6196
6197 .. note::
6198
6199 You can specify only a single URL in
6200 SOURCE_MIRROR_URL
6201 .
6202
6203SPDXLICENSEMAP
6204 Maps commonly used license names to their SPDX counterparts found in
6205 ``meta/files/common-licenses/``. For the default ``SPDXLICENSEMAP``
6206 mappings, see the ``meta/conf/licenses.conf`` file.
6207
6208 For additional information, see the ```LICENSE`` <#var-LICENSE>`__
6209 variable.
6210
6211SPECIAL_PKGSUFFIX
6212 A list of prefixes for ```PN`` <#var-PN>`__ used by the OpenEmbedded
6213 build system to create variants of recipes or packages. The list
6214 specifies the prefixes to strip off during certain circumstances such
6215 as the generation of the ```BPN`` <#var-BPN>`__ variable.
6216
6217SPL_BINARY
6218 The file type for the Secondary Program Loader (SPL). Some devices
6219 use an SPL from which to boot (e.g. the BeagleBone development
6220 board). For such cases, you can declare the file type of the SPL
6221 binary in the ``u-boot.inc`` include file, which is used in the
6222 U-Boot recipe.
6223
6224 The SPL file type is set to "null" by default in the ``u-boot.inc``
6225 file as follows: # Some versions of u-boot build an SPL (Second
6226 Program Loader) image that # should be packaged along with the u-boot
6227 binary as well as placed in the # deploy directory. For those
6228 versions they can set the following variables # to allow packaging
6229 the SPL. SPL_BINARY ?= "" SPL_BINARYNAME ?=
6230 "${@os.path.basename(d.getVar("SPL_BINARY"))}" SPL_IMAGE ?=
6231 "${SPL_BINARYNAME}-${MACHINE}-${PV}-${PR}" SPL_SYMLINK ?=
6232 "${SPL_BINARYNAME}-${MACHINE}" The ``SPL_BINARY`` variable helps form
6233 various ``SPL_*`` variables used by the OpenEmbedded build system.
6234
6235 See the BeagleBone machine configuration example in the "`Creating a
6236 new BSP Layer Using the ``bitbake-layers``
6237 Script <&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script>`__"
6238 section in the Yocto Project Board Support Package Developer's Guide
6239 for additional information.
6240
6241SRC_URI
6242 The list of source files - local or remote. This variable tells the
6243 OpenEmbedded build system which bits to pull in for the build and how
6244 to pull them in. For example, if the recipe or append file only needs
6245 to fetch a tarball from the Internet, the recipe or append file uses
6246 a single ``SRC_URI`` entry. On the other hand, if the recipe or
6247 append file needs to fetch a tarball, apply two patches, and include
6248 a custom file, the recipe or append file would include four instances
6249 of the variable.
6250
6251 The following list explains the available URI protocols. URI
6252 protocols are highly dependent on particular BitBake Fetcher
6253 submodules. Depending on the fetcher BitBake uses, various URL
6254 parameters are employed. For specifics on the supported Fetchers, see
6255 the "`Fetchers <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__" section in the
6256 BitBake User Manual.
6257
6258 - *``file://`` -* Fetches files, which are usually files shipped
6259 with the `Metadata <#metadata>`__, from the local machine (e.g.
6260 `patch <&YOCTO_DOCS_OM_URL;#patching-dev-environment>`__ files).
6261 The path is relative to the ```FILESPATH`` <#var-FILESPATH>`__
6262 variable. Thus, the build system searches, in order, from the
6263 following directories, which are assumed to be a subdirectories of
6264 the directory in which the recipe file (``.bb``) or append file
6265 (``.bbappend``) resides:
6266
6267 - *``${BPN}`` -* The base recipe name without any special suffix
6268 or version numbers.
6269
6270 - *``${BP}`` -* ``${BPN}-${PV}``. The base recipe name and
6271 version but without any special package name suffix.
6272
6273 - *files -* Files within a directory, which is named ``files``
6274 and is also alongside the recipe or append file.
6275
6276 .. note::
6277
6278 If you want the build system to pick up files specified through
6279 a
6280 SRC_URI
6281 statement from your append file, you need to be sure to extend
6282 the
6283 FILESPATH
6284 variable by also using the
6285 FILESEXTRAPATHS
6286 variable from within your append file.
6287
6288 - *``bzr://`` -* Fetches files from a Bazaar revision control
6289 repository.
6290
6291 - *``git://`` -* Fetches files from a Git revision control
6292 repository.
6293
6294 - *``osc://`` -* Fetches files from an OSC (OpenSUSE Build service)
6295 revision control repository.
6296
6297 - *``repo://`` -* Fetches files from a repo (Git) repository.
6298
6299 - *``ccrc://`` -* Fetches files from a ClearCase repository.
6300
6301 - *``http://`` -* Fetches files from the Internet using ``http``.
6302
6303 - *``https://`` -* Fetches files from the Internet using ``https``.
6304
6305 - *``ftp://`` -* Fetches files from the Internet using ``ftp``.
6306
6307 - *``cvs://`` -* Fetches files from a CVS revision control
6308 repository.
6309
6310 - *``hg://`` -* Fetches files from a Mercurial (``hg``) revision
6311 control repository.
6312
6313 - *``p4://`` -* Fetches files from a Perforce (``p4``) revision
6314 control repository.
6315
6316 - *``ssh://`` -* Fetches files from a secure shell.
6317
6318 - *``svn://`` -* Fetches files from a Subversion (``svn``) revision
6319 control repository.
6320
6321 - *``npm://`` -* Fetches JavaScript modules from a registry.
6322
6323 Standard and recipe-specific options for ``SRC_URI`` exist. Here are
6324 standard options:
6325
6326 - *``apply`` -* Whether to apply the patch or not. The default
6327 action is to apply the patch.
6328
6329 - *``striplevel`` -* Which striplevel to use when applying the
6330 patch. The default level is 1.
6331
6332 - *``patchdir`` -* Specifies the directory in which the patch should
6333 be applied. The default is ``${``\ ```S`` <#var-S>`__\ ``}``.
6334
6335 Here are options specific to recipes building code from a revision
6336 control system:
6337
6338 - *``mindate`` -* Apply the patch only if
6339 ```SRCDATE`` <#var-SRCDATE>`__ is equal to or greater than
6340 ``mindate``.
6341
6342 - *``maxdate`` -* Apply the patch only if ``SRCDATE`` is not later
6343 than ``maxdate``.
6344
6345 - *``minrev`` -* Apply the patch only if ``SRCREV`` is equal to or
6346 greater than ``minrev``.
6347
6348 - *``maxrev`` -* Apply the patch only if ``SRCREV`` is not later
6349 than ``maxrev``.
6350
6351 - *``rev`` -* Apply the patch only if ``SRCREV`` is equal to
6352 ``rev``.
6353
6354 - *``notrev`` -* Apply the patch only if ``SRCREV`` is not equal to
6355 ``rev``.
6356
6357 Here are some additional options worth mentioning:
6358
6359 - *``unpack`` -* Controls whether or not to unpack the file if it is
6360 an archive. The default action is to unpack the file.
6361
6362 - *``destsuffix`` -* Places the file (or extracts its contents) into
6363 the specified subdirectory of ```WORKDIR`` <#var-WORKDIR>`__ when
6364 the Git fetcher is used.
6365
6366 - *``subdir`` -* Places the file (or extracts its contents) into the
6367 specified subdirectory of ``WORKDIR`` when the local (``file://``)
6368 fetcher is used.
6369
6370 - *``localdir`` -* Places the file (or extracts its contents) into
6371 the specified subdirectory of ``WORKDIR`` when the CVS fetcher is
6372 used.
6373
6374 - *``subpath`` -* Limits the checkout to a specific subpath of the
6375 tree when using the Git fetcher is used.
6376
6377 - *``name`` -* Specifies a name to be used for association with
6378 ``SRC_URI`` checksums when you have more than one file specified
6379 in ``SRC_URI``.
6380
6381 - *``downloadfilename`` -* Specifies the filename used when storing
6382 the downloaded file.
6383
6384SRC_URI_OVERRIDES_PACKAGE_ARCH
6385 By default, the OpenEmbedded build system automatically detects
6386 whether ``SRC_URI`` contains files that are machine-specific. If so,
6387 the build system automatically changes ``PACKAGE_ARCH``. Setting this
6388 variable to "0" disables this behavior.
6389
6390SRCDATE
6391 The date of the source code used to build the package. This variable
6392 applies only if the source was fetched from a Source Code Manager
6393 (SCM).
6394
6395SRCPV
6396 Returns the version string of the current package. This string is
6397 used to help define the value of ```PV`` <#var-PV>`__.
6398
6399 The ``SRCPV`` variable is defined in the ``meta/conf/bitbake.conf``
6400 configuration file in the `Source Directory <#source-directory>`__ as
6401 follows: SRCPV = "${@bb.fetch2.get_srcrev(d)}"
6402
6403 Recipes that need to define ``PV`` do so with the help of the
6404 ``SRCPV``. For example, the ``ofono`` recipe (``ofono_git.bb``)
6405 located in ``meta/recipes-connectivity`` in the Source Directory
6406 defines ``PV`` as follows: PV = "0.12-git${SRCPV}"
6407
6408SRCREV
6409 The revision of the source code used to build the package. This
6410 variable applies to Subversion, Git, Mercurial, and Bazaar only. Note
6411 that if you want to build a fixed revision and you want to avoid
6412 performing a query on the remote repository every time BitBake parses
6413 your recipe, you should specify a ``SRCREV`` that is a full revision
6414 identifier and not just a tag.
6415
6416 .. note::
6417
6418 For information on limitations when inheriting the latest revision
6419 of software using
6420 SRCREV
6421 , see the
6422 AUTOREV
6423 variable description and the "
6424 Automatically Incrementing a Binary Package Revision Number
6425 " section, which is in the Yocto Project Development Tasks Manual.
6426
6427SSTATE_DIR
6428 The directory for the shared state cache.
6429
6430SSTATE_MIRROR_ALLOW_NETWORK
6431 If set to "1", allows fetches from mirrors that are specified in
6432 ```SSTATE_MIRRORS`` <#var-SSTATE_MIRRORS>`__ to work even when
6433 fetching from the network is disabled by setting ``BB_NO_NETWORK`` to
6434 "1". Using the ``SSTATE_MIRROR_ALLOW_NETWORK`` variable is useful if
6435 you have set ``SSTATE_MIRRORS`` to point to an internal server for
6436 your shared state cache, but you want to disable any other fetching
6437 from the network.
6438
6439SSTATE_MIRRORS
6440 Configures the OpenEmbedded build system to search other mirror
6441 locations for prebuilt cache data objects before building out the
6442 data. This variable works like fetcher ```MIRRORS`` <#var-MIRRORS>`__
6443 and ```PREMIRRORS`` <#var-PREMIRRORS>`__ and points to the cache
6444 locations to check for the shared state (sstate) objects.
6445
6446 You can specify a filesystem directory or a remote URL such as HTTP
6447 or FTP. The locations you specify need to contain the shared state
6448 cache (sstate-cache) results from previous builds. The sstate-cache
6449 you point to can also be from builds on other machines.
6450
6451 When pointing to sstate build artifacts on another machine that uses
6452 a different GCC version for native builds, you must configure
6453 ``SSTATE_MIRRORS`` with a regular expression that maps local search
6454 paths to server paths. The paths need to take into account
6455 ```NATIVELSBSTRING`` <#var-NATIVELSBSTRING>`__ set by the
6456 ```uninative`` <#ref-classes-uninative>`__ class. For example, the
6457 following maps the local search path ``universal-4.9`` to the
6458 server-provided path server_url_sstate_path: SSTATE_MIRRORS ?=
6459 file://universal-4.9/(.*)
6460 http://server_url_sstate_path/universal-4.8/\1 \\n
6461
6462 If a mirror uses the same structure as
6463 ```SSTATE_DIR`` <#var-SSTATE_DIR>`__, you need to add "PATH" at the
6464 end as shown in the examples below. The build system substitutes the
6465 correct path within the directory structure. SSTATE_MIRRORS ?= "\\
6466 file://.\*
6467 http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \\n \\
6468 file://.\* file:///some-local-dir/sstate/PATH"
6469
6470SSTATE_SCAN_FILES
6471 Controls the list of files the OpenEmbedded build system scans for
6472 hardcoded installation paths. The variable uses a space-separated
6473 list of filenames (not paths) with standard wildcard characters
6474 allowed.
6475
6476 During a build, the OpenEmbedded build system creates a shared state
6477 (sstate) object during the first stage of preparing the sysroots.
6478 That object is scanned for hardcoded paths for original installation
6479 locations. The list of files that are scanned for paths is controlled
6480 by the ``SSTATE_SCAN_FILES`` variable. Typically, recipes add files
6481 they want to be scanned to the value of ``SSTATE_SCAN_FILES`` rather
6482 than the variable being comprehensively set. The
6483 ```sstate`` <#ref-classes-sstate>`__ class specifies the default list
6484 of files.
6485
6486 For details on the process, see the
6487 ```staging`` <#ref-classes-staging>`__ class.
6488
6489STAGING_BASE_LIBDIR_NATIVE
6490 Specifies the path to the ``/lib`` subdirectory of the sysroot
6491 directory for the build host.
6492
6493STAGING_BASELIBDIR
6494 Specifies the path to the ``/lib`` subdirectory of the sysroot
6495 directory for the target for which the current recipe is being built
6496 (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6497
6498STAGING_BINDIR
6499 Specifies the path to the ``/usr/bin`` subdirectory of the sysroot
6500 directory for the target for which the current recipe is being built
6501 (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6502
6503STAGING_BINDIR_CROSS
6504 Specifies the path to the directory containing binary configuration
6505 scripts. These scripts provide configuration information for other
6506 software that wants to make use of libraries or include files
6507 provided by the software associated with the script.
6508
6509 .. note::
6510
6511 This style of build configuration has been largely replaced by
6512 pkg-config
6513 . Consequently, if
6514 pkg-config
6515 is supported by the library to which you are linking, it is
6516 recommended you use
6517 pkg-config
6518 instead of a provided configuration script.
6519
6520STAGING_BINDIR_NATIVE
6521 Specifies the path to the ``/usr/bin`` subdirectory of the sysroot
6522 directory for the build host.
6523
6524STAGING_DATADIR
6525 Specifies the path to the ``/usr/share`` subdirectory of the sysroot
6526 directory for the target for which the current recipe is being built
6527 (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6528
6529STAGING_DATADIR_NATIVE
6530 Specifies the path to the ``/usr/share`` subdirectory of the sysroot
6531 directory for the build host.
6532
6533STAGING_DIR
6534 Helps construct the ``recipe-sysroots`` directory, which is used
6535 during packaging.
6536
6537 For information on how staging for recipe-specific sysroots occurs,
6538 see the ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__
6539 task, the "`Sharing Files Between
6540 Recipes <&YOCTO_DOCS_DEV_URL;#new-sharing-files-between-recipes>`__"
6541 section in the Yocto Project Development Tasks Manual, the
6542 "`Configuration, Compilation, and
6543 Staging <&YOCTO_DOCS_OM_URL;#configuration-compilation-and-staging-dev-environment>`__"
6544 section in the Yocto Project Overview and Concepts Manual, and the
6545 ```SYSROOT_DIRS`` <#var-SYSROOT_DIRS>`__ variable.
6546
6547 .. note::
6548
6549 Recipes should never write files directly under the
6550 STAGING_DIR
6551 directory because the OpenEmbedded build system manages the
6552 directory automatically. Instead, files should be installed to
6553 ${
6554 D
6555 }
6556 within your recipe's
6557 do_install
6558 task and then the OpenEmbedded build system will stage a subset of
6559 those files into the sysroot.
6560
6561STAGING_DIR_HOST
6562 Specifies the path to the sysroot directory for the system on which
6563 the component is built to run (the system that hosts the component).
6564 For most recipes, this sysroot is the one in which that recipe's
6565 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task copies
6566 files. Exceptions include ``-native`` recipes, where the
6567 ``do_populate_sysroot`` task instead uses
6568 ```STAGING_DIR_NATIVE`` <#var-STAGING_DIR_NATIVE>`__. Depending on
6569 the type of recipe and the build target, ``STAGING_DIR_HOST`` can
6570 have the following values:
6571
6572 - For recipes building for the target machine, the value is
6573 "${`STAGING_DIR <#var-STAGING_DIR>`__}/${`MACHINE <#var-MACHINE>`__}".
6574
6575 - For native recipes building for the build host, the value is empty
6576 given the assumption that when building for the build host, the
6577 build host's own directories should be used.
6578
6579 .. note::
6580
6581 ``-native`` recipes are not installed into host paths like such
6582 as ``/usr``. Rather, these recipes are installed into
6583 ``STAGING_DIR_NATIVE``. When compiling ``-native`` recipes,
6584 standard build environment variables such as
6585 ```CPPFLAGS`` <#var-CPPFLAGS>`__ and
6586 ```CFLAGS`` <#var-CFLAGS>`__ are set up so that both host paths
6587 and ``STAGING_DIR_NATIVE`` are searched for libraries and
6588 headers using, for example, GCC's ``-isystem`` option.
6589
6590 Thus, the emphasis is that the ``STAGING_DIR*`` variables
6591 should be viewed as input variables by tasks such as
6592 ```do_configure`` <#ref-tasks-configure>`__,
6593 ```do_compile`` <#ref-tasks-compile>`__, and
6594 ```do_install`` <#ref-tasks-install>`__. Having the real system
6595 root correspond to ``STAGING_DIR_HOST`` makes conceptual sense
6596 for ``-native`` recipes, as they make use of host headers and
6597 libraries.
6598
6599STAGING_DIR_NATIVE
6600 Specifies the path to the sysroot directory used when building
6601 components that run on the build host itself.
6602
6603STAGING_DIR_TARGET
6604 Specifies the path to the sysroot used for the system for which the
6605 component generates code. For components that do not generate code,
6606 which is the majority, ``STAGING_DIR_TARGET`` is set to match
6607 ```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__.
6608
6609 Some recipes build binaries that can run on the target system but
6610 those binaries in turn generate code for another different system
6611 (e.g. cross-canadian recipes). Using terminology from GNU, the
6612 primary system is referred to as the "HOST" and the secondary, or
6613 different, system is referred to as the "TARGET". Thus, the binaries
6614 run on the "HOST" system and generate binaries for the "TARGET"
6615 system. The ``STAGING_DIR_HOST`` variable points to the sysroot used
6616 for the "HOST" system, while ``STAGING_DIR_TARGET`` points to the
6617 sysroot used for the "TARGET" system.
6618
6619STAGING_ETCDIR_NATIVE
6620 Specifies the path to the ``/etc`` subdirectory of the sysroot
6621 directory for the build host.
6622
6623STAGING_EXECPREFIXDIR
6624 Specifies the path to the ``/usr`` subdirectory of the sysroot
6625 directory for the target for which the current recipe is being built
6626 (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6627
6628STAGING_INCDIR
6629 Specifies the path to the ``/usr/include`` subdirectory of the
6630 sysroot directory for the target for which the current recipe being
6631 built (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6632
6633STAGING_INCDIR_NATIVE
6634 Specifies the path to the ``/usr/include`` subdirectory of the
6635 sysroot directory for the build host.
6636
6637STAGING_KERNEL_BUILDDIR
6638 Points to the directory containing the kernel build artifacts.
6639 Recipes building software that needs to access kernel build artifacts
6640 (e.g. ``systemtap-uprobes``) can look in the directory specified with
6641 the ``STAGING_KERNEL_BUILDDIR`` variable to find these artifacts
6642 after the kernel has been built.
6643
6644STAGING_KERNEL_DIR
6645 The directory with kernel headers that are required to build
6646 out-of-tree modules.
6647
6648STAGING_LIBDIR
6649 Specifies the path to the ``/usr/lib`` subdirectory of the sysroot
6650 directory for the target for which the current recipe is being built
6651 (```STAGING_DIR_HOST`` <#var-STAGING_DIR_HOST>`__).
6652
6653STAGING_LIBDIR_NATIVE
6654 Specifies the path to the ``/usr/lib`` subdirectory of the sysroot
6655 directory for the build host.
6656
6657STAMP
6658 Specifies the base path used to create recipe stamp files. The path
6659 to an actual stamp file is constructed by evaluating this string and
6660 then appending additional information. Currently, the default
6661 assignment for ``STAMP`` as set in the ``meta/conf/bitbake.conf``
6662 file is: STAMP =
6663 "${STAMPS_DIR}/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
6664
6665 For information on how BitBake uses stamp files to determine if a
6666 task should be rerun, see the "`Stamp Files and the Rerunning of
6667 Tasks <&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks>`__"
6668 section in the Yocto Project Overview and Concepts Manual.
6669
6670 See ```STAMPS_DIR`` <#var-STAMPS_DIR>`__,
6671 ```MULTIMACH_TARGET_SYS`` <#var-MULTIMACH_TARGET_SYS>`__,
6672 ```PN`` <#var-PN>`__, ```EXTENDPE`` <#var-EXTENDPE>`__,
6673 ```PV`` <#var-PV>`__, and ```PR`` <#var-PR>`__ for related variable
6674 information.
6675
6676STAMPS_DIR
6677 Specifies the base directory in which the OpenEmbedded build system
6678 places stamps. The default directory is ``${TMPDIR}/stamps``.
6679
6680STRIP
6681 The minimal command and arguments to run ``strip``, which is used to
6682 strip symbols.
6683
6684SUMMARY
6685 The short (72 characters or less) summary of the binary package for
6686 packaging systems such as ``opkg``, ``rpm``, or ``dpkg``. By default,
6687 ``SUMMARY`` is used to define the
6688 ```DESCRIPTION`` <#var-DESCRIPTION>`__ variable if ``DESCRIPTION`` is
6689 not set in the recipe.
6690
6691SVNDIR
6692 The directory in which files checked out of a Subversion system are
6693 stored.
6694
6695SYSLINUX_DEFAULT_CONSOLE
6696 Specifies the kernel boot default console. If you want to use a
6697 console other than the default, set this variable in your recipe as
6698 follows where "X" is the console number you want to use:
6699 SYSLINUX_DEFAULT_CONSOLE = "console=ttyX"
6700
6701 The ```syslinux`` <#ref-classes-syslinux>`__ class initially sets
6702 this variable to null but then checks for a value later.
6703
6704SYSLINUX_OPTS
6705 Lists additional options to add to the syslinux file. You need to set
6706 this variable in your recipe. If you want to list multiple options,
6707 separate the options with a semicolon character (``;``).
6708
6709 The ```syslinux`` <#ref-classes-syslinux>`__ class uses this variable
6710 to create a set of options.
6711
6712SYSLINUX_SERIAL
6713 Specifies the alternate serial port or turns it off. To turn off
6714 serial, set this variable to an empty string in your recipe. The
6715 variable's default value is set in the
6716 ```syslinux`` <#ref-classes-syslinux>`__ class as follows:
6717 SYSLINUX_SERIAL ?= "0 115200"
6718
6719 The class checks for and uses the variable as needed.
6720
6721SYSLINUX_SPLASH
6722 An ``.LSS`` file used as the background for the VGA boot menu when
6723 you use the boot menu. You need to set this variable in your recipe.
6724
6725 The ```syslinux`` <#ref-classes-syslinux>`__ class checks for this
6726 variable and if found, the OpenEmbedded build system installs the
6727 splash screen.
6728
6729SYSLINUX_SERIAL_TTY
6730 Specifies the alternate console=tty... kernel boot argument. The
6731 variable's default value is set in the
6732 ```syslinux`` <#ref-classes-syslinux>`__ class as follows:
6733 SYSLINUX_SERIAL_TTY ?= "console=ttyS0,115200"
6734
6735 The class checks for and uses the variable as needed.
6736
6737SYSROOT_DESTDIR
6738 Points to the temporary directory under the work directory (default
6739 "``${``\ ```WORKDIR`` <#var-WORKDIR>`__\ ``}/sysroot-destdir``")
6740 where the files populated into the sysroot are assembled during the
6741 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task.
6742
6743SYSROOT_DIRS
6744 Directories that are staged into the sysroot by the
6745 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task. By
6746 default, the following directories are staged: SYSROOT_DIRS = " \\
6747 ${includedir} \\ ${libdir} \\ ${base_libdir} \\
6748 ${nonarch_base_libdir} \\ ${datadir} \\ "
6749
6750SYSROOT_DIRS_BLACKLIST
6751 Directories that are not staged into the sysroot by the
6752 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task. You
6753 can use this variable to exclude certain subdirectories of
6754 directories listed in ```SYSROOT_DIRS`` <#var-SYSROOT_DIRS>`__ from
6755 staging. By default, the following directories are not staged:
6756 SYSROOT_DIRS_BLACKLIST = " \\ ${mandir} \\ ${docdir} \\ ${infodir} \\
6757 ${datadir}/locale \\ ${datadir}/applications \\ ${datadir}/fonts \\
6758 ${datadir}/pixmaps \\ "
6759
6760SYSROOT_DIRS_NATIVE
6761 Extra directories staged into the sysroot by the
6762 ```do_populate_sysroot`` <#ref-tasks-populate_sysroot>`__ task for
6763 ``-native`` recipes, in addition to those specified in
6764 ```SYSROOT_DIRS`` <#var-SYSROOT_DIRS>`__. By default, the following
6765 extra directories are staged: SYSROOT_DIRS_NATIVE = " \\ ${bindir} \\
6766 ${sbindir} \\ ${base_bindir} \\ ${base_sbindir} \\ ${libexecdir} \\
6767 ${sysconfdir} \\ ${localstatedir} \\ "
6768
6769 .. note::
6770
6771 Programs built by
6772 -native
6773 recipes run directly from the sysroot (
6774 STAGING_DIR_NATIVE
6775 ), which is why additional directories containing program
6776 executables and supporting files need to be staged.
6777
6778SYSROOT_PREPROCESS_FUNCS
6779 A list of functions to execute after files are staged into the
6780 sysroot. These functions are usually used to apply additional
6781 processing on the staged files, or to stage additional files.
6782
6783SYSTEMD_AUTO_ENABLE
6784 When inheriting the ```systemd`` <#ref-classes-systemd>`__ class,
6785 this variable specifies whether the specified service in
6786 ```SYSTEMD_SERVICE`` <#var-SYSTEMD_SERVICE>`__ should start
6787 automatically or not. By default, the service is enabled to
6788 automatically start at boot time. The default setting is in the
6789 ```systemd`` <#ref-classes-systemd>`__ class as follows:
6790 SYSTEMD_AUTO_ENABLE ??= "enable"
6791
6792 You can disable the service by setting the variable to "disable".
6793
6794SYSTEMD_BOOT_CFG
6795 When ```EFI_PROVIDER`` <#var-EFI_PROVIDER>`__ is set to
6796 "systemd-boot", the ``SYSTEMD_BOOT_CFG`` variable specifies the
6797 configuration file that should be used. By default, the
6798 ```systemd-boot`` <#ref-classes-systemd-boot>`__ class sets the
6799 ``SYSTEMD_BOOT_CFG`` as follows: SYSTEMD_BOOT_CFG ?=
6800 "${`S <#var-S>`__}/loader.conf"
6801
6802 For information on Systemd-boot, see the `Systemd-boot
6803 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
6804
6805SYSTEMD_BOOT_ENTRIES
6806 When ```EFI_PROVIDER`` <#var-EFI_PROVIDER>`__ is set to
6807 "systemd-boot", the ``SYSTEMD_BOOT_ENTRIES`` variable specifies a
6808 list of entry files (``*.conf``) to install that contain one boot
6809 entry per file. By default, the
6810 ```systemd-boot`` <#ref-classes-systemd-boot>`__ class sets the
6811 ``SYSTEMD_BOOT_ENTRIES`` as follows: SYSTEMD_BOOT_ENTRIES ?= ""
6812
6813 For information on Systemd-boot, see the `Systemd-boot
6814 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
6815
6816SYSTEMD_BOOT_TIMEOUT
6817 When ```EFI_PROVIDER`` <#var-EFI_PROVIDER>`__ is set to
6818 "systemd-boot", the ``SYSTEMD_BOOT_TIMEOUT`` variable specifies the
6819 boot menu timeout in seconds. By default, the
6820 ```systemd-boot`` <#ref-classes-systemd-boot>`__ class sets the
6821 ``SYSTEMD_BOOT_TIMEOUT`` as follows: SYSTEMD_BOOT_TIMEOUT ?= "10"
6822
6823 For information on Systemd-boot, see the `Systemd-boot
6824 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
6825
6826SYSTEMD_PACKAGES
6827 When inheriting the ```systemd`` <#ref-classes-systemd>`__ class,
6828 this variable locates the systemd unit files when they are not found
6829 in the main recipe's package. By default, the ``SYSTEMD_PACKAGES``
6830 variable is set such that the systemd unit files are assumed to
6831 reside in the recipes main package: SYSTEMD_PACKAGES ?= "${PN}"
6832
6833 If these unit files are not in this recipe's main package, you need
6834 to use ``SYSTEMD_PACKAGES`` to list the package or packages in which
6835 the build system can find the systemd unit files.
6836
6837SYSTEMD_SERVICE
6838 When inheriting the ```systemd`` <#ref-classes-systemd>`__ class,
6839 this variable specifies the systemd service name for a package.
6840
6841 When you specify this file in your recipe, use a package name
6842 override to indicate the package to which the value applies. Here is
6843 an example from the connman recipe: SYSTEMD_SERVICE_${PN} =
6844 "connman.service"
6845
6846SYSVINIT_ENABLED_GETTYS
6847 When using
6848 `SysVinit <&YOCTO_DOCS_DEV_URL;#new-recipe-enabling-system-services>`__,
6849 specifies a space-separated list of the virtual terminals that should
6850 run a `getty <http://en.wikipedia.org/wiki/Getty_%28Unix%29>`__
6851 (allowing login), assuming ```USE_VT`` <#var-USE_VT>`__ is not set to
6852 "0".
6853
6854 The default value for ``SYSVINIT_ENABLED_GETTYS`` is "1" (i.e. only
6855 run a getty on the first virtual terminal).
6856
6857T
6858 This variable points to a directory were BitBake places temporary
6859 files, which consist mostly of task logs and scripts, when building a
6860 particular recipe. The variable is typically set as follows: T =
6861 "${WORKDIR}/temp"
6862
6863 The ```WORKDIR`` <#var-WORKDIR>`__ is the directory into which
6864 BitBake unpacks and builds the recipe. The default ``bitbake.conf``
6865 file sets this variable.
6866
6867 The ``T`` variable is not to be confused with the
6868 ```TMPDIR`` <#var-TMPDIR>`__ variable, which points to the root of
6869 the directory tree where BitBake places the output of an entire
6870 build.
6871
6872TARGET_ARCH
6873 The target machine's architecture. The OpenEmbedded build system
6874 supports many architectures. Here is an example list of architectures
6875 supported. This list is by no means complete as the architecture is
6876 configurable: arm i586 x86_64 powerpc powerpc64 mips mipsel
6877
6878 For additional information on machine architectures, see the
6879 ```TUNE_ARCH`` <#var-TUNE_ARCH>`__ variable.
6880
6881TARGET_AS_ARCH
6882 Specifies architecture-specific assembler flags for the target
6883 system. ``TARGET_AS_ARCH`` is initialized from
6884 ```TUNE_ASARGS`` <#var-TUNE_ASARGS>`__ by default in the BitBake
6885 configuration file (``meta/conf/bitbake.conf``): TARGET_AS_ARCH =
6886 "${TUNE_ASARGS}"
6887
6888TARGET_CC_ARCH
6889 Specifies architecture-specific C compiler flags for the target
6890 system. ``TARGET_CC_ARCH`` is initialized from
6891 ```TUNE_CCARGS`` <#var-TUNE_CCARGS>`__ by default.
6892
6893 .. note::
6894
6895 It is a common workaround to append
6896 LDFLAGS
6897 to
6898 TARGET_CC_ARCH
6899 in recipes that build software for the target that would not
6900 otherwise respect the exported
6901 LDFLAGS
6902 variable.
6903
6904TARGET_CC_KERNEL_ARCH
6905 This is a specific kernel compiler flag for a CPU or Application
6906 Binary Interface (ABI) tune. The flag is used rarely and only for
6907 cases where a userspace ```TUNE_CCARGS`` <#var-TUNE_CCARGS>`__ is not
6908 compatible with the kernel compilation. The ``TARGET_CC_KERNEL_ARCH``
6909 variable allows the kernel (and associated modules) to use a
6910 different configuration. See the
6911 ``meta/conf/machine/include/arm/feature-arm-thumb.inc`` file in the
6912 `Source Directory <#source-directory>`__ for an example.
6913
6914TARGET_CFLAGS
6915 Specifies the flags to pass to the C compiler when building for the
6916 target. When building in the target context,
6917 ```CFLAGS`` <#var-CFLAGS>`__ is set to the value of this variable by
6918 default.
6919
6920 Additionally, the SDK's environment setup script sets the ``CFLAGS``
6921 variable in the environment to the ``TARGET_CFLAGS`` value so that
6922 executables built using the SDK also have the flags applied.
6923
6924TARGET_CPPFLAGS
6925 Specifies the flags to pass to the C pre-processor (i.e. to both the
6926 C and the C++ compilers) when building for the target. When building
6927 in the target context, ```CPPFLAGS`` <#var-CPPFLAGS>`__ is set to the
6928 value of this variable by default.
6929
6930 Additionally, the SDK's environment setup script sets the
6931 ``CPPFLAGS`` variable in the environment to the ``TARGET_CPPFLAGS``
6932 value so that executables built using the SDK also have the flags
6933 applied.
6934
6935TARGET_CXXFLAGS
6936 Specifies the flags to pass to the C++ compiler when building for the
6937 target. When building in the target context,
6938 ```CXXFLAGS`` <#var-CXXFLAGS>`__ is set to the value of this variable
6939 by default.
6940
6941 Additionally, the SDK's environment setup script sets the
6942 ``CXXFLAGS`` variable in the environment to the ``TARGET_CXXFLAGS``
6943 value so that executables built using the SDK also have the flags
6944 applied.
6945
6946TARGET_FPU
6947 Specifies the method for handling FPU code. For FPU-less targets,
6948 which include most ARM CPUs, the variable must be set to "soft". If
6949 not, the kernel emulation gets used, which results in a performance
6950 penalty.
6951
6952TARGET_LD_ARCH
6953 Specifies architecture-specific linker flags for the target system.
6954 ``TARGET_LD_ARCH`` is initialized from
6955 ```TUNE_LDARGS`` <#var-TUNE_LDARGS>`__ by default in the BitBake
6956 configuration file (``meta/conf/bitbake.conf``): TARGET_LD_ARCH =
6957 "${TUNE_LDARGS}"
6958
6959TARGET_LDFLAGS
6960 Specifies the flags to pass to the linker when building for the
6961 target. When building in the target context,
6962 ```LDFLAGS`` <#var-LDFLAGS>`__ is set to the value of this variable
6963 by default.
6964
6965 Additionally, the SDK's environment setup script sets the
6966 ```LDFLAGS`` <#var-LDFLAGS>`__ variable in the environment to the
6967 ``TARGET_LDFLAGS`` value so that executables built using the SDK also
6968 have the flags applied.
6969
6970TARGET_OS
6971 Specifies the target's operating system. The variable can be set to
6972 "linux" for glibc-based systems (GNU C Library) and to "linux-musl"
6973 for musl libc. For ARM/EABI targets, "linux-gnueabi" and
6974 "linux-musleabi" possible values exist.
6975
6976TARGET_PREFIX
6977 Specifies the prefix used for the toolchain binary target tools.
6978
6979 Depending on the type of recipe and the build target,
6980 ``TARGET_PREFIX`` is set as follows:
6981
6982 - For recipes building for the target machine, the value is
6983 "${`TARGET_SYS <#var-TARGET_SYS>`__}-".
6984
6985 - For native recipes, the build system sets the variable to the
6986 value of ``BUILD_PREFIX``.
6987
6988 - For native SDK recipes (``nativesdk``), the build system sets the
6989 variable to the value of ``SDK_PREFIX``.
6990
6991TARGET_SYS
6992 Specifies the system, including the architecture and the operating
6993 system, for which the build is occurring in the context of the
6994 current recipe.
6995
6996 The OpenEmbedded build system automatically sets this variable based
6997 on ```TARGET_ARCH`` <#var-TARGET_ARCH>`__,
6998 ```TARGET_VENDOR`` <#var-TARGET_VENDOR>`__, and
6999 ```TARGET_OS`` <#var-TARGET_OS>`__ variables.
7000
7001 .. note::
7002
7003 You do not need to set the
7004 TARGET_SYS
7005 variable yourself.
7006
7007 Consider these two examples:
7008
7009 - Given a native recipe on a 32-bit, x86 machine running Linux, the
7010 value is "i686-linux".
7011
7012 - Given a recipe being built for a little-endian, MIPS target
7013 running Linux, the value might be "mipsel-linux".
7014
7015TARGET_VENDOR
7016 Specifies the name of the target vendor.
7017
7018TCLIBC
7019 Specifies the GNU standard C library (``libc``) variant to use during
7020 the build process. This variable replaces ``POKYLIBC``, which is no
7021 longer supported.
7022
7023 You can select "glibc", "musl", "newlib", or "baremetal"
7024
7025TCLIBCAPPEND
7026 Specifies a suffix to be appended onto the
7027 ```TMPDIR`` <#var-TMPDIR>`__ value. The suffix identifies the
7028 ``libc`` variant for building. When you are building for multiple
7029 variants with the same `Build Directory <#build-directory>`__, this
7030 mechanism ensures that output for different ``libc`` variants is kept
7031 separate to avoid potential conflicts.
7032
7033 In the ``defaultsetup.conf`` file, the default value of
7034 ``TCLIBCAPPEND`` is "-${TCLIBC}". However, distros such as poky,
7035 which normally only support one ``libc`` variant, set
7036 ``TCLIBCAPPEND`` to "" in their distro configuration file resulting
7037 in no suffix being applied.
7038
7039TCMODE
7040 Specifies the toolchain selector. ``TCMODE`` controls the
7041 characteristics of the generated packages and images by telling the
7042 OpenEmbedded build system which toolchain profile to use. By default,
7043 the OpenEmbedded build system builds its own internal toolchain. The
7044 variable's default value is "default", which uses that internal
7045 toolchain.
7046
7047 .. note::
7048
7049 If
7050 TCMODE
7051 is set to a value other than "default", then it is your
7052 responsibility to ensure that the toolchain is compatible with the
7053 default toolchain. Using older or newer versions of these
7054 components might cause build problems. See the Release Notes for
7055 the Yocto Project release for the specific components with which
7056 the toolchain must be compatible. To access the Release Notes, go
7057 to the
7058 Downloads
7059 page on the Yocto Project website and click on the "RELEASE
7060 INFORMATION" link for the appropriate release.
7061
7062 The ``TCMODE`` variable is similar to ```TCLIBC`` <#var-TCLIBC>`__,
7063 which controls the variant of the GNU standard C library (``libc``)
7064 used during the build process: ``glibc`` or ``musl``.
7065
7066 With additional layers, it is possible to use a pre-compiled external
7067 toolchain. One example is the Sourcery G++ Toolchain. The support for
7068 this toolchain resides in the separate Mentor Graphics
7069 ``meta-sourcery`` layer at
7070 ` <http://github.com/MentorEmbedded/meta-sourcery/>`__.
7071
7072 The layer's ``README`` file contains information on how to use the
7073 Sourcery G++ Toolchain as an external toolchain. In summary, you must
7074 be sure to add the layer to your ``bblayers.conf`` file in front of
7075 the ``meta`` layer and then set the ``EXTERNAL_TOOLCHAIN`` variable
7076 in your ``local.conf`` file to the location in which you installed
7077 the toolchain.
7078
7079 The fundamentals used for this example apply to any external
7080 toolchain. You can use ``meta-sourcery`` as a template for adding
7081 support for other external toolchains.
7082
7083TEST_EXPORT_DIR
7084 The location the OpenEmbedded build system uses to export tests when
7085 the ```TEST_EXPORT_ONLY`` <#var-TEST_EXPORT_ONLY>`__ variable is set
7086 to "1".
7087
7088 The ``TEST_EXPORT_DIR`` variable defaults to
7089 ``"${TMPDIR}/testimage/${PN}"``.
7090
7091TEST_EXPORT_ONLY
7092 Specifies to export the tests only. Set this variable to "1" if you
7093 do not want to run the tests but you want them to be exported in a
7094 manner that you to run them outside of the build system.
7095
7096TEST_LOG_DIR
7097 Holds the SSH log and the boot log for QEMU machines. The
7098 ``TEST_LOG_DIR`` variable defaults to ``"${WORKDIR}/testimage"``.
7099
7100 .. note::
7101
7102 Actual test results reside in the task log (
7103 log.do_testimage
7104 ), which is in the
7105 ${WORKDIR}/temp/
7106 directory.
7107
7108TEST_POWERCONTROL_CMD
7109 For automated hardware testing, specifies the command to use to
7110 control the power of the target machine under test. Typically, this
7111 command would point to a script that performs the appropriate action
7112 (e.g. interacting with a web-enabled power strip). The specified
7113 command should expect to receive as the last argument "off", "on" or
7114 "cycle" specifying to power off, on, or cycle (power off and then
7115 power on) the device, respectively.
7116
7117TEST_POWERCONTROL_EXTRA_ARGS
7118 For automated hardware testing, specifies additional arguments to
7119 pass through to the command specified in
7120 ```TEST_POWERCONTROL_CMD`` <#var-TEST_POWERCONTROL_CMD>`__. Setting
7121 ``TEST_POWERCONTROL_EXTRA_ARGS`` is optional. You can use it if you
7122 wish, for example, to separate the machine-specific and
7123 non-machine-specific parts of the arguments.
7124
7125TEST_QEMUBOOT_TIMEOUT
7126 The time in seconds allowed for an image to boot before automated
7127 runtime tests begin to run against an image. The default timeout
7128 period to allow the boot process to reach the login prompt is 500
7129 seconds. You can specify a different value in the ``local.conf``
7130 file.
7131
7132 For more information on testing images, see the "`Performing
7133 Automated Runtime
7134 Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
7135 section in the Yocto Project Development Tasks Manual.
7136
7137TEST_SERIALCONTROL_CMD
7138 For automated hardware testing, specifies the command to use to
7139 connect to the serial console of the target machine under test. This
7140 command simply needs to connect to the serial console and forward
7141 that connection to standard input and output as any normal terminal
7142 program does.
7143
7144 For example, to use the Picocom terminal program on serial device
7145 ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:
7146 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
7147
7148TEST_SERIALCONTROL_EXTRA_ARGS
7149 For automated hardware testing, specifies additional arguments to
7150 pass through to the command specified in
7151 ```TEST_SERIALCONTROL_CMD`` <#var-TEST_SERIALCONTROL_CMD>`__. Setting
7152 ``TEST_SERIALCONTROL_EXTRA_ARGS`` is optional. You can use it if you
7153 wish, for example, to separate the machine-specific and
7154 non-machine-specific parts of the command.
7155
7156TEST_SERVER_IP
7157 The IP address of the build machine (host machine). This IP address
7158 is usually automatically detected. However, if detection fails, this
7159 variable needs to be set to the IP address of the build machine (i.e.
7160 where the build is taking place).
7161
7162 .. note::
7163
7164 The
7165 TEST_SERVER_IP
7166 variable is only used for a small number of tests such as the
7167 "dnf" test suite, which needs to download packages from
7168 WORKDIR/oe-rootfs-repo
7169 .
7170
7171TEST_TARGET
7172 Specifies the target controller to use when running tests against a
7173 test image. The default controller to use is "qemu": TEST_TARGET =
7174 "qemu"
7175
7176 A target controller is a class that defines how an image gets
7177 deployed on a target and how a target is started. A layer can extend
7178 the controllers by adding a module in the layer's
7179 ``/lib/oeqa/controllers`` directory and by inheriting the
7180 ``BaseTarget`` class, which is an abstract class that cannot be used
7181 as a value of ``TEST_TARGET``.
7182
7183 You can provide the following arguments with ``TEST_TARGET``:
7184
7185 - *"qemu":* Boots a QEMU image and runs the tests. See the
7186 "`Enabling Runtime Tests on
7187 QEMU <&YOCTO_DOCS_DEV_URL;#qemu-image-enabling-tests>`__" section
7188 in the Yocto Project Development Tasks Manual for more
7189 information.
7190
7191 - *"simpleremote":* Runs the tests on target hardware that is
7192 already up and running. The hardware can be on the network or it
7193 can be a device running an image on QEMU. You must also set
7194 ```TEST_TARGET_IP`` <#var-TEST_TARGET_IP>`__ when you use
7195 "simpleremote".
7196
7197 .. note::
7198
7199 This argument is defined in
7200 meta/lib/oeqa/controllers/simpleremote.py
7201 .
7202
7203 For information on running tests on hardware, see the "`Enabling
7204 Runtime Tests on
7205 Hardware <&YOCTO_DOCS_DEV_URL;#hardware-image-enabling-tests>`__"
7206 section in the Yocto Project Development Tasks Manual.
7207
7208TEST_TARGET_IP
7209 The IP address of your hardware under test. The ``TEST_TARGET_IP``
7210 variable has no effect when ```TEST_TARGET`` <#var-TEST_TARGET>`__ is
7211 set to "qemu".
7212
7213 When you specify the IP address, you can also include a port. Here is
7214 an example: TEST_TARGET_IP = "192.168.1.4:2201" Specifying a port is
7215 useful when SSH is started on a non-standard port or in cases when
7216 your hardware under test is behind a firewall or network that is not
7217 directly accessible from your host and you need to do port address
7218 translation.
7219
7220TEST_SUITES
7221 An ordered list of tests (modules) to run against an image when
7222 performing automated runtime testing.
7223
7224 The OpenEmbedded build system provides a core set of tests that can
7225 be used against images.
7226
7227 .. note::
7228
7229 Currently, there is only support for running these tests under
7230 QEMU.
7231
7232 Tests include ``ping``, ``ssh``, ``df`` among others. You can add
7233 your own tests to the list of tests by appending ``TEST_SUITES`` as
7234 follows: TEST_SUITES_append = " mytest" Alternatively, you can
7235 provide the "auto" option to have all applicable tests run against
7236 the image. TEST_SUITES_append = " auto" Using this option causes the
7237 build system to automatically run tests that are applicable to the
7238 image. Tests that are not applicable are skipped.
7239
7240 The order in which tests are run is important. Tests that depend on
7241 another test must appear later in the list than the test on which
7242 they depend. For example, if you append the list of tests with two
7243 tests (``test_A`` and ``test_B``) where ``test_B`` is dependent on
7244 ``test_A``, then you must order the tests as follows: TEST_SUITES = "
7245 test_A test_B"
7246
7247 For more information on testing images, see the "`Performing
7248 Automated Runtime
7249 Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
7250 section in the Yocto Project Development Tasks Manual.
7251
7252TESTIMAGE_AUTO
7253 Automatically runs the series of automated tests for images when an
7254 image is successfully built. Setting ``TESTIMAGE_AUTO`` to "1" causes
7255 any image that successfully builds to automatically boot under QEMU.
7256 Using the variable also adds in dependencies so that any SDK for
7257 which testing is requested is automatically built first.
7258
7259 These tests are written in Python making use of the ``unittest``
7260 module, and the majority of them run commands on the target system
7261 over ``ssh``. You can set this variable to "1" in your ``local.conf``
7262 file in the `Build Directory <#build-directory>`__ to have the
7263 OpenEmbedded build system automatically run these tests after an
7264 image successfully builds: TESTIMAGE_AUTO = "1" For more information
7265 on enabling, running, and writing these tests, see the "`Performing
7266 Automated Runtime
7267 Testing <&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing>`__"
7268 section in the Yocto Project Development Tasks Manual and the
7269 "```testimage*.bbclass`` <#ref-classes-testimage*>`__" section.
7270
7271THISDIR
7272 The directory in which the file BitBake is currently parsing is
7273 located. Do not manually set this variable.
7274
7275TIME
7276 The time the build was started. Times appear using the hour, minute,
7277 and second (HMS) format (e.g. "140159" for one minute and fifty-nine
7278 seconds past 1400 hours).
7279
7280TMPDIR
7281 This variable is the base directory the OpenEmbedded build system
7282 uses for all build output and intermediate files (other than the
7283 shared state cache). By default, the ``TMPDIR`` variable points to
7284 ``tmp`` within the `Build Directory <#build-directory>`__.
7285
7286 If you want to establish this directory in a location other than the
7287 default, you can uncomment and edit the following statement in the
7288 ``conf/local.conf`` file in the `Source
7289 Directory <#source-directory>`__: #TMPDIR = "${TOPDIR}/tmp" An
7290 example use for this scenario is to set ``TMPDIR`` to a local disk,
7291 which does not use NFS, while having the Build Directory use NFS.
7292
7293 The filesystem used by ``TMPDIR`` must have standard filesystem
7294 semantics (i.e. mixed-case files are unique, POSIX file locking, and
7295 persistent inodes). Due to various issues with NFS and bugs in some
7296 implementations, NFS does not meet this minimum requirement.
7297 Consequently, ``TMPDIR`` cannot be on NFS.
7298
7299TOOLCHAIN_HOST_TASK
7300 This variable lists packages the OpenEmbedded build system uses when
7301 building an SDK, which contains a cross-development environment. The
7302 packages specified by this variable are part of the toolchain set
7303 that runs on the ```SDKMACHINE`` <#var-SDKMACHINE>`__, and each
7304 package should usually have the prefix ``nativesdk-``. For example,
7305 consider the following command when building an SDK: $ bitbake -c
7306 populate_sdk imagename In this case, a default list of packages is
7307 set in this variable, but you can add additional packages to the
7308 list. See the "`Adding Individual Packages to the Standard
7309 SDK <&YOCTO_DOCS_SDK_URL;#sdk-adding-individual-packages>`__" section
7310 in the Yocto Project Application Development and the Extensible
7311 Software Development Kit (eSDK) manual for more information.
7312
7313 For background information on cross-development toolchains in the
7314 Yocto Project development environment, see the "`Cross-Development
7315 Toolchain
7316 Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
7317 section in the Yocto Project Overview and Concepts Manual. For
7318 information on setting up a cross-development environment, see the
7319 `Yocto Project Application Development and the Extensible Software
7320 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
7321
7322TOOLCHAIN_OUTPUTNAME
7323 This variable defines the name used for the toolchain output. The
7324 ```populate_sdk_base`` <#ref-classes-populate-sdk-*>`__ class sets
7325 the ``TOOLCHAIN_OUTPUTNAME`` variable as follows:
7326 TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-toolchain-${SDK_VERSION}" See
7327 the ```SDK_NAME`` <#var-SDK_NAME>`__ and
7328 ```SDK_VERSION`` <#var-SDK_VERSION>`__ variables for additional
7329 information.
7330
7331TOOLCHAIN_TARGET_TASK
7332 This variable lists packages the OpenEmbedded build system uses when
7333 it creates the target part of an SDK (i.e. the part built for the
7334 target hardware), which includes libraries and headers. Use this
7335 variable to add individual packages to the part of the SDK that runs
7336 on the target. See the "`Adding Individual Packages to the Standard
7337 SDK <&YOCTO_DOCS_SDK_URL;#sdk-adding-individual-packages>`__" section
7338 in the Yocto Project Application Development and the Extensible
7339 Software Development Kit (eSDK) manual for more information.
7340
7341 For background information on cross-development toolchains in the
7342 Yocto Project development environment, see the "`Cross-Development
7343 Toolchain
7344 Generation <&YOCTO_DOCS_OM_URL;#cross-development-toolchain-generation>`__"
7345 section in the Yocto Project Overview and Concepts Manual. For
7346 information on setting up a cross-development environment, see the
7347 `Yocto Project Application Development and the Extensible Software
7348 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__ manual.
7349
7350TOPDIR
7351 The top-level `Build Directory <#build-directory>`__. BitBake
7352 automatically sets this variable when you initialize your build
7353 environment using ````` <#structure-core-script>`__.
7354
7355TRANSLATED_TARGET_ARCH
7356 A sanitized version of ```TARGET_ARCH`` <#var-TARGET_ARCH>`__. This
7357 variable is used where the architecture is needed in a value where
7358 underscores are not allowed, for example within package filenames. In
7359 this case, dash characters replace any underscore characters used in
7360 ``TARGET_ARCH``.
7361
7362 Do not edit this variable.
7363
7364TUNE_ARCH
7365 The GNU canonical architecture for a specific architecture (i.e.
7366 ``arm``, ``armeb``, ``mips``, ``mips64``, and so forth). BitBake uses
7367 this value to setup configuration.
7368
7369 ``TUNE_ARCH`` definitions are specific to a given architecture. The
7370 definitions can be a single static definition, or can be dynamically
7371 adjusted. You can see details for a given CPU family by looking at
7372 the architecture's ``README`` file. For example, the
7373 ``meta/conf/machine/include/mips/README`` file in the `Source
7374 Directory <#source-directory>`__ provides information for
7375 ``TUNE_ARCH`` specific to the ``mips`` architecture.
7376
7377 ``TUNE_ARCH`` is tied closely to
7378 ```TARGET_ARCH`` <#var-TARGET_ARCH>`__, which defines the target
7379 machine's architecture. The BitBake configuration file
7380 (``meta/conf/bitbake.conf``) sets ``TARGET_ARCH`` as follows:
7381 TARGET_ARCH = "${TUNE_ARCH}"
7382
7383 The following list, which is by no means complete since architectures
7384 are configurable, shows supported machine architectures: arm i586
7385 x86_64 powerpc powerpc64 mips mipsel
7386
7387TUNE_ASARGS
7388 Specifies architecture-specific assembler flags for the target
7389 system. The set of flags is based on the selected tune features.
7390 ``TUNE_ASARGS`` is set using the tune include files, which are
7391 typically under ``meta/conf/machine/include/`` and are influenced
7392 through ```TUNE_FEATURES`` <#var-TUNE_FEATURES>`__. For example, the
7393 ``meta/conf/machine/include/x86/arch-x86.inc`` file defines the flags
7394 for the x86 architecture as follows: TUNE_ASARGS +=
7395 "${@bb.utils.contains("TUNE_FEATURES", "mx32", "-x32", "", d)}"
7396
7397 .. note::
7398
7399 Board Support Packages (BSPs) select the tune. The selected tune,
7400 in turn, affects the tune variables themselves (i.e. the tune can
7401 supply its own set of flags).
7402
7403TUNE_CCARGS
7404 Specifies architecture-specific C compiler flags for the target
7405 system. The set of flags is based on the selected tune features.
7406 ``TUNE_CCARGS`` is set using the tune include files, which are
7407 typically under ``meta/conf/machine/include/`` and are influenced
7408 through ```TUNE_FEATURES`` <#var-TUNE_FEATURES>`__.
7409
7410 .. note::
7411
7412 Board Support Packages (BSPs) select the tune. The selected tune,
7413 in turn, affects the tune variables themselves (i.e. the tune can
7414 supply its own set of flags).
7415
7416TUNE_LDARGS
7417 Specifies architecture-specific linker flags for the target system.
7418 The set of flags is based on the selected tune features.
7419 ``TUNE_LDARGS`` is set using the tune include files, which are
7420 typically under ``meta/conf/machine/include/`` and are influenced
7421 through ```TUNE_FEATURES`` <#var-TUNE_FEATURES>`__. For example, the
7422 ``meta/conf/machine/include/x86/arch-x86.inc`` file defines the flags
7423 for the x86 architecture as follows: TUNE_LDARGS +=
7424 "${@bb.utils.contains("TUNE_FEATURES", "mx32", "-m elf32_x86_64", "",
7425 d)}"
7426
7427 .. note::
7428
7429 Board Support Packages (BSPs) select the tune. The selected tune,
7430 in turn, affects the tune variables themselves (i.e. the tune can
7431 supply its own set of flags).
7432
7433TUNE_FEATURES
7434 Features used to "tune" a compiler for optimal use given a specific
7435 processor. The features are defined within the tune files and allow
7436 arguments (i.e. ``TUNE_*ARGS``) to be dynamically generated based on
7437 the features.
7438
7439 The OpenEmbedded build system verifies the features to be sure they
7440 are not conflicting and that they are supported.
7441
7442 The BitBake configuration file (``meta/conf/bitbake.conf``) defines
7443 ``TUNE_FEATURES`` as follows: TUNE_FEATURES ??=
7444 "${TUNE_FEATURES_tune-${DEFAULTTUNE}}" See the
7445 ```DEFAULTTUNE`` <#var-DEFAULTTUNE>`__ variable for more information.
7446
7447TUNE_PKGARCH
7448 The package architecture understood by the packaging system to define
7449 the architecture, ABI, and tuning of output packages. The specific
7450 tune is defined using the "_tune" override as follows:
7451 TUNE_PKGARCH_tune-tune = "tune"
7452
7453 These tune-specific package architectures are defined in the machine
7454 include files. Here is an example of the "core2-32" tuning as used in
7455 the ``meta/conf/machine/include/tune-core2.inc`` file:
7456 TUNE_PKGARCH_tune-core2-32 = "core2-32"
7457
7458TUNEABI
7459 An underlying Application Binary Interface (ABI) used by a particular
7460 tuning in a given toolchain layer. Providers that use prebuilt
7461 libraries can use the ``TUNEABI``,
7462 ```TUNEABI_OVERRIDE`` <#var-TUNEABI_OVERRIDE>`__, and
7463 ```TUNEABI_WHITELIST`` <#var-TUNEABI_WHITELIST>`__ variables to check
7464 compatibility of tunings against their selection of libraries.
7465
7466 If ``TUNEABI`` is undefined, then every tuning is allowed. See the
7467 ```sanity`` <#ref-classes-sanity>`__ class to see how the variable is
7468 used.
7469
7470TUNEABI_OVERRIDE
7471 If set, the OpenEmbedded system ignores the
7472 ```TUNEABI_WHITELIST`` <#var-TUNEABI_WHITELIST>`__ variable.
7473 Providers that use prebuilt libraries can use the
7474 ``TUNEABI_OVERRIDE``, ``TUNEABI_WHITELIST``, and
7475 ```TUNEABI`` <#var-TUNEABI>`__ variables to check compatibility of a
7476 tuning against their selection of libraries.
7477
7478 See the ```sanity`` <#ref-classes-sanity>`__ class to see how the
7479 variable is used.
7480
7481TUNEABI_WHITELIST
7482 A whitelist of permissible ```TUNEABI`` <#var-TUNEABI>`__ values. If
7483 ``TUNEABI_WHITELIST`` is not set, all tunes are allowed. Providers
7484 that use prebuilt libraries can use the ``TUNEABI_WHITELIST``,
7485 ```TUNEABI_OVERRIDE`` <#var-TUNEABI_OVERRIDE>`__, and ``TUNEABI``
7486 variables to check compatibility of a tuning against their selection
7487 of libraries.
7488
7489 See the ```sanity`` <#ref-classes-sanity>`__ class to see how the
7490 variable is used.
7491
7492TUNECONFLICTS[feature]
7493 Specifies CPU or Application Binary Interface (ABI) tuning features
7494 that conflict with feature.
7495
7496 Known tuning conflicts are specified in the machine include files in
7497 the `Source Directory <#source-directory>`__. Here is an example from
7498 the ``meta/conf/machine/include/mips/arch-mips.inc`` include file
7499 that lists the "o32" and "n64" features as conflicting with the "n32"
7500 feature: TUNECONFLICTS[n32] = "o32 n64"
7501
7502TUNEVALID[feature]
7503 Specifies a valid CPU or Application Binary Interface (ABI) tuning
7504 feature. The specified feature is stored as a flag. Valid features
7505 are specified in the machine include files (e.g.
7506 ``meta/conf/machine/include/arm/arch-arm.inc``). Here is an example
7507 from that file: TUNEVALID[bigendian] = "Enable big-endian mode."
7508
7509 See the machine include files in the `Source
7510 Directory <#source-directory>`__ for these features.
7511
7512UBOOT_CONFIG
7513 Configures the ```UBOOT_MACHINE`` <#var-UBOOT_MACHINE>`__ and can
7514 also define ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ for individual
7515 cases.
7516
7517 Following is an example from the ``meta-fsl-arm`` layer. UBOOT_CONFIG
7518 ??= "sd" UBOOT_CONFIG[sd] = "mx6qsabreauto_config,sdcard"
7519 UBOOT_CONFIG[eimnor] = "mx6qsabreauto_eimnor_config"
7520 UBOOT_CONFIG[nand] = "mx6qsabreauto_nand_config,ubifs"
7521 UBOOT_CONFIG[spinor] = "mx6qsabreauto_spinor_config" In this example,
7522 "sd" is selected as the configuration of the possible four for the
7523 ``UBOOT_MACHINE``. The "sd" configuration defines
7524 "mx6qsabreauto_config" as the value for ``UBOOT_MACHINE``, while the
7525 "sdcard" specifies the ``IMAGE_FSTYPES`` to use for the U-boot image.
7526
7527 For more information on how the ``UBOOT_CONFIG`` is handled, see the
7528 ```uboot-config`` <http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/uboot-config.bbclass>`__
7529 class.
7530
7531UBOOT_ENTRYPOINT
7532 Specifies the entry point for the U-Boot image. During U-Boot image
7533 creation, the ``UBOOT_ENTRYPOINT`` variable is passed as a
7534 command-line parameter to the ``uboot-mkimage`` utility.
7535
7536UBOOT_LOADADDRESS
7537 Specifies the load address for the U-Boot image. During U-Boot image
7538 creation, the ``UBOOT_LOADADDRESS`` variable is passed as a
7539 command-line parameter to the ``uboot-mkimage`` utility.
7540
7541UBOOT_LOCALVERSION
7542 Appends a string to the name of the local version of the U-Boot
7543 image. For example, assuming the version of the U-Boot image built
7544 was "2013.10", the full version string reported by U-Boot would be
7545 "2013.10-yocto" given the following statement: UBOOT_LOCALVERSION =
7546 "-yocto"
7547
7548UBOOT_MACHINE
7549 Specifies the value passed on the ``make`` command line when building
7550 a U-Boot image. The value indicates the target platform
7551 configuration. You typically set this variable from the machine
7552 configuration file (i.e. ``conf/machine/machine_name.conf``).
7553
7554 Please see the "Selection of Processor Architecture and Board Type"
7555 section in the U-Boot README for valid values for this variable.
7556
7557UBOOT_MAKE_TARGET
7558 Specifies the target called in the ``Makefile``. The default target
7559 is "all".
7560
7561UBOOT_MKIMAGE_DTCOPTS
7562 Options for the device tree compiler passed to mkimage '-D'
7563 feature while creating FIT image in ``kernel-fitimage`` class.
7564
7565UBOOT_RD_LOADADDRESS
7566 Specifies the load address for the RAM disk image.
7567 During FIT image creation, the
7568 ``UBOOT_RD_LOADADDRESS`` variable is used
7569 in ``kernel-fitimage`` class to specify the
7570 load address to be used in creating the Image Tree Source for
7571 the FIT image.
7572
7573UBOOT_RD_ENTRYPOINT
7574 Specifies the entrypoint for the RAM disk image.
7575 During FIT image creation, the
7576 ``UBOOT_RD_ENTRYPOINT`` variable is used
7577 in ``kernel-fitimage`` class to specify the
7578 entrypoint to be used in creating the Image Tree Source for
7579 the FIT image.
7580
7581UBOOT_SUFFIX
7582 Points to the generated U-Boot extension. For example, ``u-boot.sb``
7583 has a ``.sb`` extension.
7584
7585 The default U-Boot extension is ``.bin``
7586
7587UBOOT_TARGET
7588 Specifies the target used for building U-Boot. The target is passed
7589 directly as part of the "make" command (e.g. SPL and AIS). If you do
7590 not specifically set this variable, the OpenEmbedded build process
7591 passes and uses "all" for the target during the U-Boot building
7592 process.
7593
7594UBOOT_SIGN_ENABLE
7595 Enable signing of FIT image. The default value is "0".
7596
7597UBOOT_SIGN_KEYDIR
7598 Location of the directory containing the RSA key and
7599 certificate used for signing FIT image.
7600
7601UBOOT_SIGN_KEYNAME
7602 The name of keys used for signing U-boot FIT image stored in
7603 :term:`UBOOT_SIGN_KEYDIR` directory. For e.g. dev.key key and dev.crt
7604 certificate stored in :term:`UBOOT_SIGN_KEYDIR` directory will have
7605 :term:`UBOOT_SIGN_KEYNAME` set to "dev".
7606
7607UNKNOWN_CONFIGURE_WHITELIST
7608 Specifies a list of options that, if reported by the configure script
7609 as being invalid, should not generate a warning during the
7610 ```do_configure`` <#ref-tasks-configure>`__ task. Normally, invalid
7611 configure options are simply not passed to the configure script (e.g.
7612 should be removed from ```EXTRA_OECONF`` <#var-EXTRA_OECONF>`__ or
7613 ```PACKAGECONFIG_CONFARGS`` <#var-PACKAGECONFIG_CONFARGS>`__).
7614 However, common options, for example, exist that are passed to all
7615 configure scripts at a class level that might not be valid for some
7616 configure scripts. It follows that no benefit exists in seeing a
7617 warning about these options. For these cases, the options are added
7618 to ``UNKNOWN_CONFIGURE_WHITELIST``.
7619
7620 The configure arguments check that uses
7621 ``UNKNOWN_CONFIGURE_WHITELIST`` is part of the
7622 ```insane`` <#ref-classes-insane>`__ class and is only enabled if the
7623 recipe inherits the ```autotools`` <#ref-classes-autotools>`__ class.
7624
7625UPDATERCPN
7626 For recipes inheriting the
7627 ```update-rc.d`` <#ref-classes-update-rc.d>`__ class, ``UPDATERCPN``
7628 specifies the package that contains the initscript that is enabled.
7629
7630 The default value is "${PN}". Given that almost all recipes that
7631 install initscripts package them in the main package for the recipe,
7632 you rarely need to set this variable in individual recipes.
7633
7634UPSTREAM_CHECK_GITTAGREGEX
7635 You can perform a per-recipe check for what the latest upstream
7636 source code version is by calling ``bitbake -c checkpkg`` recipe. If
7637 the recipe source code is provided from Git repositories, the
7638 OpenEmbedded build system determines the latest upstream version by
7639 picking the latest tag from the list of all repository tags.
7640
7641 You can use the ``UPSTREAM_CHECK_GITTAGREGEX`` variable to provide a
7642 regular expression to filter only the relevant tags should the
7643 default filter not work correctly. UPSTREAM_CHECK_GITTAGREGEX =
7644 "git_tag_regex"
7645
7646UPSTREAM_CHECK_REGEX
7647 Use the ``UPSTREAM_CHECK_REGEX`` variable to specify a different
7648 regular expression instead of the default one when the package
7649 checking system is parsing the page found using
7650 ```UPSTREAM_CHECK_URI`` <#var-UPSTREAM_CHECK_URI>`__.
7651 UPSTREAM_CHECK_REGEX = "package_regex"
7652
7653UPSTREAM_CHECK_URI
7654 You can perform a per-recipe check for what the latest upstream
7655 source code version is by calling ``bitbake -c checkpkg`` recipe. If
7656 the source code is provided from tarballs, the latest version is
7657 determined by fetching the directory listing where the tarball is and
7658 attempting to find a later tarball. When this approach does not work,
7659 you can use ``UPSTREAM_CHECK_URI`` to provide a different URI that
7660 contains the link to the latest tarball. UPSTREAM_CHECK_URI =
7661 "recipe_url"
7662
7663USE_DEVFS
7664 Determines if ``devtmpfs`` is used for ``/dev`` population. The
7665 default value used for ``USE_DEVFS`` is "1" when no value is
7666 specifically set. Typically, you would set ``USE_DEVFS`` to "0" for a
7667 statically populated ``/dev`` directory.
7668
7669 See the "`Selecting a Device
7670 Manager <&YOCTO_DOCS_DEV_URL;#selecting-dev-manager>`__" section in
7671 the Yocto Project Development Tasks Manual for information on how to
7672 use this variable.
7673
7674USE_VT
7675 When using
7676 `SysVinit <&YOCTO_DOCS_DEV_URL;#new-recipe-enabling-system-services>`__,
7677 determines whether or not to run a
7678 `getty <http://en.wikipedia.org/wiki/Getty_%28Unix%29>`__ on any
7679 virtual terminals in order to enable logging in through those
7680 terminals.
7681
7682 The default value used for ``USE_VT`` is "1" when no default value is
7683 specifically set. Typically, you would set ``USE_VT`` to "0" in the
7684 machine configuration file for machines that do not have a graphical
7685 display attached and therefore do not need virtual terminal
7686 functionality.
7687
7688USER_CLASSES
7689 A list of classes to globally inherit. These classes are used by the
7690 OpenEmbedded build system to enable extra features (e.g.
7691 ``buildstats``, ``image-mklibs``, and so forth).
7692
7693 The default list is set in your ``local.conf`` file: USER_CLASSES ?=
7694 "buildstats image-mklibs image-prelink" For more information, see
7695 ``meta-poky/conf/local.conf.sample`` in the `Source
7696 Directory <#source-directory>`__.
7697
7698USERADD_ERROR_DYNAMIC
7699 If set to ``error``, forces the OpenEmbedded build system to produce
7700 an error if the user identification (``uid``) and group
7701 identification (``gid``) values are not defined in any of the files
7702 listed in ```USERADD_UID_TABLES`` <#var-USERADD_UID_TABLES>`__ and
7703 ```USERADD_GID_TABLES`` <#var-USERADD_GID_TABLES>`__. If set to
7704 ``warn``, a warning will be issued instead.
7705
7706 The default behavior for the build system is to dynamically apply
7707 ``uid`` and ``gid`` values. Consequently, the
7708 ``USERADD_ERROR_DYNAMIC`` variable is by default not set. If you plan
7709 on using statically assigned ``gid`` and ``uid`` values, you should
7710 set the ``USERADD_ERROR_DYNAMIC`` variable in your ``local.conf``
7711 file as follows: USERADD_ERROR_DYNAMIC = "error" Overriding the
7712 default behavior implies you are going to also take steps to set
7713 static ``uid`` and ``gid`` values through use of the
7714 ```USERADDEXTENSION`` <#var-USERADDEXTENSION>`__,
7715 ```USERADD_UID_TABLES`` <#var-USERADD_UID_TABLES>`__, and
7716 ```USERADD_GID_TABLES`` <#var-USERADD_GID_TABLES>`__ variables.
7717
7718 .. note::
7719
7720 There is a difference in behavior between setting
7721 USERADD_ERROR_DYNAMIC
7722 to
7723 error
7724 and setting it to
7725 warn
7726 . When it is set to
7727 warn
7728 , the build system will report a warning for every undefined
7729 uid
7730 and
7731 gid
7732 in any recipe. But when it is set to
7733 error
7734 , it will only report errors for recipes that are actually built.
7735 This saves you from having to add static IDs for recipes that you
7736 know will never be built.
7737
7738USERADD_GID_TABLES
7739 Specifies a password file to use for obtaining static group
7740 identification (``gid``) values when the OpenEmbedded build system
7741 adds a group to the system during package installation.
7742
7743 When applying static group identification (``gid``) values, the
7744 OpenEmbedded build system looks in ```BBPATH`` <#var-BBPATH>`__ for a
7745 ``files/group`` file and then applies those ``uid`` values. Set the
7746 variable as follows in your ``local.conf`` file: USERADD_GID_TABLES =
7747 "files/group"
7748
7749 .. note::
7750
7751 Setting the
7752 USERADDEXTENSION
7753 variable to "useradd-staticids" causes the build system to use
7754 static
7755 gid
7756 values.
7757
7758USERADD_PACKAGES
7759 When inheriting the ```useradd`` <#ref-classes-useradd>`__ class,
7760 this variable specifies the individual packages within the recipe
7761 that require users and/or groups to be added.
7762
7763 You must set this variable if the recipe inherits the class. For
7764 example, the following enables adding a user for the main package in
7765 a recipe: USERADD_PACKAGES = "${PN}"
7766
7767 .. note::
7768
7769 It follows that if you are going to use the
7770 USERADD_PACKAGES
7771 variable, you need to set one or more of the
7772 USERADD_PARAM
7773 ,
7774 GROUPADD_PARAM
7775 , or
7776 GROUPMEMS_PARAM
7777 variables.
7778
7779USERADD_PARAM
7780 When inheriting the ```useradd`` <#ref-classes-useradd>`__ class,
7781 this variable specifies for a package what parameters should pass to
7782 the ``useradd`` command if you add a user to the system when the
7783 package is installed.
7784
7785 Here is an example from the ``dbus`` recipe: USERADD_PARAM_${PN} =
7786 "--system --home ${localstatedir}/lib/dbus \\ --no-create-home
7787 --shell /bin/false \\ --user-group messagebus" For information on the
7788 standard Linux shell command ``useradd``, see
7789 ` <http://linux.die.net/man/8/useradd>`__.
7790
7791USERADD_UID_TABLES
7792 Specifies a password file to use for obtaining static user
7793 identification (``uid``) values when the OpenEmbedded build system
7794 adds a user to the system during package installation.
7795
7796 When applying static user identification (``uid``) values, the
7797 OpenEmbedded build system looks in ```BBPATH`` <#var-BBPATH>`__ for a
7798 ``files/passwd`` file and then applies those ``uid`` values. Set the
7799 variable as follows in your ``local.conf`` file: USERADD_UID_TABLES =
7800 "files/passwd"
7801
7802 .. note::
7803
7804 Setting the
7805 USERADDEXTENSION
7806 variable to "useradd-staticids" causes the build system to use
7807 static
7808 uid
7809 values.
7810
7811USERADDEXTENSION
7812 When set to "useradd-staticids", causes the OpenEmbedded build system
7813 to base all user and group additions on a static ``passwd`` and
7814 ``group`` files found in ```BBPATH`` <#var-BBPATH>`__.
7815
7816 To use static user identification (``uid``) and group identification
7817 (``gid``) values, set the variable as follows in your ``local.conf``
7818 file: USERADDEXTENSION = "useradd-staticids"
7819
7820 .. note::
7821
7822 Setting this variable to use static
7823 uid
7824 and
7825 gid
7826 values causes the OpenEmbedded build system to employ the
7827 useradd-staticids
7828 class.
7829
7830 If you use static ``uid`` and ``gid`` information, you must also
7831 specify the ``files/passwd`` and ``files/group`` files by setting the
7832 ```USERADD_UID_TABLES`` <#var-USERADD_UID_TABLES>`__ and
7833 ```USERADD_GID_TABLES`` <#var-USERADD_GID_TABLES>`__ variables.
7834 Additionally, you should also set the
7835 ```USERADD_ERROR_DYNAMIC`` <#var-USERADD_ERROR_DYNAMIC>`__ variable.
7836
7837VOLATILE_LOG_DIR
7838 Specifies the persistence of the target's ``/var/log`` directory,
7839 which is used to house postinstall target log files.
7840
7841 By default, ``VOLATILE_LOG_DIR`` is set to "yes", which means the
7842 file is not persistent. You can override this setting by setting the
7843 variable to "no" to make the log directory persistent.
7844
7845WARN_QA
7846 Specifies the quality assurance checks whose failures are reported as
7847 warnings by the OpenEmbedded build system. You set this variable in
7848 your distribution configuration file. For a list of the checks you
7849 can control with this variable, see the
7850 "```insane.bbclass`` <#ref-classes-insane>`__" section.
7851
7852WKS_FILE_DEPENDS
7853 When placed in the recipe that builds your image, this variable lists
7854 build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
7855 applicable when Wic images are active (i.e. when
7856 ```IMAGE_FSTYPES`` <#var-IMAGE_FSTYPES>`__ contains entries related
7857 to Wic). If your recipe does not create Wic images, the variable has
7858 no effect.
7859
7860 The ``WKS_FILE_DEPENDS`` variable is similar to the
7861 ```DEPENDS`` <#var-DEPENDS>`__ variable. When you use the variable in
7862 your recipe that builds the Wic image, dependencies you list in the
7863 ``WIC_FILE_DEPENDS`` variable are added to the ``DEPENDS`` variable.
7864
7865 With the ``WKS_FILE_DEPENDS`` variable, you have the possibility to
7866 specify a list of additional dependencies (e.g. native tools,
7867 bootloaders, and so forth), that are required to build Wic images.
7868 Following is an example: WKS_FILE_DEPENDS = "some-native-tool" In the
7869 previous example, some-native-tool would be replaced with an actual
7870 native tool on which the build would depend.
7871
7872WKS_FILE
7873 Specifies the location of the Wic kickstart file that is used by the
7874 OpenEmbedded build system to create a partitioned image
7875 (image\ ``.wic``). For information on how to create a partitioned
7876 image, see the "`Creating Partitioned Images Using
7877 Wic <&YOCTO_DOCS_DEV_URL;#creating-partitioned-images-using-wic>`__"
7878 section in the Yocto Project Development Tasks Manual. For details on
7879 the kickstart file format, see the "`OpenEmbedded Kickstart
7880 (``.wks``) Reference <#ref-kickstart>`__" Chapter.
7881
7882WORKDIR
7883 The pathname of the work directory in which the OpenEmbedded build
7884 system builds a recipe. This directory is located within the
7885 ```TMPDIR`` <#var-TMPDIR>`__ directory structure and is specific to
7886 the recipe being built and the system for which it is being built.
7887
7888 The ``WORKDIR`` directory is defined as follows:
7889 ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}
7890 The actual directory depends on several things:
7891
7892 - TMPDIR
7893 : The top-level build output directory
7894 - MULTIMACH_TARGET_SYS
7895 : The target system identifier
7896 - PN
7897 : The recipe name
7898 - EXTENDPE
7899 : The epoch - (if
7900 PE
7901 is not specified, which is usually the case for most recipes, then
7902 EXTENDPE
7903 is blank)
7904 - PV
7905 : The recipe version
7906 - PR
7907 : The recipe revision
7908
7909 As an example, assume a Source Directory top-level folder name
7910 ``poky``, a default Build Directory at ``poky/build``, and a
7911 ``qemux86-poky-linux`` machine target system. Furthermore, suppose
7912 your recipe is named ``foo_1.3.0-r0.bb``. In this case, the work
7913 directory the build system uses to build the package would be as
7914 follows: poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
7915
7916XSERVER
7917 Specifies the packages that should be installed to provide an X
7918 server and drivers for the current machine, assuming your image
7919 directly includes ``packagegroup-core-x11-xserver`` or, perhaps
7920 indirectly, includes "x11-base" in
7921 ```IMAGE_FEATURES`` <#var-IMAGE_FEATURES>`__.
7922
7923 The default value of ``XSERVER``, if not specified in the machine
7924 configuration, is "xserver-xorg xf86-video-fbdev xf86-input-evdev".
diff --git a/documentation/ref-manual/ref-varlocality.rst b/documentation/ref-manual/ref-varlocality.rst
new file mode 100644
index 0000000000..d98283c6fb
--- /dev/null
+++ b/documentation/ref-manual/ref-varlocality.rst
@@ -0,0 +1,164 @@
1****************
2Variable Context
3****************
4
5While you can use most variables in almost any context such as
6``.conf``, ``.bbclass``, ``.inc``, and ``.bb`` files, some variables are
7often associated with a particular locality or context. This chapter
8describes some common associations.
9
10.. _ref-varlocality-configuration:
11
12Configuration
13=============
14
15The following subsections provide lists of variables whose context is
16configuration: distribution, machine, and local.
17
18.. _ref-varlocality-config-distro:
19
20Distribution (Distro)
21---------------------
22
23This section lists variables whose configuration context is the
24distribution, or distro.
25
26- ``DISTRO``
27
28- ``DISTRO_NAME``
29
30- ``DISTRO_VERSION``
31
32- ``MAINTAINER``
33
34- ``PACKAGE_CLASSES``
35
36- ``TARGET_OS``
37
38- ``TARGET_FPU``
39
40- ``TCMODE``
41
42- ``TCLIBC``
43
44.. _ref-varlocality-config-machine:
45
46Machine
47-------
48
49This section lists variables whose configuration context is the machine.
50
51- ``TARGET_ARCH``
52
53- ``SERIAL_CONSOLES``
54
55- ``PACKAGE_EXTRA_ARCHS``
56
57- ``IMAGE_FSTYPES``
58
59- ``MACHINE_FEATURES``
60
61- ``MACHINE_EXTRA_RDEPENDS``
62
63- ``MACHINE_EXTRA_RRECOMMENDS``
64
65- ``MACHINE_ESSENTIAL_EXTRA_RDEPENDS``
66
67- ``MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS``
68
69.. _ref-varlocality-config-local:
70
71Local
72-----
73
74This section lists variables whose configuration context is the local
75configuration through the ``local.conf`` file.
76
77- ``DISTRO``
78
79- ``MACHINE``
80
81- ``DL_DIR``
82
83- ``BBFILES``
84
85- ``EXTRA_IMAGE_FEATURES``
86
87- ``PACKAGE_CLASSES``
88
89- ``BB_NUMBER_THREADS``
90
91- ``BBINCLUDELOGS``
92
93- ``ENABLE_BINARY_LOCALE_GENERATION``
94
95.. _ref-varlocality-recipes:
96
97Recipes
98=======
99
100The following subsections provide lists of variables whose context is
101recipes: required, dependencies, path, and extra build information.
102
103.. _ref-varlocality-recipe-required:
104
105Required
106--------
107
108This section lists variables that are required for recipes.
109
110- ``LICENSE``
111
112- ``LIC_FILES_CHKSUM``
113
114- ``SRC_URI`` - used in recipes that fetch local or remote files.
115
116.. _ref-varlocality-recipe-dependencies:
117
118Dependencies
119------------
120
121This section lists variables that define recipe dependencies.
122
123- ``DEPENDS``
124
125- ``RDEPENDS``
126
127- ``RRECOMMENDS``
128
129- ``RCONFLICTS``
130
131- ``RREPLACES``
132
133.. _ref-varlocality-recipe-paths:
134
135Paths
136-----
137
138This section lists variables that define recipe paths.
139
140- ``WORKDIR``
141
142- ``S``
143
144- ``FILES``
145
146.. _ref-varlocality-recipe-build:
147
148Extra Build Information
149-----------------------
150
151This section lists variables that define extra build information for
152recipes.
153
154- ``DEFAULT_PREFERENCE``
155
156- ``EXTRA_OECMAKE``
157
158- ``EXTRA_OECONF``
159
160- ``EXTRA_OEMAKE``
161
162- ``PACKAGECONFIG_CONFARGS``
163
164- ``PACKAGES``
diff --git a/documentation/ref-manual/resources.rst b/documentation/ref-manual/resources.rst
new file mode 100644
index 0000000000..9364ea9a64
--- /dev/null
+++ b/documentation/ref-manual/resources.rst
@@ -0,0 +1,207 @@
1****************************************
2Contributions and Additional Information
3****************************************
4
5.. _resources-intro:
6
7Introduction
8============
9
10The Yocto Project team is happy for people to experiment with the Yocto
11Project. A number of places exist to find help if you run into
12difficulties or find bugs. This presents information about contributing
13and participating in the Yocto Project.
14
15.. _resources-contributions:
16
17Contributions
18=============
19
20The Yocto Project gladly accepts contributions. You can submit changes
21to the project either by creating and sending pull requests, or by
22submitting patches through email. For information on how to do both as
23well as information on how to identify the maintainer for each area of
24code, see the "`Submitting a Change to the Yocto
25Project <&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change>`__" section in the
26Yocto Project Development Tasks Manual.
27
28.. _resources-bugtracker:
29
30Yocto Project Bugzilla
31======================
32
33The Yocto Project uses its own implementation of
34`Bugzilla <&YOCTO_BUGZILLA_URL;>`__ to track defects (bugs).
35Implementations of Bugzilla work well for group development because they
36track bugs and code changes, can be used to communicate changes and
37problems with developers, can be used to submit and review patches, and
38can be used to manage quality assurance.
39
40Sometimes it is helpful to submit, investigate, or track a bug against
41the Yocto Project itself (e.g. when discovering an issue with some
42component of the build system that acts contrary to the documentation or
43your expectations).
44
45A general procedure and guidelines exist for when you use Bugzilla to
46submit a bug. For information on how to use Bugzilla to submit a bug
47against the Yocto Project, see the following:
48
49- The "`Submitting a Defect Against the Yocto
50 Project <&YOCTO_DOCS_DEV_URL;#submitting-a-defect-against-the-yocto-project>`__"
51 section in the Yocto Project Development Tasks Manual.
52
53- The Yocto Project `Bugzilla wiki
54 page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__
55
56For information on Bugzilla in general, see
57` <http://www.bugzilla.org/about/>`__.
58
59.. _resources-mailinglist:
60
61Mailing lists
62=============
63
64A number of mailing lists maintained by the Yocto Project exist as well
65as related OpenEmbedded mailing lists for discussion, patch submission
66and announcements. To subscribe to one of the following mailing lists,
67click on the appropriate URL in the following list and follow the
68instructions:
69
70- ` <&YOCTO_LISTS_URL;/listinfo/yocto>`__ - General Yocto Project
71 discussion mailing list.
72
73- ` <&OE_LISTS_URL;/listinfo/openembedded-core>`__ - Discussion mailing
74 list about OpenEmbedded-Core (the core metadata).
75
76- ` <&OE_LISTS_URL;/listinfo/openembedded-devel>`__ - Discussion
77 mailing list about OpenEmbedded.
78
79- ` <&OE_LISTS_URL;/listinfo/bitbake-devel>`__ - Discussion mailing
80 list about the `BitBake <#bitbake-term>`__ build tool.
81
82- ` <&YOCTO_LISTS_URL;/listinfo/poky>`__ - Discussion mailing list
83 about `Poky <#poky>`__.
84
85- ` <&YOCTO_LISTS_URL;/listinfo/yocto-announce>`__ - Mailing list to
86 receive official Yocto Project release and milestone announcements.
87
88For more Yocto Project-related mailing lists, see the
89Yocto Project Website
90.
91.. _resources-irc:
92
93Internet Relay Chat (IRC)
94=========================
95
96Two IRC channels on freenode are available for the Yocto Project and
97Poky discussions:
98
99- ``#yocto``
100
101- ``#poky``
102
103.. _resources-links-and-related-documentation:
104
105Links and Related Documentation
106===============================
107
108Here is a list of resources you might find helpful:
109
110- `The Yocto Project website <&YOCTO_HOME_URL;>`__\ *:* The home site
111 for the Yocto Project.
112
113- `The Yocto Project Main Wiki
114 Page <&YOCTO_WIKI_URL;/wiki/Main_Page>`__\ *:* The main wiki page for
115 the Yocto Project. This page contains information about project
116 planning, release engineering, QA & automation, a reference site map,
117 and other resources related to the Yocto Project.
118
119- `OpenEmbedded <&OE_HOME_URL;>`__\ *:* The build system used by the
120 Yocto Project. This project is the upstream, generic, embedded
121 distribution from which the Yocto Project derives its build system
122 (Poky) and to which it contributes.
123
124- `BitBake <http://www.openembedded.org/wiki/BitBake>`__\ *:* The tool
125 used to process metadata.
126
127- `BitBake User Manual <&YOCTO_DOCS_BB_URL;>`__\ *:* A comprehensive
128 guide to the BitBake tool. If you want information on BitBake, see
129 this manual.
130
131- `Yocto Project Quick Build <&YOCTO_DOCS_BRIEF_URL;>`__\ *:* This
132 short document lets you experience building an image using the Yocto
133 Project without having to understand any concepts or details.
134
135- `Yocto Project Overview and Concepts
136 Manual <&YOCTO_DOCS_OM_URL;>`__\ *:* This manual provides overview
137 and conceptual information about the Yocto Project.
138
139- `Yocto Project Development Tasks
140 Manual <&YOCTO_DOCS_DEV_URL;>`__\ *:* This manual is a "how-to" guide
141 that presents procedures useful to both application and system
142 developers who use the Yocto Project.
143
144- `Yocto Project Application Development and the Extensible Software
145 Development Kit (eSDK) <&YOCTO_DOCS_SDK_URL;>`__\ *manual:* This
146 guide provides information that lets you get going with the standard
147 or extensible SDK. An SDK, with its cross-development toolchains,
148 allows you to develop projects inside or outside of the Yocto Project
149 environment.
150
151- `Yocto Project Board Support Package (BSP) Developer's
152 Guide <&YOCTO_DOCS_BSP_URL;>`__\ *:* This guide defines the structure
153 for BSP components. Having a commonly understood structure encourages
154 standardization.
155
156- `Yocto Project Linux Kernel Development
157 Manual <&YOCTO_DOCS_KERNEL_DEV_URL;>`__\ *:* This manual describes
158 how to work with Linux Yocto kernels as well as provides a bit of
159 conceptual information on the construction of the Yocto Linux kernel
160 tree.
161
162- `Yocto Project Reference Manual <&YOCTO_DOCS_REF_URL;>`__\ *:* This
163 manual provides reference material such as variable, task, and class
164 descriptions.
165
166- `Yocto Project Mega-Manual <&YOCTO_DOCS_MM_URL;>`__\ *:* This manual
167 is simply a single HTML file comprised of the bulk of the Yocto
168 Project manuals. The Mega-Manual primarily exists as a vehicle by
169 which you can easily search for phrases and terms used in the Yocto
170 Project documentation set.
171
172- `Yocto Project Profiling and Tracing
173 Manual <&YOCTO_DOCS_PROF_URL;>`__\ *:* This manual presents a set of
174 common and generally useful tracing and profiling schemes along with
175 their applications (as appropriate) to each tool.
176
177- `Toaster User Manual <&YOCTO_DOCS_TOAST_URL;>`__\ *:* This manual
178 introduces and describes how to set up and use Toaster. Toaster is an
179 Application Programming Interface (API) and web-based interface to
180 the `OpenEmbedded Build System <#build-system-term>`__, which uses
181 BitBake, that reports build information.
182
183- `FAQ <&YOCTO_WIKI_URL;/wiki/FAQ>`__\ *:* A list of commonly asked
184 questions and their answers.
185
186- *Release Notes:* Features, updates and known issues for the current
187 release of the Yocto Project. To access the Release Notes, go to the
188 `Downloads <&YOCTO_HOME_URL;/software-overview/downloads/>`__ page on
189 the Yocto Project website and click on the "RELEASE INFORMATION" link
190 for the appropriate release.
191
192- `Bugzilla <&YOCTO_BUGZILLA_URL;>`__\ *:* The bug tracking application
193 the Yocto Project uses. If you find problems with the Yocto Project,
194 you should report them using this application.
195
196- `Bugzilla Configuration and Bug Tracking Wiki
197 Page <&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking>`__\ *:*
198 Information on how to get set up and use the Yocto Project
199 implementation of Bugzilla for logging and tracking Yocto Project
200 defects.
201
202- *Internet Relay Chat (IRC):* Two IRC channels on freenode are
203 available for Yocto Project and Poky discussions: ``#yocto`` and
204 ``#poky``, respectively.
205
206- `Quick EMUlator (QEMU) <http://wiki.qemu.org/Index.html>`__\ *:* An
207 open-source machine emulator and virtualizer.
diff --git a/documentation/sdk-manual/sdk-appendix-customizing-standard.rst b/documentation/sdk-manual/sdk-appendix-customizing-standard.rst
new file mode 100644
index 0000000000..4c61925725
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-customizing-standard.rst
@@ -0,0 +1,32 @@
1****************************
2Customizing the Standard SDK
3****************************
4
5This appendix presents customizations you can apply to the standard SDK.
6
7Adding Individual Packages to the Standard SDK
8==============================================
9
10When you build a standard SDK using the ``bitbake -c populate_sdk``, a
11default set of packages is included in the resulting SDK. The
12```TOOLCHAIN_HOST_TASK`` <&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_HOST_TASK>`__
13and
14```TOOLCHAIN_TARGET_TASK`` <&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_TARGET_TASK>`__
15variables control the set of packages adding to the SDK.
16
17If you want to add individual packages to the toolchain that runs on the
18host, simply add those packages to the ``TOOLCHAIN_HOST_TASK`` variable.
19Similarly, if you want to add packages to the default set that is part
20of the toolchain that runs on the target, add the packages to the
21``TOOLCHAIN_TARGET_TASK`` variable.
22
23Adding API Documentation to the Standard SDK
24============================================
25
26You can include API documentation as well as any other documentation
27provided by recipes with the standard SDK by adding "api-documentation"
28to the
29```DISTRO_FEATURES`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES>`__
30variable: DISTRO_FEATURES_append = " api-documentation" Setting this
31variable as shown here causes the OpenEmbedded build system to build the
32documentation and then include it in the standard SDK.
diff --git a/documentation/sdk-manual/sdk-appendix-customizing.rst b/documentation/sdk-manual/sdk-appendix-customizing.rst
new file mode 100644
index 0000000000..b613a12f23
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-customizing.rst
@@ -0,0 +1,347 @@
1******************************
2Customizing the Extensible SDK
3******************************
4
5This appendix describes customizations you can apply to the extensible
6SDK.
7
8Configuring the Extensible SDK
9==============================
10
11The extensible SDK primarily consists of a pre-configured copy of the
12OpenEmbedded build system from which it was produced. Thus, the SDK's
13configuration is derived using that build system and the filters shown
14in the following list. When these filters are present, the OpenEmbedded
15build system applies them against ``local.conf`` and ``auto.conf``:
16
17- Variables whose values start with "/" are excluded since the
18 assumption is that those values are paths that are likely to be
19 specific to the `build
20 host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__.
21
22- Variables listed in
23 ```SDK_LOCAL_CONF_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_BLACKLIST>`__
24 are excluded. These variables are not allowed through from the
25 OpenEmbedded build system configuration into the extensible SDK
26 configuration. Typically, these variables are specific to the machine
27 on which the build system is running and could be problematic as part
28 of the extensible SDK configuration.
29
30 For a list of the variables excluded by default, see the
31 ```SDK_LOCAL_CONF_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_BLACKLIST>`__
32 in the glossary of the Yocto Project Reference Manual.
33
34- Variables listed in
35 ```SDK_LOCAL_CONF_WHITELIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_WHITELIST>`__
36 are included. Including a variable in the value of
37 ``SDK_LOCAL_CONF_WHITELIST`` overrides either of the previous two
38 filters. The default value is blank.
39
40- Classes inherited globally with
41 ```INHERIT`` <&YOCTO_DOCS_REF_URL;#var-INHERIT>`__ that are listed in
42 ```SDK_INHERIT_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_INHERIT_BLACKLIST>`__
43 are disabled. Using ``SDK_INHERIT_BLACKLIST`` to disable these
44 classes is the typical method to disable classes that are problematic
45 or unnecessary in the SDK context. The default value blacklists the
46 ```buildhistory`` <&YOCTO_DOCS_REF_URL;#ref-classes-buildhistory>`__
47 and ```icecc`` <&YOCTO_DOCS_REF_URL;#ref-classes-icecc>`__ classes.
48
49Additionally, the contents of ``conf/sdk-extra.conf``, when present, are
50appended to the end of ``conf/local.conf`` within the produced SDK,
51without any filtering. The ``sdk-extra.conf`` file is particularly
52useful if you want to set a variable value just for the SDK and not the
53OpenEmbedded build system used to create the SDK.
54
55Adjusting the Extensible SDK to Suit Your Build Host's Setup
56============================================================
57
58In most cases, the extensible SDK defaults should work with your `build
59host's <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ setup.
60However, some cases exist for which you might consider making
61adjustments:
62
63- If your SDK configuration inherits additional classes using the
64 ```INHERIT`` <&YOCTO_DOCS_REF_URL;#var-INHERIT>`__ variable and you
65 do not need or want those classes enabled in the SDK, you can
66 blacklist them by adding them to the
67 ```SDK_INHERIT_BLACKLIST`` <&YOCTO_DOCS_REF_URL;#var-SDK_INHERIT_BLACKLIST>`__
68 variable as described in the fourth bullet of the previous section.
69
70 .. note::
71
72 The default value of
73 SDK_INHERIT_BLACKLIST
74 is set using the "?=" operator. Consequently, you will need to
75 either define the entire list by using the "=" operator, or you
76 will need to append a value using either "_append" or the "+="
77 operator. You can learn more about these operators in the "
78 Basic Syntax
79 " section of the BitBake User Manual.
80
81 .
82
83- If you have classes or recipes that add additional tasks to the
84 standard build flow (i.e. the tasks execute as the recipe builds as
85 opposed to being called explicitly), then you need to do one of the
86 following:
87
88 - After ensuring the tasks are `shared
89 state <&YOCTO_DOCS_OM_URL;#shared-state-cache>`__ tasks (i.e. the
90 output of the task is saved to and can be restored from the shared
91 state cache) or ensuring the tasks are able to be produced quickly
92 from a task that is a shared state task, add the task name to the
93 value of
94 ```SDK_RECRDEP_TASKS`` <&YOCTO_DOCS_REF_URL;#var-SDK_RECRDEP_TASKS>`__.
95
96 - Disable the tasks if they are added by a class and you do not need
97 the functionality the class provides in the extensible SDK. To
98 disable the tasks, add the class to the ``SDK_INHERIT_BLACKLIST``
99 variable as described in the previous section.
100
101- Generally, you want to have a shared state mirror set up so users of
102 the SDK can add additional items to the SDK after installation
103 without needing to build the items from source. See the "`Providing
104 Additional Installable Extensible SDK
105 Content <#sdk-providing-additional-installable-extensible-sdk-content>`__"
106 section for information.
107
108- If you want users of the SDK to be able to easily update the SDK, you
109 need to set the
110 ```SDK_UPDATE_URL`` <&YOCTO_DOCS_REF_URL;#var-SDK_UPDATE_URL>`__
111 variable. For more information, see the "`Providing Updates to the
112 Extensible SDK After
113 Installation <#sdk-providing-updates-to-the-extensible-sdk-after-installation>`__"
114 section.
115
116- If you have adjusted the list of files and directories that appear in
117 ```COREBASE`` <&YOCTO_DOCS_REF_URL;#var-COREBASE>`__ (other than
118 layers that are enabled through ``bblayers.conf``), then you must
119 list these files in
120 ```COREBASE_FILES`` <&YOCTO_DOCS_REF_URL;#var-COREBASE_FILES>`__ so
121 that the files are copied into the SDK.
122
123- If your OpenEmbedded build system setup uses a different environment
124 setup script other than
125 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__, then you must
126 set
127 ```OE_INIT_ENV_SCRIPT`` <&YOCTO_DOCS_REF_URL;#var-OE_INIT_ENV_SCRIPT>`__
128 to point to the environment setup script you use.
129
130 .. note::
131
132 You must also reflect this change in the value used for the
133 COREBASE_FILES
134 variable as previously described.
135
136Changing the Extensible SDK Installer Title
137===========================================
138
139You can change the displayed title for the SDK installer by setting the
140```SDK_TITLE`` <&YOCTO_DOCS_REF_URL;#var-SDK_TITLE>`__ variable and then
141rebuilding the the SDK installer. For information on how to build an SDK
142installer, see the "`Building an SDK
143Installer <#sdk-building-an-sdk-installer>`__" section.
144
145By default, this title is derived from
146```DISTRO_NAME`` <&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME>`__ when it is
147set. If the ``DISTRO_NAME`` variable is not set, the title is derived
148from the ```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__ variable.
149
150The
151```populate_sdk_base`` <&YOCTO_DOCS_REF_URL;#ref-classes-populate-sdk-*>`__
152class defines the default value of the ``SDK_TITLE`` variable as
153follows: SDK_TITLE ??= "${@d.getVar('DISTRO_NAME') or
154d.getVar('DISTRO')} SDK"
155
156While several ways exist to change this variable, an efficient method is
157to set the variable in your distribution's configuration file. Doing so
158creates an SDK installer title that applies across your distribution. As
159an example, assume you have your own layer for your distribution named
160"meta-mydistro" and you are using the same type of file hierarchy as
161does the default "poky" distribution. If so, you could update the
162``SDK_TITLE`` variable in the
163``~/meta-mydistro/conf/distro/mydistro.conf`` file using the following
164form: SDK_TITLE = "your_title"
165
166Providing Updates to the Extensible SDK After Installation
167==========================================================
168
169When you make changes to your configuration or to the metadata and if
170you want those changes to be reflected in installed SDKs, you need to
171perform additional steps. These steps make it possible for anyone using
172the installed SDKs to update the installed SDKs by using the
173``devtool sdk-update`` command:
174
1751. Create a directory that can be shared over HTTP or HTTPS. You can do
176 this by setting up a web server such as an `Apache HTTP
177 Server <https://en.wikipedia.org/wiki/Apache_HTTP_Server>`__ or
178 `Nginx <https://en.wikipedia.org/wiki/Nginx>`__ server in the cloud
179 to host the directory. This directory must contain the published SDK.
180
1812. Set the
182 ```SDK_UPDATE_URL`` <&YOCTO_DOCS_REF_URL;#var-SDK_UPDATE_URL>`__
183 variable to point to the corresponding HTTP or HTTPS URL. Setting
184 this variable causes any SDK built to default to that URL and thus,
185 the user does not have to pass the URL to the ``devtool sdk-update``
186 command as described in the "`Applying Updates to an Installed
187 Extensible
188 SDK <#sdk-applying-updates-to-an-installed-extensible-sdk>`__"
189 section.
190
1913. Build the extensible SDK normally (i.e., use the
192 ``bitbake -c populate_sdk_ext`` imagename command).
193
1944. Publish the SDK using the following command: $ oe-publish-sdk
195 some_path/sdk-installer.sh path_to_shared_http_directory You must
196 repeat this step each time you rebuild the SDK with changes that you
197 want to make available through the update mechanism.
198
199Completing the above steps allows users of the existing installed SDKs
200to simply run ``devtool sdk-update`` to retrieve and apply the latest
201updates. See the "`Applying Updates to an Installed Extensible
202SDK <#sdk-applying-updates-to-an-installed-extensible-sdk>`__" section
203for further information.
204
205Changing the Default SDK Installation Directory
206===============================================
207
208When you build the installer for the Extensible SDK, the default
209installation directory for the SDK is based on the
210```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__ and
211```SDKEXTPATH`` <&YOCTO_DOCS_REF_URL;#var-SDKEXTPATH>`__ variables from
212within the
213```populate_sdk_base`` <&YOCTO_DOCS_REF_URL;#ref-classes-populate-sdk-*>`__
214class as follows: SDKEXTPATH ??= "~/${@d.getVar('DISTRO')}_sdk" You can
215change this default installation directory by specifically setting the
216``SDKEXTPATH`` variable.
217
218While a number of ways exist through which you can set this variable,
219the method that makes the most sense is to set the variable in your
220distribution's configuration file. Doing so creates an SDK installer
221default directory that applies across your distribution. As an example,
222assume you have your own layer for your distribution named
223"meta-mydistro" and you are using the same type of file hierarchy as
224does the default "poky" distribution. If so, you could update the
225``SDKEXTPATH`` variable in the
226``~/meta-mydistro/conf/distro/mydistro.conf`` file using the following
227form: SDKEXTPATH = "some_path_for_your_installed_sdk"
228
229After building your installer, running it prompts the user for
230acceptance of the some_path_for_your_installed_sdk directory as the
231default location to install the Extensible SDK.
232
233Providing Additional Installable Extensible SDK Content
234=======================================================
235
236If you want the users of an extensible SDK you build to be able to add
237items to the SDK without requiring the users to build the items from
238source, you need to do a number of things:
239
2401. Ensure the additional items you want the user to be able to install
241 are already built:
242
243 - Build the items explicitly. You could use one or more "meta"
244 recipes that depend on lists of other recipes.
245
246 - Build the "world" target and set
247 ``EXCLUDE_FROM_WORLD_pn-``\ recipename for the recipes you do not
248 want built. See the
249 ```EXCLUDE_FROM_WORLD`` <&YOCTO_DOCS_REF_URL;#var-EXCLUDE_FROM_WORLD>`__
250 variable for additional information.
251
2522. Expose the ``sstate-cache`` directory produced by the build.
253 Typically, you expose this directory by making it available through
254 an `Apache HTTP
255 Server <https://en.wikipedia.org/wiki/Apache_HTTP_Server>`__ or
256 `Nginx <https://en.wikipedia.org/wiki/Nginx>`__ server.
257
2583. Set the appropriate configuration so that the produced SDK knows how
259 to find the configuration. The variable you need to set is
260 ```SSTATE_MIRRORS`` <&YOCTO_DOCS_REF_URL;#var-SSTATE_MIRRORS>`__:
261 SSTATE_MIRRORS = "file://.\*
262 http://example.com/some_path/sstate-cache/PATH" You can set the
263 ``SSTATE_MIRRORS`` variable in two different places:
264
265 - If the mirror value you are setting is appropriate to be set for
266 both the OpenEmbedded build system that is actually building the
267 SDK and the SDK itself (i.e. the mirror is accessible in both
268 places or it will fail quickly on the OpenEmbedded build system
269 side, and its contents will not interfere with the build), then
270 you can set the variable in your ``local.conf`` or custom distro
271 configuration file. You can then "whitelist" the variable through
272 to the SDK by adding the following: SDK_LOCAL_CONF_WHITELIST =
273 "SSTATE_MIRRORS"
274
275 - Alternatively, if you just want to set the ``SSTATE_MIRRORS``
276 variable's value for the SDK alone, create a
277 ``conf/sdk-extra.conf`` file either in your `Build
278 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ or within any
279 layer and put your ``SSTATE_MIRRORS`` setting within that file.
280
281 .. note::
282
283 This second option is the safest option should you have any
284 doubts as to which method to use when setting
285 SSTATE_MIRRORS
286 .
287
288Minimizing the Size of the Extensible SDK Installer Download
289============================================================
290
291By default, the extensible SDK bundles the shared state artifacts for
292everything needed to reconstruct the image for which the SDK was built.
293This bundling can lead to an SDK installer file that is a Gigabyte or
294more in size. If the size of this file causes a problem, you can build
295an SDK that has just enough in it to install and provide access to the
296``devtool command`` by setting the following in your configuration:
297SDK_EXT_TYPE = "minimal" Setting
298```SDK_EXT_TYPE`` <&YOCTO_DOCS_REF_URL;#var-SDK_EXT_TYPE>`__ to
299"minimal" produces an SDK installer that is around 35 Mbytes in size,
300which downloads and installs quickly. You need to realize, though, that
301the minimal installer does not install any libraries or tools out of the
302box. These libraries and tools must be installed either "on the fly" or
303through actions you perform using ``devtool`` or explicitly with the
304``devtool sdk-install`` command.
305
306In most cases, when building a minimal SDK you need to also enable
307bringing in the information on a wider range of packages produced by the
308system. Requiring this wider range of information is particularly true
309so that ``devtool add`` is able to effectively map dependencies it
310discovers in a source tree to the appropriate recipes. Additionally, the
311information enables the ``devtool search`` command to return useful
312results.
313
314To facilitate this wider range of information, you would need to set the
315following: SDK_INCLUDE_PKGDATA = "1" See the
316```SDK_INCLUDE_PKGDATA`` <&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_PKGDATA>`__
317variable for additional information.
318
319Setting the ``SDK_INCLUDE_PKGDATA`` variable as shown causes the "world"
320target to be built so that information for all of the recipes included
321within it are available. Having these recipes available increases build
322time significantly and increases the size of the SDK installer by 30-80
323Mbytes depending on how many recipes are included in your configuration.
324
325You can use ``EXCLUDE_FROM_WORLD_pn-``\ recipename for recipes you want
326to exclude. However, it is assumed that you would need to be building
327the "world" target if you want to provide additional items to the SDK.
328Consequently, building for "world" should not represent undue overhead
329in most cases.
330
331.. note::
332
333 If you set
334 SDK_EXT_TYPE
335 to "minimal", then providing a shared state mirror is mandatory so
336 that items can be installed as needed. See the "
337 Providing Additional Installable Extensible SDK Content
338 " section for more information.
339
340You can explicitly control whether or not to include the toolchain when
341you build an SDK by setting the
342```SDK_INCLUDE_TOOLCHAIN`` <&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_TOOLCHAIN>`__
343variable to "1". In particular, it is useful to include the toolchain
344when you have set ``SDK_EXT_TYPE`` to "minimal", which by default,
345excludes the toolchain. Also, it is helpful if you are building a small
346SDK for use with an IDE or some other tool where you do not want to take
347extra steps to install a toolchain.
diff --git a/documentation/sdk-manual/sdk-appendix-obtain.rst b/documentation/sdk-manual/sdk-appendix-obtain.rst
new file mode 100644
index 0000000000..506f5cffb7
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-obtain.rst
@@ -0,0 +1,270 @@
1*****************
2Obtaining the SDK
3*****************
4
5.. _sdk-locating-pre-built-sdk-installers:
6
7Locating Pre-Built SDK Installers
8=================================
9
10You can use existing, pre-built toolchains by locating and running an
11SDK installer script that ships with the Yocto Project. Using this
12method, you select and download an architecture-specific SDK installer
13and then run the script to hand-install the toolchain.
14
15Follow these steps to locate and hand-install the toolchain:
16
171. *Go to the Installers Directory:* Go to
18 ` <&YOCTO_TOOLCHAIN_DL_URL;>`__
19
202. *Open the Folder for Your Build Host:* Open the folder that matches
21 your `build host <&YOCTO_DOCS_REF_URL;#build-system-term>`__ (i.e.
22 ``i686`` for 32-bit machines or ``x86_64`` for 64-bit machines).
23
243. *Locate and Download the SDK Installer:* You need to find and
25 download the installer appropriate for your build host, target
26 hardware, and image type.
27
28 The installer files (``*.sh``) follow this naming convention:
29 poky-glibc-host_system-core-image-type-arch-toolchain[-ext]-release.sh
30 Where: host_system is a string representing your development system:
31 "i686" or "x86_64" type is a string representing the image: "sato" or
32 "minimal" arch is a string representing the target architecture:
33 "aarch64", "armv5e", "core2-64", "coretexa8hf-neon", "i586",
34 "mips32r2", "mips64", or "ppc7400" release is the version of Yocto
35 Project. NOTE: The standard SDK installer does not have the "-ext"
36 string as part of the filename. The toolchains provided by the Yocto
37 Project are based off of the ``core-image-sato`` and
38 ``core-image-minimal`` images and contain libraries appropriate for
39 developing against those images.
40
41 For example, if your build host is a 64-bit x86 system and you need
42 an extended SDK for a 64-bit core2 target, go into the ``x86_64``
43 folder and download the following installer:
44 poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
45
464. *Run the Installer:* Be sure you have execution privileges and run
47 the installer. Following is an example from the ``Downloads``
48 directory: $
49 ~/Downloads/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
50 During execution of the script, you choose the root location for the
51 toolchain. See the "`Installed Standard SDK Directory
52 Structure <#sdk-installed-standard-sdk-directory-structure>`__"
53 section and the "`Installed Extensible SDK Directory
54 Structure <#sdk-installed-extensible-sdk-directory-structure>`__"
55 section for more information.
56
57Building an SDK Installer
58=========================
59
60As an alternative to locating and downloading an SDK installer, you can
61build the SDK installer. Follow these steps:
62
631. *Set Up the Build Environment:* Be sure you are set up to use BitBake
64 in a shell. See the "`Preparing the Build
65 Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section
66 in the Yocto Project Development Tasks Manual for information on how
67 to get a build host ready that is either a native Linux machine or a
68 machine that uses CROPS.
69
702. *Clone the ``poky`` Repository:* You need to have a local copy of the
71 Yocto Project `Source
72 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (i.e. a local
73 ``poky`` repository). See the "`Cloning the ``poky``
74 Repository <&YOCTO_DOCS_DEV_URL;#cloning-the-poky-repository>`__" and
75 possibly the "`Checking Out by Branch in
76 Poky <&YOCTO_DOCS_DEV_URL;#checking-out-by-branch-in-poky>`__" and
77 "`Checking Out by Tag in
78 Poky <&YOCTO_DOCS_DEV_URL;#checkout-out-by-tag-in-poky>`__" sections
79 all in the Yocto Project Development Tasks Manual for information on
80 how to clone the ``poky`` repository and check out the appropriate
81 branch for your work.
82
833. *Initialize the Build Environment:* While in the root directory of
84 the Source Directory (i.e. ``poky``), run the
85 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ environment
86 setup script to define the OpenEmbedded build environment on your
87 build host. $ source OE_INIT_FILE Among other things, the script
88 creates the `Build
89 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__, which is
90 ``build`` in this case and is located in the Source Directory. After
91 the script runs, your current working directory is set to the
92 ``build`` directory.
93
944. *Make Sure You Are Building an Installer for the Correct Machine:*
95 Check to be sure that your
96 ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__ variable in the
97 ``local.conf`` file in your Build Directory matches the architecture
98 for which you are building.
99
1005. *Make Sure Your SDK Machine is Correctly Set:* If you are building a
101 toolchain designed to run on an architecture that differs from your
102 current development host machine (i.e. the build host), be sure that
103 the ```SDKMACHINE`` <&YOCTO_DOCS_REF_URL;#var-SDKMACHINE>`__ variable
104 in the ``local.conf`` file in your Build Directory is correctly set.
105
106 .. note::
107
108 If you are building an SDK installer for the Extensible SDK, the
109 SDKMACHINE
110 value must be set for the architecture of the machine you are
111 using to build the installer. If
112 SDKMACHINE
113 is not set appropriately, the build fails and provides an error
114 message similar to the following:
115 ::
116
117 The extensible SDK can currently only be built for the same architecture as the machine being built on - SDK_ARCH is
118 set to i686 (likely via setting SDKMACHINE) which is different from the architecture of the build machine (x86_64).
119 Unable to continue.
120
121
1226. *Build the SDK Installer:* To build the SDK installer for a standard
123 SDK and populate the SDK image, use the following command form. Be
124 sure to replace image with an image (e.g. "core-image-sato"): $
125 bitbake image -c populate_sdk You can do the same for the extensible
126 SDK using this command form: $ bitbake image -c populate_sdk_ext
127 These commands produce an SDK installer that contains the sysroot
128 that matches your target root filesystem.
129
130 When the ``bitbake`` command completes, the SDK installer will be in
131 ``tmp/deploy/sdk`` in the Build Directory.
132
133 .. note::
134
135 - By default, the previous BitBake command does not build static
136 binaries. If you want to use the toolchain to build these types
137 of libraries, you need to be sure your SDK has the appropriate
138 static development libraries. Use the
139 ```TOOLCHAIN_TARGET_TASK`` <&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_TARGET_TASK>`__
140 variable inside your ``local.conf`` file before building the
141 SDK installer. Doing so ensures that the eventual SDK
142 installation process installs the appropriate library packages
143 as part of the SDK. Following is an example using ``libc``
144 static development libraries: TOOLCHAIN_TARGET_TASK_append = "
145 libc-staticdev"
146
1477. *Run the Installer:* You can now run the SDK installer from
148 ``tmp/deploy/sdk`` in the Build Directory. Following is an example: $
149 cd ~/poky/build/tmp/deploy/sdk $
150 ./poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
151 During execution of the script, you choose the root location for the
152 toolchain. See the "`Installed Standard SDK Directory
153 Structure <#sdk-installed-standard-sdk-directory-structure>`__"
154 section and the "`Installed Extensible SDK Directory
155 Structure <#sdk-installed-extensible-sdk-directory-structure>`__"
156 section for more information.
157
158Extracting the Root Filesystem
159==============================
160
161After installing the toolchain, for some use cases you might need to
162separately extract a root filesystem:
163
164- You want to boot the image using NFS.
165
166- You want to use the root filesystem as the target sysroot.
167
168- You want to develop your target application using the root filesystem
169 as the target sysroot.
170
171Follow these steps to extract the root filesystem:
172
1731. *Locate and Download the Tarball for the Pre-Built Root Filesystem
174 Image File:* You need to find and download the root filesystem image
175 file that is appropriate for your target system. These files are kept
176 in machine-specific folders in the `Index of
177 Releases <&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;/machines/>`__
178 in the "machines" directory.
179
180 The machine-specific folders of the "machines" directory contain
181 tarballs (``*.tar.bz2``) for supported machines. These directories
182 also contain flattened root filesystem image files (``*.ext4``),
183 which you can use with QEMU directly.
184
185 The pre-built root filesystem image files follow these naming
186 conventions: core-image-profile-arch.tar.bz2 Where: profile is the
187 filesystem image's profile: lsb, lsb-dev, lsb-sdk, minimal,
188 minimal-dev, minimal-initramfs, sato, sato-dev, sato-sdk,
189 sato-sdk-ptest. For information on these types of image profiles, see
190 the "`Images <&YOCTO_DOCS_REF_URL;#ref-images>`__" chapter in the
191 Yocto Project Reference Manual. arch is a string representing the
192 target architecture: beaglebone-yocto, beaglebone-yocto-lsb,
193 edgerouter, edgerouter-lsb, genericx86, genericx86-64,
194 genericx86-64-lsb, genericx86-lsb and qemu*. The root filesystems
195 provided by the Yocto Project are based off of the
196 ``core-image-sato`` and ``core-image-minimal`` images.
197
198 For example, if you plan on using a BeagleBone device as your target
199 hardware and your image is a ``core-image-sato-sdk`` image, you can
200 download the following file:
201 core-image-sato-sdk-beaglebone-yocto.tar.bz2
202
2032. *Initialize the Cross-Development Environment:* You must ``source``
204 the cross-development environment setup script to establish necessary
205 environment variables.
206
207 This script is located in the top-level directory in which you
208 installed the toolchain (e.g. ``poky_sdk``).
209
210 Following is an example based on the toolchain installed in the
211 "`Locating Pre-Built SDK
212 Installers <#sdk-locating-pre-built-sdk-installers>`__" section: $
213 source ~/poky_sdk/environment-setup-core2-64-poky-linux
214
2153. *Extract the Root Filesystem:* Use the ``runqemu-extract-sdk``
216 command and provide the root filesystem image.
217
218 Following is an example command that extracts the root filesystem
219 from a previously built root filesystem image that was downloaded
220 from the `Index of Releases <&YOCTO_DOCS_OM_URL;#index-downloads>`__.
221 This command extracts the root filesystem into the ``core2-64-sato``
222 directory: $ runqemu-extract-sdk
223 ~/Downloads/core-image-sato-sdk-beaglebone-yocto.tar.bz2
224 ~/beaglebone-sato You could now point to the target sysroot at
225 ``beablebone-sato``.
226
227Installed Standard SDK Directory Structure
228==========================================
229
230The following figure shows the resulting directory structure after you
231install the Standard SDK by running the ``*.sh`` SDK installation
232script:
233
234The installed SDK consists of an environment setup script for the SDK, a
235configuration file for the target, a version file for the target, and
236the root filesystem (``sysroots``) needed to develop objects for the
237target system.
238
239Within the figure, italicized text is used to indicate replaceable
240portions of the file or directory name. For example, install_dir/version
241is the directory where the SDK is installed. By default, this directory
242is ``/opt/poky/``. And, version represents the specific snapshot of the
243SDK (e.g. ````). Furthermore, target represents the target architecture
244(e.g. ``i586``) and host represents the development system's
245architecture (e.g. ``x86_64``). Thus, the complete names of the two
246directories within the ``sysroots`` could be ``i586-poky-linux`` and
247``x86_64-pokysdk-linux`` for the target and host, respectively.
248
249Installed Extensible SDK Directory Structure
250============================================
251
252The following figure shows the resulting directory structure after you
253install the Extensible SDK by running the ``*.sh`` SDK installation
254script:
255
256The installed directory structure for the extensible SDK is quite
257different than the installed structure for the standard SDK. The
258extensible SDK does not separate host and target parts in the same
259manner as does the standard SDK. The extensible SDK uses an embedded
260copy of the OpenEmbedded build system, which has its own sysroots.
261
262Of note in the directory structure are an environment setup script for
263the SDK, a configuration file for the target, a version file for the
264target, and log files for the OpenEmbedded build system preparation
265script run by the installer and BitBake.
266
267Within the figure, italicized text is used to indicate replaceable
268portions of the file or directory name. For example, install_dir is the
269directory where the SDK is installed, which is ``poky_sdk`` by default,
270and target represents the target architecture (e.g. ``i586``).
diff --git a/documentation/sdk-manual/sdk-extensible.rst b/documentation/sdk-manual/sdk-extensible.rst
new file mode 100644
index 0000000000..db6bfb4394
--- /dev/null
+++ b/documentation/sdk-manual/sdk-extensible.rst
@@ -0,0 +1,1230 @@
1************************
2Using the Extensible SDK
3************************
4
5This chapter describes the extensible SDK and how to install it.
6Information covers the pieces of the SDK, how to install it, and
7presents a look at using the ``devtool`` functionality. The extensible
8SDK makes it easy to add new applications and libraries to an image,
9modify the source for an existing component, test changes on the target
10hardware, and ease integration into the rest of the `OpenEmbedded build
11system <&YOCTO_DOCS_REF_URL;#build-system-term>`__.
12
13.. note::
14
15 For a side-by-side comparison of main features supported for an
16 extensible SDK as compared to a standard SDK, see the "
17 Introduction
18 " section.
19
20In addition to the functionality available through ``devtool``, you can
21alternatively make use of the toolchain directly, for example from
22Makefile and Autotools. See the "`Using the SDK Toolchain
23Directly <#sdk-working-projects>`__" chapter for more information.
24
25.. _sdk-extensible-sdk-intro:
26
27Why use the Extensible SDK and What is in It?
28=============================================
29
30The extensible SDK provides a cross-development toolchain and libraries
31tailored to the contents of a specific image. You would use the
32Extensible SDK if you want a toolchain experience supplemented with the
33powerful set of ``devtool`` commands tailored for the Yocto Project
34environment.
35
36The installed extensible SDK consists of several files and directories.
37Basically, it contains an SDK environment setup script, some
38configuration files, an internal build system, and the ``devtool``
39functionality.
40
41.. _sdk-installing-the-extensible-sdk:
42
43Installing the Extensible SDK
44=============================
45
46The first thing you need to do is install the SDK on your `Build
47Host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ by running the
48``*.sh`` installation script.
49
50You can download a tarball installer, which includes the pre-built
51toolchain, the ``runqemu`` script, the internal build system,
52``devtool``, and support files from the appropriate
53`toolchain <&YOCTO_TOOLCHAIN_DL_URL;>`__ directory within the Index of
54Releases. Toolchains are available for several 32-bit and 64-bit
55architectures with the ``x86_64`` directories, respectively. The
56toolchains the Yocto Project provides are based off the
57``core-image-sato`` and ``core-image-minimal`` images and contain
58libraries appropriate for developing against that image.
59
60The names of the tarball installer scripts are such that a string
61representing the host system appears first in the filename and then is
62immediately followed by a string representing the target architecture.
63An extensible SDK has the string "-ext" as part of the name. Following
64is the general form:
65poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh
66Where: host_system is a string representing your development system:
67i686 or x86_64. image_type is the image for which the SDK was built:
68core-image-sato or core-image-minimal arch is a string representing the
69tuned target architecture: aarch64, armv5e, core2-64, i586, mips32r2,
70mips64, ppc7400, or cortexa8hf-neon release_version is a string
71representing the release number of the Yocto Project: DISTRO,
72DISTRO+snapshot For example, the following SDK installer is for a 64-bit
73development host system and a i586-tuned target architecture based off
74the SDK for ``core-image-sato`` and using the current DISTRO snapshot:
75poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-DISTRO.sh
76
77.. note::
78
79 As an alternative to downloading an SDK, you can build the SDK
80 installer. For information on building the installer, see the "
81 Building an SDK Installer
82 " section.
83
84The SDK and toolchains are self-contained and by default are installed
85into the ``poky_sdk`` folder in your home directory. You can choose to
86install the extensible SDK in any location when you run the installer.
87However, because files need to be written under that directory during
88the normal course of operation, the location you choose for installation
89must be writable for whichever users need to use the SDK.
90
91The following command shows how to run the installer given a toolchain
92tarball for a 64-bit x86 development host system and a 64-bit x86 target
93architecture. The example assumes the SDK installer is located in
94``~/Downloads/`` and has execution rights.
95
96.. note::
97
98 If you do not have write permissions for the directory into which you
99 are installing the SDK, the installer notifies you and exits. For
100 that case, set up the proper permissions in the directory and run the
101 installer again.
102
103$
104./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
105Poky (Yocto Project Reference Distro) Extensible SDK installer version
1062.5
107==========================================================================
108Enter target directory for SDK (default: ~/poky_sdk): You are about to
109install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
110Extracting SDK..............done Setting it up... Extracting
111buildtools... Preparing build system... Parsing recipes: 100%
112\|##################################################################\|
113Time: 0:00:52 Initialising tasks: 100%
114\|###############################################################\|
115Time: 0:00:00 Checking sstate mirror object availability: 100%
116\|#######################################\| Time: 0:00:00 Loading cache:
117100%
118\|####################################################################\|
119Time: 0:00:00 Initialising tasks: 100%
120\|###############################################################\|
121Time: 0:00:00 done SDK has been successfully set up and is ready to be
122used. Each time you wish to use the SDK in a new shell session, you need
123to source the environment setup script e.g. $ .
124/home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
125
126.. _sdk-running-the-extensible-sdk-environment-setup-script:
127
128Running the Extensible SDK Environment Setup Script
129===================================================
130
131Once you have the SDK installed, you must run the SDK environment setup
132script before you can actually use the SDK. This setup script resides in
133the directory you chose when you installed the SDK, which is either the
134default ``poky_sdk`` directory or the directory you chose during
135installation.
136
137Before running the script, be sure it is the one that matches the
138architecture for which you are developing. Environment setup scripts
139begin with the string "``environment-setup``" and include as part of
140their name the tuned target architecture. As an example, the following
141commands set the working directory to where the SDK was installed and
142then source the environment setup script. In this example, the setup
143script is for an IA-based target machine using i586 tuning: $ cd
144/home/scottrif/poky_sdk $ source environment-setup-core2-64-poky-linux
145SDK environment now set up; additionally you may now run devtool to
146perform development tasks. Run devtool --help for further details.
147Running the setup script defines many environment variables needed in
148order to use the SDK (e.g. ``PATH``,
149```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__,
150```LD`` <&YOCTO_DOCS_REF_URL;#var-LD>`__, and so forth). If you want to
151see all the environment variables the script exports, examine the
152installation file itself.
153
154Using ``devtool`` in Your SDK Workflow
155======================================
156
157The cornerstone of the extensible SDK is a command-line tool called
158``devtool``. This tool provides a number of features that help you
159build, test and package software within the extensible SDK, and
160optionally integrate it into an image built by the OpenEmbedded build
161system.
162
163.. note::
164
165 The use of
166 devtool
167 is not limited to the extensible SDK. You can use
168 devtool
169 to help you easily develop any project whose build output must be
170 part of an image built using the build system.
171
172The ``devtool`` command line is organized similarly to
173`Git <&YOCTO_DOCS_OM_URL;#git>`__ in that it has a number of
174sub-commands for each function. You can run ``devtool --help`` to see
175all the commands.
176
177.. note::
178
179 See the "
180 devtool
181  Quick Reference
182 " in the Yocto Project Reference Manual for a
183 devtool
184 quick reference.
185
186Three ``devtool`` subcommands exist that provide entry-points into
187development:
188
189- *``devtool add``*: Assists in adding new software to be built.
190
191- *``devtool modify``*: Sets up an environment to enable you to modify
192 the source of an existing component.
193
194- *``devtool upgrade``*: Updates an existing recipe so that you can
195 build it for an updated set of source files.
196
197As with the build system, "recipes" represent software packages within
198``devtool``. When you use ``devtool add``, a recipe is automatically
199created. When you use ``devtool modify``, the specified existing recipe
200is used in order to determine where to get the source code and how to
201patch it. In both cases, an environment is set up so that when you build
202the recipe a source tree that is under your control is used in order to
203allow you to make changes to the source as desired. By default, new
204recipes and the source go into a "workspace" directory under the SDK.
205
206The remainder of this section presents the ``devtool add``,
207``devtool modify``, and ``devtool upgrade`` workflows.
208
209.. _sdk-use-devtool-to-add-an-application:
210
211Use ``devtool add`` to Add an Application
212-----------------------------------------
213
214The ``devtool add`` command generates a new recipe based on existing
215source code. This command takes advantage of the
216`workspace <&YOCTO_DOCS_REF_URL;#devtool-the-workspace-layer-structure>`__
217layer that many ``devtool`` commands use. The command is flexible enough
218to allow you to extract source code into both the workspace or a
219separate local Git repository and to use existing code that does not
220need to be extracted.
221
222Depending on your particular scenario, the arguments and options you use
223with ``devtool add`` form different combinations. The following diagram
224shows common development flows you would use with the ``devtool add``
225command:
226
2271. *Generating the New Recipe*: The top part of the flow shows three
228 scenarios by which you could use ``devtool add`` to generate a recipe
229 based on existing source code.
230
231 In a shared development environment, it is typical for other
232 developers to be responsible for various areas of source code. As a
233 developer, you are probably interested in using that source code as
234 part of your development within the Yocto Project. All you need is
235 access to the code, a recipe, and a controlled area in which to do
236 your work.
237
238 Within the diagram, three possible scenarios feed into the
239 ``devtool add`` workflow:
240
241 - *Left*: The left scenario in the figure represents a common
242 situation where the source code does not exist locally and needs
243 to be extracted. In this situation, the source code is extracted
244 to the default workspace - you do not want the files in some
245 specific location outside of the workspace. Thus, everything you
246 need will be located in the workspace: $ devtool add recipe
247 fetchuri With this command, ``devtool`` extracts the upstream
248 source files into a local Git repository within the ``sources``
249 folder. The command then creates a recipe named recipe and a
250 corresponding append file in the workspace. If you do not provide
251 recipe, the command makes an attempt to determine the recipe name.
252
253 - *Middle*: The middle scenario in the figure also represents a
254 situation where the source code does not exist locally. In this
255 case, the code is again upstream and needs to be extracted to some
256 local area - this time outside of the default workspace.
257
258 .. note::
259
260 If required,
261 devtool
262 always creates a Git repository locally during the extraction.
263
264 Furthermore, the first positional argument srctree in this case
265 identifies where the ``devtool add`` command will locate the
266 extracted code outside of the workspace. You need to specify an
267 empty directory: $ devtool add recipe srctree fetchuri In summary,
268 the source code is pulled from fetchuri and extracted into the
269 location defined by srctree as a local Git repository.
270
271 Within workspace, ``devtool`` creates a recipe named recipe along
272 with an associated append file.
273
274 - *Right*: The right scenario in the figure represents a situation
275 where the srctree has been previously prepared outside of the
276 ``devtool`` workspace.
277
278 The following command provides a new recipe name and identifies
279 the existing source tree location: $ devtool add recipe srctree
280 The command examines the source code and creates a recipe named
281 recipe for the code and places the recipe into the workspace.
282
283 Because the extracted source code already exists, ``devtool`` does
284 not try to relocate the source code into the workspace - only the
285 new recipe is placed in the workspace.
286
287 Aside from a recipe folder, the command also creates an associated
288 append folder and places an initial ``*.bbappend`` file within.
289
2902. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
291 editor as defined by the ``$EDITOR`` environment variable and modify
292 the file: $ devtool edit-recipe recipe From within the editor, you
293 can make modifications to the recipe that take affect when you build
294 it later.
295
2963. *Build the Recipe or Rebuild the Image*: The next step you take
297 depends on what you are going to do with the new code.
298
299 If you need to eventually move the build output to the target
300 hardware, use the following ``devtool`` command: $ devtool build
301 recipe
302
303 On the other hand, if you want an image to contain the recipe's
304 packages from the workspace for immediate deployment onto a device
305 (e.g. for testing purposes), you can use the ``devtool build-image``
306 command: $ devtool build-image image
307
3084. *Deploy the Build Output*: When you use the ``devtool build`` command
309 to build out your recipe, you probably want to see if the resulting
310 build output works as expected on the target hardware.
311
312 .. note::
313
314 This step assumes you have a previously built image that is
315 already either running in QEMU or is running on actual hardware.
316 Also, it is assumed that for deployment of the image to the
317 target, SSH is installed in the image and, if the image is running
318 on real hardware, you have network access to and from your
319 development machine.
320
321 You can deploy your build output to that target hardware by using the
322 ``devtool deploy-target`` command: $ devtool deploy-target recipe
323 target The target is a live target machine running as an SSH server.
324
325 You can, of course, also deploy the image you build to actual
326 hardware by using the ``devtool build-image`` command. However,
327 ``devtool`` does not provide a specific command that allows you to
328 deploy the image to actual hardware.
329
3305. *Finish Your Work With the Recipe*: The ``devtool finish`` command
331 creates any patches corresponding to commits in the local Git
332 repository, moves the new recipe to a more permanent layer, and then
333 resets the recipe so that the recipe is built normally rather than
334 from the workspace. $ devtool finish recipe layer
335
336 .. note::
337
338 Any changes you want to turn into patches must be committed to the
339 Git repository in the source tree.
340
341 As mentioned, the ``devtool finish`` command moves the final recipe
342 to its permanent layer.
343
344 As a final process of the ``devtool finish`` command, the state of
345 the standard layers and the upstream source is restored so that you
346 can build the recipe from those areas rather than the workspace.
347
348 .. note::
349
350 You can use the
351 devtool reset
352 command to put things back should you decide you do not want to
353 proceed with your work. If you do use this command, realize that
354 the source tree is preserved.
355
356.. _sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component:
357
358Use ``devtool modify`` to Modify the Source of an Existing Component
359--------------------------------------------------------------------
360
361The ``devtool modify`` command prepares the way to work on existing code
362that already has a local recipe in place that is used to build the
363software. The command is flexible enough to allow you to extract code
364from an upstream source, specify the existing recipe, and keep track of
365and gather any patch files from other developers that are associated
366with the code.
367
368Depending on your particular scenario, the arguments and options you use
369with ``devtool modify`` form different combinations. The following
370diagram shows common development flows for the ``devtool modify``
371command:
372
3731. *Preparing to Modify the Code*: The top part of the flow shows three
374 scenarios by which you could use ``devtool modify`` to prepare to
375 work on source files. Each scenario assumes the following:
376
377 - The recipe exists locally in a layer external to the ``devtool``
378 workspace.
379
380 - The source files exist either upstream in an un-extracted state or
381 locally in a previously extracted state.
382
383 The typical situation is where another developer has created a layer
384 for use with the Yocto Project and their recipe already resides in
385 that layer. Furthermore, their source code is readily available
386 either upstream or locally.
387
388 - *Left*: The left scenario in the figure represents a common
389 situation where the source code does not exist locally and it
390 needs to be extracted from an upstream source. In this situation,
391 the source is extracted into the default ``devtool`` workspace
392 location. The recipe, in this scenario, is in its own layer
393 outside the workspace (i.e. ``meta-``\ layername).
394
395 The following command identifies the recipe and, by default,
396 extracts the source files: $ devtool modify recipe Once
397 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's
398 ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ statements to
399 locate the source code and any local patch files from other
400 developers.
401
402 With this scenario, no srctree argument exists. Consequently, the
403 default behavior of the ``devtool modify`` command is to extract
404 the source files pointed to by the ``SRC_URI`` statements into a
405 local Git structure. Furthermore, the location for the extracted
406 source is the default area within the ``devtool`` workspace. The
407 result is that the command sets up both the source code and an
408 append file within the workspace while the recipe remains in its
409 original location.
410
411 Additionally, if you have any non-patch local files (i.e. files
412 referred to with ``file://`` entries in ``SRC_URI`` statement
413 excluding ``*.patch/`` or ``*.diff``), these files are copied to
414 an ``oe-local-files`` folder under the newly created source tree.
415 Copying the files here gives you a convenient area from which you
416 can modify the files. Any changes or additions you make to those
417 files are incorporated into the build the next time you build the
418 software just as are other changes you might have made to the
419 source.
420
421 - *Middle*: The middle scenario in the figure represents a situation
422 where the source code also does not exist locally. In this case,
423 the code is again upstream and needs to be extracted to some local
424 area as a Git repository. The recipe, in this scenario, is again
425 local and in its own layer outside the workspace.
426
427 The following command tells ``devtool`` the recipe with which to
428 work and, in this case, identifies a local area for the extracted
429 source files that exists outside of the default ``devtool``
430 workspace: $ devtool modify recipe srctree
431
432 .. note::
433
434 You cannot provide a URL for
435 srctree
436 using the
437 devtool
438 command.
439
440 As with all extractions, the command uses the recipe's ``SRC_URI``
441 statements to locate the source files and any associated patch
442 files. Non-patch files are copied to an ``oe-local-files`` folder
443 under the newly created source tree.
444
445 Once the files are located, the command by default extracts them
446 into srctree.
447
448 Within workspace, ``devtool`` creates an append file for the
449 recipe. The recipe remains in its original location but the source
450 files are extracted to the location you provide with srctree.
451
452 - *Right*: The right scenario in the figure represents a situation
453 where the source tree (srctree) already exists locally as a
454 previously extracted Git structure outside of the ``devtool``
455 workspace. In this example, the recipe also exists elsewhere
456 locally in its own layer.
457
458 The following command tells ``devtool`` the recipe with which to
459 work, uses the "-n" option to indicate source does not need to be
460 extracted, and uses srctree to point to the previously extracted
461 source files: $ devtool modify -n recipe srctree
462
463 If an ``oe-local-files`` subdirectory happens to exist and it
464 contains non-patch files, the files are used. However, if the
465 subdirectory does not exist and you run the ``devtool finish``
466 command, any non-patch files that might exist next to the recipe
467 are removed because it appears to ``devtool`` that you have
468 deleted those files.
469
470 Once the ``devtool modify`` command finishes, it creates only an
471 append file for the recipe in the ``devtool`` workspace. The
472 recipe and the source code remain in their original locations.
473
4742. *Edit the Source*: Once you have used the ``devtool modify`` command,
475 you are free to make changes to the source files. You can use any
476 editor you like to make and save your source code modifications.
477
4783. *Build the Recipe or Rebuild the Image*: The next step you take
479 depends on what you are going to do with the new code.
480
481 If you need to eventually move the build output to the target
482 hardware, use the following ``devtool`` command: $ devtool build
483 recipe
484
485 On the other hand, if you want an image to contain the recipe's
486 packages from the workspace for immediate deployment onto a device
487 (e.g. for testing purposes), you can use the ``devtool build-image``
488 command: $ devtool build-image image
489
4904. *Deploy the Build Output*: When you use the ``devtool build`` command
491 to build out your recipe, you probably want to see if the resulting
492 build output works as expected on target hardware.
493
494 .. note::
495
496 This step assumes you have a previously built image that is
497 already either running in QEMU or running on actual hardware.
498 Also, it is assumed that for deployment of the image to the
499 target, SSH is installed in the image and if the image is running
500 on real hardware that you have network access to and from your
501 development machine.
502
503 You can deploy your build output to that target hardware by using the
504 ``devtool deploy-target`` command: $ devtool deploy-target recipe
505 target The target is a live target machine running as an SSH server.
506
507 You can, of course, use other methods to deploy the image you built
508 using the ``devtool build-image`` command to actual hardware.
509 ``devtool`` does not provide a specific command to deploy the image
510 to actual hardware.
511
5125. *Finish Your Work With the Recipe*: The ``devtool finish`` command
513 creates any patches corresponding to commits in the local Git
514 repository, updates the recipe to point to them (or creates a
515 ``.bbappend`` file to do so, depending on the specified destination
516 layer), and then resets the recipe so that the recipe is built
517 normally rather than from the workspace. $ devtool finish recipe
518 layer
519
520 .. note::
521
522 Any changes you want to turn into patches must be staged and
523 committed within the local Git repository before you use the
524 devtool finish
525 command.
526
527 Because there is no need to move the recipe, ``devtool finish``
528 either updates the original recipe in the original layer or the
529 command creates a ``.bbappend`` file in a different layer as provided
530 by layer. Any work you did in the ``oe-local-files`` directory is
531 preserved in the original files next to the recipe during the
532 ``devtool finish`` command.
533
534 As a final process of the ``devtool finish`` command, the state of
535 the standard layers and the upstream source is restored so that you
536 can build the recipe from those areas rather than from the workspace.
537
538 .. note::
539
540 You can use the
541 devtool reset
542 command to put things back should you decide you do not want to
543 proceed with your work. If you do use this command, realize that
544 the source tree is preserved.
545
546.. _sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software:
547
548Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
549-------------------------------------------------------------------------------------------------------
550
551The ``devtool upgrade`` command upgrades an existing recipe to that of a
552more up-to-date version found upstream. Throughout the life of software,
553recipes continually undergo version upgrades by their upstream
554publishers. You can use the ``devtool upgrade`` workflow to make sure
555your recipes you are using for builds are up-to-date with their upstream
556counterparts.
557
558.. note::
559
560 Several methods exist by which you can upgrade recipes -
561 devtool upgrade
562 happens to be one. You can read about all the methods by which you
563 can upgrade recipes in the "
564 Upgrading Recipes
565 " section of the Yocto Project Development Tasks Manual.
566
567The ``devtool upgrade`` command is flexible enough to allow you to
568specify source code revision and versioning schemes, extract code into
569or out of the ``devtool``
570`workspace <&YOCTO_DOCS_REF_URL;#devtool-the-workspace-layer-structure>`__,
571and work with any source file forms that the
572`fetchers <&YOCTO_DOCS_BB_URL;#bb-fetchers>`__ support.
573
574The following diagram shows the common development flow used with the
575``devtool upgrade`` command:
576
5771. *Initiate the Upgrade*: The top part of the flow shows the typical
578 scenario by which you use the ``devtool upgrade`` command. The
579 following conditions exist:
580
581 - The recipe exists in a local layer external to the ``devtool``
582 workspace.
583
584 - The source files for the new release exist in the same location
585 pointed to by ```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__
586 in the recipe (e.g. a tarball with the new version number in the
587 name, or as a different revision in the upstream Git repository).
588
589 A common situation is where third-party software has undergone a
590 revision so that it has been upgraded. The recipe you have access to
591 is likely in your own layer. Thus, you need to upgrade the recipe to
592 use the newer version of the software: $ devtool upgrade -V version
593 recipe By default, the ``devtool upgrade`` command extracts source
594 code into the ``sources`` directory in the
595 `workspace <&YOCTO_DOCS_REF_URL;#devtool-the-workspace-layer-structure>`__.
596 If you want the code extracted to any other location, you need to
597 provide the srctree positional argument with the command as follows:
598 $ devtool upgrade -V version recipe srctree
599
600 .. note::
601
602 In this example, the "-V" option specifies the new version. If you
603 don't use "-V", the command upgrades the recipe to the latest
604 version.
605
606 If the source files pointed to by the ``SRC_URI`` statement in the
607 recipe are in a Git repository, you must provide the "-S" option and
608 specify a revision for the software.
609
610 Once ``devtool`` locates the recipe, it uses the ``SRC_URI`` variable
611 to locate the source code and any local patch files from other
612 developers. The result is that the command sets up the source code,
613 the new version of the recipe, and an append file all within the
614 workspace.
615
616 Additionally, if you have any non-patch local files (i.e. files
617 referred to with ``file://`` entries in ``SRC_URI`` statement
618 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
619 ``oe-local-files`` folder under the newly created source tree.
620 Copying the files here gives you a convenient area from which you can
621 modify the files. Any changes or additions you make to those files
622 are incorporated into the build the next time you build the software
623 just as are other changes you might have made to the source.
624
6252. *Resolve any Conflicts created by the Upgrade*: Conflicts could exist
626 due to the software being upgraded to a new version. Conflicts occur
627 if your recipe specifies some patch files in ``SRC_URI`` that
628 conflict with changes made in the new version of the software. For
629 such cases, you need to resolve the conflicts by editing the source
630 and following the normal ``git rebase`` conflict resolution process.
631
632 Before moving onto the next step, be sure to resolve any such
633 conflicts created through use of a newer or different version of the
634 software.
635
6363. *Build the Recipe or Rebuild the Image*: The next step you take
637 depends on what you are going to do with the new code.
638
639 If you need to eventually move the build output to the target
640 hardware, use the following ``devtool`` command: $ devtool build
641 recipe
642
643 On the other hand, if you want an image to contain the recipe's
644 packages from the workspace for immediate deployment onto a device
645 (e.g. for testing purposes), you can use the ``devtool build-image``
646 command: $ devtool build-image image
647
6484. *Deploy the Build Output*: When you use the ``devtool build`` command
649 or ``bitbake`` to build your recipe, you probably want to see if the
650 resulting build output works as expected on target hardware.
651
652 .. note::
653
654 This step assumes you have a previously built image that is
655 already either running in QEMU or running on actual hardware.
656 Also, it is assumed that for deployment of the image to the
657 target, SSH is installed in the image and if the image is running
658 on real hardware that you have network access to and from your
659 development machine.
660
661 You can deploy your build output to that target hardware by using the
662 ``devtool deploy-target`` command: $ devtool deploy-target recipe
663 target The target is a live target machine running as an SSH server.
664
665 You can, of course, also deploy the image you build using the
666 ``devtool build-image`` command to actual hardware. However,
667 ``devtool`` does not provide a specific command that allows you to do
668 this.
669
6705. *Finish Your Work With the Recipe*: The ``devtool finish`` command
671 creates any patches corresponding to commits in the local Git
672 repository, moves the new recipe to a more permanent layer, and then
673 resets the recipe so that the recipe is built normally rather than
674 from the workspace.
675
676 Any work you did in the ``oe-local-files`` directory is preserved in
677 the original files next to the recipe during the ``devtool finish``
678 command.
679
680 If you specify a destination layer that is the same as the original
681 source, then the old version of the recipe and associated files are
682 removed prior to adding the new version. $ devtool finish recipe
683 layer
684
685 .. note::
686
687 Any changes you want to turn into patches must be committed to the
688 Git repository in the source tree.
689
690 As a final process of the ``devtool finish`` command, the state of
691 the standard layers and the upstream source is restored so that you
692 can build the recipe from those areas rather than the workspace.
693
694 .. note::
695
696 You can use the
697 devtool reset
698 command to put things back should you decide you do not want to
699 proceed with your work. If you do use this command, realize that
700 the source tree is preserved.
701
702.. _sdk-a-closer-look-at-devtool-add:
703
704A Closer Look at ``devtool add``
705================================
706
707The ``devtool add`` command automatically creates a recipe based on the
708source tree you provide with the command. Currently, the command has
709support for the following:
710
711- Autotools (``autoconf`` and ``automake``)
712
713- CMake
714
715- Scons
716
717- ``qmake``
718
719- Plain ``Makefile``
720
721- Out-of-tree kernel module
722
723- Binary package (i.e. "-b" option)
724
725- Node.js module
726
727- Python modules that use ``setuptools`` or ``distutils``
728
729Apart from binary packages, the determination of how a source tree
730should be treated is automatic based on the files present within that
731source tree. For example, if a ``CMakeLists.txt`` file is found, then
732the source tree is assumed to be using CMake and is treated accordingly.
733
734.. note::
735
736 In most cases, you need to edit the automatically generated recipe in
737 order to make it build properly. Typically, you would go through
738 several edit and build cycles until the recipe successfully builds.
739 Once the recipe builds, you could use possible further iterations to
740 test the recipe on the target device.
741
742The remainder of this section covers specifics regarding how parts of
743the recipe are generated.
744
745.. _sdk-name-and-version:
746
747Name and Version
748----------------
749
750If you do not specify a name and version on the command line,
751``devtool add`` uses various metadata within the source tree in an
752attempt to determine the name and version of the software being built.
753Based on what the tool determines, ``devtool`` sets the name of the
754created recipe file accordingly.
755
756If ``devtool`` cannot determine the name and version, the command prints
757an error. For such cases, you must re-run the command and provide the
758name and version, just the name, or just the version as part of the
759command line.
760
761Sometimes the name or version determined from the source tree might be
762incorrect. For such a case, you must reset the recipe: $ devtool reset
763-n recipename After running the ``devtool reset`` command, you need to
764run ``devtool add`` again and provide the name or the version.
765
766.. _sdk-dependency-detection-and-mapping:
767
768Dependency Detection and Mapping
769--------------------------------
770
771The ``devtool add`` command attempts to detect build-time dependencies
772and map them to other recipes in the system. During this mapping, the
773command fills in the names of those recipes as part of the
774```DEPENDS`` <&YOCTO_DOCS_REF_URL;#var-DEPENDS>`__ variable within the
775recipe. If a dependency cannot be mapped, ``devtool`` places a comment
776in the recipe indicating such. The inability to map a dependency can
777result from naming not being recognized or because the dependency simply
778is not available. For cases where the dependency is not available, you
779must use the ``devtool add`` command to add an additional recipe that
780satisfies the dependency. Once you add that recipe, you need to update
781the ``DEPENDS`` variable in the original recipe to include the new
782recipe.
783
784If you need to add runtime dependencies, you can do so by adding the
785following to your recipe: RDEPENDS_${PN} += "dependency1 dependency2
786..."
787
788.. note::
789
790 The
791 devtool add
792 command often cannot distinguish between mandatory and optional
793 dependencies. Consequently, some of the detected dependencies might
794 in fact be optional. When in doubt, consult the documentation or the
795 configure script for the software the recipe is building for further
796 details. In some cases, you might find you can substitute the
797 dependency with an option that disables the associated functionality
798 passed to the configure script.
799
800.. _sdk-license-detection:
801
802License Detection
803-----------------
804
805The ``devtool add`` command attempts to determine if the software you
806are adding is able to be distributed under a common, open-source
807license. If so, the command sets the
808```LICENSE`` <&YOCTO_DOCS_REF_URL;#var-LICENSE>`__ value accordingly.
809You should double-check the value added by the command against the
810documentation or source files for the software you are building and, if
811necessary, update that ``LICENSE`` value.
812
813The ``devtool add`` command also sets the
814```LIC_FILES_CHKSUM`` <&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM>`__
815value to point to all files that appear to be license-related. Realize
816that license statements often appear in comments at the top of source
817files or within the documentation. In such cases, the command does not
818recognize those license statements. Consequently, you might need to
819amend the ``LIC_FILES_CHKSUM`` variable to point to one or more of those
820comments if present. Setting ``LIC_FILES_CHKSUM`` is particularly
821important for third-party software. The mechanism attempts to ensure
822correct licensing should you upgrade the recipe to a newer upstream
823version in future. Any change in licensing is detected and you receive
824an error prompting you to check the license text again.
825
826If the ``devtool add`` command cannot determine licensing information,
827``devtool`` sets the ``LICENSE`` value to "CLOSED" and leaves the
828``LIC_FILES_CHKSUM`` value unset. This behavior allows you to continue
829with development even though the settings are unlikely to be correct in
830all cases. You should check the documentation or source files for the
831software you are building to determine the actual license.
832
833.. _sdk-adding-makefile-only-software:
834
835Adding Makefile-Only Software
836-----------------------------
837
838The use of Make by itself is very common in both proprietary and
839open-source software. Unfortunately, Makefiles are often not written
840with cross-compilation in mind. Thus, ``devtool add`` often cannot do
841very much to ensure that these Makefiles build correctly. It is very
842common, for example, to explicitly call ``gcc`` instead of using the
843```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__ variable. Usually, in a
844cross-compilation environment, ``gcc`` is the compiler for the build
845host and the cross-compiler is named something similar to
846``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
847point to the associated sysroot for the target machine).
848
849When writing a recipe for Makefile-only software, keep the following in
850mind:
851
852- You probably need to patch the Makefile to use variables instead of
853 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
854
855- The environment in which Make runs is set up with various standard
856 variables for compilation (e.g. ``CC``, ``CXX``, and so forth) in a
857 similar manner to the environment set up by the SDK's environment
858 setup script. One easy way to see these variables is to run the
859 ``devtool build`` command on the recipe and then look in
860 ``oe-logs/run.do_compile``. Towards the top of this file, a list of
861 environment variables exists that are being set. You can take
862 advantage of these variables within the Makefile.
863
864- If the Makefile sets a default for a variable using "=", that default
865 overrides the value set in the environment, which is usually not
866 desirable. For this case, you can either patch the Makefile so it
867 sets the default using the "?=" operator, or you can alternatively
868 force the value on the ``make`` command line. To force the value on
869 the command line, add the variable setting to
870 ```EXTRA_OEMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE>`__ or
871 ```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
872 within the recipe. Here is an example using ``EXTRA_OEMAKE``:
873 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" In the above example,
874 single quotes are used around the variable settings as the values are
875 likely to contain spaces because required default options are passed
876 to the compiler.
877
878- Hardcoding paths inside Makefiles is often problematic in a
879 cross-compilation environment. This is particularly true because
880 those hardcoded paths often point to locations on the build host and
881 thus will either be read-only or will introduce contamination into
882 the cross-compilation because they are specific to the build host
883 rather than the target. Patching the Makefile to use prefix variables
884 or other path variables is usually the way to handle this situation.
885
886- Sometimes a Makefile runs target-specific commands such as
887 ``ldconfig``. For such cases, you might be able to apply patches that
888 remove these commands from the Makefile.
889
890.. _sdk-adding-native-tools:
891
892Adding Native Tools
893-------------------
894
895Often, you need to build additional tools that run on the `build
896host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ as opposed to
897the target. You should indicate this requirement by using one of the
898following methods when you run ``devtool add``:
899
900- Specify the name of the recipe such that it ends with "-native".
901 Specifying the name like this produces a recipe that only builds for
902 the build host.
903
904- Specify the "DASHDASHalso-native" option with the ``devtool add``
905 command. Specifying this option creates a recipe file that still
906 builds for the target but also creates a variant with a "-native"
907 suffix that builds for the build host.
908
909.. note::
910
911 If you need to add a tool that is shipped as part of a source tree
912 that builds code for the target, you can typically accomplish this by
913 building the native and target parts separately rather than within
914 the same compilation process. Realize though that with the
915 "DASHDASHalso-native" option, you can add the tool using just one
916 recipe file.
917
918.. _sdk-adding-node-js-modules:
919
920Adding Node.js Modules
921----------------------
922
923You can use the ``devtool add`` command two different ways to add
924Node.js modules: 1) Through ``npm`` and, 2) from a repository or local
925source.
926
927Use the following form to add Node.js modules through ``npm``: $ devtool
928add "npm://registry.npmjs.org;name=forever;version=0.15.1" The name and
929version parameters are mandatory. Lockdown and shrinkwrap files are
930generated and pointed to by the recipe in order to freeze the version
931that is fetched for the dependencies according to the first time. This
932also saves checksums that are verified on future fetches. Together,
933these behaviors ensure the reproducibility and integrity of the build.
934
935.. note::
936
937 - You must use quotes around the URL. The ``devtool add`` does not
938 require the quotes, but the shell considers ";" as a splitter
939 between multiple commands. Thus, without the quotes,
940 ``devtool add`` does not receive the other parts, which results in
941 several "command not found" errors.
942
943 - In order to support adding Node.js modules, a ``nodejs`` recipe
944 must be part of your SDK.
945
946As mentioned earlier, you can also add Node.js modules directly from a
947repository or local source tree. To add modules this way, use
948``devtool add`` in the following form: $ devtool add
949https://github.com/diversario/node-ssdp In this example, ``devtool``
950fetches the specified Git repository, detects the code as Node.js code,
951fetches dependencies using ``npm``, and sets
952```SRC_URI`` <&YOCTO_DOCS_REF_URL;#var-SRC_URI>`__ accordingly.
953
954.. _sdk-working-with-recipes:
955
956Working With Recipes
957====================
958
959When building a recipe using the ``devtool build`` command, the typical
960build progresses as follows:
961
9621. Fetch the source
963
9642. Unpack the source
965
9663. Configure the source
967
9684. Compile the source
969
9705. Install the build output
971
9726. Package the installed output
973
974For recipes in the workspace, fetching and unpacking is disabled as the
975source tree has already been prepared and is persistent. Each of these
976build steps is defined as a function (task), usually with a "do_" prefix
977(e.g. ```do_fetch`` <&YOCTO_DOCS_REF_URL;#ref-tasks-fetch>`__,
978```do_unpack`` <&YOCTO_DOCS_REF_URL;#ref-tasks-unpack>`__, and so
979forth). These functions are typically shell scripts but can instead be
980written in Python.
981
982If you look at the contents of a recipe, you will see that the recipe
983does not include complete instructions for building the software.
984Instead, common functionality is encapsulated in classes inherited with
985the ``inherit`` directive. This technique leaves the recipe to describe
986just the things that are specific to the software being built. A
987```base`` <&YOCTO_DOCS_REF_URL;#ref-classes-base>`__ class exists that
988is implicitly inherited by all recipes and provides the functionality
989that most recipes typically need.
990
991The remainder of this section presents information useful when working
992with recipes.
993
994.. _sdk-finding-logs-and-work-files:
995
996Finding Logs and Work Files
997---------------------------
998
999After the first run of the ``devtool build`` command, recipes that were
1000previously created using the ``devtool add`` command or whose sources
1001were modified using the ``devtool modify`` command contain symbolic
1002links created within the source tree:
1003
1004- ``oe-logs``: This link points to the directory in which log files and
1005 run scripts for each build step are created.
1006
1007- ``oe-workdir``: This link points to the temporary work area for the
1008 recipe. The following locations under ``oe-workdir`` are particularly
1009 useful:
1010
1011 - ``image/``: Contains all of the files installed during the
1012 ```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ stage.
1013 Within a recipe, this directory is referred to by the expression
1014 ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}``.
1015
1016 - ``sysroot-destdir/``: Contains a subset of files installed within
1017 ``do_install`` that have been put into the shared sysroot. For
1018 more information, see the "`Sharing Files Between
1019 Recipes <#sdk-sharing-files-between-recipes>`__" section.
1020
1021 - ``packages-split/``: Contains subdirectories for each package
1022 produced by the recipe. For more information, see the
1023 "`Packaging <#sdk-packaging>`__" section.
1024
1025You can use these links to get more information on what is happening at
1026each build step.
1027
1028.. _sdk-setting-configure-arguments:
1029
1030Setting Configure Arguments
1031---------------------------
1032
1033If the software your recipe is building uses GNU autoconf, then a fixed
1034set of arguments is passed to it to enable cross-compilation plus any
1035extras specified by
1036```EXTRA_OECONF`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF>`__ or
1037```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
1038set within the recipe. If you wish to pass additional options, add them
1039to ``EXTRA_OECONF`` or ``PACKAGECONFIG_CONFARGS``. Other supported build
1040tools have similar variables (e.g.
1041```EXTRA_OECMAKE`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE>`__ for
1042CMake, ```EXTRA_OESCONS`` <&YOCTO_DOCS_REF_URL;#var-EXTRA_OESCONS>`__
1043for Scons, and so forth). If you need to pass anything on the ``make``
1044command line, you can use ``EXTRA_OEMAKE`` or the
1045```PACKAGECONFIG_CONFARGS`` <&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS>`__
1046variables to do so.
1047
1048You can use the ``devtool configure-help`` command to help you set the
1049arguments listed in the previous paragraph. The command determines the
1050exact options being passed, and shows them to you along with any custom
1051arguments specified through ``EXTRA_OECONF`` or
1052``PACKAGECONFIG_CONFARGS``. If applicable, the command also shows you
1053the output of the configure script's "DASHDASHhelp" option as a
1054reference.
1055
1056.. _sdk-sharing-files-between-recipes:
1057
1058Sharing Files Between Recipes
1059-----------------------------
1060
1061Recipes often need to use files provided by other recipes on the `build
1062host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__. For example,
1063an application linking to a common library needs access to the library
1064itself and its associated headers. The way this access is accomplished
1065within the extensible SDK is through the sysroot. One sysroot exists per
1066"machine" for which the SDK is being built. In practical terms, this
1067means a sysroot exists for the target machine, and a sysroot exists for
1068the build host.
1069
1070Recipes should never write files directly into the sysroot. Instead,
1071files should be installed into standard locations during the
1072```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task within
1073the ``${``\ ```D`` <&YOCTO_DOCS_REF_URL;#var-D>`__\ ``}`` directory. A
1074subset of these files automatically goes into the sysroot. The reason
1075for this limitation is that almost all files that go into the sysroot
1076are cataloged in manifests in order to ensure they can be removed later
1077when a recipe is modified or removed. Thus, the sysroot is able to
1078remain free from stale files.
1079
1080.. _sdk-packaging:
1081
1082Packaging
1083---------
1084
1085Packaging is not always particularly relevant within the extensible SDK.
1086However, if you examine how build output gets into the final image on
1087the target device, it is important to understand packaging because the
1088contents of the image are expressed in terms of packages and not
1089recipes.
1090
1091During the ```do_package`` <&YOCTO_DOCS_REF_URL;#ref-tasks-package>`__
1092task, files installed during the
1093```do_install`` <&YOCTO_DOCS_REF_URL;#ref-tasks-install>`__ task are
1094split into one main package, which is almost always named the same as
1095the recipe, and into several other packages. This separation exists
1096because not all of those installed files are useful in every image. For
1097example, you probably do not need any of the documentation installed in
1098a production image. Consequently, for each recipe the documentation
1099files are separated into a ``-doc`` package. Recipes that package
1100software containing optional modules or plugins might undergo additional
1101package splitting as well.
1102
1103After building a recipe, you can see where files have gone by looking in
1104the ``oe-workdir/packages-split`` directory, which contains a
1105subdirectory for each package. Apart from some advanced cases, the
1106```PACKAGES`` <&YOCTO_DOCS_REF_URL;#var-PACKAGES>`__ and
1107```FILES`` <&YOCTO_DOCS_REF_URL;#var-FILES>`__ variables controls
1108splitting. The ``PACKAGES`` variable lists all of the packages to be
1109produced, while the ``FILES`` variable specifies which files to include
1110in each package by using an override to specify the package. For
1111example, ``FILES_${PN}`` specifies the files to go into the main package
1112(i.e. the main package has the same name as the recipe and
1113``${``\ ```PN`` <&YOCTO_DOCS_REF_URL;#var-PN>`__\ ``}`` evaluates to the
1114recipe name). The order of the ``PACKAGES`` value is significant. For
1115each installed file, the first package whose ``FILES`` value matches the
1116file is the package into which the file goes. Defaults exist for both
1117the ``PACKAGES`` and ``FILES`` variables. Consequently, you might find
1118you do not even need to set these variables in your recipe unless the
1119software the recipe is building installs files into non-standard
1120locations.
1121
1122.. _sdk-restoring-the-target-device-to-its-original-state:
1123
1124Restoring the Target Device to its Original State
1125=================================================
1126
1127If you use the ``devtool deploy-target`` command to write a recipe's
1128build output to the target, and you are working on an existing component
1129of the system, then you might find yourself in a situation where you
1130need to restore the original files that existed prior to running the
1131``devtool deploy-target`` command. Because the ``devtool deploy-target``
1132command backs up any files it overwrites, you can use the
1133``devtool undeploy-target`` command to restore those files and remove
1134any other files the recipe deployed. Consider the following example: $
1135devtool undeploy-target lighttpd root@192.168.7.2 If you have deployed
1136multiple applications, you can remove them all using the "-a" option
1137thus restoring the target device to its original state: $ devtool
1138undeploy-target -a root@192.168.7.2 Information about files deployed to
1139the target as well as any backed up files are stored on the target
1140itself. This storage, of course, requires some additional space on the
1141target machine.
1142
1143.. note::
1144
1145 The
1146 devtool deploy-target
1147 and
1148 devtool undeploy-target
1149 commands do not currently interact with any package management system
1150 on the target device (e.g. RPM or OPKG). Consequently, you should not
1151 intermingle
1152 devtool deploy-target
1153 and package manager operations on the target device. Doing so could
1154 result in a conflicting set of files.
1155
1156.. _sdk-installing-additional-items-into-the-extensible-sdk:
1157
1158Installing Additional Items Into the Extensible SDK
1159===================================================
1160
1161Out of the box the extensible SDK typically only comes with a small
1162number of tools and libraries. A minimal SDK starts mostly empty and is
1163populated on-demand. Sometimes you must explicitly install extra items
1164into the SDK. If you need these extra items, you can first search for
1165the items using the ``devtool search`` command. For example, suppose you
1166need to link to libGL but you are not sure which recipe provides libGL.
1167You can use the following command to find out: $ devtool search libGL
1168mesa A free implementation of the OpenGL API Once you know the recipe
1169(i.e. ``mesa`` in this example), you can install it: $ devtool
1170sdk-install mesa By default, the ``devtool sdk-install`` command assumes
1171the item is available in pre-built form from your SDK provider. If the
1172item is not available and it is acceptable to build the item from
1173source, you can add the "-s" option as follows: $ devtool sdk-install -s
1174mesa It is important to remember that building the item from source
1175takes significantly longer than installing the pre-built artifact. Also,
1176if no recipe exists for the item you want to add to the SDK, you must
1177instead add the item using the ``devtool add`` command.
1178
1179.. _sdk-applying-updates-to-an-installed-extensible-sdk:
1180
1181Applying Updates to an Installed Extensible SDK
1182===============================================
1183
1184If you are working with an installed extensible SDK that gets
1185occasionally updated (e.g. a third-party SDK), then you will need to
1186manually "pull down" the updates into the installed SDK.
1187
1188To update your installed SDK, use ``devtool`` as follows: $ devtool
1189sdk-update The previous command assumes your SDK provider has set the
1190default update URL for you through the
1191```SDK_UPDATE_URL`` <&YOCTO_DOCS_REF_URL;#var-SDK_UPDATE_URL>`__
1192variable as described in the "`Providing Updates to the Extensible SDK
1193After
1194Installation <#sdk-providing-updates-to-the-extensible-sdk-after-installation>`__"
1195section. If the SDK provider has not set that default URL, you need to
1196specify it yourself in the command as follows: $ devtool sdk-update
1197path_to_update_directory
1198
1199.. note::
1200
1201 The URL needs to point specifically to a published SDK and not to an
1202 SDK installer that you would download and install.
1203
1204.. _sdk-creating-a-derivative-sdk-with-additional-components:
1205
1206Creating a Derivative SDK With Additional Components
1207====================================================
1208
1209You might need to produce an SDK that contains your own custom
1210libraries. A good example would be if you were a vendor with customers
1211that use your SDK to build their own platform-specific software and
1212those customers need an SDK that has custom libraries. In such a case,
1213you can produce a derivative SDK based on the currently installed SDK
1214fairly easily by following these steps:
1215
12161. If necessary, install an extensible SDK that you want to use as a
1217 base for your derivative SDK.
1218
12192. Source the environment script for the SDK.
1220
12213. Add the extra libraries or other components you want by using the
1222 ``devtool add`` command.
1223
12244. Run the ``devtool build-sdk`` command.
1225
1226The previous steps take the recipes added to the workspace and construct
1227a new SDK installer that contains those recipes and the resulting binary
1228artifacts. The recipes go into their own separate layer in the
1229constructed derivative SDK, which leaves the workspace clean and ready
1230for users to add their own recipes.
diff --git a/documentation/sdk-manual/sdk-intro.rst b/documentation/sdk-manual/sdk-intro.rst
new file mode 100644
index 0000000000..f2641b7ade
--- /dev/null
+++ b/documentation/sdk-manual/sdk-intro.rst
@@ -0,0 +1,223 @@
1************
2Introduction
3************
4
5.. _sdk-manual-intro:
6
7Introduction
8============
9
10Welcome to the Yocto Project Application Development and the Extensible
11Software Development Kit (eSDK) manual. This manual provides information
12that explains how to use both the Yocto Project extensible and standard
13SDKs to develop applications and images.
14
15.. note::
16
17 Prior to the 2.0 Release of the Yocto Project, application
18 development was primarily accomplished through the use of the
19 Application Development Toolkit (ADT) and the availability of
20 stand-alone cross-development toolchains and other tools. With the
21 2.1 Release of the Yocto Project, application development has
22 transitioned to within a tool-rich extensible SDK and the more
23 traditional standard SDK.
24
25All SDKs consist of the following:
26
27- *Cross-Development Toolchain*: This toolchain contains a compiler,
28 debugger, and various miscellaneous tools.
29
30- *Libraries, Headers, and Symbols*: The libraries, headers, and
31 symbols are specific to the image (i.e. they match the image).
32
33- *Environment Setup Script*: This ``*.sh`` file, once run, sets up the
34 cross-development environment by defining variables and preparing for
35 SDK use.
36
37Additionally, an extensible SDK has tools that allow you to easily add
38new applications and libraries to an image, modify the source of an
39existing component, test changes on the target hardware, and easily
40integrate an application into the `OpenEmbedded build
41system <&YOCTO_DOCS_REF_URL;#build-system-term>`__.
42
43You can use an SDK to independently develop and test code that is
44destined to run on some target machine. SDKs are completely
45self-contained. The binaries are linked against their own copy of
46``libc``, which results in no dependencies on the target system. To
47achieve this, the pointer to the dynamic loader is configured at install
48time since that path cannot be dynamically altered. This is the reason
49for a wrapper around the ``populate_sdk`` and ``populate_sdk_ext``
50archives.
51
52Another feature for the SDKs is that only one set of cross-compiler
53toolchain binaries are produced for any given architecture. This feature
54takes advantage of the fact that the target hardware can be passed to
55``gcc`` as a set of compiler options. Those options are set up by the
56environment script and contained in variables such as
57```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__ and
58```LD`` <&YOCTO_DOCS_REF_URL;#var-LD>`__. This reduces the space needed
59for the tools. Understand, however, that every target still needs a
60sysroot because those binaries are target-specific.
61
62The SDK development environment consists of the following:
63
64- The self-contained SDK, which is an architecture-specific
65 cross-toolchain and matching sysroots (target and native) all built
66 by the OpenEmbedded build system (e.g. the SDK). The toolchain and
67 sysroots are based on a `Metadata <&YOCTO_DOCS_REF_URL;#metadata>`__
68 configuration and extensions, which allows you to cross-develop on
69 the host machine for the target hardware. Additionally, the
70 extensible SDK contains the ``devtool`` functionality.
71
72- The Quick EMUlator (QEMU), which lets you simulate target hardware.
73 QEMU is not literally part of the SDK. You must build and include
74 this emulator separately. However, QEMU plays an important role in
75 the development process that revolves around use of the SDK.
76
77In summary, the extensible and standard SDK share many features.
78However, the extensible SDK has powerful development tools to help you
79more quickly develop applications. Following is a table that summarizes
80the primary differences between the standard and extensible SDK types
81when considering which to build:
82
83+-----------------------+-----------------------+-----------------------+
84| *Feature* | *Standard SDK* | *Extensible SDK* |
85+=======================+=======================+=======================+
86| Toolchain | Yes | Yes\* |
87+-----------------------+-----------------------+-----------------------+
88| Debugger | Yes | Yes\* |
89+-----------------------+-----------------------+-----------------------+
90| Size | 100+ MBytes | 1+ GBytes (or 300+ |
91| | | MBytes for minimal |
92| | | w/toolchain) |
93+-----------------------+-----------------------+-----------------------+
94| ``devtool`` | No | Yes |
95+-----------------------+-----------------------+-----------------------+
96| Build Images | No | Yes |
97+-----------------------+-----------------------+-----------------------+
98| Updateable | No | Yes |
99+-----------------------+-----------------------+-----------------------+
100| Managed Sysroot*\* | No | Yes |
101+-----------------------+-----------------------+-----------------------+
102| Installed Packages | No**\* | Yes***\* |
103+-----------------------+-----------------------+-----------------------+
104| Construction | Packages | Shared State |
105+-----------------------+-----------------------+-----------------------+
106
107\* Extensible SDK contains the toolchain and debugger if
108```SDK_EXT_TYPE`` <&YOCTO_DOCS_REF_URL;#var-SDK_EXT_TYPE>`__ is "full"
109or
110```SDK_INCLUDE_TOOLCHAIN`` <&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_TOOLCHAIN>`__
111is "1", which is the default. \*\* Sysroot is managed through the use of
112``devtool``. Thus, it is less likely that you will corrupt your SDK
113sysroot when you try to add additional libraries. \**\* You can add
114runtime package management to the standard SDK but it is not supported
115by default. \***\* You must build and make the shared state available to
116extensible SDK users for "packages" you want to enable users to install.
117
118The Cross-Development Toolchain
119-------------------------------
120
121The `Cross-Development
122Toolchain <&YOCTO_DOCS_REF_URL;#cross-development-toolchain>`__ consists
123of a cross-compiler, cross-linker, and cross-debugger that are used to
124develop user-space applications for targeted hardware. Additionally, for
125an extensible SDK, the toolchain also has built-in ``devtool``
126functionality. This toolchain is created by running a SDK installer
127script or through a `Build
128Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__ that is based on
129your metadata configuration or extension for your targeted device. The
130cross-toolchain works with a matching target sysroot.
131
132.. _sysroot:
133
134Sysroots
135--------
136
137The native and target sysroots contain needed headers and libraries for
138generating binaries that run on the target architecture. The target
139sysroot is based on the target root filesystem image that is built by
140the OpenEmbedded build system and uses the same metadata configuration
141used to build the cross-toolchain.
142
143The QEMU Emulator
144-----------------
145
146The QEMU emulator allows you to simulate your hardware while running
147your application or image. QEMU is not part of the SDK but is made
148available a number of different ways:
149
150- If you have cloned the ``poky`` Git repository to create a `Source
151 Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ and you have
152 sourced the environment setup script, QEMU is installed and
153 automatically available.
154
155- If you have downloaded a Yocto Project release and unpacked it to
156 create a Source Directory and you have sourced the environment setup
157 script, QEMU is installed and automatically available.
158
159- If you have installed the cross-toolchain tarball and you have
160 sourced the toolchain's setup environment script, QEMU is also
161 installed and automatically available.
162
163SDK Development Model
164=====================
165
166Fundamentally, the SDK fits into the development process as follows: The
167SDK is installed on any machine and can be used to develop applications,
168images, and kernels. An SDK can even be used by a QA Engineer or Release
169Engineer. The fundamental concept is that the machine that has the SDK
170installed does not have to be associated with the machine that has the
171Yocto Project installed. A developer can independently compile and test
172an object on their machine and then, when the object is ready for
173integration into an image, they can simply make it available to the
174machine that has the Yocto Project. Once the object is available, the
175image can be rebuilt using the Yocto Project to produce the modified
176image.
177
178You just need to follow these general steps:
179
1801. *Install the SDK for your target hardware:* For information on how to
181 install the SDK, see the "`Installing the
182 SDK <#sdk-installing-the-sdk>`__" section.
183
1842. *Download or Build the Target Image:* The Yocto Project supports
185 several target architectures and has many pre-built kernel images and
186 root filesystem images.
187
188 If you are going to develop your application on hardware, go to the
189 ```machines`` <&YOCTO_MACHINES_DL_URL;>`__ download area and choose a
190 target machine area from which to download the kernel image and root
191 filesystem. This download area could have several files in it that
192 support development using actual hardware. For example, the area
193 might contain ``.hddimg`` files that combine the kernel image with
194 the filesystem, boot loaders, and so forth. Be sure to get the files
195 you need for your particular development process.
196
197 If you are going to develop your application and then run and test it
198 using the QEMU emulator, go to the
199 ```machines/qemu`` <&YOCTO_QEMU_DL_URL;>`__ download area. From this
200 area, go down into the directory for your target architecture (e.g.
201 ``qemux86_64`` for an Intel-based 64-bit architecture). Download the
202 kernel, root filesystem, and any other files you need for your
203 process.
204
205 .. note::
206
207 To use the root filesystem in QEMU, you need to extract it. See
208 the "
209 Extracting the Root Filesystem
210 " section for information on how to extract the root filesystem.
211
2123. *Develop and Test your Application:* At this point, you have the
213 tools to develop your application. If you need to separately install
214 and use the QEMU emulator, you can go to `QEMU Home
215 Page <http://wiki.qemu.org/Main_Page>`__ to download and learn about
216 the emulator. See the "`Using the Quick EMUlator
217 (QEMU) <&YOCTO_DOCS_DEV_URL;#dev-manual-qemu>`__" chapter in the
218 Yocto Project Development Tasks Manual for information on using QEMU
219 within the Yocto Project.
220
221The remainder of this manual describes how to use the extensible and
222standard SDKs. Information also exists in appendix form that describes
223how you can build, install, and modify an SDK.
diff --git a/documentation/sdk-manual/sdk-manual.rst b/documentation/sdk-manual/sdk-manual.rst
new file mode 100644
index 0000000000..2573b76d96
--- /dev/null
+++ b/documentation/sdk-manual/sdk-manual.rst
@@ -0,0 +1,15 @@
1========================================================================================
2Yocto Project Application Development and the Extensible Software Development Kit (eSDK)
3========================================================================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 sdk-intro
10 sdk-extensible
11 sdk-using
12 sdk-working-projects
13 sdk-appendix-obtain
14 sdk-appendix-customizing
15 sdk-appendix-customizing-standard
diff --git a/documentation/sdk-manual/sdk-using.rst b/documentation/sdk-manual/sdk-using.rst
new file mode 100644
index 0000000000..afe72d2b61
--- /dev/null
+++ b/documentation/sdk-manual/sdk-using.rst
@@ -0,0 +1,136 @@
1**********************
2Using the Standard SDK
3**********************
4
5This chapter describes the standard SDK and how to install it.
6Information includes unique installation and setup aspects for the
7standard SDK.
8
9.. note::
10
11 For a side-by-side comparison of main features supported for a
12 standard SDK as compared to an extensible SDK, see the "
13 Introduction
14 " section.
15
16You can use a standard SDK to work on Makefile and Autotools-based
17projects. See the "`Using the SDK Toolchain
18Directly <#sdk-working-projects>`__" chapter for more information.
19
20.. _sdk-standard-sdk-intro:
21
22Why use the Standard SDK and What is in It?
23===========================================
24
25The Standard SDK provides a cross-development toolchain and libraries
26tailored to the contents of a specific image. You would use the Standard
27SDK if you want a more traditional toolchain experience as compared to
28the extensible SDK, which provides an internal build system and the
29``devtool`` functionality.
30
31The installed Standard SDK consists of several files and directories.
32Basically, it contains an SDK environment setup script, some
33configuration files, and host and target root filesystems to support
34usage. You can see the directory structure in the "`Installed Standard
35SDK Directory
36Structure <#sdk-installed-standard-sdk-directory-structure>`__" section.
37
38.. _sdk-installing-the-sdk:
39
40Installing the SDK
41==================
42
43The first thing you need to do is install the SDK on your `Build
44Host <&YOCTO_DOCS_REF_URL;#hardware-build-system-term>`__ by running the
45``*.sh`` installation script.
46
47You can download a tarball installer, which includes the pre-built
48toolchain, the ``runqemu`` script, and support files from the
49appropriate `toolchain <&YOCTO_TOOLCHAIN_DL_URL;>`__ directory within
50the Index of Releases. Toolchains are available for several 32-bit and
5164-bit architectures with the ``x86_64`` directories, respectively. The
52toolchains the Yocto Project provides are based off the
53``core-image-sato`` and ``core-image-minimal`` images and contain
54libraries appropriate for developing against that image.
55
56The names of the tarball installer scripts are such that a string
57representing the host system appears first in the filename and then is
58immediately followed by a string representing the target architecture.
59poky-glibc-host_system-image_type-arch-toolchain-release_version.sh
60Where: host_system is a string representing your development system:
61i686 or x86_64. image_type is the image for which the SDK was built:
62core-image-minimal or core-image-sato. arch is a string representing the
63tuned target architecture: aarch64, armv5e, core2-64, i586, mips32r2,
64mips64, ppc7400, or cortexa8hf-neon. release_version is a string
65representing the release number of the Yocto Project: DISTRO,
66DISTRO+snapshot For example, the following SDK installer is for a 64-bit
67development host system and a i586-tuned target architecture based off
68the SDK for ``core-image-sato`` and using the current DISTRO snapshot:
69poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
70
71.. note::
72
73 As an alternative to downloading an SDK, you can build the SDK
74 installer. For information on building the installer, see the "
75 Building an SDK Installer
76 " section.
77
78The SDK and toolchains are self-contained and by default are installed
79into the ``poky_sdk`` folder in your home directory. You can choose to
80install the extensible SDK in any location when you run the installer.
81However, because files need to be written under that directory during
82the normal course of operation, the location you choose for installation
83must be writable for whichever users need to use the SDK.
84
85The following command shows how to run the installer given a toolchain
86tarball for a 64-bit x86 development host system and a 64-bit x86 target
87architecture. The example assumes the SDK installer is located in
88``~/Downloads/`` and has execution rights.
89
90.. note::
91
92 If you do not have write permissions for the directory into which you
93 are installing the SDK, the installer notifies you and exits. For
94 that case, set up the proper permissions in the directory and run the
95 installer again.
96
97$ ./Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
98Poky (Yocto Project Reference Distro) SDK installer version DISTRO
99=============================================================== Enter
100target directory for SDK (default: /opt/poky/DISTRO): You are about to
101install the SDK to "/opt/poky/DISTRO". Proceed [Y/n]? Y Extracting
102SDK........................................
103..............................done Setting it up...done SDK has been
104successfully set up and is ready to be used. Each time you wish to use
105the SDK in a new shell session, you need to source the environment setup
106script e.g. $ . /opt/poky/DISTRO/environment-setup-i586-poky-linux
107
108Again, reference the "`Installed Standard SDK Directory
109Structure <#sdk-installed-standard-sdk-directory-structure>`__" section
110for more details on the resulting directory structure of the installed
111SDK.
112
113.. _sdk-running-the-sdk-environment-setup-script:
114
115Running the SDK Environment Setup Script
116========================================
117
118Once you have the SDK installed, you must run the SDK environment setup
119script before you can actually use the SDK. This setup script resides in
120the directory you chose when you installed the SDK, which is either the
121default ``/opt/poky/DISTRO`` directory or the directory you chose during
122installation.
123
124Before running the script, be sure it is the one that matches the
125architecture for which you are developing. Environment setup scripts
126begin with the string "``environment-setup``" and include as part of
127their name the tuned target architecture. As an example, the following
128commands set the working directory to where the SDK was installed and
129then source the environment setup script. In this example, the setup
130script is for an IA-based target machine using i586 tuning: $ source
131/opt/poky/DISTRO/environment-setup-i586-poky-linux When you run the
132setup script, the same environment variables are defined as are when you
133run the setup script for an extensible SDK. See the "`Running the
134Extensible SDK Environment Setup
135Script <#sdk-running-the-extensible-sdk-environment-setup-script>`__"
136section for more information.
diff --git a/documentation/sdk-manual/sdk-working-projects.rst b/documentation/sdk-manual/sdk-working-projects.rst
new file mode 100644
index 0000000000..42f8bfeb46
--- /dev/null
+++ b/documentation/sdk-manual/sdk-working-projects.rst
@@ -0,0 +1,284 @@
1********************************
2Using the SDK Toolchain Directly
3********************************
4
5You can use the SDK toolchain directly with Makefile and Autotools-based
6projects.
7
8Autotools-Based Projects
9========================
10
11Once you have a suitable `cross-development
12toolchain <&YOCTO_DOCS_REF_URL;#cross-development-toolchain>`__
13installed, it is very easy to develop a project using the `GNU
14Autotools-based <https://en.wikipedia.org/wiki/GNU_Build_System>`__
15workflow, which is outside of the `OpenEmbedded build
16system <&YOCTO_DOCS_REF_URL;#build-system-term>`__.
17
18The following figure presents a simple Autotools workflow.
19
20Follow these steps to create a simple Autotools-based "Hello World"
21project:
22
23.. note::
24
25 For more information on the GNU Autotools workflow, see the same
26 example on the
27 GNOME Developer
28 site.
29
301. *Create a Working Directory and Populate It:* Create a clean
31 directory for your project and then make that directory your working
32 location. $ mkdir $HOME/helloworld $ cd $HOME/helloworld After
33 setting up the directory, populate it with files needed for the flow.
34 You need a project source file, a file to help with configuration,
35 and a file to help create the Makefile, and a README file:
36 ``hello.c``, ``configure.ac``, ``Makefile.am``, and ``README``,
37 respectively.
38
39 Use the following command to create an empty README file, which is
40 required by GNU Coding Standards: $ touch README Create the remaining
41 three files as follows:
42
43 - *``hello.c``:* #include <stdio.h> main() { printf("Hello
44 World!\n"); }
45
46 - *``configure.ac``:* AC_INIT(hello,0.1) AM_INIT_AUTOMAKE([foreign])
47 AC_PROG_CC AC_CONFIG_FILES(Makefile) AC_OUTPUT
48
49 - *``Makefile.am``:* bin_PROGRAMS = hello hello_SOURCES = hello.c
50
512. *Source the Cross-Toolchain Environment Setup File:* As described
52 earlier in the manual, installing the cross-toolchain creates a
53 cross-toolchain environment setup script in the directory that the
54 SDK was installed. Before you can use the tools to develop your
55 project, you must source this setup script. The script begins with
56 the string "environment-setup" and contains the machine architecture,
57 which is followed by the string "poky-linux". For this example, the
58 command sources a script from the default SDK installation directory
59 that uses the 32-bit Intel x86 Architecture and the DISTRO_NAME Yocto
60 Project release: $ source
61 /opt/poky/DISTRO/environment-setup-i586-poky-linux
62
633. *Create the ``configure`` Script:* Use the ``autoreconf`` command to
64 generate the ``configure`` script. $ autoreconf The ``autoreconf``
65 tool takes care of running the other Autotools such as ``aclocal``,
66 ``autoconf``, and ``automake``.
67
68 .. note::
69
70 If you get errors from
71 configure.ac
72 , which
73 autoreconf
74 runs, that indicate missing files, you can use the "-i" option,
75 which ensures missing auxiliary files are copied to the build
76 host.
77
784. *Cross-Compile the Project:* This command compiles the project using
79 the cross-compiler. The
80 ```CONFIGURE_FLAGS`` <&YOCTO_DOCS_REF_URL;#var-CONFIGURE_FLAGS>`__
81 environment variable provides the minimal arguments for GNU
82 configure: $ ./configure ${CONFIGURE_FLAGS} For an Autotools-based
83 project, you can use the cross-toolchain by just passing the
84 appropriate host option to ``configure.sh``. The host option you use
85 is derived from the name of the environment setup script found in the
86 directory in which you installed the cross-toolchain. For example,
87 the host option for an ARM-based target that uses the GNU EABI is
88 ``armv5te-poky-linux-gnueabi``. You will notice that the name of the
89 script is ``environment-setup-armv5te-poky-linux-gnueabi``. Thus, the
90 following command works to update your project and rebuild it using
91 the appropriate cross-toolchain tools: $ ./configure
92 --host=armv5te-poky-linux-gnueabi --with-libtool-sysroot=sysroot_dir
93
945. *Make and Install the Project:* These two commands generate and
95 install the project into the destination directory: $ make $ make
96 install DESTDIR=./tmp
97
98 .. note::
99
100 To learn about environment variables established when you run the
101 cross-toolchain environment setup script and how they are used or
102 overridden when the Makefile, see the "
103 Makefile-Based Projects
104 " section.
105
106 This next command is a simple way to verify the installation of your
107 project. Running the command prints the architecture on which the
108 binary file can run. This architecture should be the same
109 architecture that the installed cross-toolchain supports. $ file
110 ./tmp/usr/local/bin/hello
111
1126. *Execute Your Project:* To execute the project, you would need to run
113 it on your target hardware. If your target hardware happens to be
114 your build host, you could run the project as follows: $
115 ./tmp/usr/local/bin/hello As expected, the project displays the
116 "Hello World!" message.
117
118Makefile-Based Projects
119=======================
120
121Simple Makefile-based projects use and interact with the cross-toolchain
122environment variables established when you run the cross-toolchain
123environment setup script. The environment variables are subject to
124general ``make`` rules.
125
126This section presents a simple Makefile development flow and provides an
127example that lets you see how you can use cross-toolchain environment
128variables and Makefile variables during development.
129
130The main point of this section is to explain the following three cases
131regarding variable behavior:
132
133- *Case 1 - No Variables Set in the ``Makefile`` Map to Equivalent
134 Environment Variables Set in the SDK Setup Script:* Because matching
135 variables are not specifically set in the ``Makefile``, the variables
136 retain their values based on the environment setup script.
137
138- *Case 2 - Variables Are Set in the Makefile that Map to Equivalent
139 Environment Variables from the SDK Setup Script:* Specifically
140 setting matching variables in the ``Makefile`` during the build
141 results in the environment settings of the variables being
142 overwritten. In this case, the variables you set in the ``Makefile``
143 are used.
144
145- *Case 3 - Variables Are Set Using the Command Line that Map to
146 Equivalent Environment Variables from the SDK Setup Script:*
147 Executing the ``Makefile`` from the command line results in the
148 environment variables being overwritten. In this case, the
149 command-line content is used.
150
151.. note::
152
153 Regardless of how you set your variables, if you use the "-e" option
154 with
155 make
156 , the variables from the SDK setup script take precedence:
157 ::
158
159 $ make -e target
160
161
162The remainder of this section presents a simple Makefile example that
163demonstrates these variable behaviors.
164
165In a new shell environment variables are not established for the SDK
166until you run the setup script. For example, the following commands show
167a null value for the compiler variable (i.e.
168```CC`` <&YOCTO_DOCS_REF_URL;#var-CC>`__). $ echo ${CC} $ Running the
169SDK setup script for a 64-bit build host and an i586-tuned target
170architecture for a ``core-image-sato`` image using the current DISTRO
171Yocto Project release and then echoing that variable shows the value
172established through the script: $ source
173/opt/poky/DISTRO/environment-setup-i586-poky-linux $ echo ${CC}
174i586-poky-linux-gcc -m32 -march=i586
175--sysroot=/opt/poky/2.5/sysroots/i586-poky-linux
176
177To illustrate variable use, work through this simple "Hello World!"
178example:
179
1801. *Create a Working Directory and Populate It:* Create a clean
181 directory for your project and then make that directory your working
182 location. $ mkdir $HOME/helloworld $ cd $HOME/helloworld After
183 setting up the directory, populate it with files needed for the flow.
184 You need a ``main.c`` file from which you call your function, a
185 ``module.h`` file to contain headers, and a ``module.c`` that defines
186 your function.
187
188 Create the three files as follows:
189
190 - *``main.c``:* #include "module.h" void sample_func(); int main() {
191 sample_func(); return 0; }
192
193 - *``module.h``:* #include <stdio.h> void sample_func();
194
195 - *``module.c``:* #include "module.h" void sample_func() {
196 printf("Hello World!"); printf("\n"); }
197
1982. *Source the Cross-Toolchain Environment Setup File:* As described
199 earlier in the manual, installing the cross-toolchain creates a
200 cross-toolchain environment setup script in the directory that the
201 SDK was installed. Before you can use the tools to develop your
202 project, you must source this setup script. The script begins with
203 the string "environment-setup" and contains the machine architecture,
204 which is followed by the string "poky-linux". For this example, the
205 command sources a script from the default SDK installation directory
206 that uses the 32-bit Intel x86 Architecture and the DISTRO_NAME Yocto
207 Project release: $ source
208 /opt/poky/DISTRO/environment-setup-i586-poky-linux
209
2103. *Create the ``Makefile``:* For this example, the Makefile contains
211 two lines that can be used to set the ``CC`` variable. One line is
212 identical to the value that is set when you run the SDK environment
213 setup script, and the other line sets ``CC`` to "gcc", the default
214 GNU compiler on the build host: # CC=i586-poky-linux-gcc -m32
215 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux #
216 CC="gcc" all: main.o module.o ${CC} main.o module.o -o target_bin
217 main.o: main.c module.h ${CC} -I . -c main.c module.o: module.c
218 module.h ${CC} -I . -c module.c clean: rm -rf \*.o rm target_bin
219
2204. *Make the Project:* Use the ``make`` command to create the binary
221 output file. Because variables are commented out in the Makefile, the
222 value used for ``CC`` is the value set when the SDK environment setup
223 file was run: $ make i586-poky-linux-gcc -m32 -march=i586
224 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
225 i586-poky-linux-gcc -m32 -march=i586
226 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
227 i586-poky-linux-gcc -m32 -march=i586
228 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o
229 target_bin From the results of the previous command, you can see that
230 the compiler used was the compiler established through the ``CC``
231 variable defined in the setup script.
232
233 You can override the ``CC`` environment variable with the same
234 variable as set from the Makefile by uncommenting the line in the
235 Makefile and running ``make`` again. $ make clean rm -rf \*.o rm
236 target_bin # # Edit the Makefile by uncommenting the line that sets
237 CC to "gcc" # $ make gcc -I . -c main.c gcc -I . -c module.c gcc
238 main.o module.o -o target_bin As shown in the previous example, the
239 cross-toolchain compiler is not used. Rather, the default compiler is
240 used.
241
242 This next case shows how to override a variable by providing the
243 variable as part of the command line. Go into the Makefile and
244 re-insert the comment character so that running ``make`` uses the
245 established SDK compiler. However, when you run ``make``, use a
246 command-line argument to set ``CC`` to "gcc": $ make clean rm -rf
247 \*.o rm target_bin # # Edit the Makefile to comment out the line
248 setting CC to "gcc" # $ make i586-poky-linux-gcc -m32 -march=i586
249 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
250 i586-poky-linux-gcc -m32 -march=i586
251 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
252 i586-poky-linux-gcc -m32 -march=i586
253 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o
254 target_bin $ make clean rm -rf \*.o rm target_bin $ make CC="gcc" gcc
255 -I . -c main.c gcc -I . -c module.c gcc main.o module.o -o target_bin
256 In the previous case, the command-line argument overrides the SDK
257 environment variable.
258
259 In this last case, edit Makefile again to use the "gcc" compiler but
260 then use the "-e" option on the ``make`` command line: $ make clean
261 rm -rf \*.o rm target_bin # # Edit the Makefile to use "gcc" # $ make
262 gcc -I . -c main.c gcc -I . -c module.c gcc main.o module.o -o
263 target_bin $ make clean rm -rf \*.o rm target_bin $ make -e
264 i586-poky-linux-gcc -m32 -march=i586
265 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
266 i586-poky-linux-gcc -m32 -march=i586
267 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
268 i586-poky-linux-gcc -m32 -march=i586
269 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o
270 target_bin In the previous case, the "-e" option forces ``make`` to
271 use the SDK environment variables regardless of the values in the
272 Makefile.
273
2745. *Execute Your Project:* To execute the project (i.e. ``target_bin``),
275 use the following command: $ ./target_bin Hello World!
276
277 .. note::
278
279 If you used the cross-toolchain compiler to build
280 target_bin
281 and your build host differs in architecture from that of the
282 target machine, you need to run your project on the target device.
283
284 As expected, the project displays the "Hello World!" message.
diff --git a/documentation/test-manual/test-manual-intro.rst b/documentation/test-manual/test-manual-intro.rst
new file mode 100644
index 0000000000..491c4bad9a
--- /dev/null
+++ b/documentation/test-manual/test-manual-intro.rst
@@ -0,0 +1,486 @@
1*****************************************
2The Yocto Project Test Environment Manual
3*****************************************
4
5.. _test-welcome:
6
7Welcome
8=======
9
10Welcome to the Yocto Project Test Environment Manual! This manual is a
11work in progress. The manual contains information about the testing
12environment used by the Yocto Project to make sure each major and minor
13release works as intended. All the project’s testing infrastructure and
14processes are publicly visible and available so that the community can
15see what testing is being performed, how it’s being done and the current
16status of the tests and the project at any given time. It is intended
17that Other organizations can leverage off the process and testing
18environment used by the Yocto Project to create their own automated,
19production test environment, building upon the foundations from the
20project core.
21
22Currently, the Yocto Project Test Environment Manual has no projected
23release date. This manual is a work-in-progress and is being initially
24loaded with information from the `README <>`__ files and notes from key
25engineers:
26
27- *``yocto-autobuilder2``:* This
28 ```README.md`` <http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md>`__
29 is the main README which detials how to set up the Yocto Project
30 Autobuilder. The ``yocto-autobuilder2`` repository represents the
31 Yocto Project's console UI plugin to Buildbot and the configuration
32 necessary to configure Buildbot to perform the testing the project
33 requires.
34
35- *``yocto-autobuilder-helper``:* This
36 ```README`` <http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README>`__
37 and repository contains Yocto Project Autobuilder Helper scripts and
38 configuration. The ``yocto-autobuilder-helper`` repository contains
39 the "glue" logic that defines which tests to run and how to run them.
40 As a result, it can be used by any Continuous Improvement (CI) system
41 to run builds, support getting the correct code revisions, configure
42 builds and layers, run builds, and collect results. The code is
43 independent of any CI system, which means the code can work Buildbot,
44 Jenkins, or others. This repository has a branch per release of the
45 project defining the tests to run on a per release basis.
46
47.. _test-yocto-project-autobuilder-overview:
48
49Yocto Project Autobuilder Overview
50==================================
51
52The Yocto Project Autobuilder collectively refers to the software,
53tools, scripts, and procedures used by the Yocto Project to test
54released software across supported hardware in an automated and regular
55fashion. Basically, during the development of a Yocto Project release,
56the Autobuilder tests if things work. The Autobuilder builds all test
57targets and runs all the tests.
58
59The Yocto Project uses now uses standard upstream
60`Buildbot <https://docs.buildbot.net/0.9.15.post1/>`__ (version 9) to
61drive its integration and testing. Buildbot Nine has a plug-in interface
62that the Yocto Project customizes using code from the
63``yocto-autobuilder2`` repository, adding its own console UI plugin. The
64resulting UI plug-in allows you to visualize builds in a way suited to
65the project's needs.
66
67A ``helper`` layer provides configuration and job management through
68scripts found in the ``yocto-autobuilder-helper`` repository. The
69``helper`` layer contains the bulk of the build configuration
70information and is release-specific, which makes it highly customizable
71on a per-project basis. The layer is CI system-agnostic and contains a
72number of Helper scripts that can generate build configurations from
73simple JSON files.
74
75.. note::
76
77 The project uses Buildbot for historical reasons but also because
78 many of the project developers have knowledge of python. It is
79 possible to use the outer layers from another Continuous Integration
80 (CI) system such as
81 `Jenkins <https://en.wikipedia.org/wiki/Jenkins_(software)>`__
82 instead of Buildbot.
83
84The following figure shows the Yocto Project Autobuilder stack with a
85topology that includes a controller and a cluster of workers:
86
87.. _test-project-tests:
88
89Yocto Project Tests - Types of Testing Overview
90===============================================
91
92The Autobuilder tests different elements of the project by using
93thefollowing types of tests:
94
95- *Build Testing:* Tests whether specific configurations build by
96 varying ```MACHINE`` <&YOCTO_DOCS_REF_URL;#var-MACHINE>`__,
97 ```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__, other configuration
98 options, and the specific target images being built (or world). Used
99 to trigger builds of all the different test configurations on the
100 Autobuilder. Builds usually cover many different targets for
101 different architectures, machines, and distributions, as well as
102 different configurations, such as different init systems. The
103 Autobuilder tests literally hundreds of configurations and targets.
104
105 - *Sanity Checks During the Build Process:* Tests initiated through
106 the ```insane`` <&YOCTO_DOCS_REF_URL;#ref-classes-insane>`__
107 class. These checks ensure the output of the builds are correct.
108 For example, does the ELF architecture in the generated binaries
109 match the target system? ARM binaries would not work in a MIPS
110 system!
111
112- *Build Performance Testing:* Tests whether or not commonly used steps
113 during builds work efficiently and avoid regressions. Tests to time
114 commonly used usage scenarios are run through ``oe-build-perf-test``.
115 These tests are run on isolated machines so that the time
116 measurements of the tests are accurate and no other processes
117 interfere with the timing results. The project currently tests
118 performance on two different distributions, Fedora and Ubuntu, to
119 ensure we have no single point of failure and can ensure the
120 different distros work effectively.
121
122- *eSDK Testing:* Image tests initiated through the following command:
123 $ bitbake image -c testsdkext The tests utilize the ``testsdkext``
124 class and the ``do_testsdkext`` task.
125
126- *Feature Testing:* Various scenario-based tests are run through the
127 `OpenEmbedded
128 Self-Test <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__
129 (oe-selftest). We test oe-selftest on each of the main distrubutions
130 we support.
131
132- *Image Testing:* Image tests initiated through the following command:
133 $ bitbake image -c testimage The tests utilize the
134 ```testimage*`` <&YOCTO_DOCS_REF_URL;#ref-classes-testimage*>`__
135 classes and the
136 ```do_testimage`` <&YOCTO_DOCS_REF_URL;#ref-tasks-testimage>`__ task.
137
138- *Layer Testing:* The Autobuilder has the possibility to test whether
139 specific layers work with the test of the system. The layers tested
140 may be selected by members of the project. Some key community layers
141 are also tested periodically.
142
143- *Package Testing:* A Package Test (ptest) runs tests against packages
144 built by the OpenEmbedded build system on the target machine. See the
145 "`Testing Packages With
146 ptest <&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest>`__" section
147 in the Yocto Project Development Tasks Manual and the
148 "`Ptest <&YOCTO_WIKI_URL;/wiki/Ptest>`__" Wiki page for more
149 information on Ptest.
150
151- *SDK Testing:* Image tests initiated through the following command: $
152 bitbake image -c testsdk The tests utilize the
153 ```testsdk`` <&YOCTO_DOCS_REF_URL;#ref-classes-testsdk>`__ class and
154 the ``do_testsdk`` task.
155
156- *Unit Testing:* Unit tests on various components of the system run
157 through ``oe-selftest`` and
158 ```bitbake-selftest`` <&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance>`__.
159
160- *Automatic Upgrade Helper:* This target tests whether new versions of
161 software are available and whether we can automatically upgrade to
162 those new versions. If so, this target emails the maintainers with a
163 patch to let them know this is possible.
164
165.. _test-test-mapping:
166
167How Tests Map to Areas of Code
168==============================
169
170Tests map into the codebase as follows:
171
172- *bitbake-selftest*:
173
174 These tests are self-contained and test BitBake as well as its APIs,
175 which include the fetchers. The tests are located in
176 ``bitbake/lib/*/tests``.
177
178 From within the BitBake repository, run the following: $
179 bitbake-selftest
180
181 To skip tests that access the Internet, use the ``BB_SKIP_NETTEST``
182 variable when running "bitbake-selftest" as follows: $
183 BB_SKIP_NETTEST=yes bitbake-selftest
184
185 The default output is quiet and just prints a summary of what was
186 run. To see more information, there is a verbose option:$
187 bitbake-selftest -v
188
189 Use this option when you wish to skip tests that access the network,
190 which are mostly necessary to test the fetcher modules. To specify
191 individual test modules to run, append the test module name to the
192 "bitbake-selftest" command. For example, to specify the tests for the
193 bb.data.module, run: $ bitbake-selftest bb.test.data.moduleYou can
194 also specify individual tests by defining the full name and module
195 plus the class path of the test, for example: $ bitbake-selftest
196 bb.tests.data.TestOverrides.test_one_override
197
198 The tests are based on `Python
199 unittest <https://docs.python.org/3/library/unittest.html>`__.
200
201- *oe-selftest*:
202
203 - These tests use OE to test the workflows, which include testing
204 specific features, behaviors of tasks, and API unit tests.
205
206 - The tests can take advantage of parallelism through the "-j"
207 option, which can specify a number of threads to spread the tests
208 across. Note that all tests from a given class of tests will run
209 in the same thread. To parallelize large numbers of tests you can
210 split the class into multiple units.
211
212 - The tests are based on Python unittest.
213
214 - The code for the tests resides in
215 ``meta/lib/oeqa/selftest/cases/``.
216
217 - To run all the tests, enter the following command: $ oe-selftest
218 -a
219
220 - To run a specific test, use the following command form where
221 testname is the name of the specific test: $ oe-selftest -r
222 testname For example, the following command would run the tinfoil
223 getVar API test:$ oe-selftest -r
224 tinfoil.TinfoilTests.test_getvarIt is also possible to run a set
225 of tests. For example the following command will run all of the
226 tinfoil tests:$ oe-selftest -r tinfoil
227
228- *testimage:*
229
230 - These tests build an image, boot it, and run tests against the
231 image's content.
232
233 - The code for these tests resides in
234 ``meta/lib/oeqa/runtime/cases/``.
235
236 - You need to set the
237 ```IMAGE_CLASSES`` <&YOCTO_DOCS_REF_URL;#var-IMAGE_CLASSES>`__
238 variable as follows: IMAGE_CLASSES += "testimage"
239
240 - Run the tests using the following command form: $ bitbake image -c
241 testimage
242
243- *testsdk:*
244
245 - These tests build an SDK, install it, and then run tests against
246 that SDK.
247
248 - The code for these tests resides in ``meta/lib/oeqa/sdk/cases/``.
249
250 - Run the test using the following command form: $ bitbake image -c
251 testsdk
252
253- *testsdk_ext:*
254
255 - These tests build an extended SDK (eSDK), install that eSDK, and
256 run tests against the eSDK.
257
258 - The code for these tests resides in ``meta/lib/oeqa/esdk``.
259
260 - To run the tests, use the following command form: $ bitbake image
261 -c testsdkext
262
263- *oe-build-perf-test:*
264
265 - These tests run through commonly used usage scenarios and measure
266 the performance times.
267
268 - The code for these tests resides in ``meta/lib/oeqa/buildperf``.
269
270 - To run the tests, use the following command form: $
271 oe-build-perf-test optionsThe command takes a number of options,
272 such as where to place the test results. The Autobuilder Helper
273 Scripts include the ``build-perf-test-wrapper`` script with
274 examples of how to use the oe-build-perf-test from the command
275 line.
276
277 Use the ``oe-git-archive`` command to store test results into a
278 Git repository.
279
280 Use the ``oe-build-perf-report`` command to generate text reports
281 and HTML reports with graphs of the performance data. For
282 examples, see
283 `http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html <#>`__
284 and
285 `http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt <#>`__.
286
287 - The tests are contained in ``lib/oeqa/buildperf/test_basic.py``.
288
289Test Examples
290=============
291
292This section provides example tests for each of the tests listed in the
293`How Tests Map to Areas of Code <#test-test-mapping>`__ section.
294
295For oeqa tests, testcases for each area reside in the main test
296directory at ``meta/lib/oeqa/selftest/cases`` directory.
297
298For oe-selftest. bitbake testcases reside in the ``lib/bb/tests/``
299directory.
300
301.. _bitbake-selftest-example:
302
303``bitbake-selftest``
304--------------------
305
306A simple test example from ``lib/bb/tests/data.py`` is: class
307DataExpansions(unittest.TestCase): def setUp(self): self.d =
308bb.data.init() self.d["foo"] = "value_of_foo" self.d["bar"] =
309"value_of_bar" self.d["value_of_foo"] = "value_of_'value_of_foo'" def
310test_one_var(self): val = self.d.expand("${foo}")
311self.assertEqual(str(val), "value_of_foo")
312
313In this example, a ```DataExpansions`` <>`__ class of tests is created,
314derived from standard python unittest. The class has a common ``setUp``
315function which is shared by all the tests in the class. A simple test is
316then added to test that when a variable is expanded, the correct value
317is found.
318
319Bitbake selftests are straightforward python unittest. Refer to the
320Python unittest documentation for additional information on writing
321these tests at: `https://docs.python.org/3/library/unittest.html <#>`__.
322
323.. _oe-selftest-example:
324
325``oe-selftest``
326---------------
327
328These tests are more complex due to the setup required behind the scenes
329for full builds. Rather than directly using Python's unittest, the code
330wraps most of the standard objects. The tests can be simple, such as
331testing a command from within the OE build environment using the
332following example:class BitbakeLayers(OESelftestTestCase): def
333test_bitbakelayers_showcrossdepends(self): result =
334runCmd('bitbake-layers show-cross-depends') self.assertTrue('aspell' in
335result.output, msg = "No dependencies were shown. bitbake-layers
336show-cross-depends output: %s"% result.output)
337
338This example, taken from ``meta/lib/oeqa/selftest/cases/bblayers.py``,
339creates a testcase from the ```OESelftestTestCase`` <>`__ class, derived
340from ``unittest.TestCase``, which runs the ``bitbake-layers`` command
341and checks the output to ensure it contains something we know should be
342here.
343
344The ``oeqa.utils.commands`` module contains Helpers which can assist
345with common tasks, including:
346
347- *Obtaining the value of a bitbake variable:* Use
348 ``oeqa.utils.commands.get_bb_var()`` or use
349 ``oeqa.utils.commands.get_bb_vars()`` for more than one variable
350
351- *Running a bitbake invocation for a build:* Use
352 ``oeqa.utils.commands.bitbake()``
353
354- *Running a command:* Use ``oeqa.utils.commandsrunCmd()``
355
356There is also a ``oeqa.utils.commands.runqemu()`` function for launching
357the ``runqemu`` command for testing things within a running, virtualized
358image.
359
360You can run these tests in parallel. Parallelism works per test class,
361so tests within a given test class should always run in the same build,
362while tests in different classes or modules may be split into different
363builds. There is no data store available for these tests since the tests
364launch the ``bitbake`` command and exist outside of its context. As a
365result, common bitbake library functions (bb.*) are also unavailable.
366
367.. _testimage-example:
368
369``testimage``
370-------------
371
372These tests are run once an image is up and running, either on target
373hardware or under QEMU. As a result, they are assumed to be running in a
374target image environment, as opposed to a host build environment. A
375simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains
376the following:class PythonTest(OERuntimeTestCase):
377@OETestDepends(['ssh.SSHTest.test_ssh']) @OEHasPackage(['python3-core'])
378def test_python3(self): cmd = "python3 -c \\"import codecs;
379print(codecs.encode('Uryyb, jbeyq', 'rot13'))\"" status, output =
380self.target.run(cmd) msg = 'Exit status was not 0. Output: %s' % output
381self.assertEqual(status, 0, msg=msg)
382
383In this example, the ```OERuntimeTestCase`` <>`__ class wraps
384``unittest.TestCase``. Within the test, ``self.target`` represents the
385target system, where commands can be run on it using the ``run()``
386method.
387
388To ensure certain test or package dependencies are met, you can use the
389``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test
390in this example would only make sense if python3-core is installed in
391the image.
392
393.. _testsdk_ext-example:
394
395``testsdk_ext``
396---------------
397
398These tests are run against built extensible SDKs (eSDKs). The tests can
399assume that the eSDK environment has already been setup. An example from
400``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following:class
401DevtoolTest(OESDKExtTestCase): @classmethod def setUpClass(cls):
402myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp") cls.myapp_dst =
403os.path.join(cls.tc.sdk_dir, "myapp") shutil.copytree(myapp_src,
404cls.myapp_dst) subprocess.check_output(['git', 'init', '.'],
405cwd=cls.myapp_dst) subprocess.check_output(['git', 'add', '.'],
406cwd=cls.myapp_dst) subprocess.check_output(['git', 'commit', '-m',
407"'test commit'"], cwd=cls.myapp_dst) @classmethod def
408tearDownClass(cls): shutil.rmtree(cls.myapp_dst) def
409\_test_devtool_build(self, directory): self._run('devtool add myapp %s'
410% directory) try: self._run('devtool build myapp') finally:
411self._run('devtool reset myapp') def test_devtool_build_make(self):
412self._test_devtool_build(self.myapp_dst)In this example, the ``devtool``
413command is tested to see whether a sample application can be built with
414the ``devtool build`` command within the eSDK.
415
416.. _testsdk-example:
417
418``testsdk``
419-----------
420
421These tests are run against built SDKs. The tests can assume that an SDK
422has already been extracted and its environment file has been sourced. A
423simple example from ``meta/lib/oeqa/sdk/cases/python2.py`` contains the
424following:class Python3Test(OESDKTestCase): def setUp(self): if not
425(self.tc.hasHostPackage("nativesdk-python3-core") or
426self.tc.hasHostPackage("python3-core-native")): raise
427unittest.SkipTest("No python3 package in the SDK") def
428test_python3(self): cmd = "python3 -c \\"import codecs;
429print(codecs.encode('Uryyb, jbeyq', 'rot13'))\"" output = self._run(cmd)
430self.assertEqual(output, "Hello, world\n")In this example, if
431nativesdk-python3-core has been installed into the SDK, the code runs
432the python3 interpreter with a basic command to check it is working
433correctly. The test would only run if python3 is installed in the SDK.
434
435.. _oe-build-perf-test-example:
436
437``oe-build-perf-test``
438----------------------
439
440The performance tests usually measure how long operations take and the
441resource utilisation as that happens. An example from
442``meta/lib/oeqa/buildperf/test_basic.py`` contains the following:class
443Test3(BuildPerfTestCase): def test3(self): """Bitbake parsing (bitbake
444-p)""" # Drop all caches and parse self.rm_cache()
445oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
446self.measure_cmd_resources(['bitbake', '-p'], 'parse_1', 'bitbake -p (no
447caches)') # Drop tmp/cache
448oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
449self.measure_cmd_resources(['bitbake', '-p'], 'parse_2', 'bitbake -p (no
450tmp/cache)') # Parse with fully cached data
451self.measure_cmd_resources(['bitbake', '-p'], 'parse_3', 'bitbake -p
452(cached)')This example shows how three specific parsing timings are
453measured, with and without various caches, to show how BitBake’s parsing
454performance trends over time.
455
456.. _test-writing-considerations:
457
458Considerations When Writing Tests
459=================================
460
461When writing good tests, there are several things to keep in mind. Since
462things running on the Autobuilder are accessed concurrently by multiple
463workers, consider the following:
464
465**Running "cleanall" is not permitted.**
466
467This can delete files from DL_DIR which would potentially break other
468builds running in parallel. If this is required, DL_DIR must be set to
469an isolated directory.
470
471**Running "cleansstate" is not permitted.**
472
473This can delete files from SSTATE_DIR which would potentially break
474other builds running in parallel. If this is required, SSTATE_DIR must
475be set to an isolated directory. Alternatively, you can use the "-f"
476option with the ``bitbake`` command to "taint" tasks by changing the
477sstate checksums to ensure sstate cache items will not be reused.
478
479**Tests should not change the metadata.**
480
481This is particularly true for oe-selftests since these can run in
482parallel and changing metadata leads to changing checksums, which
483confuses BitBake while running in parallel. If this is necessary, copy
484layers to a temporary location and modify them. Some tests need to
485change metadata, such as the devtool tests. To prevent the metadate from
486changes, set up temporary copies of that data first.
diff --git a/documentation/test-manual/test-manual-test-process.rst b/documentation/test-manual/test-manual-test-process.rst
new file mode 100644
index 0000000000..19c9b565de
--- /dev/null
+++ b/documentation/test-manual/test-manual-test-process.rst
@@ -0,0 +1,103 @@
1***********************************
2Project Testing and Release Process
3***********************************
4
5.. _test-daily-devel:
6
7Day to Day Development
8======================
9
10This section details how the project tests changes, through automation
11on the Autobuilder or with the assistance of QA teams, through to making
12releases.
13
14The project aims to test changes against our test matrix before those
15changes are merged into the master branch. As such, changes are queued
16up in batches either in the ``master-next`` branch in the main trees, or
17in user trees such as ``ross/mut`` in ``poky-contrib`` (Ross Burton
18helps review and test patches and this is his testing tree).
19
20We have two broad categories of test builds, including "full" and
21"quick". On the Autobuilder, these can be seen as "a-quick" and
22"a-full", simply for ease of sorting in the UI. Use our Autobuilder
23console view to see where me manage most test-related items, available
24at: `https://autobuilder.yoctoproject.org/typhoon/#/console <#>`__.
25
26Builds are triggered manually when the test branches are ready. The
27builds are monitored by the SWAT team. For additional information, see
28`https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team <#>`__.
29If successful, the changes would usually be merged to the ``master``
30branch. If not successful, someone would respond to the changes on the
31mailing list explaining that there was a failure in testing. The choice
32of quick or full would depend on the type of changes and the speed with
33which the result was required.
34
35The Autobuilder does build the ``master`` branch once daily for several
36reasons, in particular, to ensure the current ``master`` branch does
37build, but also to keep ``yocto-testresults``
38(`http://git.yoctoproject.org/cgit.cgi/yocto-testresults/ <#>`__),
39buildhistory
40(`http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/ <#>`__), and
41our sstate up to date. On the weekend, there is a master-next build
42instead to ensure the test results are updated for the less frequently
43run targets.
44
45Performance builds (buildperf-\* targets in the console) are triggered
46separately every six hours and automatically push their results to the
47buildstats repository at:
48`http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/ <#>`__.
49
50The 'quick' targets have been selected to be the ones which catch the
51most failures or give the most valuable data. We run 'fast' ptests in
52this case for example but not the ones which take a long time. The quick
53target doesn't include \*-lsb builds for all architectures, some world
54builds and doesn't trigger performance tests or ltp testing. The full
55build includes all these things and is slower but more comprehensive.
56
57.. _test-yocto-project-autobuilder-overview:
58
59Release Builds
60==============
61
62The project typically has two major releases a year with a six month
63cadence in April and October. Between these there would be a number of
64milestone releases (usually four) with the final one being stablization
65only along with point releases of our stable branches.
66
67The build and release process for these project releases is similar to
68that in `Day to Day Development <#test-daily-devel>`__, in that the
69a-full target of the Autobuilder is used but in addition the form is
70configured to generate and publish artefacts and the milestone number,
71version, release candidate number and other information is entered. The
72box to "generate an email to QA"is also checked.
73
74When the build completes, an email is sent out using the send-qa-email
75script in the ``yocto-autobuilder-helper`` repository to the list of
76people configured for that release. Release builds are placed into a
77directory in `https://autobuilder.yocto.io/pub/releases <#>`__ on the
78Autobuilder which is included in the email. The process from here is
79more manual and control is effectively passed to release engineering.
80The next steps include:
81
82- QA teams respond to the email saying which tests they plan to run and
83 when the results will be available.
84
85- QA teams run their tests and share their results in the yocto-
86 testresults-contrib repository, along with a summary of their
87 findings.
88
89- Release engineering prepare the release as per their process.
90
91- Test results from the QA teams are included into the release in
92 separate directories and also uploaded to the yocto-testresults
93 repository alongside the other test results for the given revision.
94
95- The QA report in the final release is regenerated using resulttool to
96 include the new test results and the test summaries from the teams
97 (as headers to the generated report).
98
99- The release is checked against the release checklist and release
100 readiness criteria.
101
102- A final decision on whether to release is made by the YP TSC who have
103 final oversight on release readiness.
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.rst b/documentation/test-manual/test-manual-understand-autobuilder.rst
new file mode 100644
index 0000000000..69700088aa
--- /dev/null
+++ b/documentation/test-manual/test-manual-understand-autobuilder.rst
@@ -0,0 +1,287 @@
1*******************************************
2Understanding the Yocto Project Autobuilder
3*******************************************
4
5Execution Flow within the Autobuilder
6=====================================
7
8The “a-full” and “a-quick” targets are the usual entry points into the
9Autobuilder and it makes sense to follow the process through the system
10starting there. This is best visualised from the Autobuilder Console
11view (`https://autobuilder.yoctoproject.org/typhoon/#/console <#>`__).
12
13Each item along the top of that view represents some “target build” and
14these targets are all run in parallel. The ‘full’ build will trigger the
15majority of them, the “quick” build will trigger some subset of them.
16The Autobuilder effectively runs whichever configuration is defined for
17each of those targets on a seperate buildbot worker. To understand the
18configuration, you need to look at the entry on ``config.json`` file
19within the ``yocto-autobuilder-helper`` repository. The targets are
20defined in the ‘overrides’ section, a quick example could be qemux86-64
21which looks like:"qemux86-64" : { "MACHINE" : "qemux86-64", "TEMPLATE" :
22"arch-qemu", "step1" : { "extravars" : [ "IMAGE_FSTYPES_append = ' wic
23wic.bmap'" ] } },And to expand that, you need the “arch-qemu” entry from
24the “templates” section, which looks like:"arch-qemu" : { "BUILDINFO" :
25true, "BUILDHISTORY" : true, "step1" : { "BBTARGETS" : "core-image-sato
26core-image-sato-dev core-image-sato-sdk core-image-minimal
27core-image-minimal-dev core-image-sato:do_populate_sdk", "SANITYTARGETS"
28: "core-image-minimal:do_testimage core-image-sato:do_testimage
29core-image-sato-sdk:do_testimage core-image-sato:do_testsdk" }, "step2"
30: { "SDKMACHINE" : "x86_64", "BBTARGETS" :
31"core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext
32core-image-sato:do_populate_sdk_ext", "SANITYTARGETS" :
33"core-image-sato:do_testsdk core-image-minimal:do_testsdkext
34core-image-sato:do_testsdkext" }, "step3" : { "BUILDHISTORY" : false,
35"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest
36${HELPERSTMACHTARGS} -j 15"], "ADDLAYER" :
37["${BUILDDIR}/../meta-selftest"] } },Combining these two entries you can
38see that “qemux86-64” is a three step build where the
39``bitbake BBTARGETS`` would be run, then ``bitbake
40 SANITYTARGETS`` for each step; all for
41``MACHINE=”qemx86-64”`` but with differing SDKMACHINE settings. In step
421 an extra variable is added to the ``auto.conf`` file to enable wic
43image generation.
44
45While not every detail of this is covered here, you can see how the
46templating mechanism allows quite complex configurations to be built up
47yet allows duplication and repetition to be kept to a minimum.
48
49The different build targets are designed to allow for parallelisation,
50so different machines are usually built in parallel, operations using
51the same machine and metadata are built sequentially, with the aim of
52trying to optimise build efficiency as much as possible.
53
54The ``config.json`` file is processed by the scripts in the Helper
55repository in the ``scripts`` directory. The following section details
56how this works.
57
58.. _test-autobuilder-target-exec-overview:
59
60Autobuilder Target Execution Overview
61=====================================
62
63For each given target in a build, the Autobuilder executes several
64steps. These are configured in ``yocto-autobuilder2/builders.py`` and
65roughly consist of:
66
671. *Run ``clobberdir``*
68
69 This cleans out any previous build. Old builds are left around to
70 allow easier debugging of failed builds. For additional information,
71 see ```clobberdir`` <#test-clobberdir>`__.
72
732. *Obtain yocto-autobuilder-helper*
74
75 This step clones the ``yocto-autobuilder-helper`` git repository.
76 This is necessary to prevent the requirement to maintain all the
77 release or project-specific code within Buildbot. The branch chosen
78 matches the release being built so we can support older releases and
79 still make changes in newer ones.
80
813. *Write layerinfo.json*
82
83 This transfers data in the Buildbot UI when the build was configured
84 to the Helper.
85
864. *Call scripts/shared-repo-unpack*
87
88 This is a call into the Helper scripts to set up a checkout of all
89 the pieces this build might need. It might clone the BitBake
90 repository and the OpenEmbedded-Core repository. It may clone the
91 Poky repository, as well as additional layers. It will use the data
92 from the ``layerinfo.json`` file to help understand the
93 configuration. It will also use a local cache of repositories to
94 speed up the clone checkouts. For additional information, see
95 `Autobuilder Clone Cache <#test-autobuilder-clone-cache>`__.
96
97 This step has two possible modes of operation. If the build is part
98 of a parent build, its possible that all the repositories needed may
99 already be available, ready in a pre-prepared directory. An "a-quick"
100 or "a-full" build would prepare this before starting the other
101 sub-target builds. This is done for two reasons:
102
103 - the upstream may change during a build, for example, from a forced
104 push and this ensures we have matching content for the whole build
105
106 - if 15 Workers all tried to pull the same data from the same repos,
107 we can hit resource limits on upstream servers as they can think
108 they are under some kind of network attack
109
110 This pre-prepared directory is shared among the Workers over NFS. If
111 the build is an individual build and there is no "shared" directory
112 available, it would clone from the cache and the upstreams as
113 necessary. This is considered the fallback mode.
114
1155. *Call scripts/run-config*
116
117 This is another call into the Helper scripts where its expected that
118 the main functionality of this target will be executed.
119
120.. _test-autobuilder-tech:
121
122Autobuilder Technology
123======================
124
125The Autobuilder has Yocto Project-specific functionality to allow builds
126to operate with increased efficiency and speed.
127
128.. _test-clobberdir:
129
130clobberdir
131----------
132
133When deleting files, the Autobuilder uses ``clobberdir``, which is a
134special script that moves files to a special location, rather than
135deleting them. Files in this location are deleted by an ``rm`` command,
136which is run under ``ionice -c 3``. For example, the deletion only
137happens when there is idle IO capacity on the Worker. The Autobuilder
138Worker Janitor runs this deletion. See `Autobuilder Worker
139Janitor <#test-autobuilder-worker-janitor>`__.
140
141.. _test-autobuilder-clone-cache:
142
143Autobuilder Clone Cache
144-----------------------
145
146Cloning repositories from scratch each time they are required was slow
147on the Autobuilder. We therefore have a stash of commonly used
148repositories pre-cloned on the Workers. Data is fetched from these
149during clones first, then "topped up" with later revisions from any
150upstream when necesary. The cache is maintained by the Autobuilder
151Worker Janitor. See `Autobuilder Worker
152Janitor <#test-autobuilder-worker-janitor>`__.
153
154.. _test-autobuilder-worker-janitor:
155
156Autobuilder Worker Janitor
157--------------------------
158
159This is a process running on each Worker that performs two basic
160operations, including background file deletion at IO idle (see `Target
161Execution: clobberdir <#test-list-tgt-exec-clobberdir>`__) and
162maintainenance of a cache of cloned repositories to improve the speed
163the system can checkout repositories.
164
165.. _test-shared-dl-dir:
166
167Shared DL_DIR
168-------------
169
170The Workers are all connected over NFS which allows DL_DIR to be shared
171between them. This reduces network accesses from the system and allows
172the build to be sped up. Usage of the directory within the build system
173is designed to be able to be shared over NFS.
174
175.. _test-shared-sstate-cache:
176
177Shared SSTATE_DIR
178-----------------
179
180The Workers are all connected over NFS which allows the ``sstate``
181directory to be shared between them. This means once a Worker has built
182an artefact, all the others can benefit from it. Usage of the directory
183within the directory is designed for sharing over NFS.
184
185.. _test-resulttool:
186
187Resulttool
188----------
189
190All of the different tests run as part of the build generate output into
191``testresults.json`` files. This allows us to determine which tests ran
192in a given build and their status. Additional information, such as
193failure logs or the time taken to run the tests, may also be included.
194
195Resulttool is part of OpenEmbedded-Core and is used to manipulate these
196json results files. It has the ability to merge files together, display
197reports of the test results and compare different result files.
198
199For details, see `https://wiki.yoctoproject.org/wiki/Resulttool <#>`__.
200
201.. _test-run-config-tgt-execution:
202
203run-config Target Execution
204===========================
205
206The ``scripts/run-config`` execution is where most of the work within
207the Autobuilder happens. It runs through a number of steps; the first
208are general setup steps that are run once and include:
209
2101. Set up any ``buildtools-tarball`` if configured.
211
2122. Call "buildhistory-init" if buildhistory is configured.
213
214For each step that is configured in ``config.json``, it will perform the
215following:
216
217## WRITER's question: What does "logging in as stepXa" and others refer
218to below? ##
219
2201. Add any layers that are specified using the
221 ``bitbake-layers add-layer`` command (logging as stepXa)
222
2232. Call the ``scripts/setup-config`` script to generate the necessary
224 ``auto.conf`` configuration file for the build
225
2263. Run the ``bitbake BBTARGETS`` command (logging as stepXb)
227
2284. Run the ``bitbake SANITYTARGETS`` command (logging as stepXc)
229
2305. Run the ``EXTRACMDS`` command, which are run within the BitBake build
231 environment (logging as stepXd)
232
2336. Run the ``EXTRAPLAINCMDS`` command(s), which are run outside the
234 BitBake build environment (logging as stepXd)
235
2367. Remove any layers added in `step
237 1 <#test-run-config-add-layers-step>`__ using the
238 ``bitbake-layers remove-layer`` command (logging as stepXa)
239
240Once the execution steps above complete, ``run-config`` executes a set
241of post-build steps, including:
242
2431. Call ``scripts/publish-artifacts`` to collect any output which is to
244 be saved from the build.
245
2462. Call ``scripts/collect-results`` to collect any test results to be
247 saved from the build.
248
2493. Call ``scripts/upload-error-reports`` to send any error reports
250 generated to the remote server.
251
2524. Cleanup the build directory using
253 ```clobberdir`` <#test-clobberdir>`__ if the build was successful,
254 else rename it to “build-renamed” for potential future debugging.
255
256.. _test-deploying-yp-autobuilder:
257
258Deploying Yocto Autobuilder
259===========================
260
261The most up to date information about how to setup and deploy your own
262Autbuilder can be found in README.md in the ``yocto-autobuilder2``
263repository.
264
265We hope that people can use the ``yocto-autobuilder2`` code directly but
266it is inevitable that users will end up needing to heavily customise the
267``yocto-autobuilder-helper`` repository, particularly the
268``config.json`` file as they will want to define their own test matrix.
269
270The Autobuilder supports wo customization options:
271
272- variable substitution
273
274- overlaying configuration files
275
276The standard ``config.json`` minimally attempts to allow substitution of
277the paths. The Helper script repository includes a
278``local-example.json`` file to show how you could override these from a
279separate configuration file. Pass the following into the environment of
280the Autobuilder:$ ABHELPER_JSON="config.json local-example.json"As
281another example, you could also pass the following into the
282environment:$ ABHELPER_JSON="config.json /some/location/local.json"One
283issue users often run into is validation of the ``config.json`` files. A
284tip for minimizing issues from invalid json files is to use a Git
285``pre-commit-hook.sh`` script to verify the JSON file before committing
286it. Create a symbolic link as follows:$ ln -s
287../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
diff --git a/documentation/test-manual/test-manual.rst b/documentation/test-manual/test-manual.rst
new file mode 100644
index 0000000000..1bca408106
--- /dev/null
+++ b/documentation/test-manual/test-manual.rst
@@ -0,0 +1,12 @@
1=====================================
2Yocto Project Test Environment Manual
3=====================================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 test-manual-intro
10 test-manual-test-process
11 test-manual-understand-autobuilder
12
diff --git a/documentation/toaster-manual/toaster-manual-intro.rst b/documentation/toaster-manual/toaster-manual-intro.rst
new file mode 100644
index 0000000000..92d8c94c52
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-intro.rst
@@ -0,0 +1,97 @@
1************
2Introduction
3************
4
5Toaster is a web interface to the Yocto Project's `OpenEmbedded build
6system <&YOCTO_DOCS_REF_URL;#build-system-term>`__. The interface
7enables you to configure and run your builds. Information about builds
8is collected and stored in a database. You can use Toaster to configure
9and start builds on multiple remote build servers.
10
11.. _intro-features:
12
13Toaster Features
14================
15
16Toaster allows you to configure and run builds, and it provides
17extensive information about the build process.
18
19- *Configure and Run Builds:* You can use the Toaster web interface to
20 configure and start your builds. Builds started using the Toaster web
21 interface are organized into projects. When you create a project, you
22 are asked to select a release, or version of the build system you
23 want to use for the project builds. As shipped, Toaster supports
24 Yocto Project releases 1.8 and beyond. With the Toaster web
25 interface, you can:
26
27 - Browse layers listed in the various `layer
28 sources <#layer-source>`__ that are available in your project
29 (e.g. the OpenEmbedded Layer Index at
30 ` <http://layers.openembedded.org/layerindex/>`__).
31
32 - Browse images, recipes, and machines provided by those layers.
33
34 - Import your own layers for building.
35
36 - Add and remove layers from your configuration.
37
38 - Set configuration variables.
39
40 - Select a target or multiple targets to build.
41
42 - Start your builds.
43
44 Toaster also allows you to configure and run your builds from the
45 command line, and switch between the command line and the web
46 interface at any time. Builds started from the command line appear
47 within a special Toaster project called "Command line builds".
48
49- *Information About the Build Process:* Toaster also records extensive
50 information about your builds. Toaster collects data for builds you
51 start from the web interface and from the command line as long as
52 Toaster is running.
53
54 .. note::
55
56 You must start Toaster before the build or it will not collect
57 build data.
58
59 With Toaster you can:
60
61 - See what was built (recipes and packages) and what packages were
62 installed into your final image.
63
64 - Browse the directory structure of your image.
65
66 - See the value of all variables in your build configuration, and
67 which files set each value.
68
69 - Examine error, warning, and trace messages to aid in debugging.
70
71 - See information about the BitBake tasks executed and reused during
72 your build, including those that used shared state.
73
74 - See dependency relationships between recipes, packages, and tasks.
75
76 - See performance information such as build time, task time, CPU
77 usage, and disk I/O.
78
79For an overview of Toaster shipped with the Yocto Project DISTRO
80Release, see the "`Toaster - Yocto Project
812.2 <https://youtu.be/BlXdOYLgPxA>`__" video.
82
83.. _toaster-installation-options:
84
85Installation Options
86====================
87
88You can set Toaster up to run as a local instance or as a shared hosted
89service.
90
91When Toaster is set up as a local instance, all the components reside on
92a single build host. Fundamentally, a local instance of Toaster is
93suited for a single user developing on a single build host.
94
95Toaster as a hosted service is suited for multiple users developing
96across several build hosts. When Toaster is set up as a hosted service,
97its components can be spread across several machines:
diff --git a/documentation/toaster-manual/toaster-manual-reference.rst b/documentation/toaster-manual/toaster-manual-reference.rst
new file mode 100644
index 0000000000..bc39903adf
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-reference.rst
@@ -0,0 +1,515 @@
1**********************
2Concepts and Reference
3**********************
4
5In order to configure and use Toaster, you should understand some
6concepts and have some basic command reference material available. This
7final chapter provides conceptual information on layer sources,
8releases, and JSON configuration files. Also provided is a quick look at
9some useful ``manage.py`` commands that are Toaster-specific.
10Information on ``manage.py`` commands does exist across the Web and the
11information in this manual by no means attempts to provide a command
12comprehensive reference.
13
14Layer Source
15============
16
17In general, a "layer source" is a source of information about existing
18layers. In particular, we are concerned with layers that you can use
19with the Yocto Project and Toaster. This chapter describes a particular
20type of layer source called a "layer index."
21
22A layer index is a web application that contains information about a set
23of custom layers. A good example of an existing layer index is the
24OpenEmbedded Layer Index. A public instance of this layer index exists
25at ` <http://layers.openembedded.org>`__. You can find the code for this
26layer index's web application at
27` <http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/>`__.
28
29When you tie a layer source into Toaster, it can query the layer source
30through a
31`REST <http://en.wikipedia.org/wiki/Representational_state_transfer>`__
32API, store the information about the layers in the Toaster database, and
33then show the information to users. Users are then able to view that
34information and build layers from Toaster itself without worrying about
35cloning or editing the BitBake layers configuration file
36``bblayers.conf``.
37
38Tying a layer source into Toaster is convenient when you have many
39custom layers that need to be built on a regular basis by a community of
40developers. In fact, Toaster comes pre-configured with the OpenEmbedded
41Metadata Index.
42
43.. note::
44
45 You do not have to use a layer source to use Toaster. Tying into a
46 layer source is optional.
47
48.. _layer-source-using-with-toaster:
49
50Setting Up and Using a Layer Source
51-----------------------------------
52
53To use your own layer source, you need to set up the layer source and
54then tie it into Toaster. This section describes how to tie into a layer
55index in a manner similar to the way Toaster ties into the OpenEmbedded
56Metadata Index.
57
58Understanding Your Layers
59~~~~~~~~~~~~~~~~~~~~~~~~~
60
61The obvious first step for using a layer index is to have several custom
62layers that developers build and access using the Yocto Project on a
63regular basis. This set of layers needs to exist and you need to be
64familiar with where they reside. You will need that information when you
65set up the code for the web application that "hooks" into your set of
66layers.
67
68For general information on layers, see the "`The Yocto Project Layer
69Model <&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model>`__" section in
70the Yocto Project Overview and Concepts Manual. For information on how
71to create layers, see the "`Understanding and Creating
72Layers <&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers>`__"
73section in the Yocto Project Development Tasks Manual.
74
75.. _configuring-toaster-to-hook-into-your-layer-source:
76
77Configuring Toaster to Hook Into Your Layer Index
78~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
79
80If you want Toaster to use your layer index, you must host the web
81application in a server to which Toaster can connect. You also need to
82give Toaster the information about your layer index. In other words, you
83have to configure Toaster to use your layer index. This section
84describes two methods by which you can configure and use your layer
85index.
86
87In the previous section, the code for the OpenEmbedded Metadata Index
88(i.e. ` <http://layers.openembedded.org>`__) was referenced. You can use
89this code, which is at
90` <http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/>`__, as a
91base to create your own layer index.
92
93Use the Administration Interface
94^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
95
96Access the administration interface through a browser by entering the
97URL of your Toaster instance and adding "``/admin``" to the end of the
98URL. As an example, if you are running Toaster locally, use the
99following URL: http://127.0.0.1:8000/admin
100
101The administration interface has a "Layer sources" section that includes
102an "Add layer source" button. Click that button and provide the required
103information. Make sure you select "layerindex" as the layer source type.
104
105Use the Fixture Feature
106^^^^^^^^^^^^^^^^^^^^^^^
107
108The Django fixture feature overrides the default layer server when you
109use it to specify a custom URL. To use the fixture feature, create (or
110edit) the file ``bitbake/lib/toaster.orm/fixtures/custom.xml``, and then
111set the following Toaster setting to your custom URL: <?xml
112version="1.0" ?> <django-objects version="1.0"> <object
113model="orm.toastersetting" pk="100"> <field name="name"
114type="CharField">CUSTOM_LAYERINDEX_SERVER</field> <field name="value"
115type="CharField">https://layers.my_organization.org/layerindex/branch/master/layers/</field>
116</object> <django-objects> When you start Toaster for the first time, or
117if you delete the file ``toaster.sqlite`` and restart, the database will
118populate cleanly from this layer index server.
119
120Once the information has been updated, verify the new layer information
121is available by using the Toaster web interface. To do that, visit the
122"All compatible layers" page inside a Toaster project. The layers from
123your layer source should be listed there.
124
125If you change the information in your layer index server, refresh the
126Toaster database by running the following command: $
127bitbake/lib/toaster/manage.py lsupdates If Toaster can reach the API
128URL, you should see a message telling you that Toaster is updating the
129layer source information.
130
131.. _toaster-releases:
132
133Releases
134========
135
136When you create a Toaster project using the web interface, you are asked
137to choose a "Release." In the context of Toaster, the term "Release"
138refers to a set of layers and a BitBake version the OpenEmbedded build
139system uses to build something. As shipped, Toaster is pre-configured
140with releases that correspond to Yocto Project release branches.
141However, you can modify, delete, and create new releases according to
142your needs. This section provides some background information on
143releases.
144
145.. _toaster-releases-supported:
146
147Pre-Configured Releases
148-----------------------
149
150As shipped, Toaster is configured to use a specific set of releases. Of
151course, you can always configure Toaster to use any release. For
152example, you might want your project to build against a specific commit
153of any of the "out-of-the-box" releases. Or, you might want your project
154to build against different revisions of OpenEmbedded and BitBake.
155
156As shipped, Toaster is configured to work with the following releases:
157
158- *Yocto Project DISTRO "DISTRO_NAME" or OpenEmbedded "DISTRO_NAME":*
159 This release causes your Toaster projects to build against the head
160 of the DISTRO_NAME_NO_CAP branch at
161 ` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/?h=rocko>`__ or
162 ` <http://git.openembedded.org/openembedded-core/commit/?h=rocko>`__.
163
164- *Yocto Project "Master" or OpenEmbedded "Master":* This release
165 causes your Toaster Projects to build against the head of the master
166 branch, which is where active development takes place, at
167 ` <&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/log/>`__ or
168 ` <http://git.openembedded.org/openembedded-core/log/>`__.
169
170- *Local Yocto Project or Local OpenEmbedded:* This release causes your
171 Toaster Projects to build against the head of the ``poky`` or
172 ``openembedded-core`` clone you have local to the machine running
173 Toaster.
174
175Configuring Toaster
176===================
177
178In order to use Toaster, you must configure the database with the
179default content. The following subsections describe various aspects of
180Toaster configuration.
181
182Configuring the Workflow
183------------------------
184
185The ``bldcontrol/management/commands/checksettings.py`` file controls
186workflow configuration. The following steps outline the process to
187initially populate this database.
188
1891. The default project settings are set from
190 ``orm/fixtures/settings.xml``.
191
1922. The default project distro and layers are added from
193 ``orm/fixtures/poky.xml`` if poky is installed. If poky is not
194 installed, they are added from ``orm/fixtures/oe-core.xml``.
195
1963. If the ``orm/fixtures/custom.xml`` file exists, then its values are
197 added.
198
1994. The layer index is then scanned and added to the database.
200
201Once these steps complete, Toaster is set up and ready to use.
202
203Customizing Pre-Set Data
204------------------------
205
206The pre-set data for Toaster is easily customizable. You can create the
207``orm/fixtures/custom.xml`` file to customize the values that go into to
208the database. Customization is additive, and can either extend or
209completely replace the existing values.
210
211You use the ``orm/fixtures/custom.xml`` file to change the default
212project settings for the machine, distro, file images, and layers. When
213creating a new project, you can use the file to define the offered
214alternate project release selections. For example, you can add one or
215more additional selections that present custom layer sets or distros,
216and any other local or proprietary content.
217
218Additionally, you can completely disable the content from the
219``oe-core.xml`` and ``poky.xml`` files by defining the section shown
220below in the ``settings.xml`` file. For example, this option is
221particularly useful if your custom configuration defines fewer releases
222or layers than the default fixture files.
223
224The following example sets "name" to "CUSTOM_XML_ONLY" and its value to
225"True". <object model="orm.toastersetting" pk="99"> <field
226type="CharField" name="name">CUSTOM_XML_ONLY</field> <field
227type="CharField" name="value">True</field> </object>
228
229Understanding Fixture File Format
230---------------------------------
231
232The following is an overview of the file format used by the
233``oe-core.xml``, ``poky.xml``, and ``custom.xml`` files.
234
235The following subsections describe each of the sections in the fixture
236files, and outline an example section of the XML code. you can use to
237help understand this information and create a local ``custom.xml`` file.
238
239Defining the Default Distro and Other Values
240~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
241
242This section defines the default distro value for new projects. By
243default, it reserves the first Toaster Setting record "1". The following
244demonstrates how to set the project default value for
245```DISTRO`` <&YOCTO_DOCS_REF_URL;#var-DISTRO>`__: <!-- Set the project
246default value for DISTRO --> <object model="orm.toastersetting" pk="1">
247<field type="CharField" name="name">DEFCONF_DISTRO</field> <field
248type="CharField" name="value">poky</field> </object> You can override
249other default project values by adding additional Toaster Setting
250sections such as any of the settings coming from the ``settings.xml``
251file. Also, you can add custom values that are included in the BitBake
252environment. The "pk" values must be unique. By convention, values that
253set default project values have a "DEFCONF" prefix.
254
255Defining BitBake Version
256~~~~~~~~~~~~~~~~~~~~~~~~
257
258The following defines which version of BitBake is used for the following
259release selection: <!-- Bitbake versions which correspond to the
260metadata release --> <object model="orm.bitbakeversion" pk="1"> <field
261type="CharField" name="name">rocko</field> <field type="CharField"
262name="giturl">git://git.yoctoproject.org/poky</field> <field
263type="CharField" name="branch">rocko</field> <field type="CharField"
264name="dirpath">bitbake</field> </object>
265
266.. _defining-releases:
267
268Defining Release
269~~~~~~~~~~~~~~~~
270
271The following defines the releases when you create a new project. <!--
272Releases available --> <object model="orm.release" pk="1"> <field
273type="CharField" name="name">rocko</field> <field type="CharField"
274name="description">Yocto Project 2.4 "Rocko"</field> <field
275rel="ManyToOneRel" to="orm.bitbakeversion"
276name="bitbake_version">1</field> <field type="CharField"
277name="branch_name">rocko</field> <field type="TextField"
278name="helptext">Toaster will run your builds using the tip of the <a
279href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=rocko">Yocto
280Project Rocko branch</a>.</field> </object> The "pk" value must match
281the above respective BitBake version record.
282
283Defining the Release Default Layer Names
284~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
285
286The following defines the default layers for each release: <!-- Default
287project layers for each release --> <object
288model="orm.releasedefaultlayer" pk="1"> <field rel="ManyToOneRel"
289to="orm.release" name="release">1</field> <field type="CharField"
290name="layer_name">openembedded-core</field> </object> The 'pk' values in
291the example above should start at "1" and increment uniquely. You can
292use the same layer name in multiple releases.
293
294Defining Layer Definitions
295~~~~~~~~~~~~~~~~~~~~~~~~~~
296
297Layer definitions are the most complex. The following defines each of
298the layers, and then defines the exact layer version of the layer used
299for each respective release. You must have one ``orm.layer`` entry for
300each layer. Then, with each entry you need a set of
301``orm.layer_version`` entries that connects the layer with each release
302that includes the layer. In general all releases include the layer.
303<object model="orm.layer" pk="1"> <field type="CharField"
304name="name">openembedded-core</field> <field type="CharField"
305name="layer_index_url"></field> <field type="CharField"
306name="vcs_url">git://git.yoctoproject.org/poky</field> <field
307type="CharField"
308name="vcs_web_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky</field>
309<field type="CharField"
310name="vcs_web_tree_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field>
311<field type="CharField"
312name="vcs_web_file_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field>
313</object> <object model="orm.layer_version" pk="1"> <field
314rel="ManyToOneRel" to="orm.layer" name="layer">1</field> <field
315type="IntegerField" name="layer_source">0</field> <field
316rel="ManyToOneRel" to="orm.release" name="release">1</field> <field
317type="CharField" name="branch">rocko</field> <field type="CharField"
318name="dirpath">meta</field> </object> <object model="orm.layer_version"
319pk="2"> <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
320<field type="IntegerField" name="layer_source">0</field> <field
321rel="ManyToOneRel" to="orm.release" name="release">2</field> <field
322type="CharField" name="branch">HEAD</field> <field type="CharField"
323name="commit">HEAD</field> <field type="CharField"
324name="dirpath">meta</field> </object> <object model="orm.layer_version"
325pk="3"> <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
326<field type="IntegerField" name="layer_source">0</field> <field
327rel="ManyToOneRel" to="orm.release" name="release">3</field> <field
328type="CharField" name="branch">master</field> <field type="CharField"
329name="dirpath">meta</field> </object> The layer "pk" values above must
330be unique, and typically start at "1". The layer version "pk" values
331must also be unique across all layers, and typically start at "1".
332
333Remote Toaster Monitoring
334=========================
335
336Toaster has an API that allows remote management applications to
337directly query the state of the Toaster server and its builds in a
338machine-to-machine manner. This API uses the
339`REST <http://en.wikipedia.org/wiki/Representational_state_transfer>`__
340interface and the transfer of JSON files. For example, you might monitor
341a build inside a container through well supported known HTTP ports in
342order to easily access a Toaster server inside the container. In this
343example, when you use this direct JSON API, you avoid having web page
344parsing against the display the user sees.
345
346Checking Health
347---------------
348
349Before you use remote Toaster monitoring, you should do a health check.
350To do this, ping the Toaster server using the following call to see if
351it is still alive: http://host:port/health Be sure to provide values for
352host and port. If the server is alive, you will get the response HTML:
353<!DOCTYPE html> <html lang="en"> <head><title>Toaster
354Health</title></head> <body>Ok</body> </html>
355
356Determining Status of Builds in Progress
357----------------------------------------
358
359Sometimes it is useful to determine the status of a build in progress.
360To get the status of pending builds, use the following call:
361http://host:port/toastergui/api/building Be sure to provide values for
362host and port. The output is a JSON file that itemizes all builds in
363progress. This file includes the time in seconds since each respective
364build started as well as the progress of the cloning, parsing, and task
365execution. The following is sample output for a build in progress:
366{"count": 1, "building": [ {"machine": "beaglebone", "seconds":
367"463.869", "task": "927:2384", "distro": "poky", "clone": "1:1", "id":
3682, "start": "2017-09-22T09:31:44.887Z", "name": "20170922093200",
369"parse": "818:818", "project": "my_rocko", "target":
370"core-image-minimal" }] } The JSON data for this query is returned in a
371single line. In the previous example the line has been artificially
372split for readability.
373
374Checking Status of Builds Completed
375-----------------------------------
376
377Once a build is completed, you get the status when you use the following
378call: http://host:port/toastergui/api/builds Be sure to provide values
379for host and port. The output is a JSON file that itemizes all complete
380builds, and includes build summary information. The following is sample
381output for a completed build: {"count": 1, "builds": [ {"distro":
382"poky", "errors": 0, "machine": "beaglebone", "project": "my_rocko",
383"stop": "2017-09-22T09:26:36.017Z", "target": "quilt-native", "seconds":
384"78.193", "outcome": "Succeeded", "id": 1, "start":
385"2017-09-22T09:25:17.824Z", "warnings": 1, "name": "20170922092618" }] }
386The JSON data for this query is returned in a single line. In the
387previous example the line has been artificially split for readability.
388
389Determining Status of a Specific Build
390--------------------------------------
391
392Sometimes it is useful to determine the status of a specific build. To
393get the status of a specific build, use the following call:
394http://host:port/toastergui/api/build/ID Be sure to provide values for
395host, port, and ID. You can find the value for ID from the Builds
396Completed query. See the "`Checking Status of Builds
397Completed <#checking-status-of-builds-completed>`__" section for more
398information.
399
400The output is a JSON file that itemizes the specific build and includes
401build summary information. The following is sample output for a specific
402build: {"build": {"distro": "poky", "errors": 0, "machine":
403"beaglebone", "project": "my_rocko", "stop": "2017-09-22T09:26:36.017Z",
404"target": "quilt-native", "seconds": "78.193", "outcome": "Succeeded",
405"id": 1, "start": "2017-09-22T09:25:17.824Z", "warnings": 1, "name":
406"20170922092618", "cooker_log":
407"/opt/user/poky/build-toaster-2/tmp/log/cooker/beaglebone/build_20170922_022607.991.log"
408} } The JSON data for this query is returned in a single line. In the
409previous example the line has been artificially split for readability.
410
411.. _toaster-useful-commands:
412
413Useful Commands
414===============
415
416In addition to the web user interface and the scripts that start and
417stop Toaster, command-line commands exist through the ``manage.py``
418management script. You can find general documentation on ``manage.py``
419at the
420`Django <https://docs.djangoproject.com/en/1.7/topics/settings/>`__
421site. However, several ``manage.py`` commands have been created that are
422specific to Toaster and are used to control configuration and back-end
423tasks. You can locate these commands in the `Source
424Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``) at
425``bitbake/lib/manage.py``. This section documents those commands.
426
427.. note::
428
429 - When using ``manage.py`` commands given a default configuration,
430 you must be sure that your working directory is set to the `Build
431 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. Using
432 ``manage.py`` commands from the Build Directory allows Toaster to
433 find the ``toaster.sqlite`` file, which is located in the Build
434 Directory.
435
436 - For non-default database configurations, it is possible that you
437 can use ``manage.py`` commands from a directory other than the
438 Build Directory. To do so, the ``toastermain/settings.py`` file
439 must be configured to point to the correct database backend.
440
441.. _toaster-command-buildslist:
442
443``buildslist``
444--------------
445
446The ``buildslist`` command lists all builds that Toaster has recorded.
447Access the command as follows: $ bitbake/lib/toaster/manage.py
448buildslist The command returns a list, which includes numeric
449identifications, of the builds that Toaster has recorded in the current
450database.
451
452You need to run the ``buildslist`` command first to identify existing
453builds in the database before using the
454```builddelete`` <#toaster-command-builddelete>`__ command. Here is an
455example that assumes default repository and build directory names: $ cd
456~/poky/build $ python ../bitbake/lib/toaster/manage.py buildslist If
457your Toaster database had only one build, the above ``buildslist``
458command would return something like the following: 1: qemux86 poky
459core-image-minimal
460
461.. _toaster-command-builddelete:
462
463``builddelete``
464---------------
465
466The ``builddelete`` command deletes data associated with a build. Access
467the command as follows: $ bitbake/lib/toaster/manage.py builddelete
468build_id The command deletes all the build data for the specified
469build_id. This command is useful for removing old and unused data from
470the database.
471
472Prior to running the ``builddelete`` command, you need to get the ID
473associated with builds by using the
474```buildslist`` <#toaster-command-buildslist>`__ command.
475
476.. _toaster-command-perf:
477
478``perf``
479--------
480
481The ``perf`` command measures Toaster performance. Access the command as
482follows: $ bitbake/lib/toaster/manage.py perf The command is a sanity
483check that returns page loading times in order to identify performance
484problems.
485
486.. _toaster-command-checksettings:
487
488``checksettings``
489-----------------
490
491The ``checksettings`` command verifies existing Toaster settings. Access
492the command as follows: $ bitbake/lib/toaster/manage.py checksettings
493Toaster uses settings that are based on the database to configure the
494building tasks. The ``checksettings`` command verifies that the database
495settings are valid in the sense that they have the minimal information
496needed to start a build.
497
498In order for the ``checksettings`` command to work, the database must be
499correctly set up and not have existing data. To be sure the database is
500ready, you can run the following: $ bitbake/lib/toaster/mana​ge.py
501syncdb $ bitbake/lib/toaster/mana​ge.py migrate orm $
502bitbake/lib/toaster/mana​ge.py migrate bldcontrol After running these
503commands, you can run the ``checksettings`` command.
504
505.. _toaster-command-runbuilds:
506
507``runbuilds``
508-------------
509
510The ``runbuilds`` command launches scheduled builds. Access the command
511as follows: $ bitbake/lib/toaster/manage.py runbuilds The ``runbuilds``
512command checks if scheduled builds exist in the database and then
513launches them per schedule. The command returns after the builds start
514but before they complete. The Toaster Logging Interface records and
515updates the database when the builds complete.
diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.rst b/documentation/toaster-manual/toaster-manual-setup-and-use.rst
new file mode 100644
index 0000000000..b36160b697
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.rst
@@ -0,0 +1,495 @@
1****************************
2Setting Up and Using Toaster
3****************************
4
5Starting Toaster for Local Development
6======================================
7
8Once you have set up the Yocto Project and installed the Toaster system
9dependencies as described in the "`Preparing to Use
10Toaster <#toaster-manual-start>`__" chapter, you are ready to start
11Toaster.
12
13Navigate to the root of your `Source
14Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g. ``poky``): $
15cd poky Once in that directory, source the build environment script: $
16source oe-init-build-env Next, from the build directory (e.g.
17``poky/build``), start Toaster using this command: $ source toaster
18start You can now run your builds from the command line, or with Toaster
19as explained in section "`Using the Toaster Web
20Interface <#using-the-toaster-web-interface>`__".
21
22To access the Toaster web interface, open your favorite browser and
23enter the following: http://127.0.0.1:8000
24
25Setting a Different Port
26========================
27
28By default, Toaster starts on port 8000. You can use the ``WEBPORT``
29parameter to set a different port. For example, the following command
30sets the port to "8400": $ source toaster start webport=8400
31
32Setting Up Toaster Without a Web Server
33=======================================
34
35You can start a Toaster environment without starting its web server.
36This is useful for the following:
37
38- Capturing a command-line build’s statistics into the Toaster database
39 for examination later.
40
41- Capturing a command-line build’s statistics when the Toaster server
42 is already running.
43
44- Having one instance of the Toaster web server track and capture
45 multiple command-line builds, where each build is started in its own
46 “noweb” Toaster environment.
47
48The following commands show how to start a Toaster environment without
49starting its web server, perform BitBake operations, and then shut down
50the Toaster environment. Once the build is complete, you can close the
51Toaster environment. Before closing the environment, however, you should
52allow a few minutes to ensure the complete transfer of its BitBake build
53statistics to the Toaster database. If you have a separate Toaster web
54server instance running, you can watch this command-line build’s
55progress and examine the results as soon as they are posted: $ source
56toaster start noweb $ bitbake target $ source toaster stop
57
58Setting Up Toaster Without a Build Server
59=========================================
60
61You can start a Toaster environment with the “New Projects” feature
62disabled. Doing so is useful for the following:
63
64- Sharing your build results over the web server while blocking others
65 from starting builds on your host.
66
67- Allowing only local command-line builds to be captured into the
68 Toaster database.
69
70Use the following command to set up Toaster without a build server: $
71source toaster start nobuild webport=port
72
73Setting up External Access
74==========================
75
76By default, Toaster binds to the loop back address (i.e. localhost),
77which does not allow access from external hosts. To allow external
78access, use the ``WEBPORT`` parameter to open an address that connects
79to the network, specifically the IP address that your NIC uses to
80connect to the network. You can also bind to all IP addresses the
81computer supports by using the shortcut "0.0.0.0:port".
82
83The following example binds to all IP addresses on the host: $ source
84toaster start webport=0.0.0.0:8400 This example binds to a specific IP
85address on the host's NIC: $ source toaster start
86webport=192.168.1.1:8400
87
88The Directory for Cloning Layers
89================================
90
91Toaster creates a ``_toaster_clones`` directory inside your Source
92Directory (i.e. ``poky``) to clone any layers needed for your builds.
93
94Alternatively, if you would like all of your Toaster related files and
95directories to be in a particular location other than the default, you
96can set the ``TOASTER_DIR`` environment variable, which takes precedence
97over your current working directory. Setting this environment variable
98causes Toaster to create and use ``$TOASTER_DIR./_toaster_clones``.
99
100.. _toaster-the-build-directory:
101
102The Build Directory
103===================
104
105Toaster creates a build directory within your Source Directory (e.g.
106``poky``) to execute the builds.
107
108Alternatively, if you would like all of your Toaster related files and
109directories to be in a particular location, you can set the
110``TOASTER_DIR`` environment variable, which takes precedence over your
111current working directory. Setting this environment variable causes
112Toaster to use ``$TOASTER_DIR/build`` as the build directory.
113
114.. _toaster-creating-a-django-super-user:
115
116Creating a Django Superuser
117===========================
118
119Toaster is built on the `Django
120framework <https://www.djangoproject.com/>`__. Django provides an
121administration interface you can use to edit Toaster configuration
122parameters.
123
124To access the Django administration interface, you must create a
125superuser by following these steps:
126
1271. If you used ``pip3``, which is recommended, to set up the Toaster
128 system dependencies, you need be sure the local user path is in your
129 ``PATH`` list. To append the pip3 local user path, use the following
130 command: $ export PATH=$PATH:$HOME/.local/bin
131
1322. From the directory containing the Toaster database, which by default
133 is the `Build Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__,
134 invoke the ``createsuperuser`` command from ``manage.py``: $ cd
135 ~/poky/build $ ../bitbake/lib/toaster/manage.py createsuperuser
136
1373. Django prompts you for the username, which you need to provide.
138
1394. Django prompts you for an email address, which is optional.
140
1415. Django prompts you for a password, which you must provide.
142
1436. Django prompts you to re-enter your password for verification.
144
145After completing these steps, the following confirmation message
146appears: Superuser created successfully.
147
148Creating a superuser allows you to access the Django administration
149interface through a browser. The URL for this interface is the same as
150the URL used for the Toaster instance with "/admin" on the end. For
151example, if you are running Toaster locally, use the following URL:
152http://127.0.0.1:8000/admin You can use the Django administration
153interface to set Toaster configuration parameters such as the build
154directory, layer sources, default variable values, and BitBake versions.
155
156.. _toaster-setting-up-a-production-instance-of-toaster:
157
158Setting Up a Production Instance of Toaster
159===========================================
160
161You can use a production instance of Toaster to share the Toaster
162instance with remote users, multiple users, or both. The production
163instance is also the setup that can handle heavier loads on the web
164service. Use the instructions in the following sections to set up
165Toaster to run builds through the Toaster web interface.
166
167.. _toaster-production-instance-requirements:
168
169Requirements
170------------
171
172Be sure you meet the following requirements:
173
174.. note::
175
176 You must comply with all Apache,
177 mod-wsgi
178 , and Mysql requirements.
179
180- Have all the build requirements as described in the "`Preparing to
181 Use Toaster <#toaster-manual-start>`__" chapter.
182
183- Have an Apache webserver.
184
185- Have ``mod-wsgi`` for the Apache webserver.
186
187- Use the Mysql database server.
188
189- If you are using Ubuntu 16.04, run the following: $ sudo apt-get
190 install apache2 libapache2-mod-wsgi-py3 mysql-server python3-pip
191 libmysqlclient-dev
192
193- If you are using Fedora 24 or a RedHat distribution, run the
194 following: $ sudo dnf install httpd python3-mod_wsgi python3-pip
195 mariadb-server mariadb-devel python3-devel
196
197- If you are using openSUSE Leap 42.1, run the following: $ sudo zypper
198 install apache2 apache2-mod_wsgi-python3 python3-pip mariadb
199 mariadb-client python3-devel
200
201.. _toaster-installation-steps:
202
203Installation
204------------
205
206Perform the following steps to install Toaster:
207
2081. Create toaster user and set its home directory to
209 ``/var/www/toaster``: $ sudo /usr/sbin/useradd toaster -md
210 /var/www/toaster -s /bin/false $ sudo su - toaster -s /bin/bash
211
2122. Checkout a copy of ``poky`` into the web server directory. You will
213 be using ``/var/www/toaster``: $ git clone
214 git://git.yoctoproject.org/poky $ git checkout DISTRO_NAME_NO_CAP
215
2163. Install Toaster dependencies using the --user flag which keeps the
217 Python packages isolated from your system-provided packages: $ cd
218 /var/www/toaster/ $ pip3 install --user -r
219 ./poky/bitbake/toaster-requirements.txt $ pip3 install --user
220 mysqlclient
221
222 .. note::
223
224 Isolating these packages is not required but is recommended.
225 Alternatively, you can use your operating system's package
226 manager to install the packages.
227
2284. Configure Toaster by editing
229 ``/var/www/toaster/poky/bitbake/lib/toaster/toastermain/settings.py``
230 as follows:
231
232 - Edit the
233 `DATABASES <https://docs.djangoproject.com/en/1.11/ref/settings/#databases>`__
234 settings: DATABASES = { 'default': { 'ENGINE':
235 'django.db.backends.mysql', 'NAME': 'toaster_data', 'USER':
236 'toaster', 'PASSWORD': 'yourpasswordhere', 'HOST': 'localhost',
237 'PORT': '3306', } }
238
239 - Edit the
240 `SECRET_KEY <https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-SECRET_KEY>`__:
241 SECRET_KEY = 'your_secret_key'
242
243 - Edit the
244 `STATIC_ROOT <https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-STATIC_ROOT>`__:
245 STATIC_ROOT = '/var/www/toaster/static_files/'
246
2475. Add the database and user to the ``mysql`` server defined earlier: $
248 mysql -u root -p mysql> CREATE DATABASE toaster_data; mysql> CREATE
249 USER 'toaster'@'localhost' identified by 'yourpasswordhere'; mysql>
250 GRANT all on toaster_data.\* to 'toaster'@'localhost'; mysql> quit
251
2526. Get Toaster to create the database schema, default data, and gather
253 the statically-served files: $ cd /var/www/toaster/poky/ $
254 ./bitbake/lib/toaster/manage.py migrate $ TOASTER_DIR=`pwd\`
255 TEMPLATECONF='poky' \\ ./bitbake/lib/toaster/manage.py checksettings
256 $ ./bitbake/lib/toaster/manage.py collectstatic In the previous
257 example, from the ``poky`` directory, the ``migrate`` command
258 ensures the database schema changes have propagated correctly (i.e.
259 migrations). The next line sets the Toaster root directory
260 ``TOASTER_DIR`` and the location of the Toaster configuration file
261 ``TOASTER_CONF``, which is relative to ``TOASTER_DIR``. The
262 ``TEMPLATECONF`` value reflects the contents of
263 ``poky/.templateconf``, and by default, should include the string
264 "poky". For more information on the Toaster configuration file, see
265 the "`Configuring Toaster <#configuring-toaster>`__" section.
266
267 This line also runs the ``checksettings`` command, which configures
268 the location of the Toaster `Build
269 Directory <&YOCTO_DOCS_REF_URL;#build-directory>`__. The Toaster
270 root directory ``TOASTER_DIR`` determines where the Toaster build
271 directory is created on the file system. In the example above,
272 ``TOASTER_DIR`` is set as follows: /var/www/toaster/poky This
273 setting causes the Toaster build directory to be:
274 /var/www/toaster/poky/build
275
276 Finally, the ``collectstatic`` command is a Django framework command
277 that collects all the statically served files into a designated
278 directory to be served up by the Apache web server as defined by
279 ``STATIC_ROOT``.
280
2817. Test and/or use the Mysql integration with Toaster’s Django web
282 server. At this point, you can start up the normal Toaster Django
283 web server with the Toaster database in Mysql. You can use this web
284 server to confirm that the database migration and data population
285 from the Layer Index is complete.
286
287 To start the default Toaster Django web server with the Toaster
288 database now in Mysql, use the standard start commands: $ source
289 oe-init-build-env $ source toaster start Additionally, if Django is
290 sufficient for your requirements, you can use it for your release
291 system and migrate later to Apache as your requirements change.
292
2938. Add an Apache configuration file for Toaster to your Apache web
294 server's configuration directory. If you are using Ubuntu or Debian,
295 put the file here: /etc/apache2/conf-available/toaster.conf If you
296 are using Fedora or RedHat, put it here:
297 /etc/httpd/conf.d/toaster.conf If you are using OpenSUSE, put it
298 here: /etc/apache2/conf.d/toaster.conf Following is a sample Apache
299 configuration for Toaster you can follow: Alias /static
300 /var/www/toaster/static_files <Directory
301 /var/www/toaster/static_files> <IfModule mod_access_compat.c> Order
302 allow,deny Allow from all </IfModule> <IfModule
303 !mod_access_compat.c> Require all granted </IfModule> </Directory>
304 <Directory /var/www/toaster/poky/bitbake/lib/toaster/toastermain>
305 <Files "wsgi.py"> Require all granted </Files> </Directory>
306 WSGIDaemonProcess toaster_wsgi
307 python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/.local/lib/python3.4/site-packages
308 WSGIScriptAlias /
309 "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py"
310 <Location /> WSGIProcessGroup toaster_wsgi </Location> If you are
311 using Ubuntu or Debian, you will need to enable the config and
312 module for Apache: $ sudo a2enmod wsgi $ sudo a2enconf toaster $
313 chmod +x bitbake/lib/toaster/toastermain/wsgi.py Finally, restart
314 Apache to make sure all new configuration is loaded. For Ubuntu,
315 Debian, and OpenSUSE use: $ sudo service apache2 restart For Fedora
316 and RedHat use: $ sudo service httpd restart
317
3189. Prepare the systemd service to run Toaster builds. Here is a sample
319 configuration file for the service: [Unit] Description=Toaster
320 runbuilds [Service] Type=forking User=toaster
321 ExecStart=/usr/bin/screen -d -m -S runbuilds
322 /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
323 ExecStop=/usr/bin/screen -S runbuilds -X quit
324 WorkingDirectory=/var/www/toaster/poky [Install]
325 WantedBy=multi-user.target Prepare the ``runbuilds-service.sh``
326 script that you need to place in the
327 ``/var/www/toaster/poky/bitbake/lib/toaster/`` directory by setting
328 up executable permissions: #!/bin/bash #export
329 http_proxy=http://proxy.host.com:8080 #export
330 https_proxy=http://proxy.host.com:8080 #export
331 GIT_PROXY_COMMAND=$HOME/bin/gitproxy cd ~/poky/ source
332 ./oe-init-build-env build source ../bitbake/bin/toaster $1 noweb [
333 "$1" == 'start' ] && /bin/bash
334
33510. Run the service: # service runbuilds start Since the service is
336 running in a detached screen session, you can attach to it using
337 this command: $ sudo su - toaster $ screen -rS runbuilds You can
338 detach from the service again using "Ctrl-a" followed by "d" key
339 combination.
340
341You can now open up a browser and start using Toaster.
342
343Using the Toaster Web Interface
344===============================
345
346The Toaster web interface allows you to do the following:
347
348- Browse published layers in the `OpenEmbedded Layer
349 Index <http://layers.openembedded.org>`__ that are available for your
350 selected version of the build system.
351
352- Import your own layers for building.
353
354- Add and remove layers from your configuration.
355
356- Set configuration variables.
357
358- Select a target or multiple targets to build.
359
360- Start your builds.
361
362- See what was built (recipes and packages) and what packages were
363 installed into your final image.
364
365- Browse the directory structure of your image.
366
367- See the value of all variables in your build configuration, and which
368 files set each value.
369
370- Examine error, warning and trace messages to aid in debugging.
371
372- See information about the BitBake tasks executed and reused during
373 your build, including those that used shared state.
374
375- See dependency relationships between recipes, packages and tasks.
376
377- See performance information such as build time, task time, CPU usage,
378 and disk I/O.
379
380.. _web-interface-videos:
381
382Toaster Web Interface Videos
383----------------------------
384
385Following are several videos that show how to use the Toaster GUI:
386
387- *Build Configuration:* This
388 `video <https://www.youtube.com/watch?v=qYgDZ8YzV6w>`__ overviews and
389 demonstrates build configuration for Toaster.
390
391- *Build Custom Layers:* This
392 `video <https://www.youtube.com/watch?v=QJzaE_XjX5c>`__ shows you how
393 to build custom layers that are used with Toaster.
394
395- *Toaster Homepage and Table Controls:* This
396 `video <https://www.youtube.com/watch?v=QEARDnrR1Xw>`__ goes over the
397 Toaster entry page, and provides an overview of the data manipulation
398 capabilities of Toaster, which include search, sorting and filtering
399 by different criteria.
400
401- *Build Dashboard:* This
402 `video <https://www.youtube.com/watch?v=KKqHYcnp2gE>`__ shows you the
403 build dashboard, a page providing an overview of the information
404 available for a selected build.
405
406- *Image Information:* This
407 `video <https://www.youtube.com/watch?v=XqYGFsmA0Rw>`__ walks through
408 the information Toaster provides about images: packages installed and
409 root file system.
410
411- *Configuration:* This
412 `video <https://www.youtube.com/watch?v=UW-j-T2TzIg>`__ provides
413 Toaster build configuration information.
414
415- *Tasks:* This `video <https://www.youtube.com/watch?v=D4-9vGSxQtw>`__
416 shows the information Toaster provides about the tasks run by the
417 build system.
418
419- *Recipes and Packages Built:* This
420 `video <https://www.youtube.com/watch?v=x-6dx4huNnw>`__ shows the
421 information Toaster provides about recipes and packages built.
422
423- *Performance Data:* This
424 `video <https://www.youtube.com/watch?v=qWGMrJoqusQ>`__ shows the
425 build performance data provided by Toaster.
426
427.. _a-note-on-the-local-yocto-project-release:
428
429Additional Information About the Local Yocto Project Release
430------------------------------------------------------------
431
432This section only applies if you have set up Toaster for local
433development, as explained in the "`Starting Toaster for Local
434Development <#starting-toaster-for-local-development>`__" section.
435
436When you create a project in Toaster, you will be asked to provide a
437name and to select a Yocto Project release. One of the release options
438you will find is called "Local Yocto Project".
439
440When you select the "Local Yocto Project" release, Toaster will run your
441builds using the local Yocto Project clone you have in your computer:
442the same clone you are using to run Toaster. Unless you manually update
443this clone, your builds will always use the same Git revision.
444
445If you select any of the other release options, Toaster will fetch the
446tip of your selected release from the upstream `Yocto Project
447repository <https://git.yoctoproject.org>`__ every time you run a build.
448Fetching this tip effectively means that if your selected release is
449updated upstream, the Git revision you are using for your builds will
450change. If you are doing development locally, you might not want this
451change to happen. In that case, the "Local Yocto Project" release might
452be the right choice.
453
454However, the "Local Yocto Project" release will not provide you with any
455compatible layers, other than the three core layers that come with the
456Yocto Project:
457
458- `openembedded-core <http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/>`__
459
460- `meta-poky <http://layers.openembedded.org/layerindex/branch/master/layer/meta-poky/>`__
461
462- `meta-yocto-bsp <http://layers.openembedded.org/layerindex/branch/master/layer/meta-yocto-bsp/>`__
463
464If you want to build any other layers, you will need to manually import
465them into your Toaster project, using the "Import layer" page.
466
467.. _toaster-web-interface-preferred-version:
468
469Building a Specific Recipe Given Multiple Versions
470--------------------------------------------------
471
472Occasionally, a layer might provide more than one version of the same
473recipe. For example, the ``openembedded-core`` layer provides two
474versions of the ``bash`` recipe (i.e. 3.2.48 and 4.3.30-r0) and two
475versions of the ``which`` recipe (i.e. 2.21 and 2.18). The following
476figure shows this exact scenario:
477
478By default, the OpenEmbedded build system builds one of the two recipes.
479For the ``bash`` case, version 4.3.30-r0 is built by default.
480Unfortunately, Toaster as it exists, is not able to override the default
481recipe version. If you would like to build bash 3.2.48, you need to set
482the
483```PREFERRED_VERSION`` <&YOCTO_DOCS_REF_URL;#var-PREFERRED_VERSION>`__
484variable. You can do so from Toaster, using the "Add variable" form,
485which is available in the "BitBake variables" page of the project
486configuration section as shown in the following screen:
487
488To specify ``bash`` 3.2.48 as the version to build, enter
489"PREFERRED_VERSION_bash" in the "Variable" field, and "3.2.48" in the
490"Value" field. Next, click the "Add variable" button:
491
492After clicking the "Add variable" button, the settings for
493``PREFERRED_VERSION`` are added to the bottom of the BitBake variables
494list. With these settings, the OpenEmbedded build system builds the
495desired version of the recipe rather than the default version:
diff --git a/documentation/toaster-manual/toaster-manual-start.rst b/documentation/toaster-manual/toaster-manual-start.rst
new file mode 100644
index 0000000000..54f290590b
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-start.rst
@@ -0,0 +1,46 @@
1************************
2Preparing to Use Toaster
3************************
4
5This chapter describes how you need to prepare your system in order to
6use Toaster.
7
8.. _toaster-setting-up-the-basic-system-requirements:
9
10Setting Up the Basic System Requirements
11========================================
12
13Before you can use Toaster, you need to first set up your build system
14to run the Yocto Project. To do this, follow the instructions in the
15"`Preparing the Build
16Host <&YOCTO_DOCS_DEV_URL;#dev-preparing-the-build-host>`__" section of
17the Yocto Project Development Tasks Manual. For Ubuntu/Debian, you might
18also need to do an additional install of pip3. $ sudo apt-get install
19python3-pip
20
21.. _toaster-establishing-toaster-system-dependencies:
22
23Establishing Toaster System Dependencies
24========================================
25
26Toaster requires extra Python dependencies in order to run. A Toaster
27requirements file named ``toaster-requirements.txt`` defines the Python
28dependencies. The requirements file is located in the ``bitbake``
29directory, which is located in the root directory of the `Source
30Directory <&YOCTO_DOCS_REF_URL;#source-directory>`__ (e.g.
31``poky/bitbake/toaster-requirements.txt``). The dependencies appear in a
32``pip``, install-compatible format.
33
34.. _toaster-load-packages:
35
36Install Toaster Packages
37------------------------
38
39You need to install the packages that Toaster requires. Use this
40command: $ pip3 install --user -r bitbake/toaster-requirements.txt The
41previous command installs the necessary Toaster modules into a local
42python 3 cache in your ``$HOME`` directory. The caches is actually
43located in ``$HOME/.local``. To see what packages have been installed
44into your ``$HOME`` directory, do the following: $ pip3 list installed
45--local If you need to remove something, the following works: $ pip3
46uninstall PackageNameToUninstall
diff --git a/documentation/toaster-manual/toaster-manual.rst b/documentation/toaster-manual/toaster-manual.rst
new file mode 100644
index 0000000000..4b0342c578
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual.rst
@@ -0,0 +1,12 @@
1===================
2Toaster User Manual
3===================
4
5.. toctree::
6 :caption: Table of Contents
7 :numbered:
8
9 toaster-manual-intro
10 toaster-manual-start
11 toaster-manual-setup-and-use
12 toaster-manual-reference