summaryrefslogtreecommitdiffstats
path: root/documentation
diff options
context:
space:
mode:
authorNicolas Dechesne <nicolas.dechesne@linaro.org>2020-11-20 20:17:33 +0100
committerRichard Purdie <richard.purdie@linuxfoundation.org>2021-01-04 10:55:00 +0000
commitfa0cb4d34b1073f215fa3c680f2316208739d53d (patch)
treeba89c1f4289fd6456af4409a6a19caf6548dfb9c /documentation
parenta038e58f3cd82c56102444bdc5ac76c9f1550a0d (diff)
downloadpoky-fa0cb4d34b1073f215fa3c680f2316208739d53d.tar.gz
sphinx: import docs
The Yocto Project docs was migrated from Docbook to Sphinx in YP 3.2. This 3.1 is an LTS release, and since 3.1 docs are 'close to' the docs in 3.2, we agreed to backport sphinx docs onto 3.1. This first patch brings all changes done in 3.2 until: 7f64574f7 README: include detailed information about sphinx There are other changes after this commit, but they will be selectively backported in individual patches. This patch was generated with the following command: git cherry-pick -n \ $(git log --reverse --oneline \ ac352ad7f95db7eeacb53c2778caa31800bd7c26..7f64574f7 \ | cut -f1 -d' ') The following commits were applies in the dunfell docs, but not in master, so they were first reverted (and squashed into this change). A commit will reintroduce the content from these patches in the Sphinx files in a followup patch. 069c27574 Documenation: Prepared for the 3.1.1 release bd140f0f9 Documentation: Add 3.1.1 version updates missing from previous commit 17cc71a8f Documenation: Prepared for the 3.1.2 release 1a69e2c02 Documenation: Prepared for the 3.1.3 release 8910ac1c7 Documenation: Prepared for the 3.1.4 release (From yocto-docs rev: c25fe058b88b893b0d146f3ed27320b47cdec236) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r--documentation/.gitignore1
-rw-r--r--documentation/Makefile67
-rw-r--r--documentation/Makefile.sphinx35
-rw-r--r--documentation/README283
-rw-r--r--documentation/_templates/breadcrumbs.html14
-rw-r--r--documentation/_templates/layout.html7
-rw-r--r--documentation/adt-manual/adt-command.rst180
-rw-r--r--documentation/adt-manual/adt-command.xml1
-rw-r--r--documentation/adt-manual/adt-intro.rst138
-rw-r--r--documentation/adt-manual/adt-intro.xml1
-rw-r--r--documentation/adt-manual/adt-manual-customization.xsl1
-rw-r--r--documentation/adt-manual/adt-manual-eclipse-customization.xsl2
-rw-r--r--documentation/adt-manual/adt-manual-intro.rst24
-rw-r--r--documentation/adt-manual/adt-manual-intro.xml1
-rw-r--r--documentation/adt-manual/adt-manual.rst17
-rw-r--r--documentation/adt-manual/adt-manual.xml1
-rw-r--r--documentation/adt-manual/adt-package.rst70
-rw-r--r--documentation/adt-manual/adt-package.xml1
-rw-r--r--documentation/adt-manual/adt-prepare.rst752
-rw-r--r--documentation/adt-manual/adt-prepare.xml5
-rw-r--r--documentation/adt-manual/adt-style.css2
-rw-r--r--documentation/boilerplate.rst18
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-customization.xsl2
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-style.css3
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-titlepage.xsl1
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst430
-rw-r--r--documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.xml1
-rw-r--r--documentation/bsp-guide/bsp-guide-customization.xsl2
-rw-r--r--documentation/bsp-guide/bsp-guide.rst16
-rwxr-xr-xdocumentation/bsp-guide/bsp-guide.xml23
-rw-r--r--documentation/bsp-guide/bsp-style.css2
-rw-r--r--documentation/bsp-guide/bsp.rst1527
-rw-r--r--documentation/bsp-guide/bsp.xml3
-rw-r--r--documentation/bsp-guide/history.rst73
-rw-r--r--documentation/conf.py127
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.rst11803
-rw-r--r--documentation/dev-manual/dev-manual-common-tasks.xml67
-rw-r--r--documentation/dev-manual/dev-manual-customization.xsl2
-rw-r--r--documentation/dev-manual/dev-manual-intro.rst61
-rw-r--r--documentation/dev-manual/dev-manual-intro.xml1
-rw-r--r--documentation/dev-manual/dev-manual-qemu.rst470
-rw-r--r--documentation/dev-manual/dev-manual-qemu.xml9
-rw-r--r--documentation/dev-manual/dev-manual-start.rst940
-rw-r--r--documentation/dev-manual/dev-manual-start.xml1
-rw-r--r--documentation/dev-manual/dev-manual.rst19
-rwxr-xr-xdocumentation/dev-manual/dev-manual.xml23
-rw-r--r--documentation/dev-manual/dev-style.css3
-rw-r--r--documentation/dev-manual/history.rst67
-rw-r--r--documentation/figures/yp-how-it-works-new-diagram.pngbin0 -> 249657 bytes
-rw-r--r--documentation/genindex.rst3
-rw-r--r--documentation/index.rst53
-rw-r--r--documentation/kernel-dev/history.rst58
-rw-r--r--documentation/kernel-dev/kernel-dev-advanced.rst983
-rw-r--r--documentation/kernel-dev/kernel-dev-advanced.xml1
-rw-r--r--documentation/kernel-dev/kernel-dev-common.rst2078
-rw-r--r--documentation/kernel-dev/kernel-dev-common.xml1
-rw-r--r--documentation/kernel-dev/kernel-dev-concepts-appx.rst426
-rw-r--r--documentation/kernel-dev/kernel-dev-concepts-appx.xml3
-rw-r--r--documentation/kernel-dev/kernel-dev-customization.xsl2
-rw-r--r--documentation/kernel-dev/kernel-dev-faq.rst81
-rw-r--r--documentation/kernel-dev/kernel-dev-faq.xml1
-rw-r--r--documentation/kernel-dev/kernel-dev-intro.rst183
-rw-r--r--documentation/kernel-dev/kernel-dev-intro.xml1
-rw-r--r--documentation/kernel-dev/kernel-dev-maint-appx.rst239
-rw-r--r--documentation/kernel-dev/kernel-dev-maint-appx.xml1
-rw-r--r--documentation/kernel-dev/kernel-dev-style.css3
-rw-r--r--documentation/kernel-dev/kernel-dev.rst21
-rwxr-xr-xdocumentation/kernel-dev/kernel-dev.xml23
-rw-r--r--documentation/mega-manual/mega-manual-customization.xsl1
-rwxr-xr-xdocumentation/mega-manual/mega-manual.xml24
-rw-r--r--documentation/mega-manual/mega-style.css2
-rw-r--r--documentation/overview-manual/history.rst28
-rw-r--r--documentation/overview-manual/overview-manual-concepts.rst2185
-rw-r--r--documentation/overview-manual/overview-manual-concepts.xml1
-rw-r--r--documentation/overview-manual/overview-manual-customization.xsl2
-rw-r--r--documentation/overview-manual/overview-manual-development-environment.rst672
-rw-r--r--documentation/overview-manual/overview-manual-development-environment.xml13
-rw-r--r--documentation/overview-manual/overview-manual-intro.rst74
-rw-r--r--documentation/overview-manual/overview-manual-intro.xml1
-rw-r--r--documentation/overview-manual/overview-manual-style.css2
-rw-r--r--documentation/overview-manual/overview-manual-yp-intro.rst941
-rw-r--r--documentation/overview-manual/overview-manual-yp-intro.xml13
-rw-r--r--documentation/overview-manual/overview-manual.rst19
-rwxr-xr-xdocumentation/overview-manual/overview-manual.xml23
-rwxr-xr-xdocumentation/poky.ent24
-rw-r--r--documentation/poky.yaml89
-rw-r--r--documentation/profile-manual/history.rst58
-rw-r--r--documentation/profile-manual/profile-manual-arch.rst29
-rw-r--r--documentation/profile-manual/profile-manual-arch.xml1
-rw-r--r--documentation/profile-manual/profile-manual-customization.xsl2
-rw-r--r--documentation/profile-manual/profile-manual-examples.rst24
-rw-r--r--documentation/profile-manual/profile-manual-examples.xml1
-rw-r--r--documentation/profile-manual/profile-manual-intro.rst79
-rw-r--r--documentation/profile-manual/profile-manual-intro.xml1
-rw-r--r--documentation/profile-manual/profile-manual-style.css3
-rw-r--r--documentation/profile-manual/profile-manual-usage.rst2624
-rw-r--r--documentation/profile-manual/profile-manual-usage.xml1
-rw-r--r--documentation/profile-manual/profile-manual.rst19
-rwxr-xr-xdocumentation/profile-manual/profile-manual.xml23
-rw-r--r--documentation/ref-manual/examples/hello-autotools/hello_2.10.bb9
-rw-r--r--documentation/ref-manual/examples/hello-autotools/hello_2.3.bb8
-rw-r--r--documentation/ref-manual/examples/libxpm/libxpm_3.5.6.bb2
-rw-r--r--documentation/ref-manual/faq.rst451
-rw-r--r--documentation/ref-manual/faq.xml3
-rw-r--r--documentation/ref-manual/history.rst74
-rw-r--r--documentation/ref-manual/migration-1.3.rst195
-rw-r--r--documentation/ref-manual/migration-1.4.rst237
-rw-r--r--documentation/ref-manual/migration-1.5.rst355
-rw-r--r--documentation/ref-manual/migration-1.6.rst417
-rw-r--r--documentation/ref-manual/migration-1.7.rst225
-rw-r--r--documentation/ref-manual/migration-1.8.rst183
-rw-r--r--documentation/ref-manual/migration-2.0.rst281
-rw-r--r--documentation/ref-manual/migration-2.1.rst434
-rw-r--r--documentation/ref-manual/migration-2.2.rst451
-rw-r--r--documentation/ref-manual/migration-2.3.rst530
-rw-r--r--documentation/ref-manual/migration-2.4.rst327
-rw-r--r--documentation/ref-manual/migration-2.5.rst310
-rw-r--r--documentation/ref-manual/migration-2.6.rst476
-rw-r--r--documentation/ref-manual/migration-2.7.rst180
-rw-r--r--documentation/ref-manual/migration-3.0.rst321
-rw-r--r--documentation/ref-manual/migration-3.1.rst276
-rw-r--r--documentation/ref-manual/migration-general.rst54
-rw-r--r--documentation/ref-manual/migration.rst30
-rw-r--r--documentation/ref-manual/migration.xml1
-rw-r--r--documentation/ref-manual/ref-classes.rst2963
-rw-r--r--documentation/ref-manual/ref-classes.xml83
-rw-r--r--documentation/ref-manual/ref-devtool-reference.rst625
-rw-r--r--documentation/ref-manual/ref-devtool-reference.xml1
-rw-r--r--documentation/ref-manual/ref-features.rst353
-rw-r--r--documentation/ref-manual/ref-features.xml1
-rw-r--r--documentation/ref-manual/ref-images.rst139
-rw-r--r--documentation/ref-manual/ref-images.xml5
-rw-r--r--documentation/ref-manual/ref-kickstart.rst212
-rw-r--r--documentation/ref-manual/ref-kickstart.xml1
-rw-r--r--documentation/ref-manual/ref-manual-customization.xsl2
-rw-r--r--documentation/ref-manual/ref-manual.rst31
-rwxr-xr-xdocumentation/ref-manual/ref-manual.xml22
-rw-r--r--documentation/ref-manual/ref-qa-checks.rst533
-rw-r--r--documentation/ref-manual/ref-qa-checks.xml26
-rw-r--r--documentation/ref-manual/ref-release-process.rst193
-rw-r--r--documentation/ref-manual/ref-release-process.xml1
-rw-r--r--documentation/ref-manual/ref-structure.rst890
-rw-r--r--documentation/ref-manual/ref-structure.xml1
-rw-r--r--documentation/ref-manual/ref-style.css3
-rw-r--r--documentation/ref-manual/ref-system-requirements.rst437
-rw-r--r--documentation/ref-manual/ref-system-requirements.xml5
-rw-r--r--documentation/ref-manual/ref-tasks.rst875
-rw-r--r--documentation/ref-manual/ref-tasks.xml1
-rw-r--r--documentation/ref-manual/ref-terms.rst397
-rw-r--r--documentation/ref-manual/ref-terms.xml3
-rw-r--r--documentation/ref-manual/ref-variables.rst8899
-rw-r--r--documentation/ref-manual/ref-variables.xml179
-rw-r--r--documentation/ref-manual/ref-varlocality.rst166
-rw-r--r--documentation/ref-manual/ref-varlocality.xml1
-rw-r--r--documentation/ref-manual/resources.rst197
-rw-r--r--documentation/ref-manual/resources.xml1
-rw-r--r--documentation/releases.rst188
-rw-r--r--documentation/sdk-manual/history.rst40
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing-standard.rst34
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing-standard.xml1
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing.rst377
-rw-r--r--documentation/sdk-manual/sdk-appendix-customizing.xml1
-rw-r--r--documentation/sdk-manual/sdk-appendix-obtain.rst321
-rw-r--r--documentation/sdk-manual/sdk-appendix-obtain.xml1
-rw-r--r--documentation/sdk-manual/sdk-extensible.rst1356
-rw-r--r--documentation/sdk-manual/sdk-extensible.xml1
-rw-r--r--documentation/sdk-manual/sdk-intro.rst231
-rw-r--r--documentation/sdk-manual/sdk-intro.xml1
-rw-r--r--documentation/sdk-manual/sdk-manual-customization.xsl2
-rw-r--r--documentation/sdk-manual/sdk-manual.rst22
-rwxr-xr-xdocumentation/sdk-manual/sdk-manual.xml23
-rw-r--r--documentation/sdk-manual/sdk-style.css3
-rw-r--r--documentation/sdk-manual/sdk-using.rst159
-rw-r--r--documentation/sdk-manual/sdk-using.xml1
-rw-r--r--documentation/sdk-manual/sdk-working-projects.rst423
-rw-r--r--documentation/sdk-manual/sdk-working-projects.xml1
-rw-r--r--documentation/sphinx-static/YoctoProject_Logo_RGB.jpgbin0 -> 49299 bytes
-rw-r--r--documentation/sphinx-static/switchers.js233
-rw-r--r--documentation/sphinx-static/theme_overrides.css164
-rw-r--r--documentation/sphinx/yocto-vars.py47
-rw-r--r--documentation/test-manual/figures/ab-test-cluster.pngbin0 -> 18684 bytes
-rw-r--r--documentation/test-manual/figures/test-manual-title.pngbin0 -> 15382 bytes
-rw-r--r--documentation/test-manual/history.rst16
-rw-r--r--documentation/test-manual/test-manual-customization.xsl29
-rw-r--r--documentation/test-manual/test-manual-intro.rst550
-rw-r--r--documentation/test-manual/test-manual-intro.xml624
-rw-r--r--documentation/test-manual/test-manual-style.css991
-rw-r--r--documentation/test-manual/test-manual-test-process.rst103
-rw-r--r--documentation/test-manual/test-manual-test-process.xml110
-rw-r--r--documentation/test-manual/test-manual-understand-autobuilder.rst305
-rw-r--r--documentation/test-manual/test-manual-understand-autobuilder.xml314
-rw-r--r--documentation/test-manual/test-manual.rst18
-rwxr-xr-xdocumentation/test-manual/test-manual.xml104
-rw-r--r--documentation/toaster-manual/history.rst46
-rw-r--r--documentation/toaster-manual/toaster-manual-customization.xsl2
-rw-r--r--documentation/toaster-manual/toaster-manual-intro.rst105
-rw-r--r--documentation/toaster-manual/toaster-manual-intro.xml1
-rw-r--r--documentation/toaster-manual/toaster-manual-reference.rst662
-rw-r--r--documentation/toaster-manual/toaster-manual-reference.xml1
-rw-r--r--documentation/toaster-manual/toaster-manual-setup-and-use.rst651
-rw-r--r--documentation/toaster-manual/toaster-manual-setup-and-use.xml13
-rw-r--r--documentation/toaster-manual/toaster-manual-start.rst57
-rw-r--r--documentation/toaster-manual/toaster-manual-start.xml1
-rw-r--r--documentation/toaster-manual/toaster-manual-style.css3
-rw-r--r--documentation/toaster-manual/toaster-manual.rst19
-rwxr-xr-xdocumentation/toaster-manual/toaster-manual.xml23
-rw-r--r--documentation/tools/eclipse-help.sed3
-rw-r--r--documentation/tools/mega-manual.sed49
-rwxr-xr-xdocumentation/tools/poky-docbook-to-pdf5
-rw-r--r--documentation/tools/update-documentation-conf16
-rw-r--r--documentation/transitioning-to-a-custom-environment.rst116
-rw-r--r--documentation/what-i-wish-id-known.rst226
212 files changed, 59937 insertions, 369 deletions
diff --git a/documentation/.gitignore b/documentation/.gitignore
new file mode 100644
index 0000000000..69fa449dd9
--- /dev/null
+++ b/documentation/.gitignore
@@ -0,0 +1 @@
_build/
diff --git a/documentation/Makefile b/documentation/Makefile
index 15644bf926..7d4058ae75 100644
--- a/documentation/Makefile
+++ b/documentation/Makefile
@@ -1,3 +1,6 @@
1#
2# SPDX-License-Identifier: CC-BY-2.0-UK
3#
1# This is a single Makefile to handle all generated Yocto Project documents, 4# This is a single Makefile to handle all generated Yocto Project documents,
2# which includes the BitBake User Manual and the Toaster User Manual. 5# which includes the BitBake User Manual and the Toaster User Manual.
3# The Makefile needs to live in the documents directory and all figures used 6# The Makefile needs to live in the documents directory and all figures used
@@ -137,32 +140,10 @@ ALLPREQ = html tarball
137# This would be appropriate for "master" branch. 140# This would be appropriate for "master" branch.
138# 141#
139 142
140 ifeq ($(BRANCH),edison)
141TARFILES = dev-style.css dev-manual.html \
142 figures/app-dev-flow.png figures/bsp-dev-flow.png \
143 figures/dev-title.png figures/git-workflow.png \
144 figures/index-downloads.png figures/kernel-dev-flow.png \
145 figures/kernel-example-repos-edison.png \
146 figures/kernel-overview-1.png figures/kernel-overview-2.png \
147 figures/kernel-overview-3-edison.png \
148 figures/source-repos.png figures/yp-download.png \
149 figures/wip.png
150 else ifeq ($(BRANCH),denzil)
151TARFILES = dev-style.css dev-manual.html \
152 figures/app-dev-flow.png figures/bsp-dev-flow.png \
153 figures/dev-title.png figures/git-workflow.png \
154 figures/index-downloads.png figures/kernel-dev-flow.png \
155 figures/kernel-example-repos-denzil.png \
156 figures/kernel-overview-1.png figures/kernel-overview-2.png \
157 figures/kernel-overview-3-denzil.png \
158 figures/source-repos.png figures/yp-download.png \
159 figures/wip.png
160 else
161TARFILES = dev-style.css dev-manual.html figures/buildhistory-web.png \ 143TARFILES = dev-style.css dev-manual.html figures/buildhistory-web.png \
162 figures/dev-title.png figures/buildhistory.png \ 144 figures/dev-title.png figures/buildhistory.png \
163 figures/recipe-workflow.png figures/bitbake-build-flow.png \ 145 figures/recipe-workflow.png figures/bitbake-build-flow.png \
164 figures/multiconfig_files.png figures/cute-files-npm-example.png 146 figures/multiconfig_files.png figures/cute-files-npm-example.png
165 endif
166 147
167MANUALS = $(DOC)/$(DOC).html 148MANUALS = $(DOC)/$(DOC).html
168FIGURES = figures 149FIGURES = figures
@@ -178,37 +159,6 @@ XSLTOPTS = --stringparam html.stylesheet mega-style.css \
178 --xinclude 159 --xinclude
179ALLPREQ = html tarball 160ALLPREQ = html tarball
180 161
181 ifeq ($(BRANCH),edison)
182TARFILES = mega-manual.html mega-style.css figures/yocto-environment.png \
183 figures/building-an-image.png \
184 figures/using-a-pre-built-image.png \
185 figures/poky-title.png \
186 figures/adt-title.png figures/bsp-title.png \
187 figures/kernel-title.png figures/kernel-architecture-overview.png \
188 figures/app-dev-flow.png figures/bsp-dev-flow.png \
189 figures/dev-title.png figures/git-workflow.png \
190 figures/index-downloads.png figures/kernel-dev-flow.png \
191 figures/kernel-example-repos-edison.png \
192 figures/kernel-overview-1.png figures/kernel-overview-2.png \
193 figures/kernel-overview-3-edison.png \
194 figures/source-repos.png figures/yp-download.png \
195 figures/wip.png
196 else ifeq ($(BRANCH),denzil)
197TARFILES = mega-manual.html mega-style.css figures/yocto-environment.png \
198 figures/building-an-image.png \
199 figures/using-a-pre-built-image.png \
200 figures/poky-title.png \
201 figures/adt-title.png figures/bsp-title.png \
202 figures/kernel-title.png figures/kernel-architecture-overview.png \
203 figures/app-dev-flow.png figures/bsp-dev-flow.png \
204 figures/dev-title.png figures/git-workflow.png \
205 figures/index-downloads.png figures/kernel-dev-flow.png \
206 figures/kernel-example-repos-denzil.png \
207 figures/kernel-overview-1.png figures/kernel-overview-2.png \
208 figures/kernel-overview-3-denzil.png \
209 figures/source-repos.png figures/yp-download.png \
210 figures/wip.png
211 else
212TARFILES = mega-manual.html mega-style.css \ 162TARFILES = mega-manual.html mega-style.css \
213 figures/YP-flow-diagram.png \ 163 figures/YP-flow-diagram.png \
214 figures/using-a-pre-built-image.png \ 164 figures/using-a-pre-built-image.png \
@@ -266,7 +216,6 @@ TARFILES = mega-manual.html mega-style.css \
266 figures/sdk-devtool-upgrade-flow.png figures/bitbake-build-flow.png figures/bypqs-title.png \ 216 figures/sdk-devtool-upgrade-flow.png figures/bitbake-build-flow.png figures/bypqs-title.png \
267 figures/overview-manual-title.png figures/sdk-autotools-flow.png figures/sdk-makefile-flow.png \ 217 figures/overview-manual-title.png figures/sdk-autotools-flow.png figures/sdk-makefile-flow.png \
268 figures/bb_multiconfig_files.png figures/bitbake-title.png figures/cute-files-npm-example.png 218 figures/bb_multiconfig_files.png figures/bitbake-title.png figures/cute-files-npm-example.png
269 endif
270 219
271MANUALS = $(DOC)/$(DOC).html 220MANUALS = $(DOC)/$(DOC).html
272FIGURES = figures 221FIGURES = figures
@@ -355,6 +304,16 @@ STYLESHEET = $(DOC)/*.css
355endif 304endif
356 305
357 306
307ifeq ($(DOC),test-manual)
308XSLTOPTS = --xinclude
309ALLPREQ = html tarball
310TARFILES = test-manual.html test-manual-style.css \
311 figures/test-manual-title.png figures/ab-test-cluster.png
312MANUALS = $(DOC)/$(DOC).html
313FIGURES = figures
314STYLESHEET = $(DOC)/*.css
315endif
316
358## 317##
359# These URI should be rewritten by your distribution's xml catalog to 318# These URI should be rewritten by your distribution's xml catalog to
360# match your locally installed XSL stylesheets. 319# match your locally installed XSL stylesheets.
diff --git a/documentation/Makefile.sphinx b/documentation/Makefile.sphinx
new file mode 100644
index 0000000000..c9518558bb
--- /dev/null
+++ b/documentation/Makefile.sphinx
@@ -0,0 +1,35 @@
1# Minimal makefile for Sphinx documentation
2#
3
4# You can set these variables from the command line, and also
5# from the environment for the first two.
6SPHINXOPTS ?=
7SPHINXBUILD ?= sphinx-build
8SOURCEDIR = .
9BUILDDIR = _build
10DESTDIR = final
11
12ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0)
13$(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed")
14endif
15
16# Put it first so that "make" without argument is like "make help".
17help:
18 @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
19
20.PHONY: help Makefile.sphinx clean publish
21
22publish: Makefile.sphinx html singlehtml
23 rm -rf $(BUILDDIR)/$(DESTDIR)/
24 mkdir -p $(BUILDDIR)/$(DESTDIR)/
25 cp -r $(BUILDDIR)/html/* $(BUILDDIR)/$(DESTDIR)/
26 cp $(BUILDDIR)/singlehtml/index.html $(BUILDDIR)/$(DESTDIR)/singleindex.html
27 sed -i -e 's@index.html#@singleindex.html#@g' $(BUILDDIR)/$(DESTDIR)/singleindex.html
28
29clean:
30 @rm -rf $(BUILDDIR)
31
32# Catch-all target: route all unknown targets to Sphinx using the new
33# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
34%: Makefile.sphinx
35 @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/documentation/README b/documentation/README
index d64f2fd2f9..fce3cfe17e 100644
--- a/documentation/README
+++ b/documentation/README
@@ -40,16 +40,11 @@ Folders exist for individual manuals as follows:
40* kernel-dev - The Yocto Project Linux Kernel Development Tasks Manual 40* kernel-dev - The Yocto Project Linux Kernel Development Tasks Manual
41* ref-manual - The Yocto Project Reference Manual 41* ref-manual - The Yocto Project Reference Manual
42* yocto-project-qs - The Yocto Project Quick Start 42* yocto-project-qs - The Yocto Project Quick Start
43* mega-manual - The Yocto Project Mega-Manual, which is an aggregated manual comprised
44 of all YP manuals and guides
45* profile-manual - The Yocto Project Profile and Tracing Manual 43* profile-manual - The Yocto Project Profile and Tracing Manual
46* toaster-manual - The Toaster Manual 44* toaster-manual - The Toaster Manual
45* test-manual - The Test Environment Manual
47 46
48Each folder is self-contained regarding content and figures. Note that there 47Each folder is self-contained regarding content and figures.
49is a sed file needed to process the links of the mega-manual. The sed file
50is located in the tools directory. Also note that the figures folder in the
51mega-manual directory contains duplicates of all the figures in the YP folders
52directories for all YP manuals and guides.
53 48
54If you want to find HTML versions of the Yocto Project manuals on the web, 49If you want to find HTML versions of the Yocto Project manuals on the web,
55go to http://www.yoctoproject.org and click on the "Documentation" tab. From 50go to http://www.yoctoproject.org and click on the "Documentation" tab. From
@@ -60,34 +55,266 @@ currently being developed.
60In general, the Yocto Project site (http://www.yoctoproject.org) is a great 55In general, the Yocto Project site (http://www.yoctoproject.org) is a great
61reference for both information and downloads. 56reference for both information and downloads.
62 57
63Makefile 58poky.yaml
59=========
60
61This file defines variables used for documentation production. The variables
62are used to define release pathnames, URLs for the published manuals, etc.
63
64template
64======== 65========
66Contains various templates, fonts, and some old PNG files.
67
68Sphinx
69======
70
71The Yocto Project documentation was migrated from the original DocBook
72format to Sphinx based documentation for the Yocto Project 3.2
73release. This section will provide additional information related to
74the Sphinx migration, and guidelines for developers willing to
75contribute to the Yocto Project documentation.
76
77 Sphinx is a tool that makes it easy to create intelligent and
78 beautiful documentation, written by Georg Brandl and licensed under
79 the BSD license. It was originally created for the Python
80 documentation.
81
82Extensive documentation is available on the Sphinx website:
83https://www.sphinx-doc.org/en/master/. Sphinx is designed to be
84extensible thanks to the ability to write our own custom extensions,
85as Python modules, which will be executed during the generation of the
86documentation.
87
88Yocto Project documentation website
89===================================
90
91A new website has been created to host the Yocto Project
92documentation, it can be found at: https://docs.yoctoproject.org/.
93
94The entire Yocto Project documentation, as well as the BitBake manual
95is published on this website, including all previously released
96versions. A version switcher was added, as a drop-down menu on the top
97of the page to switch back and forth between the various versions of
98the current active Yocto Project releases.
99
100Transition pages have been added (as rst file) to show links to old
101versions of the Yocto Project documentation with links to each manual
102generated with DocBook.
103
104How to build the Yocto Project documentation
105============================================
106
107Sphinx is written in Python. While it might work with Python2, for
108obvious reasons, we will only support building the Yocto Project
109documentation with Python3.
110
111Sphinx might be available in your Linux distro packages repositories,
112however it is not recommend using distro packages, as they might be
113old versions, especially if you are using an LTS version of your
114distro. The recommended method to install Sphinx and all required
115dependencies is to use the Python Package Index (pip).
116
117To install all required packages run:
65 118
66The Makefile processes manual directories to create HTML, PDF, 119 $ pip3 install sphinx sphinx_rtd_theme pyyaml
67tarballs, etc. Details on how the Makefile work are documented
68inside the Makefile. See that file for more information.
69 120
70To build a manual, you run the make command and pass it the name 121To build the documentation locally, run:
71of the folder containing the manual's contents.
72For example, the following command run from the documentation directory
73creates an HTML version of the SDK manual.
74The DOC variable specifies the manual you are making:
75 122
76 $ make DOC=sdk-manual 123 $ cd documentation
124 $ make -f Makefile.sphinx html
77 125
78poky.ent 126The resulting HTML index page will be _build/html/index.html, and you
127can browse your own copy of the locally generated documentation with
128your browser.
129
130Sphinx theme and CSS customization
131==================================
132
133The Yocto Project documentation is currently based on the "Read the
134Docs" Sphinx theme, with a few changes to make sure the look and feel
135of the project documentation is preserved.
136
137Most of the theme changes can be done using the file
138'sphinx-static/theme_overrides.css'. Most CSS changes in this file
139were inherited from the DocBook CSS stylesheets.
140
141Sphinx design guidelines and principles
142=======================================
143
144The initial Docbook to Sphinx migration was done with an automated
145tool called Pandoc (https://pandoc.org/). The tool produced some clean
146output markdown text files. After the initial automated conversion
147additional changes were done to fix up headings, images and links. In
148addition Sphinx has built in mechanisms (directives) which were used
149to replace similar functions implemented in Docbook such as glossary,
150variables substitutions, notes and references.
151
152Headings
79======== 153========
80 154
81This file defines variables used for documentation production. The variables 155The layout of the Yocto Project manuals is organized as follows
82are used to define release pathnames, URLs for the published manuals, etc.
83 156
84template 157 Book
158 Chapter
159 Section
160 Section
161 Section
162
163The following headings styles are defined in Sphinx:
164
165 Book => overline ===
166 Chapter => overline ***
167 Section => ====
168 Section => ----
169 Section => ^^^^
170 Section => """" or ~~~~
171
172With this proposal, we preserve the same TOCs between Sphinx and Docbook.
173
174Built-in glossary
175=================
176
177Sphinx has a glossary directive. From
178https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:
179
180 This directive must contain a reST definition list with terms and
181 definitions. The definitions will then be referencable with the
182 [https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
183 'term' role].
184
185So anywhere in any of the Yocto Project manuals, :term:`VAR` can be
186used to refer to an item from the glossary, and a link is created
187automatically. A general index of terms is also generated by Sphinx
188automatically.
189
190Global substitutions
191====================
192
193The Yocto Project documentation makes heavy use of global
194variables. In Docbook these variables are stored in the file
195poky.ent. This Docbook feature is not handled automatically with
196Pandoc. Sphinx has builtin support for substitutions
197(https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions),
198however there are important shortcomings. For example they cannot be
199used/nested inside code-block sections.
200
201A Sphinx extension was implemented to support variable substitutions
202to mimic the DocBook based documentation behavior. Variabes
203substitutions are done while reading/parsing the .rst files. The
204pattern for variables substitutions is the same as with DocBook,
205e.g. `&VAR;`.
206
207The implementation of the extension can be found here in the file
208documentation/sphinx/yocto-vars.py, this extension is enabled by
209default when building the Yocto Project documentation. All variables
210are set in a file call poky.yaml, which was initially generated from
211poky.ent. The file was converted into YAML so that it is easier to
212process by the custom Sphinx extension (which is a Python module).
213
214For example, the following .rst content will produce the 'expected'
215content:
216
217 .. code-block::
218 $ mkdir ~/poky-&DISTRO;
219 or
220 $ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;
221
222Variables can be nested, like it was the case for DocBook:
223
224 YOCTO_HOME_URL : "http://www.yoctoproject.org"
225 YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
226
227Note directive
228==============
229
230Sphinx has a builtin 'note' directive that produces clean Note section
231in the output file. There are various types of directives such as
232"attention", "caution", "danger", "error", "hint", "important", "tip",
233"warning", "admonition" that are supported, and additional directive
234can be added as Sphinx extension if needed.
235
236Figures
237=======
238
239The Yocto Project documentation has many figures/images. Sphinx has a
240'figure' directive which is straight forward to use. To include a
241figure in the body of the documentation:
242
243 .. image:: figures/YP-flow-diagram.png
244
245Links and References
246====================
247
248The following types of links can be used: links to other locations in
249the same document, to locations in other documents and to external
250websites.
251
252More information can be found here:
253https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.
254
255References
256==========
257
258The following extension is enabed by default:
259sphinx.ext.autosectionlabel
260(https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).
261
262This extension allows you to refer sections by their titles. Note that
263autosectionlabel_prefix_document is enabled by default, so that we can
264insert references from any document.
265
266For example, to insert an HTML link to a section from
267documentaion/manual/intro.rst, use:
268
269 Please check this :ref:`manual/intro:Cross-References to Locations in the Same Document`
270
271Alternatively a custom text can be used instead of using the section
272text:
273
274 Please check this :ref:`section <manual/intro:Cross-References to Locations in the Same Document>`
275
276TIP: The following command can be used to dump all the references that
277 are defined in the project documentation:
278
279 python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv
280
281This dump contains all links and for each link it shows the default
282"Link Text" that Sphinx would use. If the default link text is not
283appropriate, a custom link text can be used in the ':ref:' directive.
284
285Extlinks
85======== 286========
86Contains various templates, fonts, and some old PNG files.
87 287
88tools 288The sphinx.ext.extlinks extension is enabled by default
89===== 289(https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension),
90Contains a tool to convert the DocBook files to PDF format. This folder also 290and it is configured with:
91contains the mega-manual.sed file, which is used by Makefile to process 291
92cross-references from within the manual that normally go to an external 292 'yocto_home': ('https://yoctoproject.org%s', None),
93manual. 293 'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
294 'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
295 'yocto_lists': ('https://lists.yoctoproject.org%s', None),
296 'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
297 'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
298 'yocto_docs': ('https://docs.yoctoproject.org%s', None),
299 'yocto_git': ('https://git.yoctoproject.org%s', None),
300 'oe_home': ('https://www.openembedded.org%s', None),
301 'oe_lists': ('https://lists.openembedded.org%s', None),
302
303It creates convenient shortcuts which can be used throughout the
304documentation rst files, as:
305
306 Please check this :yocto_wiki:`wiki page </Weekly_Status>`
307
308Intersphinx links
309=================
310
311The sphinx.ext.intersphinx extension is enabled by default
312(https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html),
313so that we can cross reference content from other Sphinx based
314documentation projects, such as the BitBake manual.
315
316References to the bitbake manual can be done like this:
317
318 See the ":ref:`-D <bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax>`" option
319or
320 :term:`bitbake:BB_NUMBER_PARSE_THREADS`
diff --git a/documentation/_templates/breadcrumbs.html b/documentation/_templates/breadcrumbs.html
new file mode 100644
index 0000000000..eb6244b74c
--- /dev/null
+++ b/documentation/_templates/breadcrumbs.html
@@ -0,0 +1,14 @@
1{% extends "!breadcrumbs.html" %}
2
3{% block breadcrumbs %}
4 <li>
5 <span class="doctype_switcher_placeholder">{{ doctype or 'single' }}</span>
6 <span class="version_switcher_placeholder">{{ release }}</span>
7 </li>
8 <li> &raquo;</li>
9 {% for doc in parents %}
10 <li><a href="{{ doc.link|e }}">{{ doc.title }}</a> &raquo;</li>
11 {% endfor %}
12 <li>{{ title }}</li>
13{% endblock %}
14
diff --git a/documentation/_templates/layout.html b/documentation/_templates/layout.html
new file mode 100644
index 0000000000..308d5c7a28
--- /dev/null
+++ b/documentation/_templates/layout.html
@@ -0,0 +1,7 @@
1{% extends "!layout.html" %}
2
3{% block extrabody %}
4<div id="outdated-warning" style="text-align: center; background-color: #FFBABA; color: #6A0E0E;">
5</div>
6{% endblock %}
7
diff --git a/documentation/adt-manual/adt-command.rst b/documentation/adt-manual/adt-command.rst
new file mode 100644
index 0000000000..de854772bb
--- /dev/null
+++ b/documentation/adt-manual/adt-command.rst
@@ -0,0 +1,180 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************
4Using the Command Line
5**********************
6
7Recall that earlier the manual discussed how to use an existing
8toolchain tarball that had been installed into the default installation
9directory, ``/opt/poky/DISTRO``, which is outside of the :term:`Build Directory`
10(see the section
11"`Using a Cross-Toolchain
12Tarball) <#using-an-existing-toolchain-tarball>`__". And, that sourcing
13your architecture-specific environment setup script initializes a
14suitable cross-toolchain development environment.
15
16During this setup, locations for the compiler, QEMU scripts, QEMU
17binary, a special version of ``pkgconfig`` and other useful utilities
18are added to the ``PATH`` variable. Also, variables to assist
19``pkgconfig`` and ``autotools`` are also defined so that, for example,
20``configure.sh`` can find pre-generated test results for tests that need
21target hardware on which to run. You can see the "`Setting Up the
22Cross-Development
23Environment <#setting-up-the-cross-development-environment>`__" section
24for the list of cross-toolchain environment variables established by the
25script.
26
27Collectively, these conditions allow you to easily use the toolchain
28outside of the OpenEmbedded build environment on both Autotools-based
29projects and Makefile-based projects. This chapter provides information
30for both these types of projects.
31
32Autotools-Based Projects
33========================
34
35Once you have a suitable cross-toolchain installed, it is very easy to
36develop a project outside of the OpenEmbedded build system. This section
37presents a simple "Helloworld" example that shows how to set up,
38compile, and run the project.
39
40Creating and Running a Project Based on GNU Autotools
41-----------------------------------------------------
42
43Follow these steps to create a simple Autotools-based project:
44
451. *Create your directory:* Create a clean directory for your project
46 and then make that directory your working location: $ mkdir
47 $HOME/helloworld $ cd $HOME/helloworld
48
492. *Populate the directory:* Create ``hello.c``, ``Makefile.am``, and
50 ``configure.in`` files as follows:
51
52 - For ``hello.c``, include these lines: #include <stdio.h> main() {
53 printf("Hello World!\n"); }
54
55 - For ``Makefile.am``, include these lines: bin_PROGRAMS = hello
56 hello_SOURCES = hello.c
57
58 - For ``configure.in``, include these lines: AC_INIT(hello.c)
59 AM_INIT_AUTOMAKE(hello,0.1) AC_PROG_CC AC_PROG_INSTALL
60 AC_OUTPUT(Makefile)
61
623. *Source the cross-toolchain environment setup file:* Installation of
63 the cross-toolchain creates a cross-toolchain environment setup
64 script in the directory that the ADT was installed. Before you can
65 use the tools to develop your project, you must source this setup
66 script. The script begins with the string "environment-setup" and
67 contains the machine architecture, which is followed by the string
68 "poky-linux". Here is an example that sources a script from the
69 default ADT installation directory that uses the 32-bit Intel x86
70 Architecture and the DISTRO_NAME Yocto Project release: $ source
71 /opt/poky/DISTRO/environment-setup-i586-poky-linux
72
734. *Generate the local aclocal.m4 files and create the configure
74 script:* The following GNU Autotools generate the local
75 ``aclocal.m4`` files and create the configure script: $ aclocal $
76 autoconf
77
785. *Generate files needed by GNU coding standards:* GNU coding
79 standards require certain files in order for the project to be
80 compliant. This command creates those files: $ touch NEWS README
81 AUTHORS ChangeLog
82
836. *Generate the configure file:* This command generates the
84 ``configure``: $ automake -a
85
867. *Cross-compile the project:* This command compiles the project using
87 the cross-compiler. The
88 :term:`CONFIGURE_FLAGS`
89 environment variable provides the minimal arguments for GNU
90 configure: $ ./configure ${CONFIGURE_FLAGS}
91
928. *Make and install the project:* These two commands generate and
93 install the project into the destination directory: $ make $ make
94 install DESTDIR=./tmp
95
969. *Verify the installation:* This command is a simple way to verify
97 the installation of your project. Running the command prints the
98 architecture on which the binary file can run. This architecture
99 should be the same architecture that the installed cross-toolchain
100 supports. $ file ./tmp/usr/local/bin/hello
101
10210. *Execute your project:* To execute the project in the shell, simply
103 enter the name. You could also copy the binary to the actual target
104 hardware and run the project there as well: $ ./hello As expected,
105 the project displays the "Hello World!" message.
106
107Passing Host Options
108--------------------
109
110For an Autotools-based project, you can use the cross-toolchain by just
111passing the appropriate host option to ``configure.sh``. The host option
112you use is derived from the name of the environment setup script found
113in the directory in which you installed the cross-toolchain. For
114example, the host option for an ARM-based target that uses the GNU EABI
115is ``armv5te-poky-linux-gnueabi``. You will notice that the name of the
116script is ``environment-setup-armv5te-poky-linux-gnueabi``. Thus, the
117following command works to update your project and rebuild it using the
118appropriate cross-toolchain tools: $ ./configure
119--host=armv5te-poky-linux-gnueabi \\ --with-libtool-sysroot=sysroot_dir
120
121.. note::
122
123 If the
124 configure
125 script results in problems recognizing the
126 --with-libtool-sysroot=
127 sysroot-dir
128 option, regenerate the script to enable the support by doing the
129 following and then run the script again:
130 ::
131
132 $ libtoolize --automake
133 $ aclocal -I ${OECORE_NATIVE_SYSROOT}/usr/share/aclocal \
134 [-I dir_containing_your_project-specific_m4_macros]
135 $ autoconf
136 $ autoheader
137 $ automake -a
138
139
140Makefile-Based Projects
141=======================
142
143For Makefile-based projects, the cross-toolchain environment variables
144established by running the cross-toolchain environment setup script are
145subject to general ``make`` rules.
146
147To illustrate this, consider the following four cross-toolchain
148environment variables:
149:term:`CC`\ =i586-poky-linux-gcc -m32
150-march=i586 --sysroot=/opt/poky/1.8/sysroots/i586-poky-linux
151:term:`LD`\ =i586-poky-linux-ld
152--sysroot=/opt/poky/1.8/sysroots/i586-poky-linux
153:term:`CFLAGS`\ =-O2 -pipe -g
154-feliminate-unused-debug-types
155:term:`CXXFLAGS`\ =-O2 -pipe -g
156-feliminate-unused-debug-types Now, consider the following three cases:
157
158- *Case 1 - No Variables Set in the ``Makefile``:* Because these
159 variables are not specifically set in the ``Makefile``, the variables
160 retain their values based on the environment.
161
162- *Case 2 - Variables Set in the ``Makefile``:* Specifically setting
163 variables in the ``Makefile`` during the build results in the
164 environment settings of the variables being overwritten.
165
166- *Case 3 - Variables Set when the ``Makefile`` is Executed from the
167 Command Line:* Executing the ``Makefile`` from the command line
168 results in the variables being overwritten with command-line content
169 regardless of what is being set in the ``Makefile``. In this case,
170 environment variables are not considered unless you use the "-e" flag
171 during the build: $ make -e file If you use this flag, then the
172 environment values of the variables override any variables
173 specifically set in the ``Makefile``.
174
175.. note::
176
177 For the list of variables set up by the cross-toolchain environment
178 setup script, see the "
179 Setting Up the Cross-Development Environment
180 " section.
diff --git a/documentation/adt-manual/adt-command.xml b/documentation/adt-manual/adt-command.xml
index c78d18a16d..b88c0ac682 100644
--- a/documentation/adt-manual/adt-command.xml
+++ b/documentation/adt-manual/adt-command.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='using-the-command-line'> 6<chapter id='using-the-command-line'>
6<title>Using the Command Line</title> 7<title>Using the Command Line</title>
diff --git a/documentation/adt-manual/adt-intro.rst b/documentation/adt-manual/adt-intro.rst
new file mode 100644
index 0000000000..5372f4f54f
--- /dev/null
+++ b/documentation/adt-manual/adt-intro.rst
@@ -0,0 +1,138 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************************
4The Application Development Toolkit (ADT)
5*****************************************
6
7Part of the Yocto Project development solution is an Application
8Development Toolkit (ADT). The ADT provides you with a custom-built,
9cross-development platform suited for developing a user-targeted product
10application.
11
12Fundamentally, the ADT consists of the following:
13
14- An architecture-specific cross-toolchain and matching sysroot both
15 built by the :term:`OpenEmbedded Build System`.
16 The toolchain and
17 sysroot are based on a `Metadata <&YOCTO_DOCS_DEV_URL;#metadata>`__
18 configuration and extensions, which allows you to cross-develop on
19 the host machine for the target hardware.
20
21- The Eclipse IDE Yocto Plug-in.
22
23- The Quick EMUlator (QEMU), which lets you simulate target hardware.
24
25- Various user-space tools that greatly enhance your application
26 development experience.
27
28The Cross-Development Toolchain
29===============================
30
31The `Cross-Development
32Toolchain <&YOCTO_DOCS_DEV_URL;#cross-development-toolchain>`__ consists
33of a cross-compiler, cross-linker, and cross-debugger that are used to
34develop user-space applications for targeted hardware. This toolchain is
35created either by running the ADT Installer script, a toolchain
36installer script, or through a :term:`Build Directory`
37that is based on
38your Metadata configuration or extension for your targeted device. The
39cross-toolchain works with a matching target sysroot.
40
41Sysroot
42=======
43
44The matching target sysroot contains needed headers and libraries for
45generating binaries that run on the target architecture. The sysroot is
46based on the target root filesystem image that is built by the
47OpenEmbedded build system and uses the same Metadata configuration used
48to build the cross-toolchain.
49
50.. _eclipse-overview:
51
52Eclipse Yocto Plug-in
53=====================
54
55The Eclipse IDE is a popular development environment and it fully
56supports development using the Yocto Project. When you install and
57configure the Eclipse Yocto Project Plug-in into the Eclipse IDE, you
58maximize your Yocto Project experience. Installing and configuring the
59Plug-in results in an environment that has extensions specifically
60designed to let you more easily develop software. These extensions allow
61for cross-compilation, deployment, and execution of your output into a
62QEMU emulation session. You can also perform cross-debugging and
63profiling. The environment also supports a suite of tools that allows
64you to perform remote profiling, tracing, collection of power data,
65collection of latency data, and collection of performance data.
66
67For information about the application development workflow that uses the
68Eclipse IDE and for a detailed example of how to install and configure
69the Eclipse Yocto Project Plug-in, see the "`Working Within
70Eclipse <&YOCTO_DOCS_DEV_URL;#adt-eclipse>`__" section of the Yocto
71Project Development Manual.
72
73The QEMU Emulator
74=================
75
76The QEMU emulator allows you to simulate your hardware while running
77your application or image. QEMU is made available a number of ways:
78
79- If you use the ADT Installer script to install ADT, you can specify
80 whether or not to install QEMU.
81
82- If you have cloned the ``poky`` Git repository to create a
83 :term:`Source Directory` and you have
84 sourced the environment setup script, QEMU is installed and
85 automatically available.
86
87- If you have downloaded a Yocto Project release and unpacked it to
88 create a :term:`Source Directory`
89 and you have sourced the environment setup script, QEMU is installed
90 and automatically available.
91
92- If you have installed the cross-toolchain tarball and you have
93 sourced the toolchain's setup environment script, QEMU is also
94 installed and automatically available.
95
96User-Space Tools
97================
98
99User-space tools are included as part of the Yocto Project. You will
100find these tools helpful during development. The tools include
101LatencyTOP, PowerTOP, OProfile, Perf, SystemTap, and Lttng-ust. These
102tools are common development tools for the Linux platform.
103
104- *LatencyTOP:* LatencyTOP focuses on latency that causes skips in
105 audio, stutters in your desktop experience, or situations that
106 overload your server even when you have plenty of CPU power left.
107
108- *PowerTOP:* Helps you determine what software is using the most
109 power. You can find out more about PowerTOP at
110 https://01.org/powertop/.
111
112- *OProfile:* A system-wide profiler for Linux systems that is capable
113 of profiling all running code at low overhead. You can find out more
114 about OProfile at http://oprofile.sourceforge.net/about/. For
115 examples on how to setup and use this tool, see the
116 "`OProfile <&YOCTO_DOCS_PROF_URL;#profile-manual-oprofile>`__"
117 section in the Yocto Project Profiling and Tracing Manual.
118
119- *Perf:* Performance counters for Linux used to keep track of certain
120 types of hardware and software events. For more information on these
121 types of counters see https://perf.wiki.kernel.org/. For
122 examples on how to setup and use this tool, see the
123 "`perf <&YOCTO_DOCS_PROF_URL;#profile-manual-perf>`__" section in the
124 Yocto Project Profiling and Tracing Manual.
125
126- *SystemTap:* A free software infrastructure that simplifies
127 information gathering about a running Linux system. This information
128 helps you diagnose performance or functional problems. SystemTap is
129 not available as a user-space tool through the Eclipse IDE Yocto
130 Plug-in. See http://sourceware.org/systemtap for more
131 information on SystemTap. For examples on how to setup and use this
132 tool, see the
133 "`SystemTap <&YOCTO_DOCS_PROF_URL;#profile-manual-systemtap>`__"
134 section in the Yocto Project Profiling and Tracing Manual.
135
136- *Lttng-ust:* A User-space Tracer designed to provide detailed
137 information on user-space activity. See http://lttng.org/ust
138 for more information on Lttng-ust.
diff --git a/documentation/adt-manual/adt-intro.xml b/documentation/adt-manual/adt-intro.xml
index 597c7120ba..eb75763db3 100644
--- a/documentation/adt-manual/adt-intro.xml
+++ b/documentation/adt-manual/adt-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='adt-intro'> 6<chapter id='adt-intro'>
6 <title>The Application Development Toolkit (ADT)</title> 7 <title>The Application Development Toolkit (ADT)</title>
diff --git a/documentation/adt-manual/adt-manual-customization.xsl b/documentation/adt-manual/adt-manual-customization.xsl
index b86be519b9..551f7e9e94 100644
--- a/documentation/adt-manual/adt-manual-customization.xsl
+++ b/documentation/adt-manual/adt-manual-customization.xsl
@@ -1,4 +1,5 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 3<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 4
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 5 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/adt-manual/adt-manual-eclipse-customization.xsl b/documentation/adt-manual/adt-manual-eclipse-customization.xsl
index 77ba5f5719..3d536d5473 100644
--- a/documentation/adt-manual/adt-manual-eclipse-customization.xsl
+++ b/documentation/adt-manual/adt-manual-eclipse-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet 4<xsl:stylesheet
3 xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 5 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
4 xmlns="http://www.w3.org/1999/xhtml" 6 xmlns="http://www.w3.org/1999/xhtml"
diff --git a/documentation/adt-manual/adt-manual-intro.rst b/documentation/adt-manual/adt-manual-intro.rst
new file mode 100644
index 0000000000..4e98da16df
--- /dev/null
+++ b/documentation/adt-manual/adt-manual-intro.rst
@@ -0,0 +1,24 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Introduction
5************
6
7Welcome to the Yocto Project Application Developer's Guide. This manual
8provides information that lets you begin developing applications using
9the Yocto Project.
10
11The Yocto Project provides an application development environment based
12on an Application Development Toolkit (ADT) and the availability of
13stand-alone cross-development toolchains and other tools. This manual
14describes the ADT and how you can configure and install it, how to
15access and use the cross-development toolchains, how to customize the
16development packages installation, how to use command-line development
17for both Autotools-based and Makefile-based projects, and an
18introduction to the Eclipse IDE Yocto Plug-in.
19
20.. note::
21
22 The ADT is distribution-neutral and does not require the Yocto
23 Project reference distribution, which is called Poky. This manual,
24 however, uses examples that use the Poky distribution.
diff --git a/documentation/adt-manual/adt-manual-intro.xml b/documentation/adt-manual/adt-manual-intro.xml
index 034fdff609..b7a25a54bd 100644
--- a/documentation/adt-manual/adt-manual-intro.xml
+++ b/documentation/adt-manual/adt-manual-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='adt-manual-intro'> 6<chapter id='adt-manual-intro'>
6<title>Introduction</title> 7<title>Introduction</title>
diff --git a/documentation/adt-manual/adt-manual.rst b/documentation/adt-manual/adt-manual.rst
new file mode 100644
index 0000000000..695230c5c4
--- /dev/null
+++ b/documentation/adt-manual/adt-manual.rst
@@ -0,0 +1,17 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3===========================================
4Yocto Project Application Developer's Guide
5===========================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 adt-manual-intro
14 adt-intro
15 adt-prepare
16 adt-package
17 adt-command
diff --git a/documentation/adt-manual/adt-manual.xml b/documentation/adt-manual/adt-manual.xml
index 972f8bf086..13202cc0de 100644
--- a/documentation/adt-manual/adt-manual.xml
+++ b/documentation/adt-manual/adt-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='adt-manual' lang='en' 6<book id='adt-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
diff --git a/documentation/adt-manual/adt-package.rst b/documentation/adt-manual/adt-package.rst
new file mode 100644
index 0000000000..787d406e65
--- /dev/null
+++ b/documentation/adt-manual/adt-package.rst
@@ -0,0 +1,70 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************************************************************
4Optionally Customizing the Development Packages Installation
5************************************************************
6
7Because the Yocto Project is suited for embedded Linux development, it
8is likely that you will need to customize your development packages
9installation. For example, if you are developing a minimal image, then
10you might not need certain packages (e.g. graphics support packages).
11Thus, you would like to be able to remove those packages from your
12target sysroot.
13
14Package Management Systems
15==========================
16
17The OpenEmbedded build system supports the generation of sysroot files
18using three different Package Management Systems (PMS):
19
20- *OPKG:* A less well known PMS whose use originated in the
21 OpenEmbedded and OpenWrt embedded Linux projects. This PMS works with
22 files packaged in an ``.ipk`` format. See
23 http://en.wikipedia.org/wiki/Opkg for more information about
24 OPKG.
25
26- *RPM:* A more widely known PMS intended for GNU/Linux distributions.
27 This PMS works with files packaged in an ``.rpm`` format. The build
28 system currently installs through this PMS by default. See
29 http://en.wikipedia.org/wiki/RPM_Package_Manager for more
30 information about RPM.
31
32- *Debian:* The PMS for Debian-based systems is built on many PMS
33 tools. The lower-level PMS tool ``dpkg`` forms the base of the Debian
34 PMS. For information on dpkg see
35 http://en.wikipedia.org/wiki/Dpkg.
36
37Configuring the PMS
38===================
39
40Whichever PMS you are using, you need to be sure that the
41:term:`PACKAGE_CLASSES`
42variable in the ``conf/local.conf`` file is set to reflect that system.
43The first value you choose for the variable specifies the package file
44format for the root filesystem at sysroot. Additional values specify
45additional formats for convenience or testing. See the
46``conf/local.conf`` configuration file for details.
47
48.. note::
49
50 For build performance information related to the PMS, see the "
51 package.bbclass
52 " section in the Yocto Project Reference Manual.
53
54As an example, consider a scenario where you are using OPKG and you want
55to add the ``libglade`` package to the target sysroot.
56
57First, you should generate the IPK file for the ``libglade`` package and
58add it into a working ``opkg`` repository. Use these commands: $ bitbake
59libglade $ bitbake package-index
60
61Next, source the cross-toolchain environment setup script found in the
62:term:`Source Directory`. Follow
63that by setting up the installation destination to point to your sysroot
64as sysroot_dir. Finally, have an OPKG configuration file conf_file that
65corresponds to the ``opkg`` repository you have just created. The
66following command forms should now work: $ opkg-cl –f conf_file -o
67sysroot_dir update $ opkg-cl –f cconf_file -o sysroot_dir \\
68--force-overwrite install libglade $ opkg-cl –f cconf_file -o
69sysroot_dir \\ --force-overwrite install libglade-dbg $ opkg-cl –f
70conf_file> -osysroot_dir> \\ --force-overwrite install libglade-dev
diff --git a/documentation/adt-manual/adt-package.xml b/documentation/adt-manual/adt-package.xml
index 68eee9b389..eaed0447b6 100644
--- a/documentation/adt-manual/adt-package.xml
+++ b/documentation/adt-manual/adt-package.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='adt-package'> 6<chapter id='adt-package'>
6<title>Optionally Customizing the Development Packages Installation</title> 7<title>Optionally Customizing the Development Packages Installation</title>
diff --git a/documentation/adt-manual/adt-prepare.rst b/documentation/adt-manual/adt-prepare.rst
new file mode 100644
index 0000000000..9b6bd05147
--- /dev/null
+++ b/documentation/adt-manual/adt-prepare.rst
@@ -0,0 +1,752 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*************************************
4Preparing for Application Development
5*************************************
6
7In order to develop applications, you need set up your host development
8system. Several ways exist that allow you to install cross-development
9tools, QEMU, the Eclipse Yocto Plug-in, and other tools. This chapter
10describes how to prepare for application development.
11
12.. _installing-the-adt:
13
14Installing the ADT and Toolchains
15=================================
16
17The following list describes installation methods that set up varying
18degrees of tool availability on your system. Regardless of the
19installation method you choose, you must ``source`` the cross-toolchain
20environment setup script, which establishes several key environment
21variables, before you use a toolchain. See the "`Setting Up the
22Cross-Development
23Environment <#setting-up-the-cross-development-environment>`__" section
24for more information.
25
26.. note::
27
28 Avoid mixing installation methods when installing toolchains for
29 different architectures. For example, avoid using the ADT Installer
30 to install some toolchains and then hand-installing cross-development
31 toolchains by running the toolchain installer for different
32 architectures. Mixing installation methods can result in situations
33 where the ADT Installer becomes unreliable and might not install the
34 toolchain.
35
36 If you must mix installation methods, you might avoid problems by
37 deleting ``/var/lib/opkg``, thus purging the ``opkg`` package
38 metadata.
39
40- *Use the ADT installer script:* This method is the recommended way to
41 install the ADT because it automates much of the process for you. For
42 example, you can configure the installation to install the QEMU
43 emulator and the user-space NFS, specify which root filesystem
44 profiles to download, and define the target sysroot location.
45
46- *Use an existing toolchain:* Using this method, you select and
47 download an architecture-specific toolchain installer and then run
48 the script to hand-install the toolchain. If you use this method, you
49 just get the cross-toolchain and QEMU - you do not get any of the
50 other mentioned benefits had you run the ADT Installer script.
51
52- *Use the toolchain from within the Build Directory:* If you already
53 have a :term:`Build Directory`,
54 you can build the cross-toolchain within the directory. However, like
55 the previous method mentioned, you only get the cross-toolchain and
56 QEMU - you do not get any of the other benefits without taking
57 separate steps.
58
59Using the ADT Installer
60-----------------------
61
62To run the ADT Installer, you need to get the ADT Installer tarball, be
63sure you have the necessary host development packages that support the
64ADT Installer, and then run the ADT Installer Script.
65
66For a list of the host packages needed to support ADT installation and
67use, see the "ADT Installer Extras" lists in the "`Required Packages for
68the Host Development
69System <&YOCTO_DOCS_REF_URL;#required-packages-for-the-host-development-system>`__"
70section of the Yocto Project Reference Manual.
71
72Getting the ADT Installer Tarball
73~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
74
75The ADT Installer is contained in the ADT Installer tarball. You can get
76the tarball using either of these methods:
77
78- *Download the Tarball:* You can download the tarball from
79 ` <&YOCTO_ADTINSTALLER_DL_URL;>`__ into any directory.
80
81- *Build the Tarball:* You can use
82 :term:`BitBake` to generate the
83 tarball inside an existing :term:`Build Directory`.
84
85 If you use BitBake to generate the ADT Installer tarball, you must
86 ``source`` the environment setup script
87 (````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
88 ```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
89 located in the Source Directory before running the ``bitbake``
90 command that creates the tarball.
91
92 The following example commands establish the
93 :term:`Source Directory`, check out the
94 current release branch, set up the build environment while also
95 creating the default Build Directory, and run the ``bitbake`` command
96 that results in the tarball
97 ``poky/build/tmp/deploy/sdk/adt_installer.tar.bz2``:
98
99 .. note::
100
101 Before using BitBake to build the ADT tarball, be sure to make
102 sure your
103 local.conf
104 file is properly configured. See the "
105 User Configuration
106 " section in the Yocto Project Reference Manual for general
107 configuration information.
108
109 $ cd ~ $ git clone git://git.yoctoproject.org/poky $ cd poky $ git
110 checkout -b DISTRO_NAME origin/DISTRO_NAME $ source OE_INIT_FILE $
111 bitbake adt-installer
112
113Configuring and Running the ADT Installer Script
114~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
115
116Before running the ADT Installer script, you need to unpack the tarball.
117You can unpack the tarball in any directory you wish. For example, this
118command copies the ADT Installer tarball from where it was built into
119the home directory and then unpacks the tarball into a top-level
120directory named ``adt-installer``: $ cd ~ $ cp
121poky/build/tmp/deploy/sdk/adt_installer.tar.bz2 $HOME $ tar -xjf
122adt_installer.tar.bz2 Unpacking it creates the directory
123``adt-installer``, which contains the ADT Installer script
124(``adt_installer``) and its configuration file (``adt_installer.conf``).
125
126Before you run the script, however, you should examine the ADT Installer
127configuration file and be sure you are going to get what you want. Your
128configurations determine which kernel and filesystem image are
129downloaded.
130
131The following list describes the configurations you can define for the
132ADT Installer. For configuration values and restrictions, see the
133comments in the ``adt-installer.conf`` file:
134
135- ``YOCTOADT_REPO``: This area includes the IPKG-based packages and the
136 root filesystem upon which the installation is based. If you want to
137 set up your own IPKG repository pointed to by ``YOCTOADT_REPO``, you
138 need to be sure that the directory structure follows the same layout
139 as the reference directory set up at
140 http://adtrepo.yoctoproject.org. Also, your repository needs
141 to be accessible through HTTP.
142
143- ``YOCTOADT_TARGETS``: The machine target architectures for which you
144 want to set up cross-development environments.
145
146- ``YOCTOADT_QEMU``: Indicates whether or not to install the emulator
147 QEMU.
148
149- ``YOCTOADT_NFS_UTIL``: Indicates whether or not to install user-mode
150 NFS. If you plan to use the Eclipse IDE Yocto plug-in against QEMU,
151 you should install NFS.
152
153 .. note::
154
155 To boot QEMU images using our userspace NFS server, you need to be
156 running
157 portmap
158 or
159 rpcbind
160 . If you are running
161 rpcbind
162 , you will also need to add the
163 -i
164 option when
165 rpcbind
166 starts up. Please make sure you understand the security
167 implications of doing this. You might also have to modify your
168 firewall settings to allow NFS booting to work.
169
170- ``YOCTOADT_ROOTFS_``\ arch: The root filesystem images you want to
171 download from the ``YOCTOADT_IPKG_REPO`` repository.
172
173- ``YOCTOADT_TARGET_SYSROOT_IMAGE_``\ arch: The particular root
174 filesystem used to extract and create the target sysroot. The value
175 of this variable must have been specified with
176 ``YOCTOADT_ROOTFS_``\ arch. For example, if you downloaded both
177 ``minimal`` and ``sato-sdk`` images by setting
178 ``YOCTOADT_ROOTFS_``\ arch to "minimal sato-sdk", then
179 ``YOCTOADT_ROOTFS_``\ arch must be set to either "minimal" or
180 "sato-sdk".
181
182- ``YOCTOADT_TARGET_SYSROOT_LOC_``\ arch: The location on the
183 development host where the target sysroot is created.
184
185After you have configured the ``adt_installer.conf`` file, run the
186installer using the following command: $ cd adt-installer $
187./adt_installer Once the installer begins to run, you are asked to enter
188the location for cross-toolchain installation. The default location is
189``/opt/poky/``\ release. After either accepting the default location or
190selecting your own location, you are prompted to run the installation
191script interactively or in silent mode. If you want to closely monitor
192the installation, choose "I" for interactive mode rather than "S" for
193silent mode. Follow the prompts from the script to complete the
194installation.
195
196Once the installation completes, the ADT, which includes the
197cross-toolchain, is installed in the selected installation directory.
198You will notice environment setup files for the cross-toolchain in the
199installation directory, and image tarballs in the ``adt-installer``
200directory according to your installer configurations, and the target
201sysroot located according to the ``YOCTOADT_TARGET_SYSROOT_LOC_``\ arch
202variable also in your configuration file.
203
204.. _using-an-existing-toolchain-tarball:
205
206Using a Cross-Toolchain Tarball
207-------------------------------
208
209If you want to simply install a cross-toolchain by hand, you can do so
210by running the toolchain installer. The installer includes the pre-built
211cross-toolchain, the ``runqemu`` script, and support files. If you use
212this method to install the cross-toolchain, you might still need to
213install the target sysroot by installing and extracting it separately.
214For information on how to install the sysroot, see the "`Extracting the
215Root Filesystem <#extracting-the-root-filesystem>`__" section.
216
217Follow these steps:
218
2191. *Get your toolchain installer using one of the following methods:*
220
221 - Go to ` <&YOCTO_TOOLCHAIN_DL_URL;>`__ and find the folder that
222 matches your host development system (i.e. ``i686`` for 32-bit
223 machines or ``x86_64`` for 64-bit machines).
224
225 Go into that folder and download the toolchain installer whose
226 name includes the appropriate target architecture. The toolchains
227 provided by the Yocto Project are based off of the
228 ``core-image-sato`` image and contain libraries appropriate for
229 developing against that image. For example, if your host
230 development system is a 64-bit x86 system and you are going to use
231 your cross-toolchain for a 32-bit x86 target, go into the
232 ``x86_64`` folder and download the following installer:
233 poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
234
235 - Build your own toolchain installer. For cases where you cannot use
236 an installer from the download area, you can build your own as
237 described in the "`Optionally Building a Toolchain
238 Installer <#optionally-building-a-toolchain-installer>`__"
239 section.
240
2412. *Once you have the installer, run it to install the toolchain:*
242
243 .. note::
244
245 You must change the permissions on the toolchain installer script
246 so that it is executable.
247
248 The following command shows how to run the installer given a
249 toolchain tarball for a 64-bit x86 development host system and a
250 32-bit x86 target architecture. The example assumes the toolchain
251 installer is located in ``~/Downloads/``. $
252 ~/Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
253 The first thing the installer prompts you for is the directory into
254 which you want to install the toolchain. The default directory used
255 is ``/opt/poky/DISTRO``. If you do not have write permissions for the
256 directory into which you are installing the toolchain, the toolchain
257 installer notifies you and exits. Be sure you have write permissions
258 in the directory and run the installer again.
259
260 When the script finishes, the cross-toolchain is installed. You will
261 notice environment setup files for the cross-toolchain in the
262 installation directory.
263
264.. _using-the-toolchain-from-within-the-build-tree:
265
266Using BitBake and the Build Directory
267-------------------------------------
268
269A final way of making the cross-toolchain available is to use BitBake to
270generate the toolchain within an existing :term:`Build Directory`.
271This method does
272not install the toolchain into the default ``/opt`` directory. As with
273the previous method, if you need to install the target sysroot, you must
274do that separately as well.
275
276Follow these steps to generate the toolchain into the Build Directory:
277
2781. *Set up the Build Environment:* Source the OpenEmbedded build
279 environment setup script (i.e.
280 ````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
281 ```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
282 located in the :term:`Source Directory`.
283
2842. *Check your Local Configuration File:* At this point, you should be
285 sure that the :term:`MACHINE`
286 variable in the ``local.conf`` file found in the ``conf`` directory
287 of the Build Directory is set for the target architecture. Comments
288 within the ``local.conf`` file list the values you can use for the
289 ``MACHINE`` variable. If you do not change the ``MACHINE`` variable,
290 the OpenEmbedded build system uses ``qemux86`` as the default target
291 machine when building the cross-toolchain.
292
293 .. note::
294
295 You can populate the Build Directory with the cross-toolchains for
296 more than a single architecture. You just need to edit the
297 MACHINE
298 variable in the
299 local.conf
300 file and re-run the
301 bitbake
302 command.
303
3043. *Make Sure Your Layers are Enabled:* Examine the
305 ``conf/bblayers.conf`` file and make sure that you have enabled all
306 the compatible layers for your target machine. The OpenEmbedded build
307 system needs to be aware of each layer you want included when
308 building images and cross-toolchains. For information on how to
309 enable a layer, see the "`Enabling Your
310 Layer <&YOCTO_DOCS_DEV_URL;#enabling-your-layer>`__" section in the
311 Yocto Project Development Manual.
312
3134. *Generate the Cross-Toolchain:* Run ``bitbake meta-ide-support`` to
314 complete the cross-toolchain generation. Once the ``bitbake`` command
315 finishes, the cross-toolchain is generated and populated within the
316 Build Directory. You will notice environment setup files for the
317 cross-toolchain that contain the string "``environment-setup``" in
318 the Build Directory's ``tmp`` folder.
319
320 Be aware that when you use this method to install the toolchain, you
321 still need to separately extract and install the sysroot filesystem.
322 For information on how to do this, see the "`Extracting the Root
323 Filesystem <#extracting-the-root-filesystem>`__" section.
324
325Setting Up the Cross-Development Environment
326============================================
327
328Before you can develop using the cross-toolchain, you need to set up the
329cross-development environment by sourcing the toolchain's environment
330setup script. If you used the ADT Installer or hand-installed
331cross-toolchain, then you can find this script in the directory you
332chose for installation. For this release, the default installation
333directory is ````. If you installed the toolchain in the
334:term:`Build Directory`, you can find the
335environment setup script for the toolchain in the Build Directory's
336``tmp`` directory.
337
338Be sure to run the environment setup script that matches the
339architecture for which you are developing. Environment setup scripts
340begin with the string "``environment-setup``" and include as part of
341their name the architecture. For example, the toolchain environment
342setup script for a 64-bit IA-based architecture installed in the default
343installation directory would be the following:
344YOCTO_ADTPATH_DIR/environment-setup-x86_64-poky-linux When you run the
345setup script, many environment variables are defined:
346:term:`SDKTARGETSYSROOT` -
347The path to the sysroot used for cross-compilation
348:term:`PKG_CONFIG_PATH` - The
349path to the target pkg-config files
350:term:`CONFIG_SITE` - A GNU
351autoconf site file preconfigured for the target
352:term:`CC` - The minimal command and
353arguments to run the C compiler
354:term:`CXX` - The minimal command and
355arguments to run the C++ compiler
356:term:`CPP` - The minimal command and
357arguments to run the C preprocessor
358:term:`AS` - The minimal command and
359arguments to run the assembler :term:`LD`
360- The minimal command and arguments to run the linker
361:term:`GDB` - The minimal command and
362arguments to run the GNU Debugger
363:term:`STRIP` - The minimal command and
364arguments to run 'strip', which strips symbols
365:term:`RANLIB` - The minimal command
366and arguments to run 'ranlib'
367:term:`OBJCOPY` - The minimal command
368and arguments to run 'objcopy'
369:term:`OBJDUMP` - The minimal command
370and arguments to run 'objdump' :term:`AR`
371- The minimal command and arguments to run 'ar'
372:term:`NM` - The minimal command and
373arguments to run 'nm'
374:term:`TARGET_PREFIX` - The
375toolchain binary prefix for the target tools
376:term:`CROSS_COMPILE` - The
377toolchain binary prefix for the target tools
378:term:`CONFIGURE_FLAGS` - The
379minimal arguments for GNU configure
380:term:`CFLAGS` - Suggested C flags
381:term:`CXXFLAGS` - Suggested C++
382flags :term:`LDFLAGS` - Suggested
383linker flags when you use CC to link
384:term:`CPPFLAGS` - Suggested
385preprocessor flags
386
387Securing Kernel and Filesystem Images
388=====================================
389
390You will need to have a kernel and filesystem image to boot using your
391hardware or the QEMU emulator. Furthermore, if you plan on booting your
392image using NFS or you want to use the root filesystem as the target
393sysroot, you need to extract the root filesystem.
394
395Getting the Images
396------------------
397
398To get the kernel and filesystem images, you either have to build them
399or download pre-built versions. For an example of how to build these
400images, see the "`Buiding
401Images <&YOCTO_DOCS_QS_URL;#qs-buiding-images>`__" section of the Yocto
402Project Quick Start. For an example of downloading pre-build versions,
403see the "`Example Using Pre-Built Binaries and
404QEMU <#using-pre-built>`__" section.
405
406The Yocto Project ships basic kernel and filesystem images for several
407architectures (``x86``, ``x86-64``, ``mips``, ``powerpc``, and ``arm``)
408that you can use unaltered in the QEMU emulator. These kernel images
409reside in the release area - ` <&YOCTO_MACHINES_DL_URL;>`__ and are
410ideal for experimentation using Yocto Project. For information on the
411image types you can build using the OpenEmbedded build system, see the
412":ref:`ref-manual/ref-images:Images`" chapter in the Yocto
413Project Reference Manual.
414
415If you are planning on developing against your image and you are not
416building or using one of the Yocto Project development images (e.g.
417``core-image-*-dev``), you must be sure to include the development
418packages as part of your image recipe.
419
420If you plan on remotely deploying and debugging your application from
421within the Eclipse IDE, you must have an image that contains the Yocto
422Target Communication Framework (TCF) agent (``tcf-agent``). You can do
423this by including the ``eclipse-debug`` image feature.
424
425.. note::
426
427 See the "
428 Image Features
429 " section in the Yocto Project Reference Manual for information on
430 image features.
431
432To include the ``eclipse-debug`` image feature, modify your
433``local.conf`` file in the :term:`Build Directory`
434so that the
435:term:`EXTRA_IMAGE_FEATURES`
436variable includes the "eclipse-debug" feature. After modifying the
437configuration file, you can rebuild the image. Once the image is
438rebuilt, the ``tcf-agent`` will be included in the image and is launched
439automatically after the boot.
440
441Extracting the Root Filesystem
442------------------------------
443
444If you install your toolchain by hand or build it using BitBake and you
445need a root filesystem, you need to extract it separately. If you use
446the ADT Installer to install the ADT, the root filesystem is
447automatically extracted and installed.
448
449Here are some cases where you need to extract the root filesystem:
450
451- You want to boot the image using NFS.
452
453- You want to use the root filesystem as the target sysroot. For
454 example, the Eclipse IDE environment with the Eclipse Yocto Plug-in
455 installed allows you to use QEMU to boot under NFS.
456
457- You want to develop your target application using the root filesystem
458 as the target sysroot.
459
460To extract the root filesystem, first ``source`` the cross-development
461environment setup script to establish necessary environment variables.
462If you built the toolchain in the Build Directory, you will find the
463toolchain environment script in the ``tmp`` directory. If you installed
464the toolchain by hand, the environment setup script is located in
465``/opt/poky/DISTRO``.
466
467After sourcing the environment script, use the ``runqemu-extract-sdk``
468command and provide the filesystem image.
469
470Following is an example. The second command sets up the environment. In
471this case, the setup script is located in the ``/opt/poky/DISTRO``
472directory. The third command extracts the root filesystem from a
473previously built filesystem that is located in the ``~/Downloads``
474directory. Furthermore, this command extracts the root filesystem into
475the ``qemux86-sato`` directory: $ cd ~ $ source
476/opt/poky/DISTRO/environment-setup-i586-poky-linux $ runqemu-extract-sdk
477\\ ~/Downloads/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2
478\\ $HOME/qemux86-sato You could now point to the target sysroot at
479``qemux86-sato``.
480
481Optionally Building a Toolchain Installer
482=========================================
483
484As an alternative to locating and downloading a toolchain installer, you
485can build the toolchain installer if you have a :term:`Build Directory`.
486
487.. note::
488
489 Although not the preferred method, it is also possible to use
490 bitbake meta-toolchain
491 to build the toolchain installer. If you do use this method, you must
492 separately install and extract the target sysroot. For information on
493 how to install the sysroot, see the "
494 Extracting the Root Filesystem
495 " section.
496
497To build the toolchain installer and populate the SDK image, use the
498following command: $ bitbake image -c populate_sdk The command results
499in a toolchain installer that contains the sysroot that matches your
500target root filesystem.
501
502Another powerful feature is that the toolchain is completely
503self-contained. The binaries are linked against their own copy of
504``libc``, which results in no dependencies on the target system. To
505achieve this, the pointer to the dynamic loader is configured at install
506time since that path cannot be dynamically altered. This is the reason
507for a wrapper around the ``populate_sdk`` archive.
508
509Another feature is that only one set of cross-canadian toolchain
510binaries are produced per architecture. This feature takes advantage of
511the fact that the target hardware can be passed to ``gcc`` as a set of
512compiler options. Those options are set up by the environment script and
513contained in variables such as :term:`CC`
514and :term:`LD`. This reduces the space
515needed for the tools. Understand, however, that a sysroot is still
516needed for every target since those binaries are target-specific.
517
518Remember, before using any BitBake command, you must source the build
519environment setup script (i.e.
520````` <&YOCTO_DOCS_REF_URL;#structure-core-script>`__ or
521```oe-init-build-env-memres`` <&YOCTO_DOCS_REF_URL;#structure-memres-core-script>`__)
522located in the Source Directory and you must make sure your
523``conf/local.conf`` variables are correct. In particular, you need to be
524sure the :term:`MACHINE` variable
525matches the architecture for which you are building and that the
526:term:`SDKMACHINE` variable is
527correctly set if you are building a toolchain designed to run on an
528architecture that differs from your current development host machine
529(i.e. the build machine).
530
531When the ``bitbake`` command completes, the toolchain installer will be
532in ``tmp/deploy/sdk`` in the Build Directory.
533
534.. note::
535
536 By default, this toolchain does not build static binaries. If you
537 want to use the toolchain to build these types of libraries, you need
538 to be sure your image has the appropriate static development
539 libraries. Use the
540 IMAGE_INSTALL
541 variable inside your
542 local.conf
543 file to install the appropriate library packages. Following is an
544 example using
545 glibc
546 static development libraries:
547 ::
548
549 IMAGE_INSTALL_append = " glibc-staticdev"
550
551
552Optionally Using an External Toolchain
553======================================
554
555You might want to use an external toolchain as part of your development.
556If this is the case, the fundamental steps you need to accomplish are as
557follows:
558
559- Understand where the installed toolchain resides. For cases where you
560 need to build the external toolchain, you would need to take separate
561 steps to build and install the toolchain.
562
563- Make sure you add the layer that contains the toolchain to your
564 ``bblayers.conf`` file through the
565 :term:`BBLAYERS` variable.
566
567- Set the
568 :term:`EXTERNAL_TOOLCHAIN`
569 variable in your ``local.conf`` file to the location in which you
570 installed the toolchain.
571
572A good example of an external toolchain used with the Yocto Project is
573Mentor Graphics Sourcery G++ Toolchain. You can see information on how
574to use that particular layer in the ``README`` file at
575http://github.com/MentorEmbedded/meta-sourcery/. You can find
576further information by reading about the
577:term:`TCMODE` variable in the Yocto
578Project Reference Manual's variable glossary.
579
580.. _using-pre-built:
581
582Example Using Pre-Built Binaries and QEMU
583=========================================
584
585If hardware, libraries and services are stable, you can get started by
586using a pre-built binary of the filesystem image, kernel, and toolchain
587and run it using the QEMU emulator. This scenario is useful for
588developing application software.
589
590|Using a Pre-Built Image|
591
592For this scenario, you need to do several things:
593
594- Install the appropriate stand-alone toolchain tarball.
595
596- Download the pre-built image that will boot with QEMU. You need to be
597 sure to get the QEMU image that matches your target machine's
598 architecture (e.g. x86, ARM, etc.).
599
600- Download the filesystem image for your target machine's architecture.
601
602- Set up the environment to emulate the hardware and then start the
603 QEMU emulator.
604
605Installing the Toolchain
606------------------------
607
608You can download a tarball installer, which includes the pre-built
609toolchain, the ``runqemu`` script, and support files from the
610appropriate directory under ` <&YOCTO_TOOLCHAIN_DL_URL;>`__. Toolchains
611are available for 32-bit and 64-bit x86 development systems from the
612``i686`` and ``x86_64`` directories, respectively. The toolchains the
613Yocto Project provides are based off the ``core-image-sato`` image and
614contain libraries appropriate for developing against that image. Each
615type of development system supports five or more target architectures.
616
617The names of the tarball installer scripts are such that a string
618representing the host system appears first in the filename and then is
619immediately followed by a string representing the target architecture.
620
621::
622
623 poky-glibc-host_system-image_type-arch-toolchain-release_version.sh
624
625 Where:
626 host_system is a string representing your development system:
627
628 i686 or x86_64.
629
630 image_type is a string representing the image you wish to
631 develop a Software Development Toolkit (SDK) for use against.
632 The Yocto Project builds toolchain installers using the
633 following BitBake command:
634
635 bitbake core-image-sato -c populate_sdk
636
637 arch is a string representing the tuned target architecture:
638
639 i586, x86_64, powerpc, mips, armv7a or armv5te
640
641 release_version is a string representing the release number of the
642 Yocto Project:
643
644 DISTRO, DISTRO+snapshot
645
646
647For example, the following toolchain installer is for a 64-bit
648development host system and a i586-tuned target architecture based off
649the SDK for ``core-image-sato``:
650poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
651
652Toolchains are self-contained and by default are installed into
653``/opt/poky``. However, when you run the toolchain installer, you can
654choose an installation directory.
655
656The following command shows how to run the installer given a toolchain
657tarball for a 64-bit x86 development host system and a 32-bit x86 target
658architecture. You must change the permissions on the toolchain installer
659script so that it is executable.
660
661The example assumes the toolchain installer is located in
662``~/Downloads/``.
663
664.. note::
665
666 If you do not have write permissions for the directory into which you
667 are installing the toolchain, the toolchain installer notifies you
668 and exits. Be sure you have write permissions in the directory and
669 run the installer again.
670
671$ ~/Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
672
673For more information on how to install tarballs, see the "`Using a
674Cross-Toolchain
675Tarball <&YOCTO_DOCS_ADT_URL;#using-an-existing-toolchain-tarball>`__"
676and "`Using BitBake and the Build
677Directory <&YOCTO_DOCS_ADT_URL;#using-the-toolchain-from-within-the-build-tree>`__"
678sections in the Yocto Project Application Developer's Guide.
679
680Downloading the Pre-Built Linux Kernel
681--------------------------------------
682
683You can download the pre-built Linux kernel suitable for running in the
684QEMU emulator from ` <&YOCTO_QEMU_DL_URL;>`__. Be sure to use the kernel
685that matches the architecture you want to simulate. Download areas exist
686for the five supported machine architectures: ``qemuarm``, ``qemumips``,
687``qemuppc``, ``qemux86``, and ``qemux86-64``.
688
689Most kernel files have one of the following forms: \*zImage-qemuarch.bin
690vmlinux-qemuarch.bin Where: arch is a string representing the target
691architecture: x86, x86-64, ppc, mips, or arm.
692
693You can learn more about downloading a Yocto Project kernel in the
694"`Yocto Project Kernel <&YOCTO_DOCS_DEV_URL;#local-kernel-files>`__"
695bulleted item in the Yocto Project Development Manual.
696
697Downloading the Filesystem
698--------------------------
699
700You can also download the filesystem image suitable for your target
701architecture from ` <&YOCTO_QEMU_DL_URL;>`__. Again, be sure to use the
702filesystem that matches the architecture you want to simulate.
703
704The filesystem image has two tarball forms: ``ext3`` and ``tar``. You
705must use the ``ext3`` form when booting an image using the QEMU
706emulator. The ``tar`` form can be flattened out in your host development
707system and used for build purposes with the Yocto Project.
708core-image-profile-qemuarch.ext3 core-image-profile-qemuarch.tar.bz2
709Where: profile is the filesystem image's profile: lsb, lsb-dev, lsb-sdk,
710lsb-qt3, minimal, minimal-dev, sato, sato-dev, or sato-sdk. For
711information on these types of image profiles, see the
712":ref:`ref-manual/ref-images:Images`" chapter in the Yocto
713Project Reference Manual. arch is a string representing the target
714architecture: x86, x86-64, ppc, mips, or arm.
715
716Setting Up the Environment and Starting the QEMU Emulator
717---------------------------------------------------------
718
719Before you start the QEMU emulator, you need to set up the emulation
720environment. The following command form sets up the emulation
721environment. $ source
722YOCTO_ADTPATH_DIR/environment-setup-arch-poky-linux-if Where: arch is a
723string representing the target architecture: i586, x86_64, ppc603e,
724mips, or armv5te. if is a string representing an embedded application
725binary interface. Not all setup scripts include this string.
726
727Finally, this command form invokes the QEMU emulator $ runqemu qemuarch
728kernel-image filesystem-image Where: qemuarch is a string representing
729the target architecture: qemux86, qemux86-64, qemuppc, qemumips, or
730qemuarm. kernel-image is the architecture-specific kernel image.
731filesystem-image is the .ext3 filesystem image.
732
733Continuing with the example, the following two commands setup the
734emulation environment and launch QEMU. This example assumes the root
735filesystem (``.ext3`` file) and the pre-built kernel image file both
736reside in your home directory. The kernel and filesystem are for a
73732-bit target architecture. $ cd $HOME $ source
738YOCTO_ADTPATH_DIR/environment-setup-i586-poky-linux $ runqemu qemux86
739bzImage-qemux86.bin \\ core-image-sato-qemux86.ext3
740
741The environment in which QEMU launches varies depending on the
742filesystem image and on the target architecture. For example, if you
743source the environment for the ARM target architecture and then boot the
744minimal QEMU image, the emulator comes up in a new shell in command-line
745mode. However, if you boot the SDK image, QEMU comes up with a GUI.
746
747.. note::
748
749 Booting the PPC image results in QEMU launching in the same shell in
750 command-line mode.
751
752.. |Using a Pre-Built Image| image:: figures/using-a-pre-built-image.png
diff --git a/documentation/adt-manual/adt-prepare.xml b/documentation/adt-manual/adt-prepare.xml
index 65df1d03e6..2dc9843259 100644
--- a/documentation/adt-manual/adt-prepare.xml
+++ b/documentation/adt-manual/adt-prepare.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='adt-prepare'> 6<chapter id='adt-prepare'>
6 7
@@ -231,7 +232,7 @@
231 own location, you are prompted to run the installation script 232 own location, you are prompted to run the installation script
232 interactively or in silent mode. 233 interactively or in silent mode.
233 If you want to closely monitor the installation, 234 If you want to closely monitor the installation,
234 choose I for interactive mode rather than S for silent mode. 235 choose "I" for interactive mode rather than "S" for silent mode.
235 Follow the prompts from the script to complete the installation. 236 Follow the prompts from the script to complete the installation.
236 </para> 237 </para>
237 238
@@ -764,7 +765,7 @@
764 <itemizedlist> 765 <itemizedlist>
765 <listitem><para>Install the appropriate stand-alone toolchain tarball.</para></listitem> 766 <listitem><para>Install the appropriate stand-alone toolchain tarball.</para></listitem>
766 <listitem><para>Download the pre-built image that will boot with QEMU. 767 <listitem><para>Download the pre-built image that will boot with QEMU.
767 You need to be sure to get the QEMU image that matches your target machines 768 You need to be sure to get the QEMU image that matches your target machine's
768 architecture (e.g. x86, ARM, etc.).</para></listitem> 769 architecture (e.g. x86, ARM, etc.).</para></listitem>
769 <listitem><para>Download the filesystem image for your target machine's architecture. 770 <listitem><para>Download the filesystem image for your target machine's architecture.
770 </para></listitem> 771 </para></listitem>
diff --git a/documentation/adt-manual/adt-style.css b/documentation/adt-manual/adt-style.css
index d722ad4b7f..9d6221ae51 100644
--- a/documentation/adt-manual/adt-style.css
+++ b/documentation/adt-manual/adt-style.css
@@ -1,4 +1,6 @@
1/* 1/*
2 SPDX-License-Identifier: CC-BY-2.0-UK
3
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 4 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 5
4 Browser wrangling and typographic design by 6 Browser wrangling and typographic design by
diff --git a/documentation/boilerplate.rst b/documentation/boilerplate.rst
new file mode 100644
index 0000000000..ddffdac242
--- /dev/null
+++ b/documentation/boilerplate.rst
@@ -0,0 +1,18 @@
1.. include:: <xhtml1-lat1.txt>
2.. include:: <xhtml1-symbol.txt>
3
4----
5
6| |project_name|
7| <docs@lists.yoctoproject.org>
8
9Permission is granted to copy, distribute and/or modify this document under the
10terms of the `Creative Commons Attribution-Share Alike 2.0 UK: England & Wales
11<http://creativecommons.org/licenses/by-sa/2.0/uk/>`_ as published by Creative
12Commons.
13
14To report any inaccuracies or problems with this (or any other Yocto Project)
15manual, or to send additions or changes, please send email/patches to the Yocto
16Project documentation mailing list at ``docs@lists.yoctoproject.org`` or
17log into the freenode ``#yocto`` channel.
18
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-customization.xsl b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-customization.xsl
index 0d57424b59..012d86384a 100644
--- a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-customization.xsl
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-style.css b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-style.css
index 386841debe..e4e79d90ef 100644
--- a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-style.css
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-titlepage.xsl b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-titlepage.xsl
index a435ac77ab..e74ac530dd 100644
--- a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-titlepage.xsl
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs-titlepage.xsl
@@ -1,4 +1,5 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
2 3
3<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exsl="http://exslt.org/common" version="1.0" exclude-result-prefixes="exsl"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:exsl="http://exslt.org/common" version="1.0" exclude-result-prefixes="exsl">
4 5
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst
new file mode 100644
index 0000000000..7e24b9e685
--- /dev/null
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.rst
@@ -0,0 +1,430 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=========================
4Yocto Project Quick Build
5=========================
6
7Welcome!
8========
9
10This short document steps you through the process for a typical
11image build using the Yocto Project. The document also introduces how to
12configure a build for specific hardware. You will use Yocto Project to
13build a reference embedded OS called Poky.
14
15.. note::
16
17 - The examples in this paper assume you are using a native Linux
18 system running a recent Ubuntu Linux distribution. If the machine
19 you want to use Yocto Project on to build an image
20 (:term:`Build Host`) is not
21 a native Linux system, you can still perform these steps by using
22 CROss PlatformS (CROPS) and setting up a Poky container. See the
23 :ref:`dev-manual/dev-manual-start:setting up to use cross platforms (crops)`
24 section
25 in the Yocto Project Development Tasks Manual for more
26 information.
27
28 - You may use Windows Subsystem For Linux v2 to set up a build host
29 using Windows 10.
30
31 .. note::
32
33 The Yocto Project is not compatible with WSLv1, it is
34 compatible but not officially supported nor validated with
35 WSLv2, if you still decide to use WSL please upgrade to WSLv2.
36
37 See the :ref:`dev-manual/dev-manual-start:setting up to use windows
38 subsystem for linux (wslv2)` section in the Yocto Project Development
39 Tasks Manual for more information.
40
41If you want more conceptual or background information on the Yocto
42Project, see the :doc:`../overview-manual/overview-manual`.
43
44Compatible Linux Distribution
45=============================
46
47Make sure your :term:`Build Host` meets the
48following requirements:
49
50- 50 Gbytes of free disk space
51
52- Runs a supported Linux distribution (i.e. recent releases of Fedora,
53 openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux
54 distributions that support the Yocto Project, see the
55 :ref:`ref-manual/ref-system-requirements:supported linux distributions`
56 section in the Yocto Project Reference Manual. For detailed
57 information on preparing your build host, see the
58 :ref:`dev-manual/dev-manual-start:preparing the build host`
59 section in the Yocto Project Development Tasks Manual.
60
61-
62
63 - Git 1.8.3.1 or greater
64 - tar 1.28 or greater
65 - Python 3.5.0 or greater.
66 - gcc 5.0 or greater.
67
68If your build host does not meet any of these three listed version
69requirements, you can take steps to prepare the system so that you
70can still use the Yocto Project. See the
71:ref:`ref-manual/ref-system-requirements:required git, tar, python and gcc versions`
72section in the Yocto Project Reference Manual for information.
73
74Build Host Packages
75===================
76
77You must install essential host packages on your build host. The
78following command installs the host packages based on an Ubuntu
79distribution:
80
81.. code-block:: shell
82
83 $ sudo apt-get install &UBUNTU_HOST_PACKAGES_ESSENTIAL;
84
85.. note::
86
87 For host package requirements on all supported Linux distributions,
88 see the :ref:`ref-manual/ref-system-requirements:required packages for the build host`
89 section in the Yocto Project Reference Manual.
90
91Use Git to Clone Poky
92=====================
93
94Once you complete the setup instructions for your machine, you need to
95get a copy of the Poky repository on your build host. Use the following
96commands to clone the Poky repository.
97
98.. code-block:: shell
99
100 $ git clone git://git.yoctoproject.org/poky
101 Cloning into 'poky'...
102 remote: Counting
103 objects: 432160, done. remote: Compressing objects: 100%
104 (102056/102056), done. remote: Total 432160 (delta 323116), reused
105 432037 (delta 323000) Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
106 Resolving deltas: 100% (323116/323116), done.
107 Checking connectivity... done.
108
109Move to the ``poky`` directory and take a look at the tags:
110
111.. code-block:: shell
112
113 $ cd poky
114 $ git fetch --tags
115 $ git tag
116 1.1_M1.final
117 1.1_M1.rc1
118 1.1_M1.rc2
119 1.1_M2.final
120 1.1_M2.rc1
121 .
122 .
123 .
124 yocto-2.5
125 yocto-2.5.1
126 yocto-2.5.2
127 yocto-2.6
128 yocto-2.6.1
129 yocto-2.6.2
130 yocto-2.7
131 yocto_1.5_M5.rc8
132
133For this example, check out the branch based on the
134``&DISTRO_REL_TAG;`` release:
135
136.. code-block:: shell
137
138 $ git checkout tags/&DISTRO_REL_TAG; -b my-&DISTRO_REL_TAG;
139 Switched to a new branch 'my-&DISTRO_REL_TAG;'
140
141The previous Git checkout command creates a local branch named
142``my-&DISTRO_REL_TAG;``. The files available to you in that branch exactly
143match the repository's files in the ``&DISTRO_NAME_NO_CAP;`` development
144branch at the time of the Yocto Project &DISTRO_REL_TAG; release.
145
146For more options and information about accessing Yocto Project related
147repositories, see the
148:ref:`dev-manual/dev-manual-start:locating yocto project source files`
149section in the Yocto Project Development Tasks Manual.
150
151Building Your Image
152===================
153
154Use the following steps to build your image. The build process creates
155an entire Linux distribution, including the toolchain, from source.
156
157.. note::
158
159 - If you are working behind a firewall and your build host is not
160 set up for proxies, you could encounter problems with the build
161 process when fetching source code (e.g. fetcher failures or Git
162 failures).
163
164 - If you do not know your proxy settings, consult your local network
165 infrastructure resources and get that information. A good starting
166 point could also be to check your web browser settings. Finally,
167 you can find more information on the
168 ":yocto_wiki:`Working Behind a Network Proxy </wiki/Working_Behind_a_Network_Proxy>`"
169 page of the Yocto Project Wiki.
170
171#. **Initialize the Build Environment:** From within the ``poky``
172 directory, run the :ref:`ref-manual/ref-structure:\`\`oe-init-build-env\`\``
173 environment
174 setup script to define Yocto Project's build environment on your
175 build host.
176
177 .. code-block:: shell
178
179 $ cd ~/poky
180 $ source &OE_INIT_FILE;
181 You had no conf/local.conf file. This configuration file has therefore been
182 created for you with some default values. You may wish to edit it to, for
183 example, select a different MACHINE (target hardware). See conf/local.conf
184 for more information as common configuration options are commented.
185
186 You had no conf/bblayers.conf file. This configuration file has therefore
187 been created for you with some default values. To add additional metadata
188 layers into your configuration please add entries to conf/bblayers.conf.
189
190 The Yocto Project has extensive documentation about OE including a reference
191 manual which can be found at:
192 http://yoctoproject.org/documentation
193
194 For more information about OpenEmbedded see their website:
195 http://www.openembedded.org/
196
197 ### Shell environment set up for builds. ###
198
199 You can now run 'bitbake <target>'
200
201 Common targets are:
202 core-image-minimal
203 core-image-sato
204 meta-toolchain
205 meta-ide-support
206
207 You can also run generated qemu images with a command like 'runqemu qemux86-64'
208
209 Among other things, the script creates the :term:`Build Directory`, which is
210 ``build`` in this case and is located in the :term:`Source Directory`. After
211 the script runs, your current working directory is set to the Build
212 Directory. Later, when the build completes, the Build Directory contains all the
213 files created during the build.
214
215#. **Examine Your Local Configuration File:** When you set up the build
216 environment, a local configuration file named ``local.conf`` becomes
217 available in a ``conf`` subdirectory of the Build Directory. For this
218 example, the defaults are set to build for a ``qemux86`` target,
219 which is suitable for emulation. The package manager used is set to
220 the RPM package manager.
221
222 .. tip::
223
224 You can significantly speed up your build and guard against fetcher
225 failures by using mirrors. To use mirrors, add these lines to your
226 local.conf file in the Build directory: ::
227
228 SSTATE_MIRRORS = "\
229 file://.* http://sstate.yoctoproject.org/dev/PATH;downloadfilename=PATH \n \
230 file://.* http://sstate.yoctoproject.org/&YOCTO_DOC_VERSION_MINUS_ONE;/PATH;downloadfilename=PATH \n \
231 file://.* http://sstate.yoctoproject.org/&YOCTO_DOC_VERSION;/PATH;downloadfilename=PATH \n \
232 "
233
234
235 The previous examples showed how to add sstate paths for Yocto Project
236 &YOCTO_DOC_VERSION_MINUS_ONE;, &YOCTO_DOC_VERSION;, and a development
237 area. For a complete index of sstate locations, see http://sstate.yoctoproject.org/.
238
239#. **Start the Build:** Continue with the following command to build an OS
240 image for the target, which is ``core-image-sato`` in this example:
241
242 .. code-block:: shell
243
244 $ bitbake core-image-sato
245
246 For information on using the ``bitbake`` command, see the
247 :ref:`usingpoky-components-bitbake` section in the Yocto Project Overview and
248 Concepts Manual, or see the ":ref:`BitBake Command
249 <bitbake:bitbake-user-manual-command>`" section in the BitBake User Manual.
250
251#. **Simulate Your Image Using QEMU:** Once this particular image is
252 built, you can start QEMU, which is a Quick EMUlator that ships with
253 the Yocto Project:
254
255 .. code-block:: shell
256
257 $ runqemu qemux86-64
258
259 If you want to learn more about running QEMU, see the
260 :ref:`dev-manual/dev-manual-qemu:using the quick emulator (qemu)` chapter in
261 the Yocto Project Development Tasks Manual.
262
263#. **Exit QEMU:** Exit QEMU by either clicking on the shutdown icon or by typing
264 ``Ctrl-C`` in the QEMU transcript window from which you evoked QEMU.
265
266Customizing Your Build for Specific Hardware
267============================================
268
269So far, all you have done is quickly built an image suitable for
270emulation only. This section shows you how to customize your build for
271specific hardware by adding a hardware layer into the Yocto Project
272development environment.
273
274In general, layers are repositories that contain related sets of
275instructions and configurations that tell the Yocto Project what to do.
276Isolating related metadata into functionally specific layers facilitates
277modular development and makes it easier to reuse the layer metadata.
278
279.. note::
280
281 By convention, layer names start with the string "meta-".
282
283Follow these steps to add a hardware layer:
284
285#. **Find a Layer:** Lots of hardware layers exist. The Yocto Project
286 :yocto_git:`Source Repositories <>` has many hardware layers.
287 This example adds the
288 `meta-altera <https://github.com/kraj/meta-altera>`__ hardware layer.
289
290#. **Clone the Layer:** Use Git to make a local copy of the layer on your
291 machine. You can put the copy in the top level of the copy of the
292 Poky repository created earlier:
293
294 .. code-block:: shell
295
296 $ cd ~/poky
297 $ git clone https://github.com/kraj/meta-altera.git
298 Cloning into 'meta-altera'...
299 remote: Counting objects: 25170, done.
300 remote: Compressing objects: 100% (350/350), done.
301 remote: Total 25170 (delta 645), reused 719 (delta 538), pack-reused 24219
302 Receiving objects: 100% (25170/25170), 41.02 MiB | 1.64 MiB/s, done.
303 Resolving deltas: 100% (13385/13385), done.
304 Checking connectivity... done.
305
306 The hardware layer now exists
307 with other layers inside the Poky reference repository on your build
308 host as ``meta-altera`` and contains all the metadata needed to
309 support hardware from Altera, which is owned by Intel.
310
311 .. note::
312
313 It is recommended for layers to have a branch per Yocto Project release.
314 Please make sure to checkout the layer branch supporting the Yocto Project
315 release you're using.
316
317#. **Change the Configuration to Build for a Specific Machine:** The
318 :term:`MACHINE` variable in the
319 ``local.conf`` file specifies the machine for the build. For this
320 example, set the ``MACHINE`` variable to ``cyclone5``. These
321 configurations are used:
322 https://github.com/kraj/meta-altera/blob/master/conf/machine/cyclone5.conf.
323
324 .. note::
325
326 See the "Examine Your Local Configuration File" step earlier for more
327 information on configuring the build.
328
329#. **Add Your Layer to the Layer Configuration File:** Before you can use
330 a layer during a build, you must add it to your ``bblayers.conf``
331 file, which is found in the
332 :term:`Build Directory` ``conf``
333 directory.
334
335 Use the ``bitbake-layers add-layer`` command to add the layer to the
336 configuration file:
337
338 .. code-block:: shell
339
340 $ cd ~/poky/build
341 $ bitbake-layers add-layer ../meta-altera
342 NOTE: Starting bitbake server...
343 Parsing recipes: 100% |##################################################################| Time: 0:00:32
344 Parsing of 918 .bb files complete (0 cached, 918 parsed). 1401 targets,
345 123 skipped, 0 masked, 0 errors.
346
347 You can find
348 more information on adding layers in the
349 :ref:`dev-manual/dev-manual-common-tasks:adding a layer using the \`\`bitbake-layers\`\` script`
350 section.
351
352Completing these steps has added the ``meta-altera`` layer to your Yocto
353Project development environment and configured it to build for the
354``cyclone5`` machine.
355
356.. note::
357
358 The previous steps are for demonstration purposes only. If you were
359 to attempt to build an image for the ``cyclone5`` machine, you should
360 read the Altera ``README``.
361
362Creating Your Own General Layer
363===============================
364
365Maybe you have an application or specific set of behaviors you need to
366isolate. You can create your own general layer using the
367``bitbake-layers create-layer`` command. The tool automates layer
368creation by setting up a subdirectory with a ``layer.conf``
369configuration file, a ``recipes-example`` subdirectory that contains an
370``example.bb`` recipe, a licensing file, and a ``README``.
371
372The following commands run the tool to create a layer named
373``meta-mylayer`` in the ``poky`` directory:
374
375.. code-block:: shell
376
377 $ cd ~/poky
378 $ bitbake-layers create-layer meta-mylayer
379 NOTE: Starting bitbake server...
380 Add your new layer with 'bitbake-layers add-layer meta-mylayer'
381
382For more information
383on layers and how to create them, see the
384:ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`
385section in the Yocto Project Development Tasks Manual.
386
387Where To Go Next
388================
389
390Now that you have experienced using the Yocto Project, you might be
391asking yourself "What now?". The Yocto Project has many sources of
392information including the website, wiki pages, and user manuals:
393
394- **Website:** The :yocto_home:`Yocto Project Website <>` provides
395 background information, the latest builds, breaking news, full
396 development documentation, and access to a rich Yocto Project
397 Development Community into which you can tap.
398
399- **Developer Screencast:** The `Getting Started with the Yocto Project -
400 New Developer Screencast Tutorial <http://vimeo.com/36450321>`__
401 provides a 30-minute video created for users unfamiliar with the
402 Yocto Project but familiar with Linux build hosts. While this
403 screencast is somewhat dated, the introductory and fundamental
404 concepts are useful for the beginner.
405
406- **Yocto Project Overview and Concepts Manual:** The
407 :doc:`../overview-manual/overview-manual` is a great
408 place to start to learn about the Yocto Project. This manual
409 introduces you to the Yocto Project and its development environment.
410 The manual also provides conceptual information for various aspects
411 of the Yocto Project.
412
413- **Yocto Project Wiki:** The :yocto_wiki:`Yocto Project Wiki <>`
414 provides additional information on where to go next when ramping up
415 with the Yocto Project, release information, project planning, and QA
416 information.
417
418- **Yocto Project Mailing Lists:** Related mailing lists provide a forum
419 for discussion, patch submission and announcements. Several mailing
420 lists exist and are grouped according to areas of concern. See the
421 :ref:`ref-manual/resources:mailing lists`
422 section in the Yocto Project Reference Manual for a complete list of
423 Yocto Project mailing lists.
424
425- **Comprehensive List of Links and Other Documentation:** The
426 :ref:`ref-manual/resources:links and related documentation`
427 section in the Yocto Project Reference Manual provides a
428 comprehensive list of all related links and other user documentation.
429
430.. include:: /boilerplate.rst
diff --git a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.xml b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.xml
index 3c83afd46b..198c7b9689 100644
--- a/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.xml
+++ b/documentation/brief-yoctoprojectqs/brief-yoctoprojectqs.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<article id='brief-yocto-project-qs-intro'> 6<article id='brief-yocto-project-qs-intro'>
6 <articleinfo> 7 <articleinfo>
diff --git a/documentation/bsp-guide/bsp-guide-customization.xsl b/documentation/bsp-guide/bsp-guide-customization.xsl
index de674a0aec..37fcbcd208 100644
--- a/documentation/bsp-guide/bsp-guide-customization.xsl
+++ b/documentation/bsp-guide/bsp-guide-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/bsp-guide/bsp-guide.rst b/documentation/bsp-guide/bsp-guide.rst
new file mode 100644
index 0000000000..435a399d52
--- /dev/null
+++ b/documentation/bsp-guide/bsp-guide.rst
@@ -0,0 +1,16 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=====================================================
4Yocto Project Board Support Package Developer's Guide
5=====================================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 bsp
14 history
15
16.. include:: /boilerplate.rst
diff --git a/documentation/bsp-guide/bsp-guide.xml b/documentation/bsp-guide/bsp-guide.xml
index eec048e021..93ba1e7fec 100755
--- a/documentation/bsp-guide/bsp-guide.xml
+++ b/documentation/bsp-guide/bsp-guide.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='bsp-guide' lang='en' 6<book id='bsp-guide' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -127,28 +128,8 @@
127 </revision> 128 </revision>
128 <revision> 129 <revision>
129 <revnumber>3.1</revnumber> 130 <revnumber>3.1</revnumber>
130 <date>April 2020</date>
131 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
132 </revision>
133 <revision>
134 <revnumber>3.1.1</revnumber>
135 <date>June 2020</date>
136 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
137 </revision>
138 <revision>
139 <revnumber>3.1.2</revnumber>
140 <date>August 2020</date>
141 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
142 </revision>
143 <revision>
144 <revnumber>3.1.3</revnumber>
145 <date>October 2020</date>
146 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
147 </revision>
148 <revision>
149 <revnumber>3.1.4</revnumber>
150 <date>&REL_MONTH_YEAR;</date> 131 <date>&REL_MONTH_YEAR;</date>
151 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 132 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
152 </revision> 133 </revision>
153 </revhistory> 134 </revhistory>
154 135
diff --git a/documentation/bsp-guide/bsp-style.css b/documentation/bsp-guide/bsp-style.css
index 0c8689b96f..4ccf5d8aef 100644
--- a/documentation/bsp-guide/bsp-style.css
+++ b/documentation/bsp-guide/bsp-style.css
@@ -1,4 +1,6 @@
1/* 1/*
2 SPDX-License-Identifier: CC-BY-2.0-UK
3
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 4 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 5
4 Browser wrangling and typographic design by 6 Browser wrangling and typographic design by
diff --git a/documentation/bsp-guide/bsp.rst b/documentation/bsp-guide/bsp.rst
new file mode 100644
index 0000000000..024a240c22
--- /dev/null
+++ b/documentation/bsp-guide/bsp.rst
@@ -0,0 +1,1527 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************************************************
4Board Support Packages (BSP) - Developer's Guide
5************************************************
6
7A Board Support Package (BSP) is a collection of information that
8defines how to support a particular hardware device, set of devices, or
9hardware platform. The BSP includes information about the hardware
10features present on the device and kernel configuration information
11along with any additional hardware drivers required. The BSP also lists
12any additional software components required in addition to a generic
13Linux software stack for both essential and optional platform features.
14
15This guide presents information about BSP layers, defines a structure
16for components so that BSPs follow a commonly understood layout,
17discusses how to customize a recipe for a BSP, addresses BSP licensing,
18and provides information that shows you how to create a BSP
19Layer using the :ref:`bitbake-layers <bsp-guide/bsp:Creating a new BSP Layer Using the \`\`bitbake-layers\`\` Script>`
20tool.
21
22BSP Layers
23==========
24
25A BSP consists of a file structure inside a base directory.
26Collectively, you can think of the base directory, its file structure,
27and the contents as a BSP layer. Although not a strict requirement, BSP
28layers in the Yocto Project use the following well-established naming
29convention: ::
30
31 meta-bsp_root_name
32
33The string "meta-" is prepended to the
34machine or platform name, which is bsp_root_name in the above form.
35
36.. note::
37
38 Because the BSP layer naming convention is well-established, it is
39 advisable to follow it when creating layers. Technically speaking, a
40 BSP layer name does not need to start with
41 meta-. However, various scripts and tools in the Yocto Project development
42 environment assume this convention.
43
44To help understand the BSP layer concept, consider the BSPs that the
45Yocto Project supports and provides with each release. You can see the
46layers in the
47:ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`
48through
49a web interface at :yocto_git:`/`. If you go to that interface,
50you will find a list of repositories under "Yocto Metadata Layers".
51
52.. note::
53
54 Layers that are no longer actively supported as part of the Yocto
55 Project appear under the heading "Yocto Metadata Layer Archive."
56
57Each repository is a BSP layer supported by the Yocto Project (e.g.
58``meta-raspberrypi`` and ``meta-intel``). Each of these layers is a
59repository unto itself and clicking on the layer name displays two URLs
60from which you can clone the layer's repository to your local system.
61Here is an example that clones the Raspberry Pi BSP layer: ::
62
63 $ git clone git://git.yoctoproject.org/meta-raspberrypi
64
65In addition to BSP layers, the ``meta-yocto-bsp`` layer is part of the
66shipped ``poky`` repository. The ``meta-yocto-bsp`` layer maintains
67several "reference" BSPs including the ARM-based Beaglebone, MIPS-based
68EdgeRouter, and generic versions of both 32-bit and 64-bit IA machines.
69
70For information on typical BSP development workflow, see the
71:ref:`bsp-guide/bsp:developing a board support package (bsp)`
72section. For more
73information on how to set up a local copy of source files from a Git
74repository, see the
75:ref:`dev-manual/dev-manual-start:locating yocto project source files`
76section in the Yocto Project Development Tasks Manual.
77
78The BSP layer's base directory (``meta-bsp_root_name``) is the root
79directory of that Layer. This directory is what you add to the
80:term:`BBLAYERS` variable in the
81``conf/bblayers.conf`` file found in your
82:term:`Build Directory`, which is
83established after you run the OpenEmbedded build environment setup
84script (i.e. :ref:`ref-manual/ref-structure:\`\`oe-init-build-env\`\`` ).
85Adding the root directory allows the :term:`OpenEmbedded Build System`
86to recognize the BSP
87layer and from it build an image. Here is an example: ::
88
89 BBLAYERS ?= " \
90 /usr/local/src/yocto/meta \
91 /usr/local/src/yocto/meta-poky \
92 /usr/local/src/yocto/meta-yocto-bsp \
93 /usr/local/src/yocto/meta-mylayer \
94 "
95
96.. note::
97
98 Ordering and ``BBFILE_PRIORITY`` for the layers listed in BBLAYERS matter. For
99 example, if multiple layers define a machine configuration, the OpenEmbedded
100 build system uses the last layer searched given similar layer priorities. The
101 build system works from the top-down through the layers listed in ``BBLAYERS``.
102
103Some BSPs require or depend on additional layers beyond the BSP's root
104layer in order to be functional. In this case, you need to specify these
105layers in the ``README`` "Dependencies" section of the BSP's root layer.
106Additionally, if any build instructions exist for the BSP, you must add
107them to the "Dependencies" section.
108
109Some layers function as a layer to hold other BSP layers. These layers
110are knows as ":term:`container layers <Container Layer>`". An example of
111this type of layer is OpenEmbedded's
112`meta-openembedded <https://github.com/openembedded/meta-openembedded>`__
113layer. The ``meta-openembedded`` layer contains many ``meta-*`` layers.
114In cases like this, you need to include the names of the actual layers
115you want to work with, such as: ::
116
117 BBLAYERS ?= " \
118 /usr/local/src/yocto/meta \
119 /usr/local/src/yocto/meta-poky \
120 /usr/local/src/yocto/meta-yocto-bsp \
121 /usr/local/src/yocto/meta-mylayer \
122 .../meta-openembedded/meta-oe \
123 .../meta-openembedded/meta-perl \
124 .../meta-openembedded/meta-networking \
125 "
126
127and so on.
128
129For more information on layers, see the
130":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
131section of the Yocto Project Development Tasks Manual.
132
133Preparing Your Build Host to Work With BSP Layers
134=================================================
135
136This section describes how to get your build host ready to work with BSP
137layers. Once you have the host set up, you can create the layer as
138described in the
139":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
140section.
141
142.. note::
143
144 For structural information on BSPs, see the Example Filesystem Layout
145 section.
146
147#. *Set Up the Build Environment:* Be sure you are set up to use BitBake
148 in a shell. See the ":ref:`dev-manual/dev-manual-start:preparing the build host`"
149 section in the Yocto Project Development Tasks Manual for information on how
150 to get a build host ready that is either a native Linux machine or a machine
151 that uses CROPS.
152
153#. *Clone the ``poky`` Repository:* You need to have a local copy of the
154 Yocto Project :term:`Source Directory` (i.e. a local
155 ``poky`` repository). See the
156 "ref:`dev-manual/dev-manual-start:cloning the ``poky`` repository`" and
157 possibly the
158 ":ref:`dev-manual/dev-manual-start:checking out by branch in poky`" or
159 ":ref:`dev-manual/dev-manual-start:checking out by tag in poky`"
160 sections
161 all in the Yocto Project Development Tasks Manual for information on
162 how to clone the ``poky`` repository and check out the appropriate
163 branch for your work.
164
165#. *Determine the BSP Layer You Want:* The Yocto Project supports many
166 BSPs, which are maintained in their own layers or in layers designed
167 to contain several BSPs. To get an idea of machine support through
168 BSP layers, you can look at the `index of
169 machines <&YOCTO_RELEASE_DL_URL;/machines>`__ for the release.
170
171#. *Optionally Clone the ``meta-intel`` BSP Layer:* If your hardware is
172 based on current Intel CPUs and devices, you can leverage this BSP
173 layer. For details on the ``meta-intel`` BSP layer, see the layer's
174 `README <http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/tree/README>`__
175 file.
176
177 #. *Navigate to Your Source Directory:* Typically, you set up the
178 ``meta-intel`` Git repository inside the :term:`Source Directory` (e.g.
179 ``poky``). ::
180
181 $ cd /home/you/poky
182
183 #. *Clone the Layer:* ::
184
185 $ git clone git://git.yoctoproject.org/meta-intel.git
186 Cloning into 'meta-intel'...
187 remote: Counting objects: 15585, done.
188 remote: Compressing objects: 100% (5056/5056), done.
189 remote: Total 15585 (delta 9123), reused 15329 (delta 8867)
190 Receiving objects: 100% (15585/15585), 4.51 MiB | 3.19 MiB/s, done.
191 Resolving deltas: 100% (9123/9123), done.
192 Checking connectivity... done.
193
194 #. *Check Out the Proper Branch:* The branch you check out for
195 ``meta-intel`` must match the same branch you are using for the
196 Yocto Project release (e.g. &DISTRO_NAME_NO_CAP;): ::
197
198 $ cd meta-intel
199 $ git checkout -b &DISTRO_NAME_NO_CAP; remotes/origin/&DISTRO_NAME_NO_CAP;
200 Branch &DISTRO_NAME_NO_CAP; set up to track remote branch
201 &DISTRO_NAME_NO_CAP; from origin.
202 Switched to a new branch '&DISTRO_NAME_NO_CAP;'
203
204 .. note::
205
206 To see the available branch names in a cloned repository, use the ``git
207 branch -al`` command. See the
208 ":ref:`dev-manual/dev-manual-start:checking out by branch in poky`"
209 section in the Yocto Project Development Tasks Manual for more
210 information.
211
212#. *Optionally Set Up an Alternative BSP Layer:* If your hardware can be
213 more closely leveraged to an existing BSP not within the
214 ``meta-intel`` BSP layer, you can clone that BSP layer.
215
216 The process is identical to the process used for the ``meta-intel``
217 layer except for the layer's name. For example, if you determine that
218 your hardware most closely matches the ``meta-raspberrypi``, clone
219 that layer: ::
220
221 $ git clone git://git.yoctoproject.org/meta-raspberrypi
222 Cloning into 'meta-raspberrypi'...
223 remote: Counting objects: 4743, done.
224 remote: Compressing objects: 100% (2185/2185), done.
225 remote: Total 4743 (delta 2447), reused 4496 (delta 2258)
226 Receiving objects: 100% (4743/4743), 1.18 MiB | 0 bytes/s, done.
227 Resolving deltas: 100% (2447/2447), done.
228 Checking connectivity... done.
229
230#. *Initialize the Build Environment:* While in the root directory of
231 the Source Directory (i.e. ``poky``), run the
232 :ref:`ref-manual/ref-structure:\`\`oe-init-build-env\`\`` environment
233 setup script to define the OpenEmbedded build environment on your
234 build host. ::
235
236 $ source &OE_INIT_FILE;
237
238 Among other things, the script creates the :term:`Build Directory`, which is
239 ``build`` in this case and is located in the :term:`Source Directory`. After
240 the script runs, your current working directory is set to the ``build``
241 directory.
242
243.. _bsp-filelayout:
244
245Example Filesystem Layout
246=========================
247
248Defining a common BSP directory structure allows end-users to understand
249and become familiar with that standard. A common format also encourages
250standardization of software support for hardware.
251
252The proposed form described in this section does have elements that are
253specific to the OpenEmbedded build system. It is intended that
254developers can use this structure with other build systems besides the
255OpenEmbedded build system. It is also intended that it will be be simple
256to extract information and convert it to other formats if required. The
257OpenEmbedded build system, through its standard :ref:`layers mechanism
258<overview-manual/overview-manual-yp-intro:the yocto project layer model>`, can
259directly accept the format described as a layer. The BSP layer captures
260all the hardware-specific details in one place using a standard format,
261which is useful for any person wishing to use the hardware platform
262regardless of the build system they are using.
263
264The BSP specification does not include a build system or other tools -
265the specification is concerned with the hardware-specific components
266only. At the end-distribution point, you can ship the BSP layer combined
267with a build system and other tools. Realize that it is important to
268maintain the distinction that the BSP layer, a build system, and tools
269are separate components that could be combined in certain end products.
270
271Before looking at the recommended form for the directory structure
272inside a BSP layer, you should be aware that some requirements do exist
273in order for a BSP layer to be considered compliant with the Yocto
274Project. For that list of requirements, see the
275":ref:`bsp-guide/bsp:released bsp requirements`" section.
276
277Below is the typical directory structure for a BSP layer. While this
278basic form represents the standard, realize that the actual layout for
279individual BSPs could differ. ::
280
281 meta-bsp_root_name/
282 meta-bsp_root_name/bsp_license_file
283 meta-bsp_root_name/README
284 meta-bsp_root_name/README.sources
285 meta-bsp_root_name/binary/bootable_images
286 meta-bsp_root_name/conf/layer.conf
287 meta-bsp_root_name/conf/machine/*.conf
288 meta-bsp_root_name/recipes-bsp/*
289 meta-bsp_root_name/recipes-core/*
290 meta-bsp_root_name/recipes-graphics/*
291 meta-bsp_root_name/recipes-kernel/linux/linux-yocto_kernel_rev.bbappend
292
293Below is an example of the Raspberry Pi BSP layer that is available from
294the :yocto_git:`Source Respositories <>`: ::
295
296 meta-raspberrypi/COPYING.MIT
297 meta-raspberrypi/README.md
298 meta-raspberrypi/classes
299 meta-raspberrypi/classes/sdcard_image-rpi.bbclass
300 meta-raspberrypi/conf/
301 meta-raspberrypi/conf/layer.conf
302 meta-raspberrypi/conf/machine/
303 meta-raspberrypi/conf/machine/raspberrypi-cm.conf
304 meta-raspberrypi/conf/machine/raspberrypi-cm3.conf
305 meta-raspberrypi/conf/machine/raspberrypi.conf
306 meta-raspberrypi/conf/machine/raspberrypi0-wifi.conf
307 meta-raspberrypi/conf/machine/raspberrypi0.conf
308 meta-raspberrypi/conf/machine/raspberrypi2.conf
309 meta-raspberrypi/conf/machine/raspberrypi3-64.conf
310 meta-raspberrypi/conf/machine/raspberrypi3.conf
311 meta-raspberrypi/conf/machine/include
312 meta-raspberrypi/conf/machine/include/rpi-base.inc
313 meta-raspberrypi/conf/machine/include/rpi-default-providers.inc
314 meta-raspberrypi/conf/machine/include/rpi-default-settings.inc
315 meta-raspberrypi/conf/machine/include/rpi-default-versions.inc
316 meta-raspberrypi/conf/machine/include/tune-arm1176jzf-s.inc
317 meta-raspberrypi/docs
318 meta-raspberrypi/docs/Makefile
319 meta-raspberrypi/docs/conf.py
320 meta-raspberrypi/docs/contributing.md
321 meta-raspberrypi/docs/extra-apps.md
322 meta-raspberrypi/docs/extra-build-config.md
323 meta-raspberrypi/docs/index.rst
324 meta-raspberrypi/docs/layer-contents.md
325 meta-raspberrypi/docs/readme.md
326 meta-raspberrypi/files
327 meta-raspberrypi/files/custom-licenses
328 meta-raspberrypi/files/custom-licenses/Broadcom
329 meta-raspberrypi/recipes-bsp
330 meta-raspberrypi/recipes-bsp/bootfiles
331 meta-raspberrypi/recipes-bsp/bootfiles/bcm2835-bootfiles.bb
332 meta-raspberrypi/recipes-bsp/bootfiles/rpi-config_git.bb
333 meta-raspberrypi/recipes-bsp/common
334 meta-raspberrypi/recipes-bsp/common/firmware.inc
335 meta-raspberrypi/recipes-bsp/formfactor
336 meta-raspberrypi/recipes-bsp/formfactor/formfactor
337 meta-raspberrypi/recipes-bsp/formfactor/formfactor/raspberrypi
338 meta-raspberrypi/recipes-bsp/formfactor/formfactor/raspberrypi/machconfig
339 meta-raspberrypi/recipes-bsp/formfactor/formfactor_0.0.bbappend
340 meta-raspberrypi/recipes-bsp/rpi-u-boot-src
341 meta-raspberrypi/recipes-bsp/rpi-u-boot-src/files
342 meta-raspberrypi/recipes-bsp/rpi-u-boot-src/files/boot.cmd.in
343 meta-raspberrypi/recipes-bsp/rpi-u-boot-src/rpi-u-boot-scr.bb
344 meta-raspberrypi/recipes-bsp/u-boot
345 meta-raspberrypi/recipes-bsp/u-boot/u-boot
346 meta-raspberrypi/recipes-bsp/u-boot/u-boot/*.patch
347 meta-raspberrypi/recipes-bsp/u-boot/u-boot_%.bbappend
348 meta-raspberrypi/recipes-connectivity
349 meta-raspberrypi/recipes-connectivity/bluez5
350 meta-raspberrypi/recipes-connectivity/bluez5/bluez5
351 meta-raspberrypi/recipes-connectivity/bluez5/bluez5/*.patch
352 meta-raspberrypi/recipes-connectivity/bluez5/bluez5/BCM43430A1.hcd
353 meta-raspberrypi/recipes-connectivity/bluez5/bluez5brcm43438.service
354 meta-raspberrypi/recipes-connectivity/bluez5/bluez5_%.bbappend
355 meta-raspberrypi/recipes-core
356 meta-raspberrypi/recipes-core/images
357 meta-raspberrypi/recipes-core/images/rpi-basic-image.bb
358 meta-raspberrypi/recipes-core/images/rpi-hwup-image.bb
359 meta-raspberrypi/recipes-core/images/rpi-test-image.bb
360 meta-raspberrypi/recipes-core/packagegroups
361 meta-raspberrypi/recipes-core/packagegroups/packagegroup-rpi-test.bb
362 meta-raspberrypi/recipes-core/psplash
363 meta-raspberrypi/recipes-core/psplash/files
364 meta-raspberrypi/recipes-core/psplash/files/psplash-raspberrypi-img.h
365 meta-raspberrypi/recipes-core/psplash/psplash_git.bbappend
366 meta-raspberrypi/recipes-core/udev
367 meta-raspberrypi/recipes-core/udev/udev-rules-rpi
368 meta-raspberrypi/recipes-core/udev/udev-rules-rpi/99-com.rules
369 meta-raspberrypi/recipes-core/udev/udev-rules-rpi.bb
370 meta-raspberrypi/recipes-devtools
371 meta-raspberrypi/recipes-devtools/bcm2835
372 meta-raspberrypi/recipes-devtools/bcm2835/bcm2835_1.52.bb
373 meta-raspberrypi/recipes-devtools/pi-blaster
374 meta-raspberrypi/recipes-devtools/pi-blaster/files
375 meta-raspberrypi/recipes-devtools/pi-blaster/files/*.patch
376 meta-raspberrypi/recipes-devtools/pi-blaster/pi-blaster_git.bb
377 meta-raspberrypi/recipes-devtools/python
378 meta-raspberrypi/recipes-devtools/python/python-rtimu
379 meta-raspberrypi/recipes-devtools/python/python-rtimu/*.patch
380 meta-raspberrypi/recipes-devtools/python/python-rtimu_git.bb
381 meta-raspberrypi/recipes-devtools/python/python-sense-hat_2.2.0.bb
382 meta-raspberrypi/recipes-devtools/python/rpi-gpio
383 meta-raspberrypi/recipes-devtools/python/rpi-gpio/*.patch
384 meta-raspberrypi/recipes-devtools/python/rpi-gpio_0.6.3.bb
385 meta-raspberrypi/recipes-devtools/python/rpio
386 meta-raspberrypi/recipes-devtools/python/rpio/*.patch
387 meta-raspberrypi/recipes-devtools/python/rpio_0.10.0.bb
388 meta-raspberrypi/recipes-devtools/wiringPi
389 meta-raspberrypi/recipes-devtools/wiringPi/files
390 meta-raspberrypi/recipes-devtools/wiringPi/files/*.patch
391 meta-raspberrypi/recipes-devtools/wiringPi/wiringpi_git.bb
392 meta-raspberrypi/recipes-graphics
393 meta-raspberrypi/recipes-graphics/eglinfo
394 meta-raspberrypi/recipes-graphics/eglinfo/eglinfo-fb_%.bbappend
395 meta-raspberrypi/recipes-graphics/eglinfo/eglinfo-x11_%.bbappend
396 meta-raspberrypi/recipes-graphics/mesa
397 meta-raspberrypi/recipes-graphics/mesa/mesa-gl_%.bbappend
398 meta-raspberrypi/recipes-graphics/mesa/mesa_%.bbappend
399 meta-raspberrypi/recipes-graphics/userland
400 meta-raspberrypi/recipes-graphics/userland/userland
401 meta-raspberrypi/recipes-graphics/userland/userland/*.patch
402 meta-raspberrypi/recipes-graphics/userland/userland_git.bb
403 meta-raspberrypi/recipes-graphics/vc-graphics
404 meta-raspberrypi/recipes-graphics/vc-graphics/files
405 meta-raspberrypi/recipes-graphics/vc-graphics/files/egl.pc
406 meta-raspberrypi/recipes-graphics/vc-graphics/files/vchiq.sh
407 meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics-hardfp.bb
408 meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics.bb
409 meta-raspberrypi/recipes-graphics/vc-graphics/vc-graphics.inc
410 meta-raspberrypi/recipes-graphics/wayland
411 meta-raspberrypi/recipes-graphics/wayland/weston_%.bbappend
412 meta-raspberrypi/recipes-graphics/xorg-xserver
413 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config
414 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi
415 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf
416 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d
417 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/10-evdev.conf
418 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/98-pitft.conf
419 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config/rpi/xorg.conf.d/99-calibration.conf
420 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
421 meta-raspberrypi/recipes-graphics/xorg-xserver/xserver-xorg_%.bbappend
422 meta-raspberrypi/recipes-kernel
423 meta-raspberrypi/recipes-kernel/linux-firmware
424 meta-raspberrypi/recipes-kernel/linux-firmware/files
425 meta-raspberrypi/recipes-kernel/linux-firmware/files/brcmfmac43430-sdio.bin
426 meta-raspberrypi/recipes-kernel/linux-firmware/files/brcfmac43430-sdio.txt
427 meta-raspberrypi/recipes-kernel/linux-firmware/linux-firmware_%.bbappend
428 meta-raspberrypi/recipes-kernel/linux
429 meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi-dev.bb
430 meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi.inc
431 meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi_4.14.bb
432 meta-raspberrypi/recipes-kernel/linux/linux-raspberrypi_4.9.bb
433 meta-raspberrypi/recipes-multimedia
434 meta-raspberrypi/recipes-multimedia/gstreamer
435 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx
436 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx/*.patch
437 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx_%.bbappend
438 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
439 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx-1.12
440 meta-raspberrypi/recipes-multimedia/gstreamer/gstreamer1.0-omx-1.12/*.patch
441 meta-raspberrypi/recipes-multimedia/omxplayer
442 meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer
443 meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer/*.patch
444 meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer_git.bb
445 meta-raspberrypi/recipes-multimedia/x264
446 meta-raspberrypi/recipes-multimedia/x264/x264_git.bbappend
447 meta-raspberrypi/wic meta-raspberrypi/wic/sdimage-raspberrypi.wks
448
449The following sections describe each part of the proposed BSP format.
450
451.. _bsp-filelayout-license:
452
453License Files
454-------------
455
456You can find these files in the BSP Layer at: ::
457
458 meta-bsp_root_name/bsp_license_file
459
460These optional files satisfy licensing requirements for the BSP. The
461type or types of files here can vary depending on the licensing
462requirements. For example, in the Raspberry Pi BSP, all licensing
463requirements are handled with the ``COPYING.MIT`` file.
464
465Licensing files can be MIT, BSD, GPLv*, and so forth. These files are
466recommended for the BSP but are optional and totally up to the BSP
467developer. For information on how to maintain license compliance, see
468the ":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
469section in the Yocto Project Development Tasks Manual.
470
471.. _bsp-filelayout-readme:
472
473README File
474-----------
475
476You can find this file in the BSP Layer at: ::
477
478 meta-bsp_root_name/README
479
480This file provides information on how to boot the live images that are
481optionally included in the ``binary/`` directory. The ``README`` file
482also provides information needed for building the image.
483
484At a minimum, the ``README`` file must contain a list of dependencies,
485such as the names of any other layers on which the BSP depends and the
486name of the BSP maintainer with his or her contact information.
487
488.. _bsp-filelayout-readme-sources:
489
490README.sources File
491-------------------
492
493You can find this file in the BSP Layer at: ::
494
495 meta-bsp_root_name/README.sources
496
497This file provides information on where to locate the BSP source files
498used to build the images (if any) that reside in
499``meta-bsp_root_name/binary``. Images in the ``binary`` would be images
500released with the BSP. The information in the ``README.sources`` file
501also helps you find the :term:`Metadata`
502used to generate the images that ship with the BSP.
503
504.. note::
505
506 If the BSP's ``binary`` directory is missing or the directory has no images, an
507 existing ``README.sources`` file is meaningless and usually does not exist.
508
509.. _bsp-filelayout-binary:
510
511Pre-built User Binaries
512-----------------------
513
514You can find these files in the BSP Layer at: ::
515
516 meta-bsp_root_name/binary/bootable_images
517
518This optional area contains useful pre-built kernels and user-space
519filesystem images released with the BSP that are appropriate to the
520target system. This directory typically contains graphical (e.g. Sato)
521and minimal live images when the BSP tarball has been created and made
522available in the :yocto_home:`Yocto Project <>` website. You can
523use these kernels and images to get a system running and quickly get
524started on development tasks.
525
526The exact types of binaries present are highly hardware-dependent. The
527:ref:`README <bsp-guide/bsp:readme file>` file should be present in the
528BSP Layer and it explains how to use the images with the target
529hardware. Additionally, the
530:ref:`README.sources <bsp-guide/bsp:readme.sources file>` file should be
531present to locate the sources used to build the images and provide
532information on the Metadata.
533
534.. _bsp-filelayout-layer:
535
536Layer Configuration File
537------------------------
538
539You can find this file in the BSP Layer at: ::
540
541 meta-bsp_root_name/conf/layer.conf
542
543The ``conf/layer.conf`` file identifies the file structure as a layer,
544identifies the contents of the layer, and contains information about how
545the build system should use it. Generally, a standard boilerplate file
546such as the following works. In the following example, you would replace
547bsp with the actual name of the BSP (i.e. bsp_root_name from the example
548template). ::
549
550 # We have a conf and classes directory, add to BBPATH
551 BBPATH .= ":${LAYERDIR}"
552
553 # We have a recipes directory, add to BBFILES
554 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
555 ${LAYERDIR}/recipes-*/*/*.bbappend"
556
557 BBFILE_COLLECTIONS += "bsp"
558 BBFILE_PATTERN_bsp = "^${LAYERDIR}/"
559 BBFILE_PRIORITY_bsp = "6"
560 LAYERDEPENDS_bsp = "intel"
561
562To illustrate the string substitutions, here are the corresponding
563statements from the Raspberry Pi ``conf/layer.conf`` file: ::
564
565 # We have a conf and classes directory, append to BBPATH
566 BBPATH .= ":${LAYERDIR}"
567
568 # We have a recipes directory containing .bb and .bbappend files, add to BBFILES
569 BBFILES += "${LAYERDIR}/recipes*/*/*.bb \
570 ${LAYERDIR}/recipes*/*/*.bbappend"
571
572 BBFILE_COLLECTIONS += "raspberrypi"
573 BBFILE_PATTERN_raspberrypi := "^${LAYERDIR}/"
574 BBFILE_PRIORITY_raspberrypi = "9"
575
576 # Additional license directories.
577 LICENSE_PATH += "${LAYERDIR}/files/custom-licenses"
578 .
579 .
580 .
581
582This file simply makes :term:`BitBake` aware of the recipes and configuration
583directories. The file must exist so that the OpenEmbedded build system can
584recognize the BSP.
585
586.. _bsp-filelayout-machine:
587
588Hardware Configuration Options
589------------------------------
590
591You can find these files in the BSP Layer at: ::
592
593 meta-bsp_root_name/conf/machine/*.conf
594
595The machine files bind together all the information contained elsewhere
596in the BSP into a format that the build system can understand. Each BSP
597Layer requires at least one machine file. If the BSP supports multiple
598machines, multiple machine configuration files can exist. These
599filenames correspond to the values to which users have set the
600:term:`MACHINE` variable.
601
602These files define things such as the kernel package to use
603(:term:`PREFERRED_PROVIDER` of
604:ref:`virtual/kernel <dev-manual/dev-manual-common-tasks:using virtual providers>`),
605the hardware drivers to include in different types of images, any
606special software components that are needed, any bootloader information,
607and also any special image format requirements.
608
609This configuration file could also include a hardware "tuning" file that
610is commonly used to define the package architecture and specify
611optimization flags, which are carefully chosen to give best performance
612on a given processor.
613
614Tuning files are found in the ``meta/conf/machine/include`` directory
615within the :term:`Source Directory`.
616For example, many ``tune-*`` files (e.g. ``tune-arm1136jf-s.inc``,
617``tune-1586-nlp.inc``, and so forth) reside in the
618``poky/meta/conf/machine/include`` directory.
619
620To use an include file, you simply include them in the machine
621configuration file. For example, the Raspberry Pi BSP
622``raspberrypi3.conf`` contains the following statement: ::
623
624 include conf/machine/include/rpi-base.inc
625
626.. _bsp-filelayout-misc-recipes:
627
628Miscellaneous BSP-Specific Recipe Files
629---------------------------------------
630
631You can find these files in the BSP Layer at: ::
632
633 meta-bsp_root_name/recipes-bsp/*
634
635This optional directory contains miscellaneous recipe files for the BSP.
636Most notably would be the formfactor files. For example, in the
637Raspberry Pi BSP, there is the ``formfactor_0.0.bbappend`` file, which
638is an append file used to augment the recipe that starts the build.
639Furthermore, there are machine-specific settings used during the build
640that are defined by the ``machconfig`` file further down in the
641directory. Here is the ``machconfig`` file for the Raspberry Pi BSP: ::
642
643 HAVE_TOUCHSCREEN=0
644 HAVE_KEYBOARD=1
645
646 DISPLAY_CAN_ROTATE=0
647 DISPLAY_ORIENTATION=0
648 DISPLAY_DPI=133
649
650.. note::
651
652 If a BSP does not have a formfactor entry, defaults are established
653 according to the formfactor configuration file that is installed by
654 the main formfactor recipe
655 ``meta/recipes-bsp/formfactor/formfactor_0.0.bb``, which is found in
656 the :term:`Source Directory`.
657
658.. _bsp-filelayout-recipes-graphics:
659
660Display Support Files
661---------------------
662
663You can find these files in the BSP Layer at: ::
664
665 meta-bsp_root_name/recipes-graphics/*
666
667This optional directory contains recipes for the BSP if it has special
668requirements for graphics support. All files that are needed for the BSP
669to support a display are kept here.
670
671.. _bsp-filelayout-kernel:
672
673Linux Kernel Configuration
674--------------------------
675
676You can find these files in the BSP Layer at: ::
677
678 meta-bsp_root_name/recipes-kernel/linux/linux*.bbappend
679 meta-bsp_root_name/recipes-kernel/linux/*.bb
680
681Append files (``*.bbappend``) modify the main kernel recipe being used
682to build the image. The ``*.bb`` files would be a developer-supplied
683kernel recipe. This area of the BSP hierarchy can contain both these
684types of files although, in practice, it is likely that you would have
685one or the other.
686
687For your BSP, you typically want to use an existing Yocto Project kernel
688recipe found in the :term:`Source Directory`
689at
690``meta/recipes-kernel/linux``. You can append machine-specific changes
691to the kernel recipe by using a similarly named append file, which is
692located in the BSP Layer for your target device (e.g. the
693``meta-bsp_root_name/recipes-kernel/linux`` directory).
694
695Suppose you are using the ``linux-yocto_4.4.bb`` recipe to build the
696kernel. In other words, you have selected the kernel in your
697bsp_root_name\ ``.conf`` file by adding
698:term:`PREFERRED_PROVIDER` and :term:`PREFERRED_VERSION`
699statements as follows: ::
700
701 PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
702 PREFERRED_VERSION_linux-yocto ?= "4.4%"
703
704.. note::
705
706 When the preferred provider is assumed by default, the ``PREFERRED_PROVIDER``
707 statement does not appear in the ``bsp_root_name`` .conf file.
708
709You would use the ``linux-yocto_4.4.bbappend`` file to append specific
710BSP settings to the kernel, thus configuring the kernel for your
711particular BSP.
712
713You can find more information on what your append file should contain in
714the ":ref:`kernel-dev/kernel-dev-common:creating the append file`" section
715in the Yocto Project Linux Kernel Development Manual.
716
717An alternate scenario is when you create your own kernel recipe for the
718BSP. A good example of this is the Raspberry Pi BSP. If you examine the
719``recipes-kernel/linux`` directory you see the following: ::
720
721 linux-raspberrypi-dev.bb
722 linux-raspberrypi.inc
723 linux-raspberrypi_4.14.bb
724 linux-raspberrypi_4.9.bb
725
726The directory contains three kernel recipes and a common include file.
727
728Developing a Board Support Package (BSP)
729========================================
730
731This section describes the high-level procedure you can follow to create
732a BSP. Although not required for BSP creation, the ``meta-intel``
733repository, which contains many BSPs supported by the Yocto Project, is
734part of the example.
735
736For an example that shows how to create a new layer using the tools, see
737the ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
738section.
739
740The following illustration and list summarize the BSP creation general
741workflow.
742
743.. image:: figures/bsp-dev-flow.png
744 :align: center
745
746#. *Set up Your Host Development System to Support Development Using the
747 Yocto Project*: See the ":ref:`dev-manual/dev-manual-start:preparing the build host`"
748 section in the Yocto Project Development Tasks Manual for options on how to
749 get a system ready to use the Yocto Project.
750
751#. *Establish the meta-intel Repository on Your System:* Having
752 local copies of these supported BSP layers on your system gives you
753 access to layers you might be able to leverage when creating your
754 BSP. For information on how to get these files, see the
755 ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
756 section.
757
758#. *Create Your Own BSP Layer Using the bitbake-layers Script:*
759 Layers are ideal for isolating and storing work for a given piece of
760 hardware. A layer is really just a location or area in which you
761 place the recipes and configurations for your BSP. In fact, a BSP is,
762 in itself, a special type of layer. The simplest way to create a new
763 BSP layer that is compliant with the Yocto Project is to use the
764 ``bitbake-layers`` script. For information about that script, see the
765 ":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
766 section.
767
768 Another example that illustrates a layer is an application. Suppose
769 you are creating an application that has library or other
770 dependencies in order for it to compile and run. The layer, in this
771 case, would be where all the recipes that define those dependencies
772 are kept. The key point for a layer is that it is an isolated area
773 that contains all the relevant information for the project that the
774 OpenEmbedded build system knows about. For more information on
775 layers, see the ":ref:`overview-manual/overview-manual-yp-intro:the yocto project layer model`"
776 section in the Yocto Project Overview and Concepts Manual. You can also
777 reference the ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
778 section in the Yocto Project Development Tasks Manual. For more
779 information on BSP layers, see the ":ref:`bsp-guide/bsp:bsp layers`"
780 section.
781
782 .. note::
783
784 - Five hardware reference BSPs exist that are part of the Yocto
785 Project release and are located in the ``poky/meta-yocto-bsp``
786 BSP layer:
787
788 - Texas Instruments Beaglebone (``beaglebone-yocto``)
789
790 - Ubiquiti Networks EdgeRouter Lite (``edgerouter``)
791
792 - Two general IA platforms (``genericx86`` and ``genericx86-64``)
793
794 - Three core Intel BSPs exist as part of the Yocto Project
795 release in the ``meta-intel`` layer:
796
797 - ``intel-core2-32``, which is a BSP optimized for the Core2
798 family of CPUs as well as all CPUs prior to the Silvermont
799 core.
800
801 - ``intel-corei7-64``, which is a BSP optimized for Nehalem
802 and later Core and Xeon CPUs as well as Silvermont and later
803 Atom CPUs, such as the Baytrail SoCs.
804
805 - ``intel-quark``, which is a BSP optimized for the Intel
806 Galileo gen1 & gen2 development boards.
807
808 When you set up a layer for a new BSP, you should follow a standard
809 layout. This layout is described in the ":ref:`bsp-guide/bsp:example filesystem layout`"
810 section. In the standard layout, notice
811 the suggested structure for recipes and configuration information.
812 You can see the standard layout for a BSP by examining any supported
813 BSP found in the ``meta-intel`` layer inside the Source Directory.
814
815#. *Make Configuration Changes to Your New BSP Layer:* The standard BSP
816 layer structure organizes the files you need to edit in ``conf`` and
817 several ``recipes-*`` directories within the BSP layer. Configuration
818 changes identify where your new layer is on the local system and
819 identifies the kernel you are going to use. When you run the
820 ``bitbake-layers`` script, you are able to interactively configure
821 many things for the BSP (e.g. keyboard, touchscreen, and so forth).
822
823#. *Make Recipe Changes to Your New BSP Layer:* Recipe changes include
824 altering recipes (``*.bb`` files), removing recipes you do not use,
825 and adding new recipes or append files (``.bbappend``) that support
826 your hardware.
827
828#. *Prepare for the Build:* Once you have made all the changes to your
829 BSP layer, there remains a few things you need to do for the
830 OpenEmbedded build system in order for it to create your image. You
831 need to get the build environment ready by sourcing an environment
832 setup script (i.e. ``oe-init-build-env``) and you need to be sure two
833 key configuration files are configured appropriately: the
834 ``conf/local.conf`` and the ``conf/bblayers.conf`` file. You must
835 make the OpenEmbedded build system aware of your new layer. See the
836 ":ref:`dev-manual/dev-manual-common-tasks:enabling your layer`"
837 section in the Yocto Project Development Tasks Manual for information
838 on how to let the build system know about your new layer.
839
840#. *Build the Image:* The OpenEmbedded build system uses the BitBake
841 tool to build images based on the type of image you want to create.
842 You can find more information about BitBake in the
843 :doc:`BitBake User Manual <bitbake:index>`.
844
845 The build process supports several types of images to satisfy
846 different needs. See the
847 ":ref:`ref-manual/ref-images:Images`" chapter in the Yocto
848 Project Reference Manual for information on supported images.
849
850Requirements and Recommendations for Released BSPs
851==================================================
852
853Certain requirements exist for a released BSP to be considered compliant
854with the Yocto Project. Additionally, recommendations also exist. This
855section describes the requirements and recommendations for released
856BSPs.
857
858Released BSP Requirements
859-------------------------
860
861Before looking at BSP requirements, you should consider the following:
862
863- The requirements here assume the BSP layer is a well-formed, "legal"
864 layer that can be added to the Yocto Project. For guidelines on
865 creating a layer that meets these base requirements, see the
866 ":ref:`bsp-guide/bsp:bsp layers`" section in this manual and the
867 ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
868 section in the Yocto Project Development Tasks Manual.
869
870- The requirements in this section apply regardless of how you package
871 a BSP. You should consult the packaging and distribution guidelines
872 for your specific release process. For an example of packaging and
873 distribution requirements, see the "`Third Party BSP Release
874 Process <https://wiki.yoctoproject.org/wiki/Third_Party_BSP_Release_Process>`__"
875 wiki page.
876
877- The requirements for the BSP as it is made available to a developer
878 are completely independent of the released form of the BSP. For
879 example, the BSP Metadata can be contained within a Git repository
880 and could have a directory structure completely different from what
881 appears in the officially released BSP layer.
882
883- It is not required that specific packages or package modifications
884 exist in the BSP layer, beyond the requirements for general
885 compliance with the Yocto Project. For example, no requirement exists
886 dictating that a specific kernel or kernel version be used in a given
887 BSP.
888
889Following are the requirements for a released BSP that conform to the
890Yocto Project:
891
892- *Layer Name:* The BSP must have a layer name that follows the Yocto
893 Project standards. For information on BSP layer names, see the
894 ":ref:`bsp-guide/bsp:bsp layers`" section.
895
896- *File System Layout:* When possible, use the same directory names in
897 your BSP layer as listed in the ``recipes.txt`` file, which is found
898 in ``poky/meta`` directory of the :term:`Source Directory`
899 or in the OpenEmbedded-Core Layer (``openembedded-core``) at
900 http://git.openembedded.org/openembedded-core/tree/meta.
901
902 You should place recipes (``*.bb`` files) and recipe modifications
903 (``*.bbappend`` files) into ``recipes-*`` subdirectories by
904 functional area as outlined in ``recipes.txt``. If you cannot find a
905 category in ``recipes.txt`` to fit a particular recipe, you can make
906 up your own ``recipes-*`` subdirectory.
907
908 Within any particular ``recipes-*`` category, the layout should match
909 what is found in the OpenEmbedded-Core Git repository
910 (``openembedded-core``) or the Source Directory (``poky``). In other
911 words, make sure you place related files in appropriately-related
912 ``recipes-*`` subdirectories specific to the recipe's function, or
913 within a subdirectory containing a set of closely-related recipes.
914 The recipes themselves should follow the general guidelines for
915 recipes used in the Yocto Project found in the "`OpenEmbedded Style
916 Guide <http://openembedded.org/wiki/Styleguide>`__".
917
918- *License File:* You must include a license file in the
919 ``meta-bsp_root_name`` directory. This license covers the BSP
920 Metadata as a whole. You must specify which license to use since no
921 default license exists when one is not specified. See the
922 :yocto_git:`COPYING.MIT </cgit.cgi/meta-raspberrypi/tree/COPYING.MIT>`
923 file for the Raspberry Pi BSP in the ``meta-raspberrypi`` BSP layer
924 as an example.
925
926- *README File:* You must include a ``README`` file in the
927 ``meta-bsp_root_name`` directory. See the
928 :yocto_git:`README.md </cgit.cgi/meta-raspberrypi/tree/README.md>`
929 file for the Raspberry Pi BSP in the ``meta-raspberrypi`` BSP layer
930 as an example.
931
932 At a minimum, the ``README`` file should contain the following:
933
934 - A brief description of the target hardware.
935
936 - A list of all the dependencies of the BSP. These dependencies are
937 typically a list of required layers needed to build the BSP.
938 However, the dependencies should also contain information
939 regarding any other dependencies the BSP might have.
940
941 - Any required special licensing information. For example, this
942 information includes information on special variables needed to
943 satisfy a EULA, or instructions on information needed to build or
944 distribute binaries built from the BSP Metadata.
945
946 - The name and contact information for the BSP layer maintainer.
947 This is the person to whom patches and questions should be sent.
948 For information on how to find the right person, see the
949 ":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
950 section in the Yocto Project Development Tasks Manual.
951
952 - Instructions on how to build the BSP using the BSP layer.
953
954 - Instructions on how to boot the BSP build from the BSP layer.
955
956 - Instructions on how to boot the binary images contained in the
957 ``binary`` directory, if present.
958
959 - Information on any known bugs or issues that users should know
960 about when either building or booting the BSP binaries.
961
962- *README.sources File:* If your BSP contains binary images in the
963 ``binary`` directory, you must include a ``README.sources`` file in
964 the ``meta-bsp_root_name`` directory. This file specifies exactly
965 where you can find the sources used to generate the binary images.
966
967- *Layer Configuration File:* You must include a ``conf/layer.conf``
968 file in the ``meta-bsp_root_name`` directory. This file identifies
969 the ``meta-bsp_root_name`` BSP layer as a layer to the build
970 system.
971
972- *Machine Configuration File:* You must include one or more
973 ``conf/machine/bsp_root_name.conf`` files in the
974 ``meta-bsp_root_name`` directory. These configuration files define
975 machine targets that can be built using the BSP layer. Multiple
976 machine configuration files define variations of machine
977 configurations that the BSP supports. If a BSP supports multiple
978 machine variations, you need to adequately describe each variation in
979 the BSP ``README`` file. Do not use multiple machine configuration
980 files to describe disparate hardware. If you do have very different
981 targets, you should create separate BSP layers for each target.
982
983 .. note::
984
985 It is completely possible for a developer to structure the working
986 repository as a conglomeration of unrelated BSP files, and to possibly
987 generate BSPs targeted for release from that directory using scripts or
988 some other mechanism (e.g. ``meta-yocto-bsp`` layer). Such considerations
989 are outside the scope of this document.
990
991Released BSP Recommendations
992----------------------------
993
994Following are recommendations for released BSPs that conform to the
995Yocto Project:
996
997- *Bootable Images:* Released BSPs can contain one or more bootable
998 images. Including bootable images allows users to easily try out the
999 BSP using their own hardware.
1000
1001 In some cases, it might not be convenient to include a bootable
1002 image. If so, you might want to make two versions of the BSP
1003 available: one that contains binary images, and one that does not.
1004 The version that does not contain bootable images avoids unnecessary
1005 download times for users not interested in the images.
1006
1007 If you need to distribute a BSP and include bootable images or build
1008 kernel and filesystems meant to allow users to boot the BSP for
1009 evaluation purposes, you should put the images and artifacts within a
1010 ``binary/`` subdirectory located in the ``meta-bsp_root_name``
1011 directory.
1012
1013 .. note::
1014
1015 If you do include a bootable image as part of the BSP and the
1016 image was built by software covered by the GPL or other open
1017 source licenses, it is your responsibility to understand and meet
1018 all licensing requirements, which could include distribution of
1019 source files.
1020
1021- *Use a Yocto Linux Kernel:* Kernel recipes in the BSP should be based
1022 on a Yocto Linux kernel. Basing your recipes on these kernels reduces
1023 the costs for maintaining the BSP and increases its scalability. See
1024 the ``Yocto Linux Kernel`` category in the
1025 :yocto_git:`Source Repositories <>` for these kernels.
1026
1027Customizing a Recipe for a BSP
1028==============================
1029
1030If you plan on customizing a recipe for a particular BSP, you need to do
1031the following:
1032
1033- Create a ``*.bbappend`` file for the modified recipe. For information on using
1034 append files, see the ":ref:`dev-manual/dev-manual-common-tasks:using
1035 .bbappend files in your layer`" section in the Yocto Project Development
1036 Tasks Manual.
1037
1038- Ensure your directory structure in the BSP layer that supports your
1039 machine is such that the OpenEmbedded build system can find it. See
1040 the example later in this section for more information.
1041
1042- Put the append file in a directory whose name matches the machine's
1043 name and is located in an appropriate sub-directory inside the BSP
1044 layer (i.e. ``recipes-bsp``, ``recipes-graphics``, ``recipes-core``,
1045 and so forth).
1046
1047- Place the BSP-specific files in the proper directory inside the BSP
1048 layer. How expansive the layer is affects where you must place these
1049 files. For example, if your layer supports several different machine
1050 types, you need to be sure your layer's directory structure includes
1051 hierarchy that separates the files according to machine. If your
1052 layer does not support multiple machines, the layer would not have
1053 that additional hierarchy and the files would obviously not be able
1054 to reside in a machine-specific directory.
1055
1056Following is a specific example to help you better understand the
1057process. This example customizes customizes a recipe by adding a
1058BSP-specific configuration file named ``interfaces`` to the
1059``init-ifupdown_1.0.bb`` recipe for machine "xyz" where the BSP layer
1060also supports several other machines:
1061
1062#. Edit the ``init-ifupdown_1.0.bbappend`` file so that it contains the
1063 following: ::
1064
1065 FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
1066
1067 The append file needs to be in the ``meta-xyz/recipes-core/init-ifupdown``
1068 directory.
1069
1070#. Create and place the new ``interfaces`` configuration file in the
1071 BSP's layer here: ::
1072
1073 meta-xyz/recipes-core/init-ifupdown/files/xyz-machine-one/interfaces
1074
1075 .. note::
1076
1077 If the meta-xyz layer did not support multiple machines, you would place
1078 the interfaces configuration file in the layer here: ::
1079
1080 meta-xyz/recipes-core/init-ifupdown/files/interfaces
1081
1082 The :term:`FILESEXTRAPATHS` variable in the append files extends the search
1083 path the build system uses to find files during the build. Consequently, for
1084 this example you need to have the ``files`` directory in the same location as
1085 your append file.
1086
1087BSP Licensing Considerations
1088============================
1089
1090In some cases, a BSP contains separately-licensed Intellectual Property
1091(IP) for a component or components. For these cases, you are required to
1092accept the terms of a commercial or other type of license that requires
1093some kind of explicit End User License Agreement (EULA). Once you accept
1094the license, the OpenEmbedded build system can then build and include
1095the corresponding component in the final BSP image. If the BSP is
1096available as a pre-built image, you can download the image after
1097agreeing to the license or EULA.
1098
1099You could find that some separately-licensed components that are
1100essential for normal operation of the system might not have an
1101unencumbered (or free) substitute. Without these essential components,
1102the system would be non-functional. Then again, you might find that
1103other licensed components that are simply 'good-to-have' or purely
1104elective do have an unencumbered, free replacement component that you
1105can use rather than agreeing to the separately-licensed component. Even
1106for components essential to the system, you might find an unencumbered
1107component that is not identical but will work as a less-capable version
1108of the licensed version in the BSP recipe.
1109
1110For cases where you can substitute a free component and still maintain
1111the system's functionality, the "DOWNLOADS" selection from the
1112"SOFTWARE" tab on the :yocto_home:`Yocto Project Website <>` makes
1113available de-featured BSPs that are completely free of any IP
1114encumbrances. For these cases, you can use the substitution directly and
1115without any further licensing requirements. If present, these fully
1116de-featured BSPs are named appropriately different as compared to the
1117names of their respective encumbered BSPs. If available, these
1118substitutions are your simplest and most preferred options. Obviously,
1119use of these substitutions assumes the resulting functionality meets
1120system requirements.
1121
1122.. note::
1123
1124 If however, a non-encumbered version is unavailable or it provides
1125 unsuitable functionality or quality, you can use an encumbered
1126 version.
1127
1128A couple different methods exist within the OpenEmbedded build system to
1129satisfy the licensing requirements for an encumbered BSP. The following
1130list describes them in order of preference:
1131
1132#. *Use the LICENSE_FLAGS Variable to Define the Recipes that Have Commercial or
1133 Other Types of Specially-Licensed Packages:* For each of those recipes, you can
1134 specify a matching license string in a ``local.conf`` variable named
1135 :term:`LICENSE_FLAGS_WHITELIST`.
1136 Specifying the matching license string signifies that you agree to
1137 the license. Thus, the build system can build the corresponding
1138 recipe and include the component in the image. See the
1139 ":ref:`dev-manual/dev-manual-common-tasks:enabling commercially licensed recipes`"
1140 section in the Yocto Project Development Tasks Manual for details on
1141 how to use these variables.
1142
1143 If you build as you normally would, without specifying any recipes in
1144 the ``LICENSE_FLAGS_WHITELIST``, the build stops and provides you
1145 with the list of recipes that you have tried to include in the image
1146 that need entries in the ``LICENSE_FLAGS_WHITELIST``. Once you enter
1147 the appropriate license flags into the whitelist, restart the build
1148 to continue where it left off. During the build, the prompt will not
1149 appear again since you have satisfied the requirement.
1150
1151 Once the appropriate license flags are on the white list in the
1152 ``LICENSE_FLAGS_WHITELIST`` variable, you can build the encumbered
1153 image with no change at all to the normal build process.
1154
1155#. *Get a Pre-Built Version of the BSP:* You can get this type of BSP by
1156 selecting the "DOWNLOADS" item from the "SOFTWARE" tab on the
1157 :yocto_home:`Yocto Project website <>`. You can download BSP tarballs
1158 that contain proprietary components after agreeing to the licensing
1159 requirements of each of the individually encumbered packages as part
1160 of the download process. Obtaining the BSP this way allows you to
1161 access an encumbered image immediately after agreeing to the
1162 click-through license agreements presented by the website. If you
1163 want to build the image yourself using the recipes contained within
1164 the BSP tarball, you will still need to create an appropriate
1165 ``LICENSE_FLAGS_WHITELIST`` to match the encumbered recipes in the
1166 BSP.
1167
1168.. note::
1169
1170 Pre-compiled images are bundled with a time-limited kernel that runs
1171 for a predetermined amount of time (10 days) before it forces the
1172 system to reboot. This limitation is meant to discourage direct
1173 redistribution of the image. You must eventually rebuild the image if
1174 you want to remove this restriction.
1175
1176Creating a new BSP Layer Using the ``bitbake-layers`` Script
1177============================================================
1178
1179The ``bitbake-layers create-layer`` script automates creating a BSP
1180layer. What makes a layer a "BSP layer" is the presence of at least one
1181machine configuration file. Additionally, a BSP layer usually has a
1182kernel recipe or an append file that leverages off an existing kernel
1183recipe. The primary requirement, however, is the machine configuration.
1184
1185Use these steps to create a BSP layer:
1186
1187- *Create a General Layer:* Use the ``bitbake-layers`` script with the
1188 ``create-layer`` subcommand to create a new general layer. For
1189 instructions on how to create a general layer using the
1190 ``bitbake-layers`` script, see the
1191 ":ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`"
1192 section in the Yocto Project Development Tasks Manual.
1193
1194- *Create a Layer Configuration File:* Every layer needs a layer
1195 configuration file. This configuration file establishes locations for
1196 the layer's recipes, priorities for the layer, and so forth. You can
1197 find examples of ``layer.conf`` files in the Yocto Project
1198 :yocto_git:`Source Repositories <>`. To get examples of what you need
1199 in your configuration file, locate a layer (e.g. "meta-ti") and
1200 examine the
1201 :yocto_git:`local.conf </cgit/cgit.cgi/meta-ti/tree/conf/layer.conf>`
1202 file.
1203
1204- *Create a Machine Configuration File:* Create a
1205 ``conf/machine/bsp_root_name.conf`` file. See
1206 :yocto_git:`meta-yocto-bsp/conf/machine </cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf/machine>`
1207 for sample ``bsp_root_name.conf`` files. Other samples such as
1208 :yocto_git:`meta-ti </cgit/cgit.cgi/meta-ti/tree/conf/machine>`
1209 and
1210 :yocto_git:`meta-freescale </cgit/cgit.cgi/meta-freescale/tree/conf/machine>`
1211 exist from other vendors that have more specific machine and tuning
1212 examples.
1213
1214- *Create a Kernel Recipe:* Create a kernel recipe in
1215 ``recipes-kernel/linux`` by either using a kernel append file or a
1216 new custom kernel recipe file (e.g. ``yocto-linux_4.12.bb``). The BSP
1217 layers mentioned in the previous step also contain different kernel
1218 examples. See the ":ref:`kernel-dev/kernel-dev-common:modifying an existing recipe`"
1219 section in the Yocto Project Linux Kernel Development Manual for
1220 information on how to create a custom kernel.
1221
1222The remainder of this section provides a description of the Yocto
1223Project reference BSP for Beaglebone, which resides in the
1224:yocto_git:`meta-yocto-bsp </cgit/cgit.cgi/poky/tree/meta-yocto-bsp>`
1225layer.
1226
1227BSP Layer Configuration Example
1228-------------------------------
1229
1230The layer's ``conf`` directory contains the ``layer.conf`` configuration
1231file. In this example, the ``conf/layer.conf`` is the following: ::
1232
1233 # We have a conf and classes directory, add to BBPATH
1234 BBPATH .= ":${LAYERDIR}"
1235
1236 # We have recipes-\* directories, add to BBFILES
1237 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
1238 ${LAYERDIR}/recipes-*/*/*.bbappend"
1239
1240 BBFILE_COLLECTIONS += "yoctobsp"
1241 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
1242 BBFILE_PRIORITY_yoctobsp = "5"
1243 LAYERVERSION_yoctobsp = "4"
1244 LAYERSERIES_COMPAT_yoctobsp = "&DISTRO_NAME_NO_CAP;"
1245
1246The variables used in this file configure the layer. A good way to learn about layer
1247configuration files is to examine various files for BSP from the
1248:yocto_git:`Source Repositories <>`.
1249
1250For a detailed description of this particular layer configuration file,
1251see ":ref:`step 3 <dev-manual/dev-manual-common-tasks:creating your own layer>`"
1252in the discussion that describes how to create layers in the Yocto
1253Project Development Tasks Manual.
1254
1255BSP Machine Configuration Example
1256---------------------------------
1257
1258As mentioned earlier in this section, the existence of a machine
1259configuration file is what makes a layer a BSP layer as compared to a
1260general or kernel layer.
1261
1262One or more machine configuration files exist in the
1263``bsp_layer/conf/machine/`` directory of the layer: ::
1264
1265 bsp_layer/conf/machine/machine1\.conf``
1266 bsp_layer/conf/machine/machine2\.conf``
1267 bsp_layer/conf/machine/machine3\.conf``
1268 ... more ...
1269
1270For example, the machine configuration file for the `BeagleBone and
1271BeagleBone Black development boards <http://beagleboard.org/bone>`__ is
1272located in the layer ``poky/meta-yocto-bsp/conf/machine`` and is named
1273``beaglebone-yocto.conf``: ::
1274
1275 #@TYPE: Machine
1276 #@NAME: Beaglebone-yocto machine
1277 #@DESCRIPTION: Reference machine configuration for http://beagleboard.org/bone and http://beagleboard.org/black boards
1278
1279 PREFERRED_PROVIDER_virtual/xserver ?= "xserver-xorg"
1280 XSERVER ?= "xserver-xorg \
1281 xf86-video-modesetting \
1282 "
1283
1284 MACHINE_EXTRA_RRECOMMENDS = "kernel-modules kernel-devicetree"
1285
1286 EXTRA_IMAGEDEPENDS += "u-boot"
1287
1288 DEFAULTTUNE ?= "cortexa8hf-neon"
1289 include conf/machine/include/tune-cortexa8.inc
1290
1291 IMAGE_FSTYPES += "tar.bz2 jffs2 wic wic.bmap"
1292 EXTRA_IMAGECMD_jffs2 = "-lnp "
1293 WKS_FILE ?= "beaglebone-yocto.wks"
1294 IMAGE_INSTALL_append = " kernel-devicetree kernel-image-zimage"
1295 do_image_wic[depends] += "mtools-native:do_populate_sysroot dosfstools-native:do_populate_sysroot"
1296
1297 SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyO0"
1298 SERIAL_CONSOLES_CHECK = "${SERIAL_CONSOLES}"
1299
1300 PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
1301 PREFERRED_VERSION_linux-yocto ?= "5.0%"
1302
1303 KERNEL_IMAGETYPE = "zImage"
1304 KERNEL_DEVICETREE = "am335x-bone.dtb am335x-boneblack.dtb am335x-bonegreen.dtb"
1305 KERNEL_EXTRA_ARGS += "LOADADDR=${UBOOT_ENTRYPOINT}"
1306
1307 SPL_BINARY = "MLO"
1308 UBOOT_SUFFIX = "img"
1309 UBOOT_MACHINE = "am335x_evm_defconfig"
1310 UBOOT_ENTRYPOINT = "0x80008000"
1311 UBOOT_LOADADDRESS = "0x80008000"
1312
1313 MACHINE_FEATURES = "usbgadget usbhost vfat alsa"
1314
1315 IMAGE_BOOT_FILES ?= "u-boot.${UBOOT_SUFFIX} MLO zImage am335x-bone.dtb am335x-boneblack.dtb am335x-bonegreen.dtb"
1316
1317The variables used to configure the machine define machine-specific properties; for
1318example, machine-dependent packages, machine tunings, the type of kernel
1319to build, and U-Boot configurations.
1320
1321The following list provides some explanation for the statements found in
1322the example reference machine configuration file for the BeagleBone
1323development boards. Realize that much more can be defined as part of a
1324machine's configuration file. In general, you can learn about related
1325variables that this example does not have by locating the variables in
1326the ":ref:`ref-manual/ref-variables:variables glossary`" in the Yocto
1327Project Reference Manual.
1328
1329- :term:`PREFERRED_PROVIDER_virtual/xserver <PREFERRED_PROVIDER>`:
1330 The recipe that provides "virtual/xserver" when more than one
1331 provider is found. In this case, the recipe that provides
1332 "virtual/xserver" is "xserver-xorg", which exists in
1333 ``poky/meta/recipes-graphics/xorg-xserver``.
1334
1335- :term:`XSERVER`: The packages that
1336 should be installed to provide an X server and drivers for the
1337 machine. In this example, the "xserver-xorg" and
1338 "xf86-video-modesetting" are installed.
1339
1340- :term:`MACHINE_EXTRA_RRECOMMENDS`:
1341 A list of machine-dependent packages not essential for booting the
1342 image. Thus, the build does not fail if the packages do not exist.
1343 However, the packages are required for a fully-featured image.
1344
1345 .. tip::
1346
1347 Many ``MACHINE\*`` variables exist that help you configure a particular piece
1348 of hardware.
1349
1350- :term:`EXTRA_IMAGEDEPENDS`:
1351 Recipes to build that do not provide packages for installing into the
1352 root filesystem but building the image depends on the recipes.
1353 Sometimes a recipe is required to build the final image but is not
1354 needed in the root filesystem. In this case, the U-Boot recipe must
1355 be built for the image.
1356
1357- :term:`DEFAULTTUNE`: Machines
1358 use tunings to optimize machine, CPU, and application performance.
1359 These features, which are collectively known as "tuning features",
1360 exist in the :term:`OpenEmbedded-Core (OE-Core)` layer (e.g.
1361 ``poky/meta/conf/machine/include``). In this example, the default
1362 tunning file is "cortexa8hf-neon".
1363
1364 .. note::
1365
1366 The include statement that pulls in the
1367 conf/machine/include/tune-cortexa8.inc file provides many tuning
1368 possibilities.
1369
1370- :term:`IMAGE_FSTYPES`: The
1371 formats the OpenEmbedded build system uses during the build when
1372 creating the root filesystem. In this example, four types of images
1373 are supported.
1374
1375- :term:`EXTRA_IMAGECMD`:
1376 Specifies additional options for image creation commands. In this
1377 example, the "-lnp " option is used when creating the
1378 `JFFS2 <https://en.wikipedia.org/wiki/JFFS2>`__ image.
1379
1380- :term:`WKS_FILE`: The location of
1381 the :ref:`Wic kickstart <ref-manual/ref-kickstart:openembedded kickstart (\`\`.wks\`\`) reference>` file used
1382 by the OpenEmbedded build system to create a partitioned image
1383 (image.wic).
1384
1385- :term:`IMAGE_INSTALL`:
1386 Specifies packages to install into an image through the
1387 :ref:`image <ref-classes-image>` class. Recipes
1388 use the ``IMAGE_INSTALL`` variable.
1389
1390- ``do_image_wic[depends]``: A task that is constructed during the
1391 build. In this example, the task depends on specific tools in order
1392 to create the sysroot when buiding a Wic image.
1393
1394- :term:`SERIAL_CONSOLES`:
1395 Defines a serial console (TTY) to enable using getty. In this case,
1396 the baud rate is "115200" and the device name is "ttyO0".
1397
1398- :term:`PREFERRED_PROVIDER_virtual/kernel <PREFERRED_PROVIDER>`:
1399 Specifies the recipe that provides "virtual/kernel" when more than
1400 one provider is found. In this case, the recipe that provides
1401 "virtual/kernel" is "linux-yocto", which exists in the layer's
1402 ``recipes-kernel/linux`` directory.
1403
1404- :term:`PREFERRED_VERSION_linux-yocto <PREFERRED_VERSION>`:
1405 Defines the version of the recipe used to build the kernel, which is
1406 "5.0" in this case.
1407
1408- :term:`KERNEL_IMAGETYPE`:
1409 The type of kernel to build for the device. In this case, the
1410 OpenEmbedded build system creates a "zImage" image type.
1411
1412- :term:`KERNEL_DEVICETREE`:
1413 The names of the generated Linux kernel device trees (i.e. the
1414 ``*.dtb``) files. All the device trees for the various BeagleBone
1415 devices are included.
1416
1417- :term:`KERNEL_EXTRA_ARGS`:
1418 Additional ``make`` command-line arguments the OpenEmbedded build
1419 system passes on when compiling the kernel. In this example,
1420 ``LOADADDR=${UBOOT_ENTRYPOINT}`` is passed as a command-line argument.
1421
1422- :term:`SPL_BINARY`: Defines the
1423 Secondary Program Loader (SPL) binary type. In this case, the SPL
1424 binary is set to "MLO", which stands for Multimedia card LOader.
1425
1426 The BeagleBone development board requires an SPL to boot and that SPL
1427 file type must be MLO. Consequently, the machine configuration needs
1428 to define ``SPL_BINARY`` as ``MLO``.
1429
1430 .. note::
1431
1432 For more information on how the SPL variables are used, see the u-boot.inc
1433 include file.
1434
1435- :term:`UBOOT_* <UBOOT_ENTRYPOINT>`: Defines
1436 various U-Boot configurations needed to build a U-Boot image. In this
1437 example, a U-Boot image is required to boot the BeagleBone device.
1438 See the following variables for more information:
1439
1440 - :term:`UBOOT_SUFFIX`:
1441 Points to the generated U-Boot extension.
1442
1443 - :term:`UBOOT_MACHINE`:
1444 Specifies the value passed on the make command line when building
1445 a U-Boot image.
1446
1447 - :term:`UBOOT_ENTRYPOINT`:
1448 Specifies the entry point for the U-Boot image.
1449
1450 - :term:`UBOOT_LOADADDRESS`:
1451 Specifies the load address for the U-Boot image.
1452
1453- :term:`MACHINE_FEATURES`:
1454 Specifies the list of hardware features the BeagleBone device is
1455 capable of supporting. In this case, the device supports "usbgadget
1456 usbhost vfat alsa".
1457
1458- :term:`IMAGE_BOOT_FILES`:
1459 Files installed into the device's boot partition when preparing the
1460 image using the Wic tool with the ``bootimg-partition`` or
1461 ``bootimg-efi`` source plugin.
1462
1463BSP Kernel Recipe Example
1464-------------------------
1465
1466The kernel recipe used to build the kernel image for the BeagleBone
1467device was established in the machine configuration: ::
1468
1469 PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
1470 PREFERRED_VERSION_linux-yocto ?= "5.0%"
1471
1472The ``meta-yocto-bsp/recipes-kernel/linux`` directory in the layer contains
1473metadata used to build the kernel. In this case, a kernel append file
1474(i.e. ``linux-yocto_5.0.bbappend``) is used to override an established
1475kernel recipe (i.e. ``linux-yocto_5.0.bb``), which is located in
1476https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/linux.
1477
1478Following is the contents of the append file: ::
1479
1480 KBRANCH_genericx86 = "v5.0/standard/base"
1481 KBRANCH_genericx86-64 = "v5.0/standard/base"
1482 KBRANCH_edgerouter = "v5.0/standard/edgerouter"
1483 KBRANCH_beaglebone-yocto = "v5.0/standard/beaglebone"
1484
1485 KMACHINE_genericx86 ?= "common-pc"
1486 KMACHINE_genericx86-64 ?= "common-pc-64"
1487 KMACHINE_beaglebone-yocto ?= "beaglebone"
1488
1489 SRCREV_machine_genericx86 ?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d"
1490 SRCREV_machine_genericx86-64 ?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d"
1491 SRCREV_machine_edgerouter ?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d"
1492 SRCREV_machine_beaglebone-yocto ?= "3df4aae6074e94e794e27fe7f17451d9353cdf3d"
1493
1494 COMPATIBLE_MACHINE_genericx86 = "genericx86"
1495 COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
1496 COMPATIBLE_MACHINE_edgerouter = "edgerouter"
1497 COMPATIBLE_MACHINE_beaglebone-yocto = "beaglebone-yocto"
1498
1499 LINUX_VERSION_genericx86 = "5.0.3"
1500 LINUX_VERSION_genericx86-64 = "5.0.3"
1501 LINUX_VERSION_edgerouter = "5.0.3"
1502 LINUX_VERSION_beaglebone-yocto = "5.0.3"
1503
1504This particular append file works for all the machines that are
1505part of the ``meta-yocto-bsp`` layer. The relevant statements are
1506appended with the "beaglebone-yocto" string. The OpenEmbedded build
1507system uses these statements to override similar statements in the
1508kernel recipe:
1509
1510- :term:`KBRANCH`: Identifies the
1511 kernel branch that is validated, patched, and configured during the
1512 build.
1513
1514- :term:`KMACHINE`: Identifies the
1515 machine name as known by the kernel, which is sometimes a different
1516 name than what is known by the OpenEmbedded build system.
1517
1518- :term:`SRCREV`: Identifies the
1519 revision of the source code used to build the image.
1520
1521- :term:`COMPATIBLE_MACHINE`:
1522 A regular expression that resolves to one or more target machines
1523 with which the recipe is compatible.
1524
1525- :term:`LINUX_VERSION`: The
1526 Linux version from kernel.org used by the OpenEmbedded build system
1527 to build the kernel image.
diff --git a/documentation/bsp-guide/bsp.xml b/documentation/bsp-guide/bsp.xml
index 96c0455f67..f5c3f31faf 100644
--- a/documentation/bsp-guide/bsp.xml
+++ b/documentation/bsp-guide/bsp.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='bsp'> 6<chapter id='bsp'>
6 7
@@ -2157,7 +2158,7 @@
2157 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_BOOT_FILES'><filename>IMAGE_BOOT_FILES</filename></ulink>: 2158 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_BOOT_FILES'><filename>IMAGE_BOOT_FILES</filename></ulink>:
2158 Files installed into the device's boot partition 2159 Files installed into the device's boot partition
2159 when preparing the image using the Wic tool 2160 when preparing the image using the Wic tool
2160 with the <filename>bootimg-partition</filename> 2161 with the <filename>bootimg-partition</filename> or <filename>bootimg-efi</filename>
2161 source plugin. 2162 source plugin.
2162 </para></listitem> 2163 </para></listitem>
2163 </itemizedlist> 2164 </itemizedlist>
diff --git a/documentation/bsp-guide/history.rst b/documentation/bsp-guide/history.rst
new file mode 100644
index 0000000000..b52006adf9
--- /dev/null
+++ b/documentation/bsp-guide/history.rst
@@ -0,0 +1,73 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 0.9
15 - November 2010
16 - The initial document released with the Yocto Project 0.9 Release
17 * - 1.0
18 - April 2011
19 - Released with the Yocto Project 1.0 Release.
20 * - 1.1
21 - October 2011
22 - Released with the Yocto Project 1.1 Release.
23 * - 1.2
24 - April 2012
25 - Released with the Yocto Project 1.2 Release.
26 * - 1.3
27 - October 2012
28 - Released with the Yocto Project 1.3 Release.
29 * - 1.4
30 - April 2013
31 - Released with the Yocto Project 1.4 Release.
32 * - 1.5
33 - October 2013
34 - Released with the Yocto Project 1.5 Release.
35 * - 1.6
36 - April 2014
37 - Released with the Yocto Project 1.6 Release.
38 * - 1.7
39 - October 2014
40 - Released with the Yocto Project 1.7 Release.
41 * - 1.8
42 - April 2015
43 - Released with the Yocto Project 1.8 Release.
44 * - 2.0
45 - October 2015
46 - Released with the Yocto Project 2.0 Release.
47 * - 2.1
48 - April 2016
49 - Released with the Yocto Project 2.1 Release.
50 * - 2.2
51 - October 2016
52 - Released with the Yocto Project 2.2 Release.
53 * - 2.3
54 - May 2017
55 - Released with the Yocto Project 2.3 Release.
56 * - 2.4
57 - October 2017
58 - Released with the Yocto Project 2.4 Release.
59 * - 2.5
60 - May 2018
61 - Released with the Yocto Project 2.5 Release.
62 * - 2.6
63 - November 2018
64 - Released with the Yocto Project 2.6 Release.
65 * - 2.7
66 - May 2019
67 - Released with the Yocto Project 2.7 Release.
68 * - 3.0
69 - October 2019
70 - Released with the Yocto Project 3.0 Release.
71 * - 3.1
72 - April 2020
73 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/conf.py b/documentation/conf.py
new file mode 100644
index 0000000000..f2dd2556fa
--- /dev/null
+++ b/documentation/conf.py
@@ -0,0 +1,127 @@
1# Configuration file for the Sphinx documentation builder.
2#
3# SPDX-License-Identifier: CC-BY-2.0-UK
4#
5# This file only contains a selection of the most common options. For a full
6# list see the documentation:
7# https://www.sphinx-doc.org/en/master/usage/configuration.html
8
9# -- Path setup --------------------------------------------------------------
10
11# If extensions (or modules to document with autodoc) are in another directory,
12# add these directories to sys.path here. If the directory is relative to the
13# documentation root, use os.path.abspath to make it absolute, like shown here.
14#
15import os
16import sys
17import datetime
18
19current_version = "dev"
20
21# String used in sidebar
22version = 'Version: ' + current_version
23if current_version == 'dev':
24 version = 'Version: Current Development'
25# Version seen in documentation_options.js and hence in js switchers code
26release = current_version
27
28
29# -- Project information -----------------------------------------------------
30project = 'The Yocto Project'
31copyright = '2010-%s, The Linux Foundation' % datetime.datetime.now().year
32author = 'The Linux Foundation'
33
34# -- General configuration ---------------------------------------------------
35
36# to load local extension from the folder 'sphinx'
37sys.path.insert(0, os.path.abspath('sphinx'))
38
39# Add any Sphinx extension module names here, as strings. They can be
40# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
41# ones.
42extensions = [
43 'sphinx.ext.autosectionlabel',
44 'sphinx.ext.extlinks',
45 'sphinx.ext.intersphinx',
46 'yocto-vars'
47]
48autosectionlabel_prefix_document = True
49
50# Add any paths that contain templates here, relative to this directory.
51templates_path = ['_templates']
52
53# List of patterns, relative to source directory, that match files and
54# directories to ignore when looking for source files.
55# This pattern also affects html_static_path and html_extra_path.
56exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'boilerplate.rst',
57 'adt-manual/*.rst']
58
59# master document name. The default changed from contents to index. so better
60# set it ourselves.
61master_doc = 'index'
62
63# create substitution for project configuration variables
64rst_prolog = """
65.. |project_name| replace:: %s
66.. |copyright| replace:: %s
67.. |author| replace:: %s
68""" % (project, copyright, author)
69
70# external links and substitutions
71extlinks = {
72 'yocto_home': ('https://yoctoproject.org%s', None),
73 'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
74 'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
75 'yocto_lists': ('https://lists.yoctoproject.org%s', None),
76 'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
77 'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
78 'yocto_docs': ('https://docs.yoctoproject.org%s', None),
79 'yocto_git': ('https://git.yoctoproject.org%s', None),
80 'oe_home': ('https://www.openembedded.org%s', None),
81 'oe_lists': ('https://lists.openembedded.org%s', None),
82}
83
84# Intersphinx config to use cross reference with Bitbake user manual
85intersphinx_mapping = {
86 'bitbake': ('https://docs.yoctoproject.org/bitbake/', None)
87}
88
89# -- Options for HTML output -------------------------------------------------
90
91# The theme to use for HTML and HTML Help pages. See the documentation for
92# a list of builtin themes.
93#
94try:
95 import sphinx_rtd_theme
96 html_theme = 'sphinx_rtd_theme'
97 html_theme_options = {
98 'sticky_navigation': False,
99 }
100except ImportError:
101 sys.stderr.write("The Sphinx sphinx_rtd_theme HTML theme was not found.\
102 \nPlease make sure to install the sphinx_rtd_theme python package.\n")
103 sys.exit(1)
104
105html_logo = 'sphinx-static/YoctoProject_Logo_RGB.jpg'
106
107# Add any paths that contain custom static files (such as style sheets) here,
108# relative to this directory. They are copied after the builtin static files,
109# so a file named "default.css" will overwrite the builtin "default.css".
110html_static_path = ['sphinx-static']
111
112html_context = {
113 'current_version': current_version,
114}
115
116# Add customm CSS and JS files
117html_css_files = ['theme_overrides.css']
118html_js_files = ['switchers.js']
119
120# Hide 'Created using Sphinx' text
121html_show_sphinx = False
122
123# Add 'Last updated' on each page
124html_last_updated_fmt = '%b %d, %Y'
125
126# Remove the trailing 'dot' in section numbers
127html_secnumber_suffix = " "
diff --git a/documentation/dev-manual/dev-manual-common-tasks.rst b/documentation/dev-manual/dev-manual-common-tasks.rst
new file mode 100644
index 0000000000..179979c763
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-common-tasks.rst
@@ -0,0 +1,11803 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Common Tasks
5************
6
7This chapter describes fundamental procedures such as creating layers,
8adding new software packages, extending or customizing images, porting
9work to new hardware (adding a new machine), and so forth. You will find
10that the procedures documented here occur often in the development cycle
11using the Yocto Project.
12
13Understanding and Creating Layers
14=================================
15
16The OpenEmbedded build system supports organizing
17:term:`Metadata` into multiple layers.
18Layers allow you to isolate different types of customizations from each
19other. For introductory information on the Yocto Project Layer Model,
20see the
21":ref:`overview-manual/overview-manual-yp-intro:the yocto project layer model`"
22section in the Yocto Project Overview and Concepts Manual.
23
24Creating Your Own Layer
25-----------------------
26
27It is very easy to create your own layers to use with the OpenEmbedded
28build system. The Yocto Project ships with tools that speed up creating
29layers. This section describes the steps you perform by hand to create
30layers so that you can better understand them. For information about the
31layer-creation tools, see the
32":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
33section in the Yocto Project Board Support Package (BSP) Developer's
34Guide and the ":ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`"
35section further down in this manual.
36
37Follow these general steps to create your layer without using tools:
38
391. *Check Existing Layers:* Before creating a new layer, you should be
40 sure someone has not already created a layer containing the Metadata
41 you need. You can see the `OpenEmbedded Metadata
42 Index <http://layers.openembedded.org/layerindex/layers/>`__ for a
43 list of layers from the OpenEmbedded community that can be used in
44 the Yocto Project. You could find a layer that is identical or close
45 to what you need.
46
472. *Create a Directory:* Create the directory for your layer. When you
48 create the layer, be sure to create the directory in an area not
49 associated with the Yocto Project :term:`Source Directory`
50 (e.g. the cloned ``poky`` repository).
51
52 While not strictly required, prepend the name of the directory with
53 the string "meta-". For example:
54 ::
55
56 meta-mylayer
57 meta-GUI_xyz
58 meta-mymachine
59
60 With rare exceptions, a layer's name follows this form:
61 ::
62
63 meta-root_name
64
65 Following this layer naming convention can save
66 you trouble later when tools, components, or variables "assume" your
67 layer name begins with "meta-". A notable example is in configuration
68 files as shown in the following step where layer names without the
69 "meta-" string are appended to several variables used in the
70 configuration.
71
723. *Create a Layer Configuration File:* Inside your new layer folder,
73 you need to create a ``conf/layer.conf`` file. It is easiest to take
74 an existing layer configuration file and copy that to your layer's
75 ``conf`` directory and then modify the file as needed.
76
77 The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project
78 :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf>`
79 demonstrates the required syntax. For your layer, you need to replace
80 "yoctobsp" with a unique identifier for your layer (e.g. "machinexyz"
81 for a layer named "meta-machinexyz"):
82 ::
83
84 # We have a conf and classes directory, add to BBPATH
85 BBPATH .= ":${LAYERDIR}"
86
87 # We have recipes-\* directories, add to BBFILES
88 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
89 ${LAYERDIR}/recipes-*/*/*.bbappend"
90
91 BBFILE_COLLECTIONS += "yoctobsp"
92 BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
93 BBFILE_PRIORITY_yoctobsp = "5"
94 LAYERVERSION_yoctobsp = "4"
95 LAYERSERIES_COMPAT_yoctobsp = "dunfell"
96
97 Following is an explanation of the layer configuration file:
98
99 - :term:`BBPATH`: Adds the layer's
100 root directory to BitBake's search path. Through the use of the
101 ``BBPATH`` variable, BitBake locates class files (``.bbclass``),
102 configuration files, and files that are included with ``include``
103 and ``require`` statements. For these cases, BitBake uses the
104 first file that matches the name found in ``BBPATH``. This is
105 similar to the way the ``PATH`` variable is used for binaries. It
106 is recommended, therefore, that you use unique class and
107 configuration filenames in your custom layer.
108
109 - :term:`BBFILES`: Defines the
110 location for all recipes in the layer.
111
112 - :term:`BBFILE_COLLECTIONS`:
113 Establishes the current layer through a unique identifier that is
114 used throughout the OpenEmbedded build system to refer to the
115 layer. In this example, the identifier "yoctobsp" is the
116 representation for the container layer named "meta-yocto-bsp".
117
118 - :term:`BBFILE_PATTERN`:
119 Expands immediately during parsing to provide the directory of the
120 layer.
121
122 - :term:`BBFILE_PRIORITY`:
123 Establishes a priority to use for recipes in the layer when the
124 OpenEmbedded build finds recipes of the same name in different
125 layers.
126
127 - :term:`LAYERVERSION`:
128 Establishes a version number for the layer. You can use this
129 version number to specify this exact version of the layer as a
130 dependency when using the
131 :term:`LAYERDEPENDS`
132 variable.
133
134 - :term:`LAYERDEPENDS`:
135 Lists all layers on which this layer depends (if any).
136
137 - :term:`LAYERSERIES_COMPAT`:
138 Lists the :yocto_wiki:`Yocto Project </wiki/Releases>`
139 releases for which the current version is compatible. This
140 variable is a good way to indicate if your particular layer is
141 current.
142
1434. *Add Content:* Depending on the type of layer, add the content. If
144 the layer adds support for a machine, add the machine configuration
145 in a ``conf/machine/`` file within the layer. If the layer adds
146 distro policy, add the distro configuration in a ``conf/distro/``
147 file within the layer. If the layer introduces new recipes, put the
148 recipes you need in ``recipes-*`` subdirectories within the layer.
149
150 .. note::
151
152 For an explanation of layer hierarchy that is compliant with the
153 Yocto Project, see the "
154 Example Filesystem Layout
155 " section in the Yocto Project Board Support Package (BSP)
156 Developer's Guide.
157
1585. *Optionally Test for Compatibility:* If you want permission to use
159 the Yocto Project Compatibility logo with your layer or application
160 that uses your layer, perform the steps to apply for compatibility.
161 See the "`Making Sure Your Layer is Compatible With Yocto
162 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
163 section for more information.
164
165.. _best-practices-to-follow-when-creating-layers:
166
167Following Best Practices When Creating Layers
168---------------------------------------------
169
170To create layers that are easier to maintain and that will not impact
171builds for other machines, you should consider the information in the
172following list:
173
174- *Avoid "Overlaying" Entire Recipes from Other Layers in Your
175 Configuration:* In other words, do not copy an entire recipe into
176 your layer and then modify it. Rather, use an append file
177 (``.bbappend``) to override only those parts of the original recipe
178 you need to modify.
179
180- *Avoid Duplicating Include Files:* Use append files (``.bbappend``)
181 for each recipe that uses an include file. Or, if you are introducing
182 a new recipe that requires the included file, use the path relative
183 to the original layer directory to refer to the file. For example,
184 use ``require recipes-core/``\ package\ ``/``\ file\ ``.inc`` instead
185 of ``require``\ file\ ``.inc``. If you're finding you have to overlay
186 the include file, it could indicate a deficiency in the include file
187 in the layer to which it originally belongs. If this is the case, you
188 should try to address that deficiency instead of overlaying the
189 include file. For example, you could address this by getting the
190 maintainer of the include file to add a variable or variables to make
191 it easy to override the parts needing to be overridden.
192
193- *Structure Your Layers:* Proper use of overrides within append files
194 and placement of machine-specific files within your layer can ensure
195 that a build is not using the wrong Metadata and negatively impacting
196 a build for a different machine. Following are some examples:
197
198 - *Modify Variables to Support a Different Machine:* Suppose you
199 have a layer named ``meta-one`` that adds support for building
200 machine "one". To do so, you use an append file named
201 ``base-files.bbappend`` and create a dependency on "foo" by
202 altering the :term:`DEPENDS`
203 variable:
204 ::
205
206 DEPENDS = "foo"
207
208 The dependency is created during any
209 build that includes the layer ``meta-one``. However, you might not
210 want this dependency for all machines. For example, suppose you
211 are building for machine "two" but your ``bblayers.conf`` file has
212 the ``meta-one`` layer included. During the build, the
213 ``base-files`` for machine "two" will also have the dependency on
214 ``foo``.
215
216 To make sure your changes apply only when building machine "one",
217 use a machine override with the ``DEPENDS`` statement: DEPENDS_one
218 = "foo" You should follow the same strategy when using ``_append``
219 and ``_prepend`` operations:
220 ::
221
222 DEPENDS_append_one = " foo"
223 DEPENDS_prepend_one = "foo "
224
225 As an actual example, here's a
226 snippet from the generic kernel include file ``linux-yocto.inc``,
227 wherein the kernel compile and link options are adjusted in the
228 case of a subset of the supported architectures:
229 ::
230
231 DEPENDS_append_aarch64 = " libgcc"
232 KERNEL_CC_append_aarch64 = " ${TOOLCHAIN_OPTIONS}"
233 KERNEL_LD_append_aarch64 = " ${TOOLCHAIN_OPTIONS}"
234
235 DEPENDS_append_nios2 = " libgcc"
236 KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
237 KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}"
238
239 DEPENDS_append_arc = " libgcc"
240 KERNEL_CC_append_arc = " ${TOOLCHAIN_OPTIONS}"
241 KERNEL_LD_append_arc = " ${TOOLCHAIN_OPTIONS}"
242
243 KERNEL_FEATURES_append_qemuall=" features/debug/printk.scc"
244
245 .. note::
246
247 Avoiding "+=" and "=+" and using machine-specific
248 \_append
249 and
250 \_prepend
251 operations is recommended as well.
252
253 - *Place Machine-Specific Files in Machine-Specific Locations:* When
254 you have a base recipe, such as ``base-files.bb``, that contains a
255 :term:`SRC_URI` statement to a
256 file, you can use an append file to cause the build to use your
257 own version of the file. For example, an append file in your layer
258 at ``meta-one/recipes-core/base-files/base-files.bbappend`` could
259 extend :term:`FILESPATH`
260 using
261 :term:`FILESEXTRAPATHS`
262 as follows: FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" The
263 build for machine "one" will pick up your machine-specific file as
264 long as you have the file in
265 ``meta-one/recipes-core/base-files/base-files/``. However, if you
266 are building for a different machine and the ``bblayers.conf``
267 file includes the ``meta-one`` layer and the location of your
268 machine-specific file is the first location where that file is
269 found according to ``FILESPATH``, builds for all machines will
270 also use that machine-specific file.
271
272 You can make sure that a machine-specific file is used for a
273 particular machine by putting the file in a subdirectory specific
274 to the machine. For example, rather than placing the file in
275 ``meta-one/recipes-core/base-files/base-files/`` as shown above,
276 put it in ``meta-one/recipes-core/base-files/base-files/one/``.
277 Not only does this make sure the file is used only when building
278 for machine "one", but the build process locates the file more
279 quickly.
280
281 In summary, you need to place all files referenced from
282 ``SRC_URI`` in a machine-specific subdirectory within the layer in
283 order to restrict those files to machine-specific builds.
284
285- *Perform Steps to Apply for Yocto Project Compatibility:* If you want
286 permission to use the Yocto Project Compatibility logo with your
287 layer or application that uses your layer, perform the steps to apply
288 for compatibility. See the "`Making Sure Your Layer is Compatible
289 With Yocto
290 Project <#making-sure-your-layer-is-compatible-with-yocto-project>`__"
291 section for more information.
292
293- *Follow the Layer Naming Convention:* Store custom layers in a Git
294 repository that use the ``meta-layer_name`` format.
295
296- *Group Your Layers Locally:* Clone your repository alongside other
297 cloned ``meta`` directories from the :term:`Source Directory`.
298
299Making Sure Your Layer is Compatible With Yocto Project
300-------------------------------------------------------
301
302When you create a layer used with the Yocto Project, it is advantageous
303to make sure that the layer interacts well with existing Yocto Project
304layers (i.e. the layer is compatible with the Yocto Project). Ensuring
305compatibility makes the layer easy to be consumed by others in the Yocto
306Project community and could allow you permission to use the Yocto
307Project Compatible Logo.
308
309.. note::
310
311 Only Yocto Project member organizations are permitted to use the
312 Yocto Project Compatible Logo. The logo is not available for general
313 use. For information on how to become a Yocto Project member
314 organization, see the
315 Yocto Project Website
316 .
317
318The Yocto Project Compatibility Program consists of a layer application
319process that requests permission to use the Yocto Project Compatibility
320Logo for your layer and application. The process consists of two parts:
321
3221. Successfully passing a script (``yocto-check-layer``) that when run
323 against your layer, tests it against constraints based on experiences
324 of how layers have worked in the real world and where pitfalls have
325 been found. Getting a "PASS" result from the script is required for
326 successful compatibility registration.
327
3282. Completion of an application acceptance form, which you can find at
329 https://www.yoctoproject.org/webform/yocto-project-compatible-registration.
330
331To be granted permission to use the logo, you need to satisfy the
332following:
333
334- Be able to check the box indicating that you got a "PASS" when
335 running the script against your layer.
336
337- Answer "Yes" to the questions on the form or have an acceptable
338 explanation for any questions answered "No".
339
340- Be a Yocto Project Member Organization.
341
342The remainder of this section presents information on the registration
343form and on the ``yocto-check-layer`` script.
344
345Yocto Project Compatible Program Application
346~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
347
348Use the form to apply for your layer's approval. Upon successful
349application, you can use the Yocto Project Compatibility Logo with your
350layer and the application that uses your layer.
351
352To access the form, use this link:
353https://www.yoctoproject.org/webform/yocto-project-compatible-registration.
354Follow the instructions on the form to complete your application.
355
356The application consists of the following sections:
357
358- *Contact Information:* Provide your contact information as the fields
359 require. Along with your information, provide the released versions
360 of the Yocto Project for which your layer is compatible.
361
362- *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the
363 items in the checklist. Space exists at the bottom of the form for
364 any explanations for items for which you answered "No".
365
366- *Recommendations:* Provide answers for the questions regarding Linux
367 kernel use and build success.
368
369``yocto-check-layer`` Script
370~~~~~~~~~~~~~~~~~~~~~~~~~~~~
371
372The ``yocto-check-layer`` script provides you a way to assess how
373compatible your layer is with the Yocto Project. You should run this
374script prior to using the form to apply for compatibility as described
375in the previous section. You need to achieve a "PASS" result in order to
376have your application form successfully processed.
377
378The script divides tests into three areas: COMMON, BSP, and DISTRO. For
379example, given a distribution layer (DISTRO), the layer must pass both
380the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP
381layer, the layer must pass the COMMON and BSP set of tests.
382
383To execute the script, enter the following commands from your build
384directory:
385::
386
387 $ source oe-init-build-env
388 $ yocto-check-layer your_layer_directory
389
390Be sure to provide the actual directory for your
391layer as part of the command.
392
393Entering the command causes the script to determine the type of layer
394and then to execute a set of specific tests against the layer. The
395following list overviews the test:
396
397- ``common.test_readme``: Tests if a ``README`` file exists in the
398 layer and the file is not empty.
399
400- ``common.test_parse``: Tests to make sure that BitBake can parse the
401 files without error (i.e. ``bitbake -p``).
402
403- ``common.test_show_environment``: Tests that the global or per-recipe
404 environment is in order without errors (i.e. ``bitbake -e``).
405
406- ``common.test_world``: Verifies that ``bitbake world`` works.
407
408- ``common.test_signatures``: Tests to be sure that BSP and DISTRO
409 layers do not come with recipes that change signatures.
410
411- ``common.test_layerseries_compat``: Verifies layer compatibility is
412 set properly.
413
414- ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine
415 configurations.
416
417- ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not
418 set the machine when the layer is added.
419
420- ``bsp.test_machine_world``: Verifies that ``bitbake world`` works
421 regardless of which machine is selected.
422
423- ``bsp.test_machine_signatures``: Verifies that building for a
424 particular machine affects only the signature of tasks specific to
425 that machine.
426
427- ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has
428 distro configurations.
429
430- ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer
431 does not set the distribution when the layer is added.
432
433Enabling Your Layer
434-------------------
435
436Before the OpenEmbedded build system can use your new layer, you need to
437enable it. To enable your layer, simply add your layer's path to the
438``BBLAYERS`` variable in your ``conf/bblayers.conf`` file, which is
439found in the :term:`Build Directory`.
440The following example shows how to enable a layer named
441``meta-mylayer``:
442::
443
444 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
445 # changes incompatibly
446 POKY_BBLAYERS_CONF_VERSION = "2"
447 BBPATH = "${TOPDIR}"
448 BBFILES ?= ""
449 BBLAYERS ?= " \
450 /home/user/poky/meta \
451 /home/user/poky/meta-poky \
452 /home/user/poky/meta-yocto-bsp \
453 /home/user/poky/meta-mylayer \
454 "
455
456BitBake parses each ``conf/layer.conf`` file from the top down as
457specified in the ``BBLAYERS`` variable within the ``conf/bblayers.conf``
458file. During the processing of each ``conf/layer.conf`` file, BitBake
459adds the recipes, classes and configurations contained within the
460particular layer to the source directory.
461
462.. _using-bbappend-files:
463
464Using .bbappend Files in Your Layer
465-----------------------------------
466
467A recipe that appends Metadata to another recipe is called a BitBake
468append file. A BitBake append file uses the ``.bbappend`` file type
469suffix, while the corresponding recipe to which Metadata is being
470appended uses the ``.bb`` file type suffix.
471
472You can use a ``.bbappend`` file in your layer to make additions or
473changes to the content of another layer's recipe without having to copy
474the other layer's recipe into your layer. Your ``.bbappend`` file
475resides in your layer, while the main ``.bb`` recipe file to which you
476are appending Metadata resides in a different layer.
477
478Being able to append information to an existing recipe not only avoids
479duplication, but also automatically applies recipe changes from a
480different layer into your layer. If you were copying recipes, you would
481have to manually merge changes as they occur.
482
483When you create an append file, you must use the same root name as the
484corresponding recipe file. For example, the append file
485``someapp_DISTRO.bbappend`` must apply to ``someapp_DISTRO.bb``. This
486means the original recipe and append file names are version
487number-specific. If the corresponding recipe is renamed to update to a
488newer version, you must also rename and possibly update the
489corresponding ``.bbappend`` as well. During the build process, BitBake
490displays an error on starting if it detects a ``.bbappend`` file that
491does not have a corresponding recipe with a matching name. See the
492:term:`BB_DANGLINGAPPENDS_WARNONLY`
493variable for information on how to handle this error.
494
495As an example, consider the main formfactor recipe and a corresponding
496formfactor append file both from the :term:`Source Directory`.
497Here is the main
498formfactor recipe, which is named ``formfactor_0.0.bb`` and located in
499the "meta" layer at ``meta/recipes-bsp/formfactor``:
500::
501
502 SUMMARY = "Device formfactor information"
503 SECTION = "base"
504 LICENSE = "MIT"
505 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
506 PR = "r45"
507
508 SRC_URI = "file://config file://machconfig"
509 S = "${WORKDIR}"
510
511 PACKAGE_ARCH = "${MACHINE_ARCH}"
512 INHIBIT_DEFAULT_DEPS = "1"
513
514 do_install() {
515 # Install file only if it has contents
516 install -d ${D}${sysconfdir}/formfactor/
517 install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
518 if [ -s "${S}/machconfig" ]; then
519 install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
520 fi
521 }
522
523In the main recipe, note the :term:`SRC_URI`
524variable, which tells the OpenEmbedded build system where to find files
525during the build.
526
527Following is the append file, which is named ``formfactor_0.0.bbappend``
528and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The
529file is in the layer at ``recipes-bsp/formfactor``:
530::
531
532 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
533
534By default, the build system uses the
535:term:`FILESPATH` variable to
536locate files. This append file extends the locations by setting the
537:term:`FILESEXTRAPATHS`
538variable. Setting this variable in the ``.bbappend`` file is the most
539reliable and recommended method for adding directories to the search
540path used by the build system to find files.
541
542The statement in this example extends the directories to include
543``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``,
544which resolves to a directory named ``formfactor`` in the same directory
545in which the append file resides (i.e.
546``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must
547have the supporting directory structure set up that will contain any
548files or patches you will be including from the layer.
549
550Using the immediate expansion assignment operator ``:=`` is important
551because of the reference to ``THISDIR``. The trailing colon character is
552important as it ensures that items in the list remain colon-separated.
553
554.. note::
555
556 BitBake automatically defines the ``THISDIR`` variable. You should
557 never set this variable yourself. Using "_prepend" as part of the
558 ``FILESEXTRAPATHS`` ensures your path will be searched prior to other
559 paths in the final list.
560
561 Also, not all append files add extra files. Many append files simply
562 exist to add build options (e.g. ``systemd``). For these cases, your
563 append file would not even use the ``FILESEXTRAPATHS`` statement.
564
565Prioritizing Your Layer
566-----------------------
567
568Each layer is assigned a priority value. Priority values control which
569layer takes precedence if there are recipe files with the same name in
570multiple layers. For these cases, the recipe file from the layer with a
571higher priority number takes precedence. Priority values also affect the
572order in which multiple ``.bbappend`` files for the same recipe are
573applied. You can either specify the priority manually, or allow the
574build system to calculate it based on the layer's dependencies.
575
576To specify the layer's priority manually, use the
577:term:`BBFILE_PRIORITY`
578variable and append the layer's root name:
579::
580
581 BBFILE_PRIORITY_mylayer = "1"
582
583.. note::
584
585 It is possible for a recipe with a lower version number
586 :term:`PV` in a layer that has a higher
587 priority to take precedence.
588
589 Also, the layer priority does not currently affect the precedence
590 order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake
591 might address this.
592
593Managing Layers
594---------------
595
596You can use the BitBake layer management tool ``bitbake-layers`` to
597provide a view into the structure of recipes across a multi-layer
598project. Being able to generate output that reports on configured layers
599with their paths and priorities and on ``.bbappend`` files and their
600applicable recipes can help to reveal potential problems.
601
602For help on the BitBake layer management tool, use the following
603command:
604::
605
606 $ bitbake-layers --help NOTE: Starting bitbake server... usage:
607 NOTE: Starting bitbake server...
608 usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ...
609
610 BitBake layers utility
611
612 optional arguments:
613 -d, --debug Enable debug output
614 -q, --quiet Print only errors
615 -F, --force Force add without recipe parse verification
616 --color COLOR Colorize output (where COLOR is auto, always, never)
617 -h, --help show this help message and exit
618
619 subcommands:
620 <subcommand>
621 layerindex-fetch Fetches a layer from a layer index along with its
622 dependent layers, and adds them to conf/bblayers.conf.
623 layerindex-show-depends
624 Find layer dependencies from layer index.
625 add-layer Add one or more layers to bblayers.conf.
626 remove-layer Remove one or more layers from bblayers.conf.
627 flatten flatten layer configuration into a separate output
628 directory.
629 show-layers show current configured layers.
630 show-overlayed list overlayed recipes (where the same recipe exists
631 in another layer)
632 show-recipes list available recipes, showing the layer they are
633 provided by
634 show-appends list bbappend files and recipe files they apply to
635 show-cross-depends Show dependencies between recipes that cross layer
636 boundaries.
637 create-layer Create a basic layer
638
639 Use bitbake-layers <subcommand> --help to get help on a specific command
640
641The following list describes the available commands:
642
643- ``help:`` Displays general help or help on a specified command.
644
645- ``show-layers:`` Shows the current configured layers.
646
647- ``show-overlayed:`` Lists overlayed recipes. A recipe is overlayed
648 when a recipe with the same name exists in another layer that has a
649 higher layer priority.
650
651- ``show-recipes:`` Lists available recipes and the layers that
652 provide them.
653
654- ``show-appends:`` Lists ``.bbappend`` files and the recipe files to
655 which they apply.
656
657- ``show-cross-depends:`` Lists dependency relationships between
658 recipes that cross layer boundaries.
659
660- ``add-layer:`` Adds a layer to ``bblayers.conf``.
661
662- ``remove-layer:`` Removes a layer from ``bblayers.conf``
663
664- ``flatten:`` Flattens the layer configuration into a separate
665 output directory. Flattening your layer configuration builds a
666 "flattened" directory that contains the contents of all layers, with
667 any overlayed recipes removed and any ``.bbappend`` files appended to
668 the corresponding recipes. You might have to perform some manual
669 cleanup of the flattened layer as follows:
670
671 - Non-recipe files (such as patches) are overwritten. The flatten
672 command shows a warning for these files.
673
674 - Anything beyond the normal layer setup has been added to the
675 ``layer.conf`` file. Only the lowest priority layer's
676 ``layer.conf`` is used.
677
678 - Overridden and appended items from ``.bbappend`` files need to be
679 cleaned up. The contents of each ``.bbappend`` end up in the
680 flattened recipe. However, if there are appended or changed
681 variable values, you need to tidy these up yourself. Consider the
682 following example. Here, the ``bitbake-layers`` command adds the
683 line ``#### bbappended ...`` so that you know where the following
684 lines originate:
685 ::
686
687 ...
688 DESCRIPTION = "A useful utility"
689 ...
690 EXTRA_OECONF = "--enable-something"
691 ...
692
693 #### bbappended from meta-anotherlayer ####
694
695 DESCRIPTION = "Customized utility"
696 EXTRA_OECONF += "--enable-somethingelse"
697
698
699 Ideally, you would tidy up these utilities as follows:
700 ::
701
702 ...
703 DESCRIPTION = "Customized utility"
704 ...
705 EXTRA_OECONF = "--enable-something --enable-somethingelse"
706 ...
707
708- ``layerindex-fetch``: Fetches a layer from a layer index, along
709 with its dependent layers, and adds the layers to the
710 ``conf/bblayers.conf`` file.
711
712- ``layerindex-show-depends``: Finds layer dependencies from the
713 layer index.
714
715- ``create-layer``: Creates a basic layer.
716
717Creating a General Layer Using the ``bitbake-layers`` Script
718------------------------------------------------------------
719
720The ``bitbake-layers`` script with the ``create-layer`` subcommand
721simplifies creating a new general layer.
722
723.. note::
724
725 - For information on BSP layers, see the ":ref:`bsp-guide/bsp:bsp layers`"
726 section in the Yocto
727 Project Board Specific (BSP) Developer's Guide.
728
729 - In order to use a layer with the OpenEmbedded build system, you
730 need to add the layer to your ``bblayers.conf`` configuration
731 file. See the ":ref:`dev-manual/dev-manual-common-tasks:adding a layer using the \`\`bitbake-layers\`\` script`"
732 section for more information.
733
734The default mode of the script's operation with this subcommand is to
735create a layer with the following:
736
737- A layer priority of 6.
738
739- A ``conf`` subdirectory that contains a ``layer.conf`` file.
740
741- A ``recipes-example`` subdirectory that contains a further
742 subdirectory named ``example``, which contains an ``example.bb``
743 recipe file.
744
745- A ``COPYING.MIT``, which is the license statement for the layer. The
746 script assumes you want to use the MIT license, which is typical for
747 most layers, for the contents of the layer itself.
748
749- A ``README`` file, which is a file describing the contents of your
750 new layer.
751
752In its simplest form, you can use the following command form to create a
753layer. The command creates a layer whose name corresponds to
754your_layer_name in the current directory: $ bitbake-layers create-layer
755your_layer_name As an example, the following command creates a layer
756named ``meta-scottrif`` in your home directory:
757::
758
759 $ cd /usr/home
760 $ bitbake-layers create-layer meta-scottrif
761 NOTE: Starting bitbake server...
762 Add your new layer with 'bitbake-layers add-layer meta-scottrif'
763
764If you want to set the priority of the layer to other than the default
765value of "6", you can either use the ``DASHDASHpriority`` option or you
766can edit the
767:term:`BBFILE_PRIORITY` value
768in the ``conf/layer.conf`` after the script creates it. Furthermore, if
769you want to give the example recipe file some name other than the
770default, you can use the ``DASHDASHexample-recipe-name`` option.
771
772The easiest way to see how the ``bitbake-layers create-layer`` command
773works is to experiment with the script. You can also read the usage
774information by entering the following:
775::
776
777 $ bitbake-layers create-layer --help
778 NOTE: Starting bitbake server...
779 usage: bitbake-layers create-layer [-h] [--priority PRIORITY]
780 [--example-recipe-name EXAMPLERECIPE]
781 layerdir
782
783 Create a basic layer
784
785 positional arguments:
786 layerdir Layer directory to create
787
788 optional arguments:
789 -h, --help show this help message and exit
790 --priority PRIORITY, -p PRIORITY
791 Layer directory to create
792 --example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE
793 Filename of the example recipe
794
795Adding a Layer Using the ``bitbake-layers`` Script
796--------------------------------------------------
797
798Once you create your general layer, you must add it to your
799``bblayers.conf`` file. Adding the layer to this configuration file
800makes the OpenEmbedded build system aware of your layer so that it can
801search it for metadata.
802
803Add your layer by using the ``bitbake-layers add-layer`` command:
804::
805
806 $ bitbake-layers add-layer your_layer_name
807
808Here is an example that adds a
809layer named ``meta-scottrif`` to the configuration file. Following the
810command that adds the layer is another ``bitbake-layers`` command that
811shows the layers that are in your ``bblayers.conf`` file:
812::
813
814 $ bitbake-layers add-layer meta-scottrif
815 NOTE: Starting bitbake server...
816 Parsing recipes: 100% |##########################################################| Time: 0:00:49
817 Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors.
818 $ bitbake-layers show-layers
819 NOTE: Starting bitbake server...
820 layer path priority
821 ==========================================================================
822 meta /home/scottrif/poky/meta 5
823 meta-poky /home/scottrif/poky/meta-poky 5
824 meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5
825 workspace /home/scottrif/poky/build/workspace 99
826 meta-scottrif /home/scottrif/poky/build/meta-scottrif 6
827
828
829Adding the layer to this file
830enables the build system to locate the layer during the build.
831
832.. note::
833
834 During a build, the OpenEmbedded build system looks in the layers
835 from the top of the list down to the bottom in that order.
836
837.. _usingpoky-extend-customimage:
838
839Customizing Images
840==================
841
842You can customize images to satisfy particular requirements. This
843section describes several methods and provides guidelines for each.
844
845.. _usingpoky-extend-customimage-localconf:
846
847Customizing Images Using ``local.conf``
848---------------------------------------
849
850Probably the easiest way to customize an image is to add a package by
851way of the ``local.conf`` configuration file. Because it is limited to
852local use, this method generally only allows you to add packages and is
853not as flexible as creating your own customized image. When you add
854packages using local variables this way, you need to realize that these
855variable changes are in effect for every build and consequently affect
856all images, which might not be what you require.
857
858To add a package to your image using the local configuration file, use
859the ``IMAGE_INSTALL`` variable with the ``_append`` operator:
860::
861
862 IMAGE_INSTALL_append = " strace"
863
864Use of the syntax is important -
865specifically, the space between the quote and the package name, which is
866``strace`` in this example. This space is required since the ``_append``
867operator does not add the space.
868
869Furthermore, you must use ``_append`` instead of the ``+=`` operator if
870you want to avoid ordering issues. The reason for this is because doing
871so unconditionally appends to the variable and avoids ordering problems
872due to the variable being set in image recipes and ``.bbclass`` files
873with operators like ``?=``. Using ``_append`` ensures the operation
874takes affect.
875
876As shown in its simplest use, ``IMAGE_INSTALL_append`` affects all
877images. It is possible to extend the syntax so that the variable applies
878to a specific image only. Here is an example:
879IMAGE_INSTALL_append_pn-core-image-minimal = " strace" This example adds
880``strace`` to the ``core-image-minimal`` image only.
881
882You can add packages using a similar approach through the
883``CORE_IMAGE_EXTRA_INSTALL`` variable. If you use this variable, only
884``core-image-*`` images are affected.
885
886.. _usingpoky-extend-customimage-imagefeatures:
887
888Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES``
889-------------------------------------------------------------------------------
890
891Another method for customizing your image is to enable or disable
892high-level image features by using the
893:term:`IMAGE_FEATURES` and
894:term:`EXTRA_IMAGE_FEATURES`
895variables. Although the functions for both variables are nearly
896equivalent, best practices dictate using ``IMAGE_FEATURES`` from within
897a recipe and using ``EXTRA_IMAGE_FEATURES`` from within your
898``local.conf`` file, which is found in the
899:term:`Build Directory`.
900
901To understand how these features work, the best reference is
902``meta/classes/core-image.bbclass``. This class lists out the available
903``IMAGE_FEATURES`` of which most map to package groups while some, such
904as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general
905configuration settings.
906
907In summary, the file looks at the contents of the ``IMAGE_FEATURES``
908variable and then maps or configures the feature accordingly. Based on
909this information, the build system automatically adds the appropriate
910packages or configurations to the
911:term:`IMAGE_INSTALL` variable.
912Effectively, you are enabling extra features by extending the class or
913creating a custom class for use with specialized image ``.bb`` files.
914
915Use the ``EXTRA_IMAGE_FEATURES`` variable from within your local
916configuration file. Using a separate area from which to enable features
917with this variable helps you avoid overwriting the features in the image
918recipe that are enabled with ``IMAGE_FEATURES``. The value of
919``EXTRA_IMAGE_FEATURES`` is added to ``IMAGE_FEATURES`` within
920``meta/conf/bitbake.conf``.
921
922To illustrate how you can use these variables to modify your image,
923consider an example that selects the SSH server. The Yocto Project ships
924with two SSH servers you can use with your images: Dropbear and OpenSSH.
925Dropbear is a minimal SSH server appropriate for resource-constrained
926environments, while OpenSSH is a well-known standard SSH server
927implementation. By default, the ``core-image-sato`` image is configured
928to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb``
929images both include OpenSSH. The ``core-image-minimal`` image does not
930contain an SSH server.
931
932You can customize your image and change these defaults. Edit the
933``IMAGE_FEATURES`` variable in your recipe or use the
934``EXTRA_IMAGE_FEATURES`` in your ``local.conf`` file so that it
935configures the image you are working with to include
936``ssh-server-dropbear`` or ``ssh-server-openssh``.
937
938.. note::
939
940 See the "
941 Images
942 " section in the Yocto Project Reference Manual for a complete list
943 of image features that ship with the Yocto Project.
944
945.. _usingpoky-extend-customimage-custombb:
946
947Customizing Images Using Custom .bb Files
948-----------------------------------------
949
950You can also customize an image by creating a custom recipe that defines
951additional software as part of the image. The following example shows
952the form for the two lines you need:
953::
954
955 IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
956 inherit core-image
957
958Defining the software using a custom recipe gives you total control over
959the contents of the image. It is important to use the correct names of
960packages in the ``IMAGE_INSTALL`` variable. You must use the
961OpenEmbedded notation and not the Debian notation for the names (e.g.
962``glibc-dev`` instead of ``libc6-dev``).
963
964The other method for creating a custom image is to base it on an
965existing image. For example, if you want to create an image based on
966``core-image-sato`` but add the additional package ``strace`` to the
967image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new
968``.bb`` and add the following line to the end of the copy:
969::
970
971 IMAGE_INSTALL += "strace"
972
973.. _usingpoky-extend-customimage-customtasks:
974
975Customizing Images Using Custom Package Groups
976----------------------------------------------
977
978For complex custom images, the best approach for customizing an image is
979to create a custom package group recipe that is used to build the image
980or images. A good example of a package group recipe is
981``meta/recipes-core/packagegroups/packagegroup-base.bb``.
982
983If you examine that recipe, you see that the ``PACKAGES`` variable lists
984the package group packages to produce. The ``inherit packagegroup``
985statement sets appropriate default values and automatically adds
986``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each
987package specified in the ``PACKAGES`` statement.
988
989.. note::
990
991 The
992 inherit packagegroup
993 line should be located near the top of the recipe, certainly before
994 the
995 PACKAGES
996 statement.
997
998For each package you specify in ``PACKAGES``, you can use ``RDEPENDS``
999and ``RRECOMMENDS`` entries to provide a list of packages the parent
1000task package should contain. You can see examples of these further down
1001in the ``packagegroup-base.bb`` recipe.
1002
1003Here is a short, fabricated example showing the same basic pieces for a
1004hypothetical packagegroup defined in ``packagegroup-custom.bb``, where
1005the variable ``PN`` is the standard way to abbreviate the reference to
1006the full packagegroup name ``packagegroup-custom``:
1007::
1008
1009 DESCRIPTION = "My Custom Package Groups"
1010
1011 inherit packagegroup
1012
1013 PACKAGES = "\
1014 ${PN}-apps \
1015 ${PN}-tools \
1016 "
1017
1018 RDEPENDS_${PN}-apps = "\
1019 dropbear \
1020 portmap \
1021 psplash"
1022
1023 RDEPENDS_${PN}-tools = "\
1024 oprofile \
1025 oprofileui-server \
1026 lttng-tools"
1027
1028 RRECOMMENDS_${PN}-tools = "\
1029 kernel-module-oprofile"
1030
1031In the previous example, two package group packages are created with
1032their dependencies and their recommended package dependencies listed:
1033``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To
1034build an image using these package group packages, you need to add
1035``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to
1036``IMAGE_INSTALL``. For other forms of image dependencies see the other
1037areas of this section.
1038
1039.. _usingpoky-extend-customimage-image-name:
1040
1041Customizing an Image Hostname
1042-----------------------------
1043
1044By default, the configured hostname (i.e. ``/etc/hostname``) in an image
1045is the same as the machine name. For example, if
1046:term:`MACHINE` equals "qemux86", the
1047configured hostname written to ``/etc/hostname`` is "qemux86".
1048
1049You can customize this name by altering the value of the "hostname"
1050variable in the ``base-files`` recipe using either an append file or a
1051configuration file. Use the following in an append file:
1052::
1053
1054 hostname = "myhostname"
1055
1056Use the following in a configuration file:
1057::
1058
1059 hostname_pn-base-files = "myhostname"
1060
1061Changing the default value of the variable "hostname" can be useful in
1062certain situations. For example, suppose you need to do extensive
1063testing on an image and you would like to easily identify the image
1064under test from existing images with typical default hostnames. In this
1065situation, you could change the default hostname to "testme", which
1066results in all the images using the name "testme". Once testing is
1067complete and you do not need to rebuild the image for test any longer,
1068you can easily reset the default hostname.
1069
1070Another point of interest is that if you unset the variable, the image
1071will have no default hostname in the filesystem. Here is an example that
1072unsets the variable in a configuration file:
1073::
1074
1075 hostname_pn-base-files = ""
1076
1077Having no default hostname in the filesystem is suitable for
1078environments that use dynamic hostnames such as virtual machines.
1079
1080.. _new-recipe-writing-a-new-recipe:
1081
1082Writing a New Recipe
1083====================
1084
1085Recipes (``.bb`` files) are fundamental components in the Yocto Project
1086environment. Each software component built by the OpenEmbedded build
1087system requires a recipe to define the component. This section describes
1088how to create, write, and test a new recipe.
1089
1090.. note::
1091
1092 For information on variables that are useful for recipes and for
1093 information about recipe naming issues, see the "
1094 Required
1095 " section of the Yocto Project Reference Manual.
1096
1097.. _new-recipe-overview:
1098
1099Overview
1100--------
1101
1102The following figure shows the basic process for creating a new recipe.
1103The remainder of the section provides details for the steps.
1104
1105.. image:: figures/recipe-workflow.png
1106 :align: center
1107
1108.. _new-recipe-locate-or-automatically-create-a-base-recipe:
1109
1110Locate or Automatically Create a Base Recipe
1111--------------------------------------------
1112
1113You can always write a recipe from scratch. However, three choices exist
1114that can help you quickly get a start on a new recipe:
1115
1116- ``devtool add``: A command that assists in creating a recipe and an
1117 environment conducive to development.
1118
1119- ``recipetool create``: A command provided by the Yocto Project that
1120 automates creation of a base recipe based on the source files.
1121
1122- *Existing Recipes:* Location and modification of an existing recipe
1123 that is similar in function to the recipe you need.
1124
1125.. note::
1126
1127 For information on recipe syntax, see the "
1128 Recipe Syntax
1129 " section.
1130
1131.. _new-recipe-creating-the-base-recipe-using-devtool:
1132
1133Creating the Base Recipe Using ``devtool add``
1134~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1135
1136The ``devtool add`` command uses the same logic for auto-creating the
1137recipe as ``recipetool create``, which is listed below. Additionally,
1138however, ``devtool add`` sets up an environment that makes it easy for
1139you to patch the source and to make changes to the recipe as is often
1140necessary when adding a recipe to build a new piece of software to be
1141included in a build.
1142
1143You can find a complete description of the ``devtool add`` command in
1144the ":ref:`sdk-a-closer-look-at-devtool-add`" section
1145in the Yocto Project Application Development and the Extensible Software
1146Development Kit (eSDK) manual.
1147
1148.. _new-recipe-creating-the-base-recipe-using-recipetool:
1149
1150Creating the Base Recipe Using ``recipetool create``
1151~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1152
1153``recipetool create`` automates creation of a base recipe given a set of
1154source code files. As long as you can extract or point to the source
1155files, the tool will construct a recipe and automatically configure all
1156pre-build information into the recipe. For example, suppose you have an
1157application that builds using Autotools. Creating the base recipe using
1158``recipetool`` results in a recipe that has the pre-build dependencies,
1159license requirements, and checksums configured.
1160
1161To run the tool, you just need to be in your
1162:term:`Build Directory` and have sourced the
1163build environment setup script (i.e.
1164`:ref:`structure-core-script`).
1165To get help on the tool, use the following command:
1166::
1167
1168 $ recipetool -h
1169 NOTE: Starting bitbake server...
1170 usage: recipetool [-d] [-q] [--color COLOR] [-h] <subcommand> ...
1171
1172 OpenEmbedded recipe tool
1173
1174 options:
1175 -d, --debug Enable debug output
1176 -q, --quiet Print only errors
1177 --color COLOR Colorize output (where COLOR is auto, always, never)
1178 -h, --help show this help message and exit
1179
1180 subcommands:
1181 create Create a new recipe
1182 newappend Create a bbappend for the specified target in the specified
1183 layer
1184 setvar Set a variable within a recipe
1185 appendfile Create/update a bbappend to replace a target file
1186 appendsrcfiles Create/update a bbappend to add or replace source files
1187 appendsrcfile Create/update a bbappend to add or replace a source file
1188 Use recipetool <subcommand> --help to get help on a specific command
1189
1190Running ``recipetool create -o`` OUTFILE creates the base recipe and
1191locates it properly in the layer that contains your source files.
1192Following are some syntax examples:
1193
1194Use this syntax to generate a recipe based on source. Once generated,
1195the recipe resides in the existing source code layer:
1196::
1197
1198 recipetool create -o OUTFILE source
1199
1200Use this syntax to generate a recipe using code that
1201you extract from source. The extracted code is placed in its own layer
1202defined by EXTERNALSRC.
1203::
1204
1205 recipetool create -o OUTFILE -x EXTERNALSRC source
1206
1207Use this syntax to generate a recipe based on source. The options
1208direct ``recipetool`` to generate debugging information. Once generated,
1209the recipe resides in the existing source code layer:
1210::
1211
1212 recipetool create -d -o OUTFILE source
1213
1214.. _new-recipe-locating-and-using-a-similar-recipe:
1215
1216Locating and Using a Similar Recipe
1217~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1218
1219Before writing a recipe from scratch, it is often useful to discover
1220whether someone else has already written one that meets (or comes close
1221to meeting) your needs. The Yocto Project and OpenEmbedded communities
1222maintain many recipes that might be candidates for what you are doing.
1223You can find a good central index of these recipes in the `OpenEmbedded
1224Layer Index <http://layers.openembedded.org>`__.
1225
1226Working from an existing recipe or a skeleton recipe is the best way to
1227get started. Here are some points on both methods:
1228
1229- *Locate and modify a recipe that is close to what you want to do:*
1230 This method works when you are familiar with the current recipe
1231 space. The method does not work so well for those new to the Yocto
1232 Project or writing recipes.
1233
1234 Some risks associated with this method are using a recipe that has
1235 areas totally unrelated to what you are trying to accomplish with
1236 your recipe, not recognizing areas of the recipe that you might have
1237 to add from scratch, and so forth. All these risks stem from
1238 unfamiliarity with the existing recipe space.
1239
1240- *Use and modify the following skeleton recipe:* If for some reason
1241 you do not want to use ``recipetool`` and you cannot find an existing
1242 recipe that is close to meeting your needs, you can use the following
1243 structure to provide the fundamental areas of a new recipe.
1244 ::
1245
1246 DESCRIPTION = ""
1247 HOMEPAGE = ""
1248 LICENSE = ""
1249 SECTION = ""
1250 DEPENDS = ""
1251 LIC_FILES_CHKSUM = ""
1252
1253 SRC_URI = ""
1254
1255.. _new-recipe-storing-and-naming-the-recipe:
1256
1257Storing and Naming the Recipe
1258-----------------------------
1259
1260Once you have your base recipe, you should put it in your own layer and
1261name it appropriately. Locating it correctly ensures that the
1262OpenEmbedded build system can find it when you use BitBake to process
1263the recipe.
1264
1265- *Storing Your Recipe:* The OpenEmbedded build system locates your
1266 recipe through the layer's ``conf/layer.conf`` file and the
1267 :term:`BBFILES` variable. This
1268 variable sets up a path from which the build system can locate
1269 recipes. Here is the typical use:
1270 ::
1271
1272 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
1273 ${LAYERDIR}/recipes-*/*/*.bbappend"
1274
1275 Consequently, you need to be sure you locate your new recipe inside
1276 your layer such that it can be found.
1277
1278 You can find more information on how layers are structured in the
1279 "`Understanding and Creating
1280 Layers <#understanding-and-creating-layers>`__" section.
1281
1282- *Naming Your Recipe:* When you name your recipe, you need to follow
1283 this naming convention: basename_version.bb Use lower-cased
1284 characters and do not include the reserved suffixes ``-native``,
1285 ``-cross``, ``-initial``, or ``-dev`` casually (i.e. do not use them
1286 as part of your recipe name unless the string applies). Here are some
1287 examples:
1288 ::
1289
1290 cups_1.7.0.bb
1291 gawk_4.0.2.bb
1292 irssi_0.8.16-rc1.bb
1293
1294.. _new-recipe-running-a-build-on-the-recipe:
1295
1296Running a Build on the Recipe
1297-----------------------------
1298
1299Creating a new recipe is usually an iterative process that requires
1300using BitBake to process the recipe multiple times in order to
1301progressively discover and add information to the recipe file.
1302
1303Assuming you have sourced the build environment setup script (i.e.
1304:ref:`structure-core-script`) and you are in
1305the :term:`Build Directory`, use
1306BitBake to process your recipe. All you need to provide is the
1307``basename`` of the recipe as described in the previous section:
1308::
1309
1310 $ bitbake basename
1311
1312During the build, the OpenEmbedded build system creates a temporary work
1313directory for each recipe
1314(``${``\ :term:`WORKDIR`\ ``}``)
1315where it keeps extracted source files, log files, intermediate
1316compilation and packaging files, and so forth.
1317
1318The path to the per-recipe temporary work directory depends on the
1319context in which it is being built. The quickest way to find this path
1320is to have BitBake return it by running the following:
1321::
1322
1323 $ bitbake -e basename \| grep ^WORKDIR=
1324
1325As an example, assume a Source Directory
1326top-level folder named ``poky``, a default Build Directory at
1327``poky/build``, and a ``qemux86-poky-linux`` machine target system.
1328Furthermore, suppose your recipe is named ``foo_1.3.0.bb``. In this
1329case, the work directory the build system uses to build the package
1330would be as follows: poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
1331Inside this directory you can find sub-directories such as ``image``,
1332``packages-split``, and ``temp``. After the build, you can examine these
1333to determine how well the build went.
1334
1335.. note::
1336
1337 You can find log files for each task in the recipe's
1338 temp
1339 directory (e.g.
1340 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp
1341 ). Log files are named
1342 log.
1343 taskname
1344 (e.g.
1345 log.do_configure
1346 ,
1347 log.do_fetch
1348 , and
1349 log.do_compile
1350 ).
1351
1352You can find more information about the build process in
1353":doc:`../overview-manual/overview-manual-development-environment`"
1354chapter of the Yocto Project Overview and Concepts Manual.
1355
1356.. _new-recipe-fetching-code:
1357
1358Fetching Code
1359-------------
1360
1361The first thing your recipe must do is specify how to fetch the source
1362files. Fetching is controlled mainly through the
1363:term:`SRC_URI` variable. Your recipe
1364must have a ``SRC_URI`` variable that points to where the source is
1365located. For a graphical representation of source locations, see the
1366":ref:`sources-dev-environment`" section in
1367the Yocto Project Overview and Concepts Manual.
1368
1369The :ref:`ref-tasks-fetch` task uses
1370the prefix of each entry in the ``SRC_URI`` variable value to determine
1371which :ref:`fetcher <bitbake:bb-fetchers>` to use to get your
1372source files. It is the ``SRC_URI`` variable that triggers the fetcher.
1373The :ref:`ref-tasks-patch` task uses
1374the variable after source is fetched to apply patches. The OpenEmbedded
1375build system uses
1376:term:`FILESOVERRIDES` for
1377scanning directory locations for local files in ``SRC_URI``.
1378
1379The ``SRC_URI`` variable in your recipe must define each unique location
1380for your source files. It is good practice to not hard-code version
1381numbers in a URL used in ``SRC_URI``. Rather than hard-code these
1382values, use ``${``\ :term:`PV`\ ``}``,
1383which causes the fetch process to use the version specified in the
1384recipe filename. Specifying the version in this manner means that
1385upgrading the recipe to a future version is as simple as renaming the
1386recipe to match the new version.
1387
1388Here is a simple example from the
1389``meta/recipes-devtools/strace/strace_5.5.bb`` recipe where the source
1390comes from a single tarball. Notice the use of the
1391:term:`PV` variable:
1392::
1393
1394 SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \\
1395
1396Files mentioned in ``SRC_URI`` whose names end in a typical archive
1397extension (e.g. ``.tar``, ``.tar.gz``, ``.tar.bz2``, ``.zip``, and so
1398forth), are automatically extracted during the
1399:ref:`ref-tasks-unpack` task. For
1400another example that specifies these types of files, see the
1401"`Autotooled Package <#new-recipe-autotooled-package>`__" section.
1402
1403Another way of specifying source is from an SCM. For Git repositories,
1404you must specify :term:`SRCREV` and
1405you should specify :term:`PV` to include
1406the revision with :term:`SRCPV`. Here
1407is an example from the recipe
1408``meta/recipes-kernel/blktrace/blktrace_git.bb``:
1409::
1410
1411 SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385"
1412
1413 PR = "r6"
1414 PV = "1.0.5+git${SRCPV}"
1415
1416 SRC_URI = "git://git.kernel.dk/blktrace.git \
1417 file://ldflags.patch"
1418
1419If your ``SRC_URI`` statement includes URLs pointing to individual files
1420fetched from a remote server other than a version control system,
1421BitBake attempts to verify the files against checksums defined in your
1422recipe to ensure they have not been tampered with or otherwise modified
1423since the recipe was written. Two checksums are used:
1424``SRC_URI[md5sum]`` and ``SRC_URI[sha256sum]``.
1425
1426If your ``SRC_URI`` variable points to more than a single URL (excluding
1427SCM URLs), you need to provide the ``md5`` and ``sha256`` checksums for
1428each URL. For these cases, you provide a name for each URL as part of
1429the ``SRC_URI`` and then reference that name in the subsequent checksum
1430statements. Here is an example combining lines from the files
1431``git.inc`` and ``git_2.24.1.bb``:
1432::
1433
1434 SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \
1435 ${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages"
1436
1437 SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b"
1438 SRC_URI[tarball.sha256sum] = "ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02"
1439 SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d"
1440 SRC_URI[manpages.sha256sum] = "9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230"
1441
1442Proper values for ``md5`` and ``sha256`` checksums might be available
1443with other signatures on the download page for the upstream source (e.g.
1444``md5``, ``sha1``, ``sha256``, ``GPG``, and so forth). Because the
1445OpenEmbedded build system only deals with ``sha256sum`` and ``md5sum``,
1446you should verify all the signatures you find by hand.
1447
1448If no ``SRC_URI`` checksums are specified when you attempt to build the
1449recipe, or you provide an incorrect checksum, the build will produce an
1450error for each missing or incorrect checksum. As part of the error
1451message, the build system provides the checksum string corresponding to
1452the fetched file. Once you have the correct checksums, you can copy and
1453paste them into your recipe and then run the build again to continue.
1454
1455.. note::
1456
1457 As mentioned, if the upstream source provides signatures for
1458 verifying the downloaded source code, you should verify those
1459 manually before setting the checksum values in the recipe and
1460 continuing with the build.
1461
1462This final example is a bit more complicated and is from the
1463``meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb`` recipe. The
1464example's ``SRC_URI`` statement identifies multiple files as the source
1465files for the recipe: a tarball, a patch file, a desktop file, and an
1466icon.
1467::
1468
1469 SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \
1470 file://xwc.patch \
1471 file://rxvt.desktop \
1472 file://rxvt.png"
1473
1474When you specify local files using the ``file://`` URI protocol, the
1475build system fetches files from the local machine. The path is relative
1476to the :term:`FILESPATH` variable
1477and searches specific directories in a certain order:
1478``${``\ :term:`BP`\ ``}``,
1479``${``\ :term:`BPN`\ ``}``, and
1480``files``. The directories are assumed to be subdirectories of the
1481directory in which the recipe or append file resides. For another
1482example that specifies these types of files, see the "`Single .c File
1483Package (Hello
1484World!) <#new-recipe-single-c-file-package-hello-world>`__" section.
1485
1486The previous example also specifies a patch file. Patch files are files
1487whose names usually end in ``.patch`` or ``.diff`` but can end with
1488compressed suffixes such as ``diff.gz`` and ``patch.bz2``, for example.
1489The build system automatically applies patches as described in the
1490"`Patching Code <#new-recipe-patching-code>`__" section.
1491
1492.. _new-recipe-unpacking-code:
1493
1494Unpacking Code
1495--------------
1496
1497During the build, the
1498:ref:`ref-tasks-unpack` task unpacks
1499the source with ``${``\ :term:`S`\ ``}``
1500pointing to where it is unpacked.
1501
1502If you are fetching your source files from an upstream source archived
1503tarball and the tarball's internal structure matches the common
1504convention of a top-level subdirectory named
1505``${``\ :term:`BPN`\ ``}-${``\ :term:`PV`\ ``}``,
1506then you do not need to set ``S``. However, if ``SRC_URI`` specifies to
1507fetch source from an archive that does not use this convention, or from
1508an SCM like Git or Subversion, your recipe needs to define ``S``.
1509
1510If processing your recipe using BitBake successfully unpacks the source
1511files, you need to be sure that the directory pointed to by ``${S}``
1512matches the structure of the source.
1513
1514.. _new-recipe-patching-code:
1515
1516Patching Code
1517-------------
1518
1519Sometimes it is necessary to patch code after it has been fetched. Any
1520files mentioned in ``SRC_URI`` whose names end in ``.patch`` or
1521``.diff`` or compressed versions of these suffixes (e.g. ``diff.gz`` are
1522treated as patches. The
1523:ref:`ref-tasks-patch` task
1524automatically applies these patches.
1525
1526The build system should be able to apply patches with the "-p1" option
1527(i.e. one directory level in the path will be stripped off). If your
1528patch needs to have more directory levels stripped off, specify the
1529number of levels using the "striplevel" option in the ``SRC_URI`` entry
1530for the patch. Alternatively, if your patch needs to be applied in a
1531specific subdirectory that is not specified in the patch file, use the
1532"patchdir" option in the entry.
1533
1534As with all local files referenced in
1535:term:`SRC_URI` using ``file://``,
1536you should place patch files in a directory next to the recipe either
1537named the same as the base name of the recipe
1538(:term:`BP` and
1539:term:`BPN`) or "files".
1540
1541.. _new-recipe-licensing:
1542
1543Licensing
1544---------
1545
1546Your recipe needs to have both the
1547:term:`LICENSE` and
1548:term:`LIC_FILES_CHKSUM`
1549variables:
1550
1551- ``LICENSE``: This variable specifies the license for the software.
1552 If you do not know the license under which the software you are
1553 building is distributed, you should go to the source code and look
1554 for that information. Typical files containing this information
1555 include ``COPYING``, ``LICENSE``, and ``README`` files. You could
1556 also find the information near the top of a source file. For example,
1557 given a piece of software licensed under the GNU General Public
1558 License version 2, you would set ``LICENSE`` as follows:
1559 ::
1560
1561 LICENSE = "GPLv2"
1562
1563 The licenses you specify within ``LICENSE`` can have any name as long
1564 as you do not use spaces, since spaces are used as separators between
1565 license names. For standard licenses, use the names of the files in
1566 ``meta/files/common-licenses/`` or the ``SPDXLICENSEMAP`` flag names
1567 defined in ``meta/conf/licenses.conf``.
1568
1569- ``LIC_FILES_CHKSUM``: The OpenEmbedded build system uses this
1570 variable to make sure the license text has not changed. If it has,
1571 the build produces an error and it affords you the chance to figure
1572 it out and correct the problem.
1573
1574 You need to specify all applicable licensing files for the software.
1575 At the end of the configuration step, the build process will compare
1576 the checksums of the files to be sure the text has not changed. Any
1577 differences result in an error with the message containing the
1578 current checksum. For more explanation and examples of how to set the
1579 ``LIC_FILES_CHKSUM`` variable, see the "`Tracking License
1580 Changes <#>`__" section.
1581
1582 To determine the correct checksum string, you can list the
1583 appropriate files in the ``LIC_FILES_CHKSUM`` variable with incorrect
1584 md5 strings, attempt to build the software, and then note the
1585 resulting error messages that will report the correct md5 strings.
1586 See the "`Fetching Code <#new-recipe-fetching-code>`__" section for
1587 additional information.
1588
1589 Here is an example that assumes the software has a ``COPYING`` file:
1590 ::
1591
1592 LIC_FILES_CHKSUM = "file://COPYING;md5=xxx"
1593
1594 When you try to build the
1595 software, the build system will produce an error and give you the
1596 correct string that you can substitute into the recipe file for a
1597 subsequent build.
1598
1599.. _new-dependencies:
1600
1601Dependencies
1602------------
1603
1604Most software packages have a short list of other packages that they
1605require, which are called dependencies. These dependencies fall into two
1606main categories: build-time dependencies, which are required when the
1607software is built; and runtime dependencies, which are required to be
1608installed on the target in order for the software to run.
1609
1610Within a recipe, you specify build-time dependencies using the
1611:term:`DEPENDS` variable. Although
1612nuances exist, items specified in ``DEPENDS`` should be names of other
1613recipes. It is important that you specify all build-time dependencies
1614explicitly. If you do not, due to the parallel nature of BitBake's
1615execution, you can end up with a race condition where the dependency is
1616present for one task of a recipe (e.g.
1617:ref:`ref-tasks-configure`) and
1618then gone when the next task runs (e.g.
1619:ref:`ref-tasks-compile`).
1620
1621Another consideration is that configure scripts might automatically
1622check for optional dependencies and enable corresponding functionality
1623if those dependencies are found. This behavior means that to ensure
1624deterministic results and thus avoid more race conditions, you need to
1625either explicitly specify these dependencies as well, or tell the
1626configure script explicitly to disable the functionality. If you wish to
1627make a recipe that is more generally useful (e.g. publish the recipe in
1628a layer for others to use), instead of hard-disabling the functionality,
1629you can use the
1630:term:`PACKAGECONFIG` variable
1631to allow functionality and the corresponding dependencies to be enabled
1632and disabled easily by other users of the recipe.
1633
1634Similar to build-time dependencies, you specify runtime dependencies
1635through a variable -
1636:term:`RDEPENDS`, which is
1637package-specific. All variables that are package-specific need to have
1638the name of the package added to the end as an override. Since the main
1639package for a recipe has the same name as the recipe, and the recipe's
1640name can be found through the
1641``${``\ :term:`PN`\ ``}`` variable, then
1642you specify the dependencies for the main package by setting
1643``RDEPENDS_${PN}``. If the package were named ``${PN}-tools``, then you
1644would set ``RDEPENDS_${PN}-tools``, and so forth.
1645
1646Some runtime dependencies will be set automatically at packaging time.
1647These dependencies include any shared library dependencies (i.e. if a
1648package "example" contains "libexample" and another package "mypackage"
1649contains a binary that links to "libexample" then the OpenEmbedded build
1650system will automatically add a runtime dependency to "mypackage" on
1651"example"). See the
1652":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
1653section in the Yocto Project Overview and Concepts Manual for further
1654details.
1655
1656.. _new-recipe-configuring-the-recipe:
1657
1658Configuring the Recipe
1659----------------------
1660
1661Most software provides some means of setting build-time configuration
1662options before compilation. Typically, setting these options is
1663accomplished by running a configure script with options, or by modifying
1664a build configuration file.
1665
1666.. note::
1667
1668 As of Yocto Project Release 1.7, some of the core recipes that
1669 package binary configuration scripts now disable the scripts due to
1670 the scripts previously requiring error-prone path substitution. The
1671 OpenEmbedded build system uses
1672 pkg-config
1673 now, which is much more robust. You can find a list of the
1674 \*-config
1675 scripts that are disabled list in the "
1676 Binary Configuration Scripts Disabled
1677 " section in the Yocto Project Reference Manual.
1678
1679A major part of build-time configuration is about checking for
1680build-time dependencies and possibly enabling optional functionality as
1681a result. You need to specify any build-time dependencies for the
1682software you are building in your recipe's
1683:term:`DEPENDS` value, in terms of
1684other recipes that satisfy those dependencies. You can often find
1685build-time or runtime dependencies described in the software's
1686documentation.
1687
1688The following list provides configuration items of note based on how
1689your software is built:
1690
1691- *Autotools:* If your source files have a ``configure.ac`` file, then
1692 your software is built using Autotools. If this is the case, you just
1693 need to worry about modifying the configuration.
1694
1695 When using Autotools, your recipe needs to inherit the
1696 :ref:`autotools <ref-classes-autotools>` class
1697 and your recipe does not have to contain a
1698 :ref:`ref-tasks-configure` task.
1699 However, you might still want to make some adjustments. For example,
1700 you can set
1701 :term:`EXTRA_OECONF` or
1702 :term:`PACKAGECONFIG_CONFARGS`
1703 to pass any needed configure options that are specific to the recipe.
1704
1705- *CMake:* If your source files have a ``CMakeLists.txt`` file, then
1706 your software is built using CMake. If this is the case, you just
1707 need to worry about modifying the configuration.
1708
1709 When you use CMake, your recipe needs to inherit the
1710 :ref:`cmake <ref-classes-cmake>` class and your
1711 recipe does not have to contain a
1712 :ref:`ref-tasks-configure` task.
1713 You can make some adjustments by setting
1714 :term:`EXTRA_OECMAKE` to
1715 pass any needed configure options that are specific to the recipe.
1716
1717 .. note::
1718
1719 If you need to install one or more custom CMake toolchain files
1720 that are supplied by the application you are building, install the
1721 files to
1722 ${D}${datadir}/cmake/
1723 Modules during
1724 do_install
1725 .
1726
1727- *Other:* If your source files do not have a ``configure.ac`` or
1728 ``CMakeLists.txt`` file, then your software is built using some
1729 method other than Autotools or CMake. If this is the case, you
1730 normally need to provide a
1731 :ref:`ref-tasks-configure` task
1732 in your recipe unless, of course, there is nothing to configure.
1733
1734 Even if your software is not being built by Autotools or CMake, you
1735 still might not need to deal with any configuration issues. You need
1736 to determine if configuration is even a required step. You might need
1737 to modify a Makefile or some configuration file used for the build to
1738 specify necessary build options. Or, perhaps you might need to run a
1739 provided, custom configure script with the appropriate options.
1740
1741 For the case involving a custom configure script, you would run
1742 ``./configure --help`` and look for the options you need to set.
1743
1744Once configuration succeeds, it is always good practice to look at the
1745``log.do_configure`` file to ensure that the appropriate options have
1746been enabled and no additional build-time dependencies need to be added
1747to ``DEPENDS``. For example, if the configure script reports that it
1748found something not mentioned in ``DEPENDS``, or that it did not find
1749something that it needed for some desired optional functionality, then
1750you would need to add those to ``DEPENDS``. Looking at the log might
1751also reveal items being checked for, enabled, or both that you do not
1752want, or items not being found that are in ``DEPENDS``, in which case
1753you would need to look at passing extra options to the configure script
1754as needed. For reference information on configure options specific to
1755the software you are building, you can consult the output of the
1756``./configure --help`` command within ``${S}`` or consult the software's
1757upstream documentation.
1758
1759.. _new-recipe-using-headers-to-interface-with-devices:
1760
1761Using Headers to Interface with Devices
1762---------------------------------------
1763
1764If your recipe builds an application that needs to communicate with some
1765device or needs an API into a custom kernel, you will need to provide
1766appropriate header files. Under no circumstances should you ever modify
1767the existing
1768``meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc`` file.
1769These headers are used to build ``libc`` and must not be compromised
1770with custom or machine-specific header information. If you customize
1771``libc`` through modified headers all other applications that use
1772``libc`` thus become affected.
1773
1774.. note::
1775
1776 Never copy and customize the
1777 libc
1778 header file (i.e.
1779 meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
1780 ).
1781
1782The correct way to interface to a device or custom kernel is to use a
1783separate package that provides the additional headers for the driver or
1784other unique interfaces. When doing so, your application also becomes
1785responsible for creating a dependency on that specific provider.
1786
1787Consider the following:
1788
1789- Never modify ``linux-libc-headers.inc``. Consider that file to be
1790 part of the ``libc`` system, and not something you use to access the
1791 kernel directly. You should access ``libc`` through specific ``libc``
1792 calls.
1793
1794- Applications that must talk directly to devices should either provide
1795 necessary headers themselves, or establish a dependency on a special
1796 headers package that is specific to that driver.
1797
1798For example, suppose you want to modify an existing header that adds I/O
1799control or network support. If the modifications are used by a small
1800number programs, providing a unique version of a header is easy and has
1801little impact. When doing so, bear in mind the guidelines in the
1802previous list.
1803
1804.. note::
1805
1806 If for some reason your changes need to modify the behavior of the
1807 libc
1808 , and subsequently all other applications on the system, use a
1809 .bbappend
1810 to modify the
1811 linux-kernel-headers.inc
1812 file. However, take care to not make the changes machine specific.
1813
1814Consider a case where your kernel is older and you need an older
1815``libc`` ABI. The headers installed by your recipe should still be a
1816standard mainline kernel, not your own custom one.
1817
1818When you use custom kernel headers you need to get them from
1819:term:`STAGING_KERNEL_DIR`,
1820which is the directory with kernel headers that are required to build
1821out-of-tree modules. Your recipe will also need the following:
1822::
1823
1824 do_configure[depends] += "virtual/kernel:do_shared_workdir"
1825
1826.. _new-recipe-compilation:
1827
1828Compilation
1829-----------
1830
1831During a build, the ``do_compile`` task happens after source is fetched,
1832unpacked, and configured. If the recipe passes through ``do_compile``
1833successfully, nothing needs to be done.
1834
1835However, if the compile step fails, you need to diagnose the failure.
1836Here are some common issues that cause failures.
1837
1838.. note::
1839
1840 For cases where improper paths are detected for configuration files
1841 or for when libraries/headers cannot be found, be sure you are using
1842 the more robust
1843 pkg-config
1844 . See the note in section "
1845 Configuring the Recipe
1846 " for additional information.
1847
1848- *Parallel build failures:* These failures manifest themselves as
1849 intermittent errors, or errors reporting that a file or directory
1850 that should be created by some other part of the build process could
1851 not be found. This type of failure can occur even if, upon
1852 inspection, the file or directory does exist after the build has
1853 failed, because that part of the build process happened in the wrong
1854 order.
1855
1856 To fix the problem, you need to either satisfy the missing dependency
1857 in the Makefile or whatever script produced the Makefile, or (as a
1858 workaround) set :term:`PARALLEL_MAKE` to an empty string:
1859 ::
1860
1861 PARALLEL_MAKE = ""
1862
1863 For information on parallel Makefile issues, see the "`Debugging
1864 Parallel Make Races <#debugging-parallel-make-races>`__" section.
1865
1866- *Improper host path usage:* This failure applies to recipes building
1867 for the target or ``nativesdk`` only. The failure occurs when the
1868 compilation process uses improper headers, libraries, or other files
1869 from the host system when cross-compiling for the target.
1870
1871 To fix the problem, examine the ``log.do_compile`` file to identify
1872 the host paths being used (e.g. ``/usr/include``, ``/usr/lib``, and
1873 so forth) and then either add configure options, apply a patch, or do
1874 both.
1875
1876- *Failure to find required libraries/headers:* If a build-time
1877 dependency is missing because it has not been declared in
1878 :term:`DEPENDS`, or because the
1879 dependency exists but the path used by the build process to find the
1880 file is incorrect and the configure step did not detect it, the
1881 compilation process could fail. For either of these failures, the
1882 compilation process notes that files could not be found. In these
1883 cases, you need to go back and add additional options to the
1884 configure script as well as possibly add additional build-time
1885 dependencies to ``DEPENDS``.
1886
1887 Occasionally, it is necessary to apply a patch to the source to
1888 ensure the correct paths are used. If you need to specify paths to
1889 find files staged into the sysroot from other recipes, use the
1890 variables that the OpenEmbedded build system provides (e.g.
1891 ``STAGING_BINDIR``, ``STAGING_INCDIR``, ``STAGING_DATADIR``, and so
1892 forth).
1893
1894.. _new-recipe-installing:
1895
1896Installing
1897----------
1898
1899During ``do_install``, the task copies the built files along with their
1900hierarchy to locations that would mirror their locations on the target
1901device. The installation process copies files from the
1902``${``\ :term:`S`\ ``}``,
1903``${``\ :term:`B`\ ``}``, and
1904``${``\ :term:`WORKDIR`\ ``}``
1905directories to the ``${``\ :term:`D`\ ``}``
1906directory to create the structure as it should appear on the target
1907system.
1908
1909How your software is built affects what you must do to be sure your
1910software is installed correctly. The following list describes what you
1911must do for installation depending on the type of build system used by
1912the software being built:
1913
1914- *Autotools and CMake:* If the software your recipe is building uses
1915 Autotools or CMake, the OpenEmbedded build system understands how to
1916 install the software. Consequently, you do not have to have a
1917 ``do_install`` task as part of your recipe. You just need to make
1918 sure the install portion of the build completes with no issues.
1919 However, if you wish to install additional files not already being
1920 installed by ``make install``, you should do this using a
1921 ``do_install_append`` function using the install command as described
1922 in the "Manual" bulleted item later in this list.
1923
1924- Other (using ``make install``): You need to define a ``do_install``
1925 function in your recipe. The function should call
1926 ``oe_runmake install`` and will likely need to pass in the
1927 destination directory as well. How you pass that path is dependent on
1928 how the ``Makefile`` being run is written (e.g. ``DESTDIR=${D}``,
1929 ``PREFIX=${D}``, ``INSTALLROOT=${D}``, and so forth).
1930
1931 For an example recipe using ``make install``, see the
1932 "`Makefile-Based Package <#new-recipe-makefile-based-package>`__"
1933 section.
1934
1935- *Manual:* You need to define a ``do_install`` function in your
1936 recipe. The function must first use ``install -d`` to create the
1937 directories under
1938 ``${``\ :term:`D`\ ``}``. Once the
1939 directories exist, your function can use ``install`` to manually
1940 install the built software into the directories.
1941
1942 You can find more information on ``install`` at
1943 http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html.
1944
1945For the scenarios that do not use Autotools or CMake, you need to track
1946the installation and diagnose and fix any issues until everything
1947installs correctly. You need to look in the default location of
1948``${D}``, which is ``${WORKDIR}/image``, to be sure your files have been
1949installed correctly.
1950
1951.. note::
1952
1953 - During the installation process, you might need to modify some of
1954 the installed files to suit the target layout. For example, you
1955 might need to replace hard-coded paths in an initscript with
1956 values of variables provided by the build system, such as
1957 replacing ``/usr/bin/`` with ``${bindir}``. If you do perform such
1958 modifications during ``do_install``, be sure to modify the
1959 destination file after copying rather than before copying.
1960 Modifying after copying ensures that the build system can
1961 re-execute ``do_install`` if needed.
1962
1963 - ``oe_runmake install``, which can be run directly or can be run
1964 indirectly by the
1965 :ref:`autotools <ref-classes-autotools>` and
1966 :ref:`cmake <ref-classes-cmake>` classes,
1967 runs ``make install`` in parallel. Sometimes, a Makefile can have
1968 missing dependencies between targets that can result in race
1969 conditions. If you experience intermittent failures during
1970 ``do_install``, you might be able to work around them by disabling
1971 parallel Makefile installs by adding the following to the recipe:
1972 PARALLEL_MAKEINST = "" See
1973 :term:`PARALLEL_MAKEINST`
1974 for additional information.
1975
1976 - If you need to install one or more custom CMake toolchain files
1977 that are supplied by the application you are building, install the
1978 files to ``${D}${datadir}/cmake/`` Modules during
1979 :ref:`ref-tasks-install`.
1980
1981.. _new-recipe-enabling-system-services:
1982
1983Enabling System Services
1984------------------------
1985
1986If you want to install a service, which is a process that usually starts
1987on boot and runs in the background, then you must include some
1988additional definitions in your recipe.
1989
1990If you are adding services and the service initialization script or the
1991service file itself is not installed, you must provide for that
1992installation in your recipe using a ``do_install_append`` function. If
1993your recipe already has a ``do_install`` function, update the function
1994near its end rather than adding an additional ``do_install_append``
1995function.
1996
1997When you create the installation for your services, you need to
1998accomplish what is normally done by ``make install``. In other words,
1999make sure your installation arranges the output similar to how it is
2000arranged on the target system.
2001
2002The OpenEmbedded build system provides support for starting services two
2003different ways:
2004
2005- *SysVinit:* SysVinit is a system and service manager that manages the
2006 init system used to control the very basic functions of your system.
2007 The init program is the first program started by the Linux kernel
2008 when the system boots. Init then controls the startup, running and
2009 shutdown of all other programs.
2010
2011 To enable a service using SysVinit, your recipe needs to inherit the
2012 :ref:`update-rc.d <ref-classes-update-rc.d>`
2013 class. The class helps facilitate safely installing the package on
2014 the target.
2015
2016 You will need to set the
2017 :term:`INITSCRIPT_PACKAGES`,
2018 :term:`INITSCRIPT_NAME`,
2019 and
2020 :term:`INITSCRIPT_PARAMS`
2021 variables within your recipe.
2022
2023- *systemd:* System Management Daemon (systemd) was designed to replace
2024 SysVinit and to provide enhanced management of services. For more
2025 information on systemd, see the systemd homepage at
2026 http://freedesktop.org/wiki/Software/systemd/.
2027
2028 To enable a service using systemd, your recipe needs to inherit the
2029 :ref:`systemd <ref-classes-systemd>` class. See
2030 the ``systemd.bbclass`` file located in your :term:`Source Directory`
2031 section for
2032 more information.
2033
2034.. _new-recipe-packaging:
2035
2036Packaging
2037---------
2038
2039Successful packaging is a combination of automated processes performed
2040by the OpenEmbedded build system and some specific steps you need to
2041take. The following list describes the process:
2042
2043- *Splitting Files*: The ``do_package`` task splits the files produced
2044 by the recipe into logical components. Even software that produces a
2045 single binary might still have debug symbols, documentation, and
2046 other logical components that should be split out. The ``do_package``
2047 task ensures that files are split up and packaged correctly.
2048
2049- *Running QA Checks*: The
2050 :ref:`insane <ref-classes-insane>` class adds a
2051 step to the package generation process so that output quality
2052 assurance checks are generated by the OpenEmbedded build system. This
2053 step performs a range of checks to be sure the build's output is free
2054 of common problems that show up during runtime. For information on
2055 these checks, see the
2056 :ref:`insane <ref-classes-insane>` class and
2057 the ":ref:`ref-manual/ref-qa-checks:qa error and warning messages`"
2058 chapter in the Yocto Project Reference Manual.
2059
2060- *Hand-Checking Your Packages*: After you build your software, you
2061 need to be sure your packages are correct. Examine the
2062 ``${``\ :term:`WORKDIR`\ ``}/packages-split``
2063 directory and make sure files are where you expect them to be. If you
2064 discover problems, you can set
2065 :term:`PACKAGES`,
2066 :term:`FILES`,
2067 ``do_install(_append)``, and so forth as needed.
2068
2069- *Splitting an Application into Multiple Packages*: If you need to
2070 split an application into several packages, see the "`Splitting an
2071 Application into Multiple
2072 Packages <#splitting-an-application-into-multiple-packages>`__"
2073 section for an example.
2074
2075- *Installing a Post-Installation Script*: For an example showing how
2076 to install a post-installation script, see the "`Post-Installation
2077 Scripts <#new-recipe-post-installation-scripts>`__" section.
2078
2079- *Marking Package Architecture*: Depending on what your recipe is
2080 building and how it is configured, it might be important to mark the
2081 packages produced as being specific to a particular machine, or to
2082 mark them as not being specific to a particular machine or
2083 architecture at all.
2084
2085 By default, packages apply to any machine with the same architecture
2086 as the target machine. When a recipe produces packages that are
2087 machine-specific (e.g. the
2088 :term:`MACHINE` value is passed
2089 into the configure script or a patch is applied only for a particular
2090 machine), you should mark them as such by adding the following to the
2091 recipe:
2092 ::
2093
2094 PACKAGE_ARCH = "${MACHINE_ARCH}"
2095
2096 On the other hand, if the recipe produces packages that do not
2097 contain anything specific to the target machine or architecture at
2098 all (e.g. recipes that simply package script files or configuration
2099 files), you should use the
2100 :ref:`allarch <ref-classes-allarch>` class to
2101 do this for you by adding this to your recipe:
2102 ::
2103
2104 inherit allarch
2105
2106 Ensuring that the package architecture is correct is not critical
2107 while you are doing the first few builds of your recipe. However, it
2108 is important in order to ensure that your recipe rebuilds (or does
2109 not rebuild) appropriately in response to changes in configuration,
2110 and to ensure that you get the appropriate packages installed on the
2111 target machine, particularly if you run separate builds for more than
2112 one target machine.
2113
2114.. _new-sharing-files-between-recipes:
2115
2116Sharing Files Between Recipes
2117-----------------------------
2118
2119Recipes often need to use files provided by other recipes on the build
2120host. For example, an application linking to a common library needs
2121access to the library itself and its associated headers. The way this
2122access is accomplished is by populating a sysroot with files. Each
2123recipe has two sysroots in its work directory, one for target files
2124(``recipe-sysroot``) and one for files that are native to the build host
2125(``recipe-sysroot-native``).
2126
2127.. note::
2128
2129 You could find the term "staging" used within the Yocto project
2130 regarding files populating sysroots (e.g. the
2131 STAGING_DIR
2132 variable).
2133
2134Recipes should never populate the sysroot directly (i.e. write files
2135into sysroot). Instead, files should be installed into standard
2136locations during the
2137:ref:`ref-tasks-install` task within
2138the ``${``\ :term:`D`\ ``}`` directory. The
2139reason for this limitation is that almost all files that populate the
2140sysroot are cataloged in manifests in order to ensure the files can be
2141removed later when a recipe is either modified or removed. Thus, the
2142sysroot is able to remain free from stale files.
2143
2144A subset of the files installed by the
2145:ref:`ref-tasks-install` task are
2146used by the
2147:ref:`ref-tasks-populate_sysroot`
2148task as defined by the the
2149:term:`SYSROOT_DIRS` variable to
2150automatically populate the sysroot. It is possible to modify the list of
2151directories that populate the sysroot. The following example shows how
2152you could add the ``/opt`` directory to the list of directories within a
2153recipe:
2154::
2155
2156 SYSROOT_DIRS += "/opt"
2157
2158For a more complete description of the
2159:ref:`ref-tasks-populate_sysroot`
2160task and its associated functions, see the
2161:ref:`staging <ref-classes-staging>` class.
2162
2163.. _metadata-virtual-providers:
2164
2165Using Virtual Providers
2166-----------------------
2167
2168Prior to a build, if you know that several different recipes provide the
2169same functionality, you can use a virtual provider (i.e. ``virtual/*``)
2170as a placeholder for the actual provider. The actual provider is
2171determined at build-time.
2172
2173A common scenario where a virtual provider is used would be for the
2174kernel recipe. Suppose you have three kernel recipes whose
2175:term:`PN` values map to ``kernel-big``,
2176``kernel-mid``, and ``kernel-small``. Furthermore, each of these recipes
2177in some way uses a :term:`PROVIDES`
2178statement that essentially identifies itself as being able to provide
2179``virtual/kernel``. Here is one way through the
2180:ref:`kernel <ref-classes-kernel>` class:
2181::
2182
2183 PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }"
2184
2185Any recipe that inherits the ``kernel`` class is
2186going to utilize a ``PROVIDES`` statement that identifies that recipe as
2187being able to provide the ``virtual/kernel`` item.
2188
2189Now comes the time to actually build an image and you need a kernel
2190recipe, but which one? You can configure your build to call out the
2191kernel recipe you want by using the
2192:term:`PREFERRED_PROVIDER`
2193variable. As an example, consider the
2194`x86-base.inc <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc>`_
2195include file, which is a machine (i.e.
2196:term:`MACHINE`) configuration file.
2197This include file is the reason all x86-based machines use the
2198``linux-yocto`` kernel. Here are the relevant lines from the include
2199file:
2200::
2201
2202 PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto"
2203 PREFERRED_VERSION_linux-yocto ??= "4.15%"
2204
2205When you use a virtual provider, you do not have to "hard code" a recipe
2206name as a build dependency. You can use the
2207:term:`DEPENDS` variable to state the
2208build is dependent on ``virtual/kernel`` for example: DEPENDS =
2209"virtual/kernel" During the build, the OpenEmbedded build system picks
2210the correct recipe needed for the ``virtual/kernel`` dependency based on
2211the ``PREFERRED_PROVIDER`` variable. If you want to use the small kernel
2212mentioned at the beginning of this section, configure your build as
2213follows: PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small"
2214
2215.. note::
2216
2217 Any recipe that
2218 PROVIDES
2219 a
2220 virtual/\*
2221 item that is ultimately not selected through
2222 PREFERRED_PROVIDER
2223 does not get built. Preventing these recipes from building is usually
2224 the desired behavior since this mechanism's purpose is to select
2225 between mutually exclusive alternative providers.
2226
2227The following lists specific examples of virtual providers:
2228
2229- ``virtual/kernel``: Provides the name of the kernel recipe to use
2230 when building a kernel image.
2231
2232- ``virtual/bootloader``: Provides the name of the bootloader to use
2233 when building an image.
2234
2235- ``virtual/libgbm``: Provides ``gbm.pc``.
2236
2237- ``virtual/egl``: Provides ``egl.pc`` and possibly ``wayland-egl.pc``.
2238
2239- ``virtual/libgl``: Provides ``gl.pc`` (i.e. libGL).
2240
2241- ``virtual/libgles1``: Provides ``glesv1_cm.pc`` (i.e. libGLESv1_CM).
2242
2243- ``virtual/libgles2``: Provides ``glesv2.pc`` (i.e. libGLESv2).
2244
2245.. note::
2246
2247 Virtual providers only apply to build time dependencies specified with
2248 :term:`PROVIDES` and :term:`DEPENDS`. They do not apply to runtime
2249 dependencies specified with :term:`RPROVIDES` and :term:`RDEPENDS`.
2250
2251Properly Versioning Pre-Release Recipes
2252---------------------------------------
2253
2254Sometimes the name of a recipe can lead to versioning problems when the
2255recipe is upgraded to a final release. For example, consider the
2256``irssi_0.8.16-rc1.bb`` recipe file in the list of example recipes in
2257the "`Storing and Naming the
2258Recipe <#new-recipe-storing-and-naming-the-recipe>`__" section. This
2259recipe is at a release candidate stage (i.e. "rc1"). When the recipe is
2260released, the recipe filename becomes ``irssi_0.8.16.bb``. The version
2261change from ``0.8.16-rc1`` to ``0.8.16`` is seen as a decrease by the
2262build system and package managers, so the resulting packages will not
2263correctly trigger an upgrade.
2264
2265In order to ensure the versions compare properly, the recommended
2266convention is to set :term:`PV` within the
2267recipe to "previous_version+current_version". You can use an additional
2268variable so that you can use the current version elsewhere. Here is an
2269example:
2270::
2271
2272 REALPV = "0.8.16-rc1"
2273 PV = "0.8.15+${REALPV}"
2274
2275.. _new-recipe-post-installation-scripts:
2276
2277Post-Installation Scripts
2278-------------------------
2279
2280Post-installation scripts run immediately after installing a package on
2281the target or during image creation when a package is included in an
2282image. To add a post-installation script to a package, add a
2283``pkg_postinst_``\ PACKAGENAME\ ``()`` function to the recipe file
2284(``.bb``) and replace PACKAGENAME with the name of the package you want
2285to attach to the ``postinst`` script. To apply the post-installation
2286script to the main package for the recipe, which is usually what is
2287required, specify
2288``${``\ :term:`PN`\ ``}`` in place of
2289PACKAGENAME.
2290
2291A post-installation function has the following structure:
2292pkg_postinst_PACKAGENAME() { # Commands to carry out }
2293
2294The script defined in the post-installation function is called when the
2295root filesystem is created. If the script succeeds, the package is
2296marked as installed.
2297
2298.. note::
2299
2300 Any RPM post-installation script that runs on the target should
2301 return a 0 exit code. RPM does not allow non-zero exit codes for
2302 these scripts, and the RPM package manager will cause the package to
2303 fail installation on the target.
2304
2305Sometimes it is necessary for the execution of a post-installation
2306script to be delayed until the first boot. For example, the script might
2307need to be executed on the device itself. To delay script execution
2308until boot time, you must explicitly mark post installs to defer to the
2309target. You can use ``pkg_postinst_ontarget()`` or call
2310``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``. Any
2311failure of a ``pkg_postinst()`` script (including exit 1) triggers an
2312error during the
2313:ref:`ref-tasks-rootfs` task.
2314
2315If you have recipes that use ``pkg_postinst`` function and they require
2316the use of non-standard native tools that have dependencies during
2317rootfs construction, you need to use the
2318:term:`PACKAGE_WRITE_DEPS`
2319variable in your recipe to list these tools. If you do not use this
2320variable, the tools might be missing and execution of the
2321post-installation script is deferred until first boot. Deferring the
2322script to first boot is undesirable and for read-only rootfs impossible.
2323
2324.. note::
2325
2326 Equivalent support for pre-install, pre-uninstall, and post-uninstall
2327 scripts exist by way of
2328 pkg_preinst
2329 ,
2330 pkg_prerm
2331 , and
2332 pkg_postrm
2333 , respectively. These scrips work in exactly the same way as does
2334 pkg_postinst
2335 with the exception that they run at different times. Also, because of
2336 when they run, they are not applicable to being run at image creation
2337 time like
2338 pkg_postinst
2339 .
2340
2341.. _new-recipe-testing:
2342
2343Testing
2344-------
2345
2346The final step for completing your recipe is to be sure that the
2347software you built runs correctly. To accomplish runtime testing, add
2348the build's output packages to your image and test them on the target.
2349
2350For information on how to customize your image by adding specific
2351packages, see the "`Customizing
2352Images <#usingpoky-extend-customimage>`__" section.
2353
2354.. _new-recipe-testing-examples:
2355
2356Examples
2357--------
2358
2359To help summarize how to write a recipe, this section provides some
2360examples given various scenarios:
2361
2362- Recipes that use local files
2363
2364- Using an Autotooled package
2365
2366- Using a Makefile-based package
2367
2368- Splitting an application into multiple packages
2369
2370- Adding binaries to an image
2371
2372.. _new-recipe-single-c-file-package-hello-world:
2373
2374Single .c File Package (Hello World!)
2375~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2376
2377Building an application from a single file that is stored locally (e.g.
2378under ``files``) requires a recipe that has the file listed in the
2379``SRC_URI`` variable. Additionally, you need to manually write the
2380``do_compile`` and ``do_install`` tasks. The ``S`` variable defines the
2381directory containing the source code, which is set to
2382:term:`WORKDIR` in this case - the
2383directory BitBake uses for the build.
2384::
2385
2386 SUMMARY = "Simple helloworld application"
2387 SECTION = "examples"
2388 LICENSE = "MIT"
2389 LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
2390
2391 SRC_URI = "file://helloworld.c"
2392
2393 S = "${WORKDIR}"
2394
2395 do_compile() {
2396 ${CC} helloworld.c -o helloworld
2397 }
2398
2399 do_install() {
2400 install -d ${D}${bindir}
2401 install -m 0755 helloworld ${D}${bindir}
2402 }
2403
2404By default, the ``helloworld``, ``helloworld-dbg``, and
2405``helloworld-dev`` packages are built. For information on how to
2406customize the packaging process, see the "`Splitting an Application into
2407Multiple Packages <#splitting-an-application-into-multiple-packages>`__"
2408section.
2409
2410.. _new-recipe-autotooled-package:
2411
2412Autotooled Package
2413~~~~~~~~~~~~~~~~~~
2414
2415Applications that use Autotools such as ``autoconf`` and ``automake``
2416require a recipe that has a source archive listed in ``SRC_URI`` and
2417also inherit the
2418:ref:`autotools <ref-classes-autotools>` class,
2419which contains the definitions of all the steps needed to build an
2420Autotool-based application. The result of the build is automatically
2421packaged. And, if the application uses NLS for localization, packages
2422with local information are generated (one package per language).
2423Following is one example: (``hello_2.3.bb``)
2424::
2425
2426 SUMMARY = "GNU Helloworld application"
2427 SECTION = "examples"
2428 LICENSE = "GPLv2+"
2429 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
2430
2431 SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
2432
2433 inherit autotools gettext
2434
2435The variable ``LIC_FILES_CHKSUM`` is used to track source license
2436changes as described in the "`Tracking License
2437Changes <#usingpoky-configuring-LIC_FILES_CHKSUM>`__" section in the
2438Yocto Project Overview and Concepts Manual. You can quickly create
2439Autotool-based recipes in a manner similar to the previous example.
2440
2441.. _new-recipe-makefile-based-package:
2442
2443Makefile-Based Package
2444~~~~~~~~~~~~~~~~~~~~~~
2445
2446Applications that use GNU ``make`` also require a recipe that has the
2447source archive listed in ``SRC_URI``. You do not need to add a
2448``do_compile`` step since by default BitBake starts the ``make`` command
2449to compile the application. If you need additional ``make`` options, you
2450should store them in the
2451:term:`EXTRA_OEMAKE` or
2452:term:`PACKAGECONFIG_CONFARGS`
2453variables. BitBake passes these options into the GNU ``make``
2454invocation. Note that a ``do_install`` task is still required.
2455Otherwise, BitBake runs an empty ``do_install`` task by default.
2456
2457Some applications might require extra parameters to be passed to the
2458compiler. For example, the application might need an additional header
2459path. You can accomplish this by adding to the ``CFLAGS`` variable. The
2460following example shows this:
2461::
2462
2463 CFLAGS_prepend = "-I ${S}/include "
2464
2465In the following example, ``mtd-utils`` is a makefile-based package:
2466::
2467
2468 SUMMARY = "Tools for managing memory technology devices"
2469 SECTION = "base"
2470 DEPENDS = "zlib lzo e2fsprogs util-linux"
2471 HOMEPAGE = "http://www.linux-mtd.infradead.org/"
2472 LICENSE = "GPLv2+"
2473 LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
2474 file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"
2475 # Use the latest version at 26 Oct, 2013
2476 SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b"
2477 SRC_URI = "git://git.infradead.org/mtd-utils.git \
2478 file://add-exclusion-to-mkfs-jffs2-git-2.patch \
2479 "
2480 PV = "1.5.1+git${SRCPV}"
2481 S = "${WORKDIR}/git"
2482 EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'"
2483 do_install () {
2484 oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir}
2485 }
2486 PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc"
2487 FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool"
2488 FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*"
2489 FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image"
2490 PARALLEL_MAKE = ""
2491 BBCLASSEXTEND = "native"
2492
2493Splitting an Application into Multiple Packages
2494~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2495
2496You can use the variables ``PACKAGES`` and ``FILES`` to split an
2497application into multiple packages.
2498
2499Following is an example that uses the ``libxpm`` recipe. By default,
2500this recipe generates a single package that contains the library along
2501with a few binaries. You can modify the recipe to split the binaries
2502into separate packages:
2503::
2504
2505 require xorg-lib-common.inc
2506 SUMMARY = "Xpm: X Pixmap extension library"
2507 LICENSE = "BSD"
2508 LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7"
2509 DEPENDS += "libxext libsm libxt"
2510 PE = "1"
2511 XORG_PN = "libXpm"
2512 PACKAGES =+ "sxpm cxpm"
2513 FILES_cxpm = "${bindir}/cxpm"
2514 FILES_sxpm = "${bindir}/sxpm"
2515
2516In the previous example, we want to ship the ``sxpm`` and ``cxpm``
2517binaries in separate packages. Since ``bindir`` would be packaged into
2518the main ``PN`` package by default, we prepend the ``PACKAGES`` variable
2519so additional package names are added to the start of list. This results
2520in the extra ``FILES_*`` variables then containing information that
2521define which files and directories go into which packages. Files
2522included by earlier packages are skipped by latter packages. Thus, the
2523main ``PN`` package does not include the above listed files.
2524
2525Packaging Externally Produced Binaries
2526~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2527
2528Sometimes, you need to add pre-compiled binaries to an image. For
2529example, suppose that binaries for proprietary code exist, which are
2530created by a particular division of a company. Your part of the company
2531needs to use those binaries as part of an image that you are building
2532using the OpenEmbedded build system. Since you only have the binaries
2533and not the source code, you cannot use a typical recipe that expects to
2534fetch the source specified in
2535:term:`SRC_URI` and then compile it.
2536
2537One method is to package the binaries and then install them as part of
2538the image. Generally, it is not a good idea to package binaries since,
2539among other things, it can hinder the ability to reproduce builds and
2540could lead to compatibility problems with ABI in the future. However,
2541sometimes you have no choice.
2542
2543The easiest solution is to create a recipe that uses the
2544:ref:`bin_package <ref-classes-bin-package>` class
2545and to be sure that you are using default locations for build artifacts.
2546In most cases, the ``bin_package`` class handles "skipping" the
2547configure and compile steps as well as sets things up to grab packages
2548from the appropriate area. In particular, this class sets ``noexec`` on
2549both the :ref:`ref-tasks-configure`
2550and :ref:`ref-tasks-compile` tasks,
2551sets ``FILES_${PN}`` to "/" so that it picks up all files, and sets up a
2552:ref:`ref-tasks-install` task, which
2553effectively copies all files from ``${S}`` to ``${D}``. The
2554``bin_package`` class works well when the files extracted into ``${S}``
2555are already laid out in the way they should be laid out on the target.
2556For more information on these variables, see the
2557:term:`FILES`,
2558:term:`PN`,
2559:term:`S`, and
2560:term:`D` variables in the Yocto Project
2561Reference Manual's variable glossary.
2562
2563.. note::
2564
2565 - Using :term:`DEPENDS` is a good
2566 idea even for components distributed in binary form, and is often
2567 necessary for shared libraries. For a shared library, listing the
2568 library dependencies in ``DEPENDS`` makes sure that the libraries
2569 are available in the staging sysroot when other recipes link
2570 against the library, which might be necessary for successful
2571 linking.
2572
2573 - Using ``DEPENDS`` also allows runtime dependencies between
2574 packages to be added automatically. See the
2575 ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
2576 section in the Yocto Project Overview and Concepts Manual for more
2577 information.
2578
2579If you cannot use the ``bin_package`` class, you need to be sure you are
2580doing the following:
2581
2582- Create a recipe where the
2583 :ref:`ref-tasks-configure` and
2584 :ref:`ref-tasks-compile` tasks do
2585 nothing: It is usually sufficient to just not define these tasks in
2586 the recipe, because the default implementations do nothing unless a
2587 Makefile is found in
2588 ``${``\ :term:`S`\ ``}``.
2589
2590 If ``${S}`` might contain a Makefile, or if you inherit some class
2591 that replaces ``do_configure`` and ``do_compile`` with custom
2592 versions, then you can use the
2593 ``[``\ :ref:`noexec <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]``
2594 flag to turn the tasks into no-ops, as follows:
2595 ::
2596
2597 do_configure[noexec] = "1"
2598 do_compile[noexec] = "1"
2599
2600 Unlike
2601 :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:deleting a task`,
2602 using the flag preserves the dependency chain from the
2603 :ref:`ref-tasks-fetch`,
2604 :ref:`ref-tasks-unpack`, and
2605 :ref:`ref-tasks-patch` tasks to the
2606 :ref:`ref-tasks-install` task.
2607
2608- Make sure your ``do_install`` task installs the binaries
2609 appropriately.
2610
2611- Ensure that you set up :term:`FILES`
2612 (usually
2613 ``FILES_${``\ :term:`PN`\ ``}``) to
2614 point to the files you have installed, which of course depends on
2615 where you have installed them and whether those files are in
2616 different locations than the defaults.
2617
2618Following Recipe Style Guidelines
2619---------------------------------
2620
2621When writing recipes, it is good to conform to existing style
2622guidelines. The `OpenEmbedded
2623Styleguide <http://www.openembedded.org/wiki/Styleguide>`__ wiki page
2624provides rough guidelines for preferred recipe style.
2625
2626It is common for existing recipes to deviate a bit from this style.
2627However, aiming for at least a consistent style is a good idea. Some
2628practices, such as omitting spaces around ``=`` operators in assignments
2629or ordering recipe components in an erratic way, are widely seen as poor
2630style.
2631
2632Recipe Syntax
2633-------------
2634
2635Understanding recipe file syntax is important for writing recipes. The
2636following list overviews the basic items that make up a BitBake recipe
2637file. For more complete BitBake syntax descriptions, see the
2638":doc:`bitbake-user-manual/bitbake-user-manual-metadata`"
2639chapter of the BitBake User Manual.
2640
2641- *Variable Assignments and Manipulations:* Variable assignments allow
2642 a value to be assigned to a variable. The assignment can be static
2643 text or might include the contents of other variables. In addition to
2644 the assignment, appending and prepending operations are also
2645 supported.
2646
2647 The following example shows some of the ways you can use variables in
2648 recipes:
2649 ::
2650
2651 S = "${WORKDIR}/postfix-${PV}"
2652 CFLAGS += "-DNO_ASM"
2653 SRC_URI_append = " file://fixup.patch"
2654
2655- *Functions:* Functions provide a series of actions to be performed.
2656 You usually use functions to override the default implementation of a
2657 task function or to complement a default function (i.e. append or
2658 prepend to an existing function). Standard functions use ``sh`` shell
2659 syntax, although access to OpenEmbedded variables and internal
2660 methods are also available.
2661
2662 The following is an example function from the ``sed`` recipe:
2663 ::
2664
2665 do_install () {
2666 autotools_do_install
2667 install -d ${D}${base_bindir}
2668 mv ${D}${bindir}/sed ${D}${base_bindir}/sed
2669 rmdir ${D}${bindir}/
2670 }
2671
2672 It is
2673 also possible to implement new functions that are called between
2674 existing tasks as long as the new functions are not replacing or
2675 complementing the default functions. You can implement functions in
2676 Python instead of shell. Both of these options are not seen in the
2677 majority of recipes.
2678
2679- *Keywords:* BitBake recipes use only a few keywords. You use keywords
2680 to include common functions (``inherit``), load parts of a recipe
2681 from other files (``include`` and ``require``) and export variables
2682 to the environment (``export``).
2683
2684 The following example shows the use of some of these keywords:
2685 ::
2686
2687 export POSTCONF = "${STAGING_BINDIR}/postconf"
2688 inherit autoconf
2689 require otherfile.inc
2690
2691- *Comments (#):* Any lines that begin with the hash character (``#``)
2692 are treated as comment lines and are ignored:
2693 ::
2694
2695 # This is a comment
2696
2697This next list summarizes the most important and most commonly used
2698parts of the recipe syntax. For more information on these parts of the
2699syntax, you can reference the
2700:doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata` chapter
2701in the BitBake User Manual.
2702
2703- *Line Continuation (\):* Use the backward slash (``\``) character to
2704 split a statement over multiple lines. Place the slash character at
2705 the end of the line that is to be continued on the next line:
2706 ::
2707
2708 VAR = "A really long \
2709 line"
2710
2711 .. note::
2712
2713 You cannot have any characters including spaces or tabs after the
2714 slash character.
2715
2716- *Using Variables (${VARNAME}):* Use the ``${VARNAME}`` syntax to
2717 access the contents of a variable:
2718 ::
2719
2720 SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
2721
2722 .. note::
2723
2724 It is important to understand that the value of a variable
2725 expressed in this form does not get substituted automatically. The
2726 expansion of these expressions happens on-demand later (e.g.
2727 usually when a function that makes reference to the variable
2728 executes). This behavior ensures that the values are most
2729 appropriate for the context in which they are finally used. On the
2730 rare occasion that you do need the variable expression to be
2731 expanded immediately, you can use the
2732 :=
2733 operator instead of
2734 =
2735 when you make the assignment, but this is not generally needed.
2736
2737- *Quote All Assignments ("value"):* Use double quotes around values in
2738 all variable assignments (e.g. ``"value"``). Following is an example:
2739 ::
2740
2741 VAR1 = "${OTHERVAR}"
2742 VAR2 = "The version is ${PV}"
2743
2744- *Conditional Assignment (?=):* Conditional assignment is used to
2745 assign a value to a variable, but only when the variable is currently
2746 unset. Use the question mark followed by the equal sign (``?=``) to
2747 make a "soft" assignment used for conditional assignment. Typically,
2748 "soft" assignments are used in the ``local.conf`` file for variables
2749 that are allowed to come through from the external environment.
2750
2751 Here is an example where ``VAR1`` is set to "New value" if it is
2752 currently empty. However, if ``VAR1`` has already been set, it
2753 remains unchanged: VAR1 ?= "New value" In this next example, ``VAR1``
2754 is left with the value "Original value":
2755 ::
2756
2757 VAR1 = "Original value"
2758 VAR1 ?= "New value"
2759
2760- *Appending (+=):* Use the plus character followed by the equals sign
2761 (``+=``) to append values to existing variables.
2762
2763 .. note::
2764
2765 This operator adds a space between the existing content of the
2766 variable and the new content.
2767
2768 Here is an example:
2769 ::
2770
2771 SRC_URI += "file://fix-makefile.patch"
2772
2773- *Prepending (=+):* Use the equals sign followed by the plus character
2774 (``=+``) to prepend values to existing variables.
2775
2776 .. note::
2777
2778 This operator adds a space between the new content and the
2779 existing content of the variable.
2780
2781 Here is an example:
2782 ::
2783
2784 VAR =+ "Starts"
2785
2786- *Appending (_append):* Use the ``_append`` operator to append values
2787 to existing variables. This operator does not add any additional
2788 space. Also, the operator is applied after all the ``+=``, and ``=+``
2789 operators have been applied and after all ``=`` assignments have
2790 occurred.
2791
2792 The following example shows the space being explicitly added to the
2793 start to ensure the appended value is not merged with the existing
2794 value:
2795 ::
2796
2797 SRC_URI_append = " file://fix-makefile.patch"
2798
2799 You can also use
2800 the ``_append`` operator with overrides, which results in the actions
2801 only being performed for the specified target or machine:
2802 ::
2803
2804 SRC_URI_append_sh4 = " file://fix-makefile.patch"
2805
2806- *Prepending (_prepend):* Use the ``_prepend`` operator to prepend
2807 values to existing variables. This operator does not add any
2808 additional space. Also, the operator is applied after all the ``+=``,
2809 and ``=+`` operators have been applied and after all ``=``
2810 assignments have occurred.
2811
2812 The following example shows the space being explicitly added to the
2813 end to ensure the prepended value is not merged with the existing
2814 value:
2815 ::
2816
2817 CFLAGS_prepend = "-I${S}/myincludes "
2818
2819 You can also use the
2820 ``_prepend`` operator with overrides, which results in the actions
2821 only being performed for the specified target or machine:
2822 ::
2823
2824 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
2825
2826- *Overrides:* You can use overrides to set a value conditionally,
2827 typically based on how the recipe is being built. For example, to set
2828 the :term:`KBRANCH` variable's
2829 value to "standard/base" for any target
2830 :term:`MACHINE`, except for
2831 qemuarm where it should be set to "standard/arm-versatile-926ejs",
2832 you would do the following:
2833 ::
2834
2835 KBRANCH = "standard/base"
2836 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
2837
2838 Overrides are also used to separate
2839 alternate values of a variable in other situations. For example, when
2840 setting variables such as
2841 :term:`FILES` and
2842 :term:`RDEPENDS` that are
2843 specific to individual packages produced by a recipe, you should
2844 always use an override that specifies the name of the package.
2845
2846- *Indentation:* Use spaces for indentation rather than than tabs. For
2847 shell functions, both currently work. However, it is a policy
2848 decision of the Yocto Project to use tabs in shell functions. Realize
2849 that some layers have a policy to use spaces for all indentation.
2850
2851- *Using Python for Complex Operations:* For more advanced processing,
2852 it is possible to use Python code during variable assignments (e.g.
2853 search and replacement on a variable).
2854
2855 You indicate Python code using the ``${@python_code}`` syntax for the
2856 variable assignment:
2857 ::
2858
2859 SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz
2860
2861- *Shell Function Syntax:* Write shell functions as if you were writing
2862 a shell script when you describe a list of actions to take. You
2863 should ensure that your script works with a generic ``sh`` and that
2864 it does not require any ``bash`` or other shell-specific
2865 functionality. The same considerations apply to various system
2866 utilities (e.g. ``sed``, ``grep``, ``awk``, and so forth) that you
2867 might wish to use. If in doubt, you should check with multiple
2868 implementations - including those from BusyBox.
2869
2870.. _platdev-newmachine:
2871
2872Adding a New Machine
2873====================
2874
2875Adding a new machine to the Yocto Project is a straightforward process.
2876This section describes how to add machines that are similar to those
2877that the Yocto Project already supports.
2878
2879.. note::
2880
2881 Although well within the capabilities of the Yocto Project, adding a
2882 totally new architecture might require changes to
2883 gcc/glibc
2884 and to the site information, which is beyond the scope of this
2885 manual.
2886
2887For a complete example that shows how to add a new machine, see the
2888":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
2889section in the Yocto Project Board Support Package (BSP) Developer's
2890Guide.
2891
2892.. _platdev-newmachine-conffile:
2893
2894Adding the Machine Configuration File
2895-------------------------------------
2896
2897To add a new machine, you need to add a new machine configuration file
2898to the layer's ``conf/machine`` directory. This configuration file
2899provides details about the device you are adding.
2900
2901The OpenEmbedded build system uses the root name of the machine
2902configuration file to reference the new machine. For example, given a
2903machine configuration file named ``crownbay.conf``, the build system
2904recognizes the machine as "crownbay".
2905
2906The most important variables you must set in your machine configuration
2907file or include from a lower-level configuration file are as follows:
2908
2909- ``TARGET_ARCH`` (e.g. "arm")
2910
2911- ``PREFERRED_PROVIDER_virtual/kernel``
2912
2913- ``MACHINE_FEATURES`` (e.g. "apm screen wifi")
2914
2915You might also need these variables:
2916
2917- ``SERIAL_CONSOLES`` (e.g. "115200;ttyS0 115200;ttyS1")
2918
2919- ``KERNEL_IMAGETYPE`` (e.g. "zImage")
2920
2921- ``IMAGE_FSTYPES`` (e.g. "tar.gz jffs2")
2922
2923You can find full details on these variables in the reference section.
2924You can leverage existing machine ``.conf`` files from
2925``meta-yocto-bsp/conf/machine/``.
2926
2927.. _platdev-newmachine-kernel:
2928
2929Adding a Kernel for the Machine
2930-------------------------------
2931
2932The OpenEmbedded build system needs to be able to build a kernel for the
2933machine. You need to either create a new kernel recipe for this machine,
2934or extend an existing kernel recipe. You can find several kernel recipe
2935examples in the Source Directory at ``meta/recipes-kernel/linux`` that
2936you can use as references.
2937
2938If you are creating a new kernel recipe, normal recipe-writing rules
2939apply for setting up a ``SRC_URI``. Thus, you need to specify any
2940necessary patches and set ``S`` to point at the source code. You need to
2941create a ``do_configure`` task that configures the unpacked kernel with
2942a ``defconfig`` file. You can do this by using a ``make defconfig``
2943command or, more commonly, by copying in a suitable ``defconfig`` file
2944and then running ``make oldconfig``. By making use of ``inherit kernel``
2945and potentially some of the ``linux-*.inc`` files, most other
2946functionality is centralized and the defaults of the class normally work
2947well.
2948
2949If you are extending an existing kernel recipe, it is usually a matter
2950of adding a suitable ``defconfig`` file. The file needs to be added into
2951a location similar to ``defconfig`` files used for other machines in a
2952given kernel recipe. A possible way to do this is by listing the file in
2953the ``SRC_URI`` and adding the machine to the expression in
2954``COMPATIBLE_MACHINE``:
2955::
2956
2957 COMPATIBLE_MACHINE = '(qemux86|qemumips)'
2958
2959For more information on ``defconfig`` files, see the
2960":ref:`kernel-dev/kernel-dev-common:changing the configuration`"
2961section in the Yocto Project Linux Kernel Development Manual.
2962
2963.. _platdev-newmachine-formfactor:
2964
2965Adding a Formfactor Configuration File
2966--------------------------------------
2967
2968A formfactor configuration file provides information about the target
2969hardware for which the image is being built and information that the
2970build system cannot obtain from other sources such as the kernel. Some
2971examples of information contained in a formfactor configuration file
2972include framebuffer orientation, whether or not the system has a
2973keyboard, the positioning of the keyboard in relation to the screen, and
2974the screen resolution.
2975
2976The build system uses reasonable defaults in most cases. However, if
2977customization is necessary, you need to create a ``machconfig`` file in
2978the ``meta/recipes-bsp/formfactor/files`` directory. This directory
2979contains directories for specific machines such as ``qemuarm`` and
2980``qemux86``. For information about the settings available and the
2981defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file
2982found in the same area.
2983
2984Following is an example for "qemuarm" machine:
2985::
2986
2987 HAVE_TOUCHSCREEN=1
2988 HAVE_KEYBOARD=1
2989 DISPLAY_CAN_ROTATE=0
2990 DISPLAY_ORIENTATION=0
2991 #DISPLAY_WIDTH_PIXELS=640
2992 #DISPLAY_HEIGHT_PIXELS=480
2993 #DISPLAY_BPP=16
2994 DISPLAY_DPI=150
2995 DISPLAY_SUBPIXEL_ORDER=vrgb
2996
2997.. _gs-upgrading-recipes:
2998
2999Upgrading Recipes
3000=================
3001
3002Over time, upstream developers publish new versions for software built
3003by layer recipes. It is recommended to keep recipes up-to-date with
3004upstream version releases.
3005
3006While several methods exist that allow you upgrade a recipe, you might
3007consider checking on the upgrade status of a recipe first. You can do so
3008using the ``devtool check-upgrade-status`` command. See the
3009":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
3010section in the Yocto Project Reference Manual for more information.
3011
3012The remainder of this section describes three ways you can upgrade a
3013recipe. You can use the Automated Upgrade Helper (AUH) to set up
3014automatic version upgrades. Alternatively, you can use
3015``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
3016you can manually upgrade a recipe by editing the recipe itself.
3017
3018.. _gs-using-the-auto-upgrade-helper:
3019
3020Using the Auto Upgrade Helper (AUH)
3021-----------------------------------
3022
3023The AUH utility works in conjunction with the OpenEmbedded build system
3024in order to automatically generate upgrades for recipes based on new
3025versions being published upstream. Use AUH when you want to create a
3026service that performs the upgrades automatically and optionally sends
3027you an email with the results.
3028
3029AUH allows you to update several recipes with a single use. You can also
3030optionally perform build and integration tests using images with the
3031results saved to your hard drive and emails of results optionally sent
3032to recipe maintainers. Finally, AUH creates Git commits with appropriate
3033commit messages in the layer's tree for the changes made to recipes.
3034
3035.. note::
3036
3037 Conditions do exist when you should not use AUH to upgrade recipes
3038 and you should instead use either
3039 devtool upgrade
3040 or upgrade your recipes manually:
3041
3042 - When AUH cannot complete the upgrade sequence. This situation
3043 usually results because custom patches carried by the recipe
3044 cannot be automatically rebased to the new version. In this case,
3045 ``devtool upgrade`` allows you to manually resolve conflicts.
3046
3047 - When for any reason you want fuller control over the upgrade
3048 process. For example, when you want special arrangements for
3049 testing.
3050
3051The following steps describe how to set up the AUH utility:
3052
30531. *Be Sure the Development Host is Set Up:* You need to be sure that
3054 your development host is set up to use the Yocto Project. For
3055 information on how to set up your host, see the "`Preparing the Build
3056 Host <#dev-preparing-the-build-host>`__" section.
3057
30582. *Make Sure Git is Configured:* The AUH utility requires Git to be
3059 configured because AUH uses Git to save upgrades. Thus, you must have
3060 Git user and email configured. The following command shows your
3061 configurations:
3062
3063 $ git config --list
3064
3065 If you do not have the user and
3066 email configured, you can use the following commands to do so:
3067 ::
3068
3069 $ git config --global user.name some_name
3070 $ git config --global user.email username@domain.com
3071
30723. *Clone the AUH Repository:* To use AUH, you must clone the repository
3073 onto your development host. The following command uses Git to create
3074 a local copy of the repository on your system:
3075 ::
3076
3077 $ git clone git://git.yoctoproject.org/auto-upgrade-helper
3078 Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done.
3079 remote: Compressing objects: 100% (300/300), done.
3080 remote: Total 768 (delta 499), reused 703 (delta 434)
3081 Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
3082 Resolving deltas: 100% (499/499), done.
3083 Checking connectivity... done.
3084
3085 AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or
3086 :term:`Poky` repositories.
3087
30884. *Create a Dedicated Build Directory:* Run the
3089 :ref:`structure-core-script`
3090 script to create a fresh build directory that you use exclusively for
3091 running the AUH utility:
3092 ::
3093
3094 $ cd ~/poky
3095 $ source oe-init-build-env
3096
3097 your_AUH_build_directory Re-using an existing build directory and its
3098 configurations is not recommended as existing settings could cause
3099 AUH to fail or behave undesirably.
3100
31015. *Make Configurations in Your Local Configuration File:* Several
3102 settings need to exist in the ``local.conf`` file in the build
3103 directory you just created for AUH. Make these following
3104 configurations:
3105
3106 - If you want to enable :ref:`Build
3107 History <dev-manual/dev-manual-common-tasks:maintaining build output quality>`,
3108 which is optional, you need the following lines in the
3109 ``conf/local.conf`` file:
3110 ::
3111
3112 INHERIT =+ "buildhistory"
3113 BUILDHISTORY_COMMIT = "1"
3114
3115 With this configuration and a successful
3116 upgrade, a build history "diff" file appears in the
3117 ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
3118 your build directory.
3119
3120 - If you want to enable testing through the
3121 :ref:`testimage <ref-classes-testimage*>`
3122 class, which is optional, you need to have the following set in
3123 your ``conf/local.conf`` file: INHERIT += "testimage"
3124
3125 .. note::
3126
3127 If your distro does not enable by default ptest, which Poky
3128 does, you need the following in your
3129 local.conf
3130 file:
3131 ::
3132
3133 DISTRO_FEATURES_append = " ptest"
3134
3135
31366. *Optionally Start a vncserver:* If you are running in a server
3137 without an X11 session, you need to start a vncserver:
3138 ::
3139
3140 $ vncserver :1
3141 $ export DISPLAY=:1
3142
31437. *Create and Edit an AUH Configuration File:* You need to have the
3144 ``upgrade-helper/upgrade-helper.conf`` configuration file in your
3145 build directory. You can find a sample configuration file in the `AUH
3146 source
3147 repository <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/>`__.
3148
3149 Read through the sample file and make configurations as needed. For
3150 example, if you enabled build history in your ``local.conf`` as
3151 described earlier, you must enable it in ``upgrade-helper.conf``.
3152
3153 Also, if you are using the default ``maintainers.inc`` file supplied
3154 with Poky and located in ``meta-yocto`` and you do not set a
3155 "maintainers_whitelist" or "global_maintainer_override" in the
3156 ``upgrade-helper.conf`` configuration, and you specify "-e all" on
3157 the AUH command-line, the utility automatically sends out emails to
3158 all the default maintainers. Please avoid this.
3159
3160This next set of examples describes how to use the AUH:
3161
3162- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
3163 following form: $ upgrade-helper.py recipe_name For example, this
3164 command upgrades the ``xmodmap`` recipe:
3165 ::
3166
3167 $ upgrade-helper.py xmodmap
3168
3169- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
3170 specific recipe to a particular version, use the following form: $
3171 upgrade-helper.py recipe_name -t version For example, this command
3172 upgrades the ``xmodmap`` recipe to version 1.2.3:
3173 ::
3174
3175 $ upgrade-helper.py xmodmap -t 1.2.3
3176
3177- *Upgrading all Recipes to the Latest Versions and Suppressing Email
3178 Notifications:* To upgrade all recipes to their most recent versions
3179 and suppress the email notifications, use the following command:
3180 ::
3181
3182 $ upgrade-helper.py all
3183
3184- *Upgrading all Recipes to the Latest Versions and Send Email
3185 Notifications:* To upgrade all recipes to their most recent versions
3186 and send email messages to maintainers for each attempted recipe as
3187 well as a status email, use the following command:
3188 ::
3189
3190 $ upgrade-helper.py -e all
3191
3192Once you have run the AUH utility, you can find the results in the AUH
3193build directory:
3194::
3195
3196 ${BUILDDIR}/upgrade-helper/timestamp
3197
3198The AUH utility
3199also creates recipe update commits from successful upgrade attempts in
3200the layer tree.
3201
3202You can easily set up to run the AUH utility on a regular basis by using
3203a cron job. See the
3204`weeklyjob.sh <http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh>`_
3205file distributed with the utility for an example.
3206
3207.. _gs-using-devtool-upgrade:
3208
3209Using ``devtool upgrade``
3210-------------------------
3211
3212As mentioned earlier, an alternative method for upgrading recipes to
3213newer versions is to use
3214:doc:`devtool upgrade <../ref-manual/ref-devtool-reference>`.
3215You can read about ``devtool upgrade`` in general in the
3216":ref:`sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software`"
3217section in the Yocto Project Application Development and the Extensible
3218Software Development Kit (eSDK) Manual.
3219
3220To see all the command-line options available with ``devtool upgrade``,
3221use the following help command:
3222::
3223
3224 $ devtool upgrade -h
3225
3226If you want to find out what version a recipe is currently at upstream
3227without any attempt to upgrade your local version of the recipe, you can
3228use the following command:
3229::
3230
3231 $ devtool latest-version recipe_name
3232
3233As mentioned in the previous section describing AUH, ``devtool upgrade``
3234works in a less-automated manner than AUH. Specifically,
3235``devtool upgrade`` only works on a single recipe that you name on the
3236command line, cannot perform build and integration testing using images,
3237and does not automatically generate commits for changes in the source
3238tree. Despite all these "limitations", ``devtool upgrade`` updates the
3239recipe file to the new upstream version and attempts to rebase custom
3240patches contained by the recipe as needed.
3241
3242.. note::
3243
3244 AUH uses much of
3245 devtool upgrade
3246 behind the scenes making AUH somewhat of a "wrapper" application for
3247 devtool upgrade
3248 .
3249
3250A typical scenario involves having used Git to clone an upstream
3251repository that you use during build operations. Because you are (or
3252have) built the recipe in the past, the layer is likely added to your
3253configuration already. If for some reason, the layer is not added, you
3254could add it easily using the
3255":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`"
3256script. For example, suppose you use the ``nano.bb`` recipe from the
3257``meta-oe`` layer in the ``meta-openembedded`` repository. For this
3258example, assume that the layer has been cloned into following area:
3259::
3260
3261 /home/scottrif/meta-openembedded
3262
3263The following command from your
3264:term:`Build Directory` adds the layer to
3265your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``):
3266::
3267
3268 $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
3269 NOTE: Starting bitbake server...
3270 Parsing recipes: 100% |##########################################| Time: 0:00:55
3271 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
3272 Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
3273 Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
3274 Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
3275 Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
3276
3277For this example, assume that the ``nano.bb`` recipe that
3278is upstream has a 2.9.3 version number. However, the version in the
3279local repository is 2.7.4. The following command from your build
3280directory automatically upgrades the recipe for you:
3281
3282.. note::
3283
3284 Using the
3285 -V
3286 option is not necessary. Omitting the version number causes
3287 devtool upgrade
3288 to upgrade the recipe to the most recent version.
3289
3290::
3291
3292 $ devtool upgrade nano -V 2.9.3
3293 NOTE: Starting bitbake server...
3294 NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
3295 Parsing recipes: 100% |##########################################| Time: 0:00:46
3296 Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
3297 NOTE: Extracting current version source...
3298 NOTE: Resolving any missing task queue dependencies
3299 .
3300 .
3301 .
3302 NOTE: Executing SetScene Tasks
3303 NOTE: Executing RunQueue Tasks
3304 NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
3305 Adding changed files: 100% |#####################################| Time: 0:00:00
3306 NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
3307 NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
3308
3309Continuing with this example, you can use ``devtool build`` to build the
3310newly upgraded recipe:
3311::
3312
3313 $ devtool build nano
3314 NOTE: Starting bitbake server...
3315 Loading cache: 100% |################################################################################################| Time: 0:00:01
3316 Loaded 2040 entries from dependency cache.
3317 Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
3318 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
3319 NOTE: Resolving any missing task queue dependencies
3320 .
3321 .
3322 .
3323 NOTE: Executing SetScene Tasks
3324 NOTE: Executing RunQueue Tasks
3325 NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
3326 NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
3327
3328Within the ``devtool upgrade`` workflow, opportunity
3329exists to deploy and test your rebuilt software. For this example,
3330however, running ``devtool finish`` cleans up the workspace once the
3331source in your workspace is clean. This usually means using Git to stage
3332and submit commits for the changes generated by the upgrade process.
3333
3334Once the tree is clean, you can clean things up in this example with the
3335following command from the ``${BUILDDIR}/workspace/sources/nano``
3336directory:
3337::
3338
3339 $ devtool finish nano meta-oe
3340 NOTE: Starting bitbake server...
3341 Loading cache: 100% |################################################################################################| Time: 0:00:00
3342 Loaded 2040 entries from dependency cache.
3343 Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
3344 Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
3345 NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
3346 NOTE: Updating recipe nano_2.9.3.bb
3347 NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
3348 NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
3349 NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
3350
3351
3352Using the ``devtool finish`` command cleans up the workspace and creates a patch
3353file based on your commits. The tool puts all patch files back into the
3354source directory in a sub-directory named ``nano`` in this case.
3355
3356.. _dev-manually-upgrading-a-recipe:
3357
3358Manually Upgrading a Recipe
3359---------------------------
3360
3361If for some reason you choose not to upgrade recipes using the `Auto
3362Upgrade Helper (AUH) <#gs-using-the-auto-upgrade-helper>`__ or by using
3363```devtool upgrade`` <#gs-using-devtool-upgrade>`__, you can manually
3364edit the recipe files to upgrade the versions.
3365
3366.. note::
3367
3368 Manually updating multiple recipes scales poorly and involves many
3369 steps. The recommendation to upgrade recipe versions is through AUH
3370 or
3371 devtool upgrade
3372 , both of which automate some steps and provide guidance for others
3373 needed for the manual process.
3374
3375To manually upgrade recipe versions, follow these general steps:
3376
33771. *Change the Version:* Rename the recipe such that the version (i.e.
3378 the :term:`PV` part of the recipe name)
3379 changes appropriately. If the version is not part of the recipe name,
3380 change the value as it is set for ``PV`` within the recipe itself.
3381
33822. Update ``SRCREV`` if Needed: If the source code your recipe builds
3383 is fetched from Git or some other version control system, update
3384 :term:`SRCREV` to point to the
3385 commit hash that matches the new version.
3386
33873. *Build the Software:* Try to build the recipe using BitBake. Typical
3388 build failures include the following:
3389
3390 - License statements were updated for the new version. For this
3391 case, you need to review any changes to the license and update the
3392 values of :term:`LICENSE` and
3393 :term:`LIC_FILES_CHKSUM`
3394 as needed.
3395
3396 .. note::
3397
3398 License changes are often inconsequential. For example, the
3399 license text's copyright year might have changed.
3400
3401 - Custom patches carried by the older version of the recipe might
3402 fail to apply to the new version. For these cases, you need to
3403 review the failures. Patches might not be necessary for the new
3404 version of the software if the upgraded version has fixed those
3405 issues. If a patch is necessary and failing, you need to rebase it
3406 into the new version.
3407
34084. *Optionally Attempt to Build for Several Architectures:* Once you
3409 successfully build the new software for a given architecture, you
3410 could test the build for other architectures by changing the
3411 :term:`MACHINE` variable and
3412 rebuilding the software. This optional step is especially important
3413 if the recipe is to be released publicly.
3414
34155. *Check the Upstream Change Log or Release Notes:* Checking both these
3416 reveals if new features exist that could break
3417 backwards-compatibility. If so, you need to take steps to mitigate or
3418 eliminate that situation.
3419
34206. *Optionally Create a Bootable Image and Test:* If you want, you can
3421 test the new software by booting it onto actual hardware.
3422
34237. *Create a Commit with the Change in the Layer Repository:* After all
3424 builds work and any testing is successful, you can create commits for
3425 any changes in the layer holding your upgraded recipe.
3426
3427.. _finding-the-temporary-source-code:
3428
3429Finding Temporary Source Code
3430=============================
3431
3432You might find it helpful during development to modify the temporary
3433source code used by recipes to build packages. For example, suppose you
3434are developing a patch and you need to experiment a bit to figure out
3435your solution. After you have initially built the package, you can
3436iteratively tweak the source code, which is located in the
3437:term:`Build Directory`, and then you can
3438force a re-compile and quickly test your altered code. Once you settle
3439on a solution, you can then preserve your changes in the form of
3440patches.
3441
3442During a build, the unpacked temporary source code used by recipes to
3443build packages is available in the Build Directory as defined by the
3444:term:`S` variable. Below is the default
3445value for the ``S`` variable as defined in the
3446``meta/conf/bitbake.conf`` configuration file in the
3447:term:`Source Directory`:
3448::
3449
3450 S = "${WORKDIR}/${BP}"
3451
3452You should be aware that many recipes override the
3453``S`` variable. For example, recipes that fetch their source from Git
3454usually set ``S`` to ``${WORKDIR}/git``.
3455
3456.. note::
3457
3458 The
3459 BP
3460 represents the base recipe name, which consists of the name and
3461 version:
3462 ::
3463
3464 BP = "${BPN}-${PV}"
3465
3466
3467The path to the work directory for the recipe
3468(:term:`WORKDIR`) is defined as
3469follows:
3470${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} The
3471actual directory depends on several things:
3472
3473- :term:`TMPDIR`: The top-level build
3474 output directory.
3475
3476- :term:`MULTIMACH_TARGET_SYS`:
3477 The target system identifier.
3478
3479- :term:`PN`: The recipe name.
3480
3481- :term:`EXTENDPE`: The epoch - (if
3482 :term:`PE` is not specified, which is
3483 usually the case for most recipes, then ``EXTENDPE`` is blank).
3484
3485- :term:`PV`: The recipe version.
3486
3487- :term:`PR`: The recipe revision.
3488
3489As an example, assume a Source Directory top-level folder named
3490``poky``, a default Build Directory at ``poky/build``, and a
3491``qemux86-poky-linux`` machine target system. Furthermore, suppose your
3492recipe is named ``foo_1.3.0.bb``. In this case, the work directory the
3493build system uses to build the package would be as follows:
3494::
3495
3496 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
3497
3498.. _using-a-quilt-workflow:
3499
3500Using Quilt in Your Workflow
3501============================
3502
3503`Quilt <http://savannah.nongnu.org/projects/quilt>`__ is a powerful tool
3504that allows you to capture source code changes without having a clean
3505source tree. This section outlines the typical workflow you can use to
3506modify source code, test changes, and then preserve the changes in the
3507form of a patch all using Quilt.
3508
3509.. note::
3510
3511 With regard to preserving changes to source files, if you clean a
3512 recipe or have
3513 rm_work
3514 enabled, the
3515 devtool
3516 workflow
3517 as described in the Yocto Project Application Development and the
3518 Extensible Software Development Kit (eSDK) manual is a safer
3519 development flow than the flow that uses Quilt.
3520
3521Follow these general steps:
3522
35231. *Find the Source Code:* Temporary source code used by the
3524 OpenEmbedded build system is kept in the
3525 :term:`Build Directory`. See the
3526 "`Finding Temporary Source
3527 Code <#finding-the-temporary-source-code>`__" section to learn how to
3528 locate the directory that has the temporary source code for a
3529 particular package.
3530
35312. *Change Your Working Directory:* You need to be in the directory that
3532 has the temporary source code. That directory is defined by the
3533 :term:`S` variable.
3534
35353. *Create a New Patch:* Before modifying source code, you need to
3536 create a new patch. To create a new patch file, use ``quilt new`` as
3537 below:
3538 :;
3539
3540 $ quilt new my_changes.patch
3541
35424. *Notify Quilt and Add Files:* After creating the patch, you need to
3543 notify Quilt about the files you plan to edit. You notify Quilt by
3544 adding the files to the patch you just created:
3545 ::
3546
3547 $ quilt add file1.c file2.c file3.c
3548
35495. *Edit the Files:* Make your changes in the source code to the files
3550 you added to the patch.
3551
35526. *Test Your Changes:* Once you have modified the source code, the
3553 easiest way to test your changes is by calling the ``do_compile``
3554 task as shown in the following example:
3555 ::
3556
3557 $ bitbake -c compile -f package
3558
3559 The ``-f`` or ``--force`` option forces the specified task to
3560 execute. If you find problems with your code, you can just keep
3561 editing and re-testing iteratively until things work as expected.
3562
3563 .. note::
3564
3565 All the modifications you make to the temporary source code
3566 disappear once you run the
3567 do_clean
3568 or
3569 do_cleanall
3570 tasks using BitBake (i.e.
3571 bitbake -c clean
3572 package
3573 and
3574 bitbake -c cleanall
3575 package
3576 ). Modifications will also disappear if you use the
3577 rm_work
3578 feature as described in the "
3579 Conserving Disk Space During Builds
3580 " section.
3581
35827. *Generate the Patch:* Once your changes work as expected, you need to
3583 use Quilt to generate the final patch that contains all your
3584 modifications.
3585 ::
3586
3587 $ quilt refresh
3588
3589 At this point, the
3590 ``my_changes.patch`` file has all your edits made to the ``file1.c``,
3591 ``file2.c``, and ``file3.c`` files.
3592
3593 You can find the resulting patch file in the ``patches/``
3594 subdirectory of the source (``S``) directory.
3595
35968. *Copy the Patch File:* For simplicity, copy the patch file into a
3597 directory named ``files``, which you can create in the same directory
3598 that holds the recipe (``.bb``) file or the append (``.bbappend``)
3599 file. Placing the patch here guarantees that the OpenEmbedded build
3600 system will find the patch. Next, add the patch into the ``SRC_URI``
3601 of the recipe. Here is an example:
3602 ::
3603
3604 SRC_URI += "file://my_changes.patch"
3605
3606.. _platdev-appdev-devshell:
3607
3608Using a Development Shell
3609=========================
3610
3611When debugging certain commands or even when just editing packages,
3612``devshell`` can be a useful tool. When you invoke ``devshell``, all
3613tasks up to and including
3614:ref:`ref-tasks-patch` are run for the
3615specified target. Then, a new terminal is opened and you are placed in
3616``${``\ :term:`S`\ ``}``, the source
3617directory. In the new terminal, all the OpenEmbedded build-related
3618environment variables are still defined so you can use commands such as
3619``configure`` and ``make``. The commands execute just as if the
3620OpenEmbedded build system were executing them. Consequently, working
3621this way can be helpful when debugging a build or preparing software to
3622be used with the OpenEmbedded build system.
3623
3624Following is an example that uses ``devshell`` on a target named
3625``matchbox-desktop``:
3626::
3627
3628 $ bitbake matchbox-desktop -c devshell
3629
3630This command spawns a terminal with a shell prompt within the
3631OpenEmbedded build environment. The
3632:term:`OE_TERMINAL` variable
3633controls what type of shell is opened.
3634
3635For spawned terminals, the following occurs:
3636
3637- The ``PATH`` variable includes the cross-toolchain.
3638
3639- The ``pkgconfig`` variables find the correct ``.pc`` files.
3640
3641- The ``configure`` command finds the Yocto Project site files as well
3642 as any other necessary files.
3643
3644Within this environment, you can run configure or compile commands as if
3645they were being run by the OpenEmbedded build system itself. As noted
3646earlier, the working directory also automatically changes to the Source
3647Directory (:term:`S`).
3648
3649To manually run a specific task using ``devshell``, run the
3650corresponding ``run.*`` script in the
3651``${``\ :term:`WORKDIR`\ ``}/temp``
3652directory (e.g., ``run.do_configure.``\ pid). If a task's script does
3653not exist, which would be the case if the task was skipped by way of the
3654sstate cache, you can create the task by first running it outside of the
3655``devshell``:
3656::
3657
3658 $ bitbake -c task
3659
3660.. note::
3661
3662 - Execution of a task's ``run.*`` script and BitBake's execution of
3663 a task are identical. In other words, running the script re-runs
3664 the task just as it would be run using the ``bitbake -c`` command.
3665
3666 - Any ``run.*`` file that does not have a ``.pid`` extension is a
3667 symbolic link (symlink) to the most recent version of that file.
3668
3669Remember, that the ``devshell`` is a mechanism that allows you to get
3670into the BitBake task execution environment. And as such, all commands
3671must be called just as BitBake would call them. That means you need to
3672provide the appropriate options for cross-compilation and so forth as
3673applicable.
3674
3675When you are finished using ``devshell``, exit the shell or close the
3676terminal window.
3677
3678.. note::
3679
3680 - It is worth remembering that when using ``devshell`` you need to
3681 use the full compiler name such as ``arm-poky-linux-gnueabi-gcc``
3682 instead of just using ``gcc``. The same applies to other
3683 applications such as ``binutils``, ``libtool`` and so forth.
3684 BitBake sets up environment variables such as ``CC`` to assist
3685 applications, such as ``make`` to find the correct tools.
3686
3687 - It is also worth noting that ``devshell`` still works over X11
3688 forwarding and similar situations.
3689
3690.. _platdev-appdev-devpyshell:
3691
3692Using a Development Python Shell
3693================================
3694
3695Similar to working within a development shell as described in the
3696previous section, you can also spawn and work within an interactive
3697Python development shell. When debugging certain commands or even when
3698just editing packages, ``devpyshell`` can be a useful tool. When you
3699invoke ``devpyshell``, all tasks up to and including
3700:ref:`ref-tasks-patch` are run for the
3701specified target. Then a new terminal is opened. Additionally, key
3702Python objects and code are available in the same way they are to
3703BitBake tasks, in particular, the data store 'd'. So, commands such as
3704the following are useful when exploring the data store and running
3705functions:
3706::
3707
3708 pydevshell> d.getVar("STAGING_DIR")
3709 '/media/build1/poky/build/tmp/sysroots'
3710 pydevshell> d.getVar("STAGING_DIR")
3711 '${TMPDIR}/sysroots'
3712 pydevshell> d.setVar("FOO", "bar")
3713 pydevshell> d.getVar("FOO")
3714 'bar'
3715 pydevshell> d.delVar("FOO")
3716 pydevshell> d.getVar("FOO")
3717 pydevshell> bb.build.exec_func("do_unpack", d)
3718 pydevshell>
3719
3720The commands execute just as if the OpenEmbedded build
3721system were executing them. Consequently, working this way can be
3722helpful when debugging a build or preparing software to be used with the
3723OpenEmbedded build system.
3724
3725Following is an example that uses ``devpyshell`` on a target named
3726``matchbox-desktop``:
3727::
3728
3729 $ bitbake matchbox-desktop -c devpyshell
3730
3731This command spawns a terminal and places you in an interactive Python
3732interpreter within the OpenEmbedded build environment. The
3733:term:`OE_TERMINAL` variable
3734controls what type of shell is opened.
3735
3736When you are finished using ``devpyshell``, you can exit the shell
3737either by using Ctrl+d or closing the terminal window.
3738
3739.. _dev-building:
3740
3741Building
3742========
3743
3744This section describes various build procedures. For example, the steps
3745needed for a simple build, a target that uses multiple configurations,
3746building an image for more than one machine, and so forth.
3747
3748.. _dev-building-a-simple-image:
3749
3750Building a Simple Image
3751-----------------------
3752
3753In the development environment, you need to build an image whenever you
3754change hardware support, add or change system libraries, or add or
3755change services that have dependencies. Several methods exist that allow
3756you to build an image within the Yocto Project. This section presents
3757the basic steps you need to build a simple image using BitBake from a
3758build host running Linux.
3759
3760.. note::
3761
3762 - For information on how to build an image using
3763 :term:`Toaster`, see the
3764 :doc:`../toaster-manual/toaster-manual`.
3765
3766 - For information on how to use ``devtool`` to build images, see the
3767 ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`"
3768 section in the Yocto Project Application Development and the
3769 Extensible Software Development Kit (eSDK) manual.
3770
3771 - For a quick example on how to build an image using the
3772 OpenEmbedded build system, see the
3773 :doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document.
3774
3775The build process creates an entire Linux distribution from source and
3776places it in your :term:`Build Directory` under
3777``tmp/deploy/images``. For detailed information on the build process
3778using BitBake, see the ":ref:`images-dev-environment`" section in the
3779Yocto Project Overview and Concepts Manual.
3780
3781The following figure and list overviews the build process:
3782
3783.. image:: figures/bitbake-build-flow.png
3784 :align: center
3785
37861. *Set up Your Host Development System to Support Development Using the
3787 Yocto Project*: See the "`Setting Up to Use the Yocto
3788 Project <#dev-manual-start>`__" section for options on how to get a
3789 build host ready to use the Yocto Project.
3790
37912. *Initialize the Build Environment:* Initialize the build environment
3792 by sourcing the build environment script (i.e.
3793 :ref:`structure-core-script`):
3794 ::
3795
3796 $ source oe-init-build-env [build_dir]
3797
3798 When you use the initialization script, the OpenEmbedded build system
3799 uses ``build`` as the default Build Directory in your current work
3800 directory. You can use a build_dir argument with the script to
3801 specify a different build directory.
3802
3803 .. note::
3804
3805 A common practice is to use a different Build Directory for
3806 different targets. For example,
3807 ~/build/x86
3808 for a
3809 qemux86
3810 target, and
3811 ~/build/arm
3812 for a
3813 qemuarm
3814 target.
3815
38163. Make Sure Your ``local.conf`` File is Correct: Ensure the
3817 ``conf/local.conf`` configuration file, which is found in the Build
3818 Directory, is set up how you want it. This file defines many aspects
3819 of the build environment including the target machine architecture
3820 through the ``MACHINE`` variable, the packaging format used during
3821 the build
3822 (:term:`PACKAGE_CLASSES`),
3823 and a centralized tarball download directory through the
3824 :term:`DL_DIR` variable.
3825
38264. *Build the Image:* Build the image using the ``bitbake`` command:
3827 ::
3828
3829 $ bitbake target
3830
3831 .. note::
3832
3833 For information on BitBake, see the
3834 BitBake User Manual
3835 .
3836
3837 The target is the name of the recipe you want to build. Common
3838 targets are the images in ``meta/recipes-core/images``,
3839 ``meta/recipes-sato/images``, and so forth all found in the
3840 :term:`Source Directory`. Or, the target
3841 can be the name of a recipe for a specific piece of software such as
3842 BusyBox. For more details about the images the OpenEmbedded build
3843 system supports, see the
3844 ":ref:`ref-manual/ref-images:Images`" chapter in the Yocto
3845 Project Reference Manual.
3846
3847 As an example, the following command builds the
3848 ``core-image-minimal`` image:
3849 ::
3850
3851 $ bitbake core-image-minimal
3852
3853 Once an
3854 image has been built, it often needs to be installed. The images and
3855 kernels built by the OpenEmbedded build system are placed in the
3856 Build Directory in ``tmp/deploy/images``. For information on how to
3857 run pre-built images such as ``qemux86`` and ``qemuarm``, see the
3858 :doc:`../sdk-manual/sdk-manual` manual. For
3859 information about how to install these images, see the documentation
3860 for your particular board or machine.
3861
3862.. _dev-building-images-for-multiple-targets-using-multiple-configurations:
3863
3864Building Images for Multiple Targets Using Multiple Configurations
3865------------------------------------------------------------------
3866
3867You can use a single ``bitbake`` command to build multiple images or
3868packages for different targets where each image or package requires a
3869different configuration (multiple configuration builds). The builds, in
3870this scenario, are sometimes referred to as "multiconfigs", and this
3871section uses that term throughout.
3872
3873This section describes how to set up for multiple configuration builds
3874and how to account for cross-build dependencies between the
3875multiconfigs.
3876
3877.. _dev-setting-up-and-running-a-multiple-configuration-build:
3878
3879Setting Up and Running a Multiple Configuration Build
3880~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3881
3882To accomplish a multiple configuration build, you must define each
3883target's configuration separately using a parallel configuration file in
3884the :term:`Build Directory`, and you
3885must follow a required file hierarchy. Additionally, you must enable the
3886multiple configuration builds in your ``local.conf`` file.
3887
3888Follow these steps to set up and execute multiple configuration builds:
3889
3890- *Create Separate Configuration Files*: You need to create a single
3891 configuration file for each build target (each multiconfig).
3892 Minimally, each configuration file must define the machine and the
3893 temporary directory BitBake uses for the build. Suggested practice
3894 dictates that you do not overlap the temporary directories used
3895 during the builds. However, it is possible that you can share the
3896 temporary directory
3897 (:term:`TMPDIR`). For example,
3898 consider a scenario with two different multiconfigs for the same
3899 :term:`MACHINE`: "qemux86" built
3900 for two distributions such as "poky" and "poky-lsb". In this case,
3901 you might want to use the same ``TMPDIR``.
3902
3903 Here is an example showing the minimal statements needed in a
3904 configuration file for a "qemux86" target whose temporary build
3905 directory is ``tmpmultix86``:
3906 ::
3907
3908 MACHINE = "qemux86"
3909 TMPDIR = "${TOPDIR}/tmpmultix86"
3910
3911 The location for these multiconfig configuration files is specific.
3912 They must reside in the current build directory in a sub-directory of
3913 ``conf`` named ``multiconfig``. Following is an example that defines
3914 two configuration files for the "x86" and "arm" multiconfigs:
3915
3916 .. image:: figures/multiconfig_files.png
3917 :align: center
3918
3919 The reason for this required file hierarchy is because the ``BBPATH``
3920 variable is not constructed until the layers are parsed.
3921 Consequently, using the configuration file as a pre-configuration
3922 file is not possible unless it is located in the current working
3923 directory.
3924
3925- *Add the BitBake Multi-configuration Variable to the Local
3926 Configuration File*: Use the
3927 :term:`BBMULTICONFIG`
3928 variable in your ``conf/local.conf`` configuration file to specify
3929 each multiconfig. Continuing with the example from the previous
3930 figure, the ``BBMULTICONFIG`` variable needs to enable two
3931 multiconfigs: "x86" and "arm" by specifying each configuration file:
3932 ::
3933
3934 BBMULTICONFIG = "x86 arm"
3935
3936 .. note::
3937
3938 A "default" configuration already exists by definition. This
3939 configuration is named: "" (i.e. empty string) and is defined by
3940 the variables coming from your
3941 local.conf
3942 file. Consequently, the previous example actually adds two
3943 additional configurations to your build: "arm" and "x86" along
3944 with "".
3945
3946- *Launch BitBake*: Use the following BitBake command form to launch
3947 the multiple configuration build:
3948 ::
3949
3950 $ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ]
3951
3952 For the example in this section, the following command applies:
3953 ::
3954
3955 $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
3956
3957 The previous BitBake command builds a ``core-image-minimal`` image
3958 that is configured through the ``x86.conf`` configuration file, a
3959 ``core-image-sato`` image that is configured through the ``arm.conf``
3960 configuration file and a ``core-image-base`` that is configured
3961 through your ``local.conf`` configuration file.
3962
3963.. note::
3964
3965 Support for multiple configuration builds in the Yocto Project DISTRO
3966 (DISTRO_NAME) Release does not include Shared State (sstate)
3967 optimizations. Consequently, if a build uses the same object twice
3968 in, for example, two different
3969 TMPDIR
3970 directories, the build either loads from an existing sstate cache for
3971 that build at the start or builds the object fresh.
3972
3973.. _dev-enabling-multiple-configuration-build-dependencies:
3974
3975Enabling Multiple Configuration Build Dependencies
3976~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3977
3978Sometimes dependencies can exist between targets (multiconfigs) in a
3979multiple configuration build. For example, suppose that in order to
3980build a ``core-image-sato`` image for an "x86" multiconfig, the root
3981filesystem of an "arm" multiconfig must exist. This dependency is
3982essentially that the
3983:ref:`ref-tasks-image` task in the
3984``core-image-sato`` recipe depends on the completion of the
3985:ref:`ref-tasks-rootfs` task of the
3986``core-image-minimal`` recipe.
3987
3988To enable dependencies in a multiple configuration build, you must
3989declare the dependencies in the recipe using the following statement
3990form:
3991::
3992
3993 task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
3994
3995To better show how to use this statement, consider the example scenario
3996from the first paragraph of this section. The following statement needs
3997to be added to the recipe that builds the ``core-image-sato`` image:
3998::
3999
4000 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
4001
4002In this example, the from_multiconfig is "x86". The to_multiconfig is "arm". The
4003task on which the ``do_image`` task in the recipe depends is the
4004``do_rootfs`` task from the ``core-image-minimal`` recipe associated
4005with the "arm" multiconfig.
4006
4007Once you set up this dependency, you can build the "x86" multiconfig
4008using a BitBake command as follows:
4009::
4010
4011 $ bitbake mc:x86:core-image-sato
4012
4013This command executes all the tasks needed to create the
4014``core-image-sato`` image for the "x86" multiconfig. Because of the
4015dependency, BitBake also executes through the ``do_rootfs`` task for the
4016"arm" multiconfig build.
4017
4018Having a recipe depend on the root filesystem of another build might not
4019seem that useful. Consider this change to the statement in the
4020``core-image-sato`` recipe:
4021::
4022
4023 do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
4024
4025In this case, BitBake must
4026create the ``core-image-minimal`` image for the "arm" build since the
4027"x86" build depends on it.
4028
4029Because "x86" and "arm" are enabled for multiple configuration builds
4030and have separate configuration files, BitBake places the artifacts for
4031each build in the respective temporary build directories (i.e.
4032:term:`TMPDIR`).
4033
4034.. _building-an-initramfs-image:
4035
4036Building an Initial RAM Filesystem (initramfs) Image
4037----------------------------------------------------
4038
4039An initial RAM filesystem (initramfs) image provides a temporary root
4040filesystem used for early system initialization (e.g. loading of modules
4041needed to locate and mount the "real" root filesystem).
4042
4043.. note::
4044
4045 The initramfs image is the successor of initial RAM disk (initrd). It
4046 is a "copy in and out" (cpio) archive of the initial filesystem that
4047 gets loaded into memory during the Linux startup process. Because
4048 Linux uses the contents of the archive during initialization, the
4049 initramfs image needs to contain all of the device drivers and tools
4050 needed to mount the final root filesystem.
4051
4052Follow these steps to create an initramfs image:
4053
40541. *Create the initramfs Image Recipe:* You can reference the
4055 ``core-image-minimal-initramfs.bb`` recipe found in the
4056 ``meta/recipes-core`` directory of the :term:`Source Directory`
4057 as an example
4058 from which to work.
4059
40602. *Decide if You Need to Bundle the initramfs Image Into the Kernel
4061 Image:* If you want the initramfs image that is built to be bundled
4062 in with the kernel image, set the
4063 :term:`INITRAMFS_IMAGE_BUNDLE`
4064 variable to "1" in your ``local.conf`` configuration file and set the
4065 :term:`INITRAMFS_IMAGE`
4066 variable in the recipe that builds the kernel image.
4067
4068 .. note::
4069
4070 It is recommended that you do bundle the initramfs image with the
4071 kernel image to avoid circular dependencies between the kernel
4072 recipe and the initramfs recipe should the initramfs image include
4073 kernel modules.
4074
4075 Setting the ``INITRAMFS_IMAGE_BUNDLE`` flag causes the initramfs
4076 image to be unpacked into the ``${B}/usr/`` directory. The unpacked
4077 initramfs image is then passed to the kernel's ``Makefile`` using the
4078 :term:`CONFIG_INITRAMFS_SOURCE`
4079 variable, allowing the initramfs image to be built into the kernel
4080 normally.
4081
4082 .. note::
4083
4084 If you choose to not bundle the initramfs image with the kernel
4085 image, you are essentially using an
4086 Initial RAM Disk (initrd)
4087 . Creating an initrd is handled primarily through the
4088 INITRD_IMAGE
4089 ,
4090 INITRD_LIVE
4091 , and
4092 INITRD_IMAGE_LIVE
4093 variables. For more information, see the
4094 image-live.bbclass
4095 file.
4096
40973. *Optionally Add Items to the initramfs Image Through the initramfs
4098 Image Recipe:* If you add items to the initramfs image by way of its
4099 recipe, you should use
4100 :term:`PACKAGE_INSTALL`
4101 rather than
4102 :term:`IMAGE_INSTALL`.
4103 ``PACKAGE_INSTALL`` gives more direct control of what is added to the
4104 image as compared to the defaults you might not necessarily want that
4105 are set by the :ref:`image <ref-classes-image>`
4106 or :ref:`core-image <ref-classes-core-image>`
4107 classes.
4108
41094. *Build the Kernel Image and the initramfs Image:* Build your kernel
4110 image using BitBake. Because the initramfs image recipe is a
4111 dependency of the kernel image, the initramfs image is built as well
4112 and bundled with the kernel image if you used the
4113 :term:`INITRAMFS_IMAGE_BUNDLE`
4114 variable described earlier.
4115
4116Building a Tiny System
4117----------------------
4118
4119Very small distributions have some significant advantages such as
4120requiring less on-die or in-package memory (cheaper), better performance
4121through efficient cache usage, lower power requirements due to less
4122memory, faster boot times, and reduced development overhead. Some
4123real-world examples where a very small distribution gives you distinct
4124advantages are digital cameras, medical devices, and small headless
4125systems.
4126
4127This section presents information that shows you how you can trim your
4128distribution to even smaller sizes than the ``poky-tiny`` distribution,
4129which is around 5 Mbytes, that can be built out-of-the-box using the
4130Yocto Project.
4131
4132.. _tiny-system-overview:
4133
4134Tiny System Overview
4135~~~~~~~~~~~~~~~~~~~~
4136
4137The following list presents the overall steps you need to consider and
4138perform to create distributions with smaller root filesystems, achieve
4139faster boot times, maintain your critical functionality, and avoid
4140initial RAM disks:
4141
4142- `Determine your goals and guiding
4143 principles. <#goals-and-guiding-principles>`__
4144
4145- `Understand what contributes to your image
4146 size. <#understand-what-gives-your-image-size>`__
4147
4148- `Reduce the size of the root
4149 filesystem. <#trim-the-root-filesystem>`__
4150
4151- `Reduce the size of the kernel. <#trim-the-kernel>`__
4152
4153- `Eliminate packaging
4154 requirements. <#remove-package-management-requirements>`__
4155
4156- `Look for other ways to minimize
4157 size. <#look-for-other-ways-to-minimize-size>`__
4158
4159- `Iterate on the process. <#iterate-on-the-process>`__
4160
4161Goals and Guiding Principles
4162~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4163
4164Before you can reach your destination, you need to know where you are
4165going. Here is an example list that you can use as a guide when creating
4166very small distributions:
4167
4168- Determine how much space you need (e.g. a kernel that is 1 Mbyte or
4169 less and a root filesystem that is 3 Mbytes or less).
4170
4171- Find the areas that are currently taking 90% of the space and
4172 concentrate on reducing those areas.
4173
4174- Do not create any difficult "hacks" to achieve your goals.
4175
4176- Leverage the device-specific options.
4177
4178- Work in a separate layer so that you keep changes isolated. For
4179 information on how to create layers, see the "`Understanding and
4180 Creating Layers <#understanding-and-creating-layers>`__" section.
4181
4182.. _understand-what-gives-your-image-size:
4183
4184Understand What Contributes to Your Image Size
4185~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4186
4187It is easiest to have something to start with when creating your own
4188distribution. You can use the Yocto Project out-of-the-box to create the
4189``poky-tiny`` distribution. Ultimately, you will want to make changes in
4190your own distribution that are likely modeled after ``poky-tiny``.
4191
4192.. note::
4193
4194 To use
4195 poky-tiny
4196 in your build, set the
4197 DISTRO
4198 variable in your
4199 local.conf
4200 file to "poky-tiny" as described in the "
4201 Creating Your Own Distribution
4202 " section.
4203
4204Understanding some memory concepts will help you reduce the system size.
4205Memory consists of static, dynamic, and temporary memory. Static memory
4206is the TEXT (code), DATA (initialized data in the code), and BSS
4207(uninitialized data) sections. Dynamic memory represents memory that is
4208allocated at runtime: stacks, hash tables, and so forth. Temporary
4209memory is recovered after the boot process. This memory consists of
4210memory used for decompressing the kernel and for the ``__init__``
4211functions.
4212
4213To help you see where you currently are with kernel and root filesystem
4214sizes, you can use two tools found in the :term:`Source Directory`
4215in the
4216``scripts/tiny/`` directory:
4217
4218- ``ksize.py``: Reports component sizes for the kernel build objects.
4219
4220- ``dirsize.py``: Reports component sizes for the root filesystem.
4221
4222This next tool and command help you organize configuration fragments and
4223view file dependencies in a human-readable form:
4224
4225- ``merge_config.sh``: Helps you manage configuration files and
4226 fragments within the kernel. With this tool, you can merge individual
4227 configuration fragments together. The tool allows you to make
4228 overrides and warns you of any missing configuration options. The
4229 tool is ideal for allowing you to iterate on configurations, create
4230 minimal configurations, and create configuration files for different
4231 machines without having to duplicate your process.
4232
4233 The ``merge_config.sh`` script is part of the Linux Yocto kernel Git
4234 repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``,
4235 ``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig``
4236 directory.
4237
4238 For more information on configuration fragments, see the
4239 ":ref:`creating-config-fragments`"
4240 section in the Yocto Project Linux Kernel Development Manual.
4241
4242- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command
4243 with these options brings up a Dependency Explorer from which you can
4244 view file dependencies. Understanding these dependencies allows you
4245 to make informed decisions when cutting out various pieces of the
4246 kernel and root filesystem.
4247
4248Trim the Root Filesystem
4249~~~~~~~~~~~~~~~~~~~~~~~~
4250
4251The root filesystem is made up of packages for booting, libraries, and
4252applications. To change things, you can configure how the packaging
4253happens, which changes the way you build them. You can also modify the
4254filesystem itself or select a different filesystem.
4255
4256First, find out what is hogging your root filesystem by running the
4257``dirsize.py`` script from your root directory:
4258::
4259
4260 $ cd root-directory-of-image
4261 $ dirsize.py 100000 > dirsize-100k.log
4262 $ cat dirsize-100k.log
4263
4264You can apply a filter to the script to ignore files
4265under a certain size. The previous example filters out any files below
4266100 Kbytes. The sizes reported by the tool are uncompressed, and thus
4267will be smaller by a relatively constant factor in a compressed root
4268filesystem. When you examine your log file, you can focus on areas of
4269the root filesystem that take up large amounts of memory.
4270
4271You need to be sure that what you eliminate does not cripple the
4272functionality you need. One way to see how packages relate to each other
4273is by using the Dependency Explorer UI with the BitBake command:
4274::
4275
4276 $ cd image-directory
4277 $ bitbake -u taskexp -g image
4278
4279Use the interface to
4280select potential packages you wish to eliminate and see their dependency
4281relationships.
4282
4283When deciding how to reduce the size, get rid of packages that result in
4284minimal impact on the feature set. For example, you might not need a VGA
4285display. Or, you might be able to get by with ``devtmpfs`` and ``mdev``
4286instead of ``udev``.
4287
4288Use your ``local.conf`` file to make changes. For example, to eliminate
4289``udev`` and ``glib``, set the following in the local configuration
4290file:
4291::
4292
4293 VIRTUAL-RUNTIME_dev_manager = ""
4294
4295Finally, you should consider exactly the type of root filesystem you
4296need to meet your needs while also reducing its size. For example,
4297consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an
4298``initramfs`` using ``initramfs``. Be aware that ``ext3`` requires a 1
4299Mbyte journal. If you are okay with running read-only, you do not need
4300this journal.
4301
4302.. note::
4303
4304 After each round of elimination, you need to rebuild your system and
4305 then use the tools to see the effects of your reductions.
4306
4307Trim the Kernel
4308~~~~~~~~~~~~~~~
4309
4310The kernel is built by including policies for hardware-independent
4311aspects. What subsystems do you enable? For what architecture are you
4312building? Which drivers do you build by default?
4313
4314.. note::
4315
4316 You can modify the kernel source if you want to help with boot time.
4317
4318Run the ``ksize.py`` script from the top-level Linux build directory to
4319get an idea of what is making up the kernel:
4320::
4321
4322 $ cd top-level-linux-build-directory
4323 $ ksize.py > ksize.log
4324 $ cat ksize.log
4325
4326When you examine the log, you will see how much space is taken up with
4327the built-in ``.o`` files for drivers, networking, core kernel files,
4328filesystem, sound, and so forth. The sizes reported by the tool are
4329uncompressed, and thus will be smaller by a relatively constant factor
4330in a compressed kernel image. Look to reduce the areas that are large
4331and taking up around the "90% rule."
4332
4333To examine, or drill down, into any particular area, use the ``-d``
4334option with the script:
4335::
4336
4337 $ ksize.py -d > ksize.log
4338
4339Using this option
4340breaks out the individual file information for each area of the kernel
4341(e.g. drivers, networking, and so forth).
4342
4343Use your log file to see what you can eliminate from the kernel based on
4344features you can let go. For example, if you are not going to need
4345sound, you do not need any drivers that support sound.
4346
4347After figuring out what to eliminate, you need to reconfigure the kernel
4348to reflect those changes during the next build. You could run
4349``menuconfig`` and make all your changes at once. However, that makes it
4350difficult to see the effects of your individual eliminations and also
4351makes it difficult to replicate the changes for perhaps another target
4352device. A better method is to start with no configurations using
4353``allnoconfig``, create configuration fragments for individual changes,
4354and then manage the fragments into a single configuration file using
4355``merge_config.sh``. The tool makes it easy for you to iterate using the
4356configuration change and build cycle.
4357
4358Each time you make configuration changes, you need to rebuild the kernel
4359and check to see what impact your changes had on the overall size.
4360
4361Remove Package Management Requirements
4362~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4363
4364Packaging requirements add size to the image. One way to reduce the size
4365of the image is to remove all the packaging requirements from the image.
4366This reduction includes both removing the package manager and its unique
4367dependencies as well as removing the package management data itself.
4368
4369To eliminate all the packaging requirements for an image, be sure that
4370"package-management" is not part of your
4371:term:`IMAGE_FEATURES`
4372statement for the image. When you remove this feature, you are removing
4373the package manager as well as its dependencies from the root
4374filesystem.
4375
4376Look for Other Ways to Minimize Size
4377~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4378
4379Depending on your particular circumstances, other areas that you can
4380trim likely exist. The key to finding these areas is through tools and
4381methods described here combined with experimentation and iteration. Here
4382are a couple of areas to experiment with:
4383
4384- ``glibc``: In general, follow this process:
4385
4386 1. Remove ``glibc`` features from
4387 :term:`DISTRO_FEATURES`
4388 that you think you do not need.
4389
4390 2. Build your distribution.
4391
4392 3. If the build fails due to missing symbols in a package, determine
4393 if you can reconfigure the package to not need those features. For
4394 example, change the configuration to not support wide character
4395 support as is done for ``ncurses``. Or, if support for those
4396 characters is needed, determine what ``glibc`` features provide
4397 the support and restore the configuration.
4398
4399 4. Rebuild and repeat the process.
4400
4401- ``busybox``: For BusyBox, use a process similar as described for
4402 ``glibc``. A difference is you will need to boot the resulting system
4403 to see if you are able to do everything you expect from the running
4404 system. You need to be sure to integrate configuration fragments into
4405 Busybox because BusyBox handles its own core features and then allows
4406 you to add configuration fragments on top.
4407
4408Iterate on the Process
4409~~~~~~~~~~~~~~~~~~~~~~
4410
4411If you have not reached your goals on system size, you need to iterate
4412on the process. The process is the same. Use the tools and see just what
4413is taking up 90% of the root filesystem and the kernel. Decide what you
4414can eliminate without limiting your device beyond what you need.
4415
4416Depending on your system, a good place to look might be Busybox, which
4417provides a stripped down version of Unix tools in a single, executable
4418file. You might be able to drop virtual terminal services or perhaps
4419ipv6.
4420
4421Building Images for More than One Machine
4422-----------------------------------------
4423
4424A common scenario developers face is creating images for several
4425different machines that use the same software environment. In this
4426situation, it is tempting to set the tunings and optimization flags for
4427each build specifically for the targeted hardware (i.e. "maxing out" the
4428tunings). Doing so can considerably add to build times and package feed
4429maintenance collectively for the machines. For example, selecting tunes
4430that are extremely specific to a CPU core used in a system might enable
4431some micro optimizations in GCC for that particular system but would
4432otherwise not gain you much of a performance difference across the other
4433systems as compared to using a more general tuning across all the builds
4434(e.g. setting :term:`DEFAULTTUNE`
4435specifically for each machine's build). Rather than "max out" each
4436build's tunings, you can take steps that cause the OpenEmbedded build
4437system to reuse software across the various machines where it makes
4438sense.
4439
4440If build speed and package feed maintenance are considerations, you
4441should consider the points in this section that can help you optimize
4442your tunings to best consider build times and package feed maintenance.
4443
4444- *Share the Build Directory:* If at all possible, share the
4445 :term:`TMPDIR` across builds. The
4446 Yocto Project supports switching between different
4447 :term:`MACHINE` values in the same
4448 ``TMPDIR``. This practice is well supported and regularly used by
4449 developers when building for multiple machines. When you use the same
4450 ``TMPDIR`` for multiple machine builds, the OpenEmbedded build system
4451 can reuse the existing native and often cross-recipes for multiple
4452 machines. Thus, build time decreases.
4453
4454 .. note::
4455
4456 If
4457 DISTRO
4458 settings change or fundamental configuration settings such as the
4459 filesystem layout, you need to work with a clean
4460 TMPDIR
4461 . Sharing
4462 TMPDIR
4463 under these circumstances might work but since it is not
4464 guaranteed, you should use a clean
4465 TMPDIR
4466 .
4467
4468- *Enable the Appropriate Package Architecture:* By default, the
4469 OpenEmbedded build system enables three levels of package
4470 architectures: "all", "tune" or "package", and "machine". Any given
4471 recipe usually selects one of these package architectures (types) for
4472 its output. Depending for what a given recipe creates packages,
4473 making sure you enable the appropriate package architecture can
4474 directly impact the build time.
4475
4476 A recipe that just generates scripts can enable "all" architecture
4477 because there are no binaries to build. To specifically enable "all"
4478 architecture, be sure your recipe inherits the
4479 :ref:`allarch <ref-classes-allarch>` class.
4480 This class is useful for "all" architectures because it configures
4481 many variables so packages can be used across multiple architectures.
4482
4483 If your recipe needs to generate packages that are machine-specific
4484 or when one of the build or runtime dependencies is already
4485 machine-architecture dependent, which makes your recipe also
4486 machine-architecture dependent, make sure your recipe enables the
4487 "machine" package architecture through the
4488 :term:`MACHINE_ARCH`
4489 variable:
4490 ::
4491
4492 PACKAGE_ARCH = "${MACHINE_ARCH}"
4493
4494 When you do not
4495 specifically enable a package architecture through the
4496 :term:`PACKAGE_ARCH`, The
4497 OpenEmbedded build system defaults to the
4498 :term:`TUNE_PKGARCH` setting:
4499 ::
4500
4501 PACKAGE_ARCH = "${TUNE_PKGARCH}"
4502
4503- *Choose a Generic Tuning File if Possible:* Some tunes are more
4504 generic and can run on multiple targets (e.g. an ``armv5`` set of
4505 packages could run on ``armv6`` and ``armv7`` processors in most
4506 cases). Similarly, ``i486`` binaries could work on ``i586`` and
4507 higher processors. You should realize, however, that advances on
4508 newer processor versions would not be used.
4509
4510 If you select the same tune for several different machines, the
4511 OpenEmbedded build system reuses software previously built, thus
4512 speeding up the overall build time. Realize that even though a new
4513 sysroot for each machine is generated, the software is not recompiled
4514 and only one package feed exists.
4515
4516- *Manage Granular Level Packaging:* Sometimes cases exist where
4517 injecting another level of package architecture beyond the three
4518 higher levels noted earlier can be useful. For example, consider how
4519 NXP (formerly Freescale) allows for the easy reuse of binary packages
4520 in their layer
4521 :yocto_git:`meta-freescale </cgit/cgit.cgi/meta-freescale/>`.
4522 In this example, the
4523 :yocto_git:`fsl-dynamic-packagearch </cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>`
4524 class shares GPU packages for i.MX53 boards because all boards share
4525 the AMD GPU. The i.MX6-based boards can do the same because all
4526 boards share the Vivante GPU. This class inspects the BitBake
4527 datastore to identify if the package provides or depends on one of
4528 the sub-architecture values. If so, the class sets the
4529 :term:`PACKAGE_ARCH` value
4530 based on the ``MACHINE_SUBARCH`` value. If the package does not
4531 provide or depend on one of the sub-architecture values but it
4532 matches a value in the machine-specific filter, it sets
4533 :term:`MACHINE_ARCH`. This
4534 behavior reduces the number of packages built and saves build time by
4535 reusing binaries.
4536
4537- *Use Tools to Debug Issues:* Sometimes you can run into situations
4538 where software is being rebuilt when you think it should not be. For
4539 example, the OpenEmbedded build system might not be using shared
4540 state between machines when you think it should be. These types of
4541 situations are usually due to references to machine-specific
4542 variables such as :term:`MACHINE`,
4543 :term:`SERIAL_CONSOLES`,
4544 :term:`XSERVER`,
4545 :term:`MACHINE_FEATURES`,
4546 and so forth in code that is supposed to only be tune-specific or
4547 when the recipe depends
4548 (:term:`DEPENDS`,
4549 :term:`RDEPENDS`,
4550 :term:`RRECOMMENDS`,
4551 :term:`RSUGGESTS`, and so forth)
4552 on some other recipe that already has
4553 :term:`PACKAGE_ARCH` defined
4554 as "${MACHINE_ARCH}".
4555
4556 .. note::
4557
4558 Patches to fix any issues identified are most welcome as these
4559 issues occasionally do occur.
4560
4561 For such cases, you can use some tools to help you sort out the
4562 situation:
4563
4564 - *sstate-diff-machines.sh:* You can find this tool in the
4565 ``scripts`` directory of the Source Repositories. See the comments
4566 in the script for information on how to use the tool.
4567
4568 - *BitBake's "-S printdiff" Option:* Using this option causes
4569 BitBake to try to establish the closest signature match it can
4570 (e.g. in the shared state cache) and then run ``bitbake-diffsigs``
4571 over the matches to determine the stamps and delta where these two
4572 stamp trees diverge.
4573
4574Building Software from an External Source
4575-----------------------------------------
4576
4577By default, the OpenEmbedded build system uses the
4578:term:`Build Directory` when building source
4579code. The build process involves fetching the source files, unpacking
4580them, and then patching them if necessary before the build takes place.
4581
4582Situations exist where you might want to build software from source
4583files that are external to and thus outside of the OpenEmbedded build
4584system. For example, suppose you have a project that includes a new BSP
4585with a heavily customized kernel. And, you want to minimize exposing the
4586build system to the development team so that they can focus on their
4587project and maintain everyone's workflow as much as possible. In this
4588case, you want a kernel source directory on the development machine
4589where the development occurs. You want the recipe's
4590:term:`SRC_URI` variable to point to
4591the external directory and use it as is, not copy it.
4592
4593To build from software that comes from an external source, all you need
4594to do is inherit the
4595:ref:`externalsrc <ref-classes-externalsrc>` class
4596and then set the
4597:term:`EXTERNALSRC` variable to
4598point to your external source code. Here are the statements to put in
4599your ``local.conf`` file:
4600::
4601
4602 INHERIT += "externalsrc"
4603 EXTERNALSRC_pn-myrecipe = "path-to-your-source-tree"
4604
4605This next example shows how to accomplish the same thing by setting
4606``EXTERNALSRC`` in the recipe itself or in the recipe's append file:
4607::
4608
4609 EXTERNALSRC = "path"
4610 EXTERNALSRC_BUILD = "path"
4611
4612.. note::
4613
4614 In order for these settings to take effect, you must globally or
4615 locally inherit the
4616 externalsrc
4617 class.
4618
4619By default, ``externalsrc.bbclass`` builds the source code in a
4620directory separate from the external source directory as specified by
4621:term:`EXTERNALSRC`. If you need
4622to have the source built in the same directory in which it resides, or
4623some other nominated directory, you can set
4624:term:`EXTERNALSRC_BUILD`
4625to point to that directory:
4626::
4627
4628 EXTERNALSRC_BUILD_pn-myrecipe = "path-to-your-source-tree"
4629
4630Replicating a Build Offline
4631---------------------------
4632
4633It can be useful to take a "snapshot" of upstream sources used in a
4634build and then use that "snapshot" later to replicate the build offline.
4635To do so, you need to first prepare and populate your downloads
4636directory your "snapshot" of files. Once your downloads directory is
4637ready, you can use it at any time and from any machine to replicate your
4638build.
4639
4640Follow these steps to populate your Downloads directory:
4641
46421. *Create a Clean Downloads Directory:* Start with an empty downloads
4643 directory (:term:`DL_DIR`). You
4644 start with an empty downloads directory by either removing the files
4645 in the existing directory or by setting ``DL_DIR`` to point to either
4646 an empty location or one that does not yet exist.
4647
46482. *Generate Tarballs of the Source Git Repositories:* Edit your
4649 ``local.conf`` configuration file as follows:
4650 ::
4651
4652 DL_DIR = "/home/your-download-dir/"
4653 BB_GENERATE_MIRROR_TARBALLS = "1"
4654
4655 During
4656 the fetch process in the next step, BitBake gathers the source files
4657 and creates tarballs in the directory pointed to by ``DL_DIR``. See
4658 the
4659 :term:`BB_GENERATE_MIRROR_TARBALLS`
4660 variable for more information.
4661
46623. *Populate Your Downloads Directory Without Building:* Use BitBake to
4663 fetch your sources but inhibit the build:
4664 ::
4665
4666 $ bitbake target --runonly=fetch
4667
4668 The downloads directory (i.e. ``${DL_DIR}``) now has
4669 a "snapshot" of the source files in the form of tarballs, which can
4670 be used for the build.
4671
46724. *Optionally Remove Any Git or other SCM Subdirectories From the
4673 Downloads Directory:* If you want, you can clean up your downloads
4674 directory by removing any Git or other Source Control Management
4675 (SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs
4676 already contain these subdirectories.
4677
4678Once your downloads directory has everything it needs regarding source
4679files, you can create your "own-mirror" and build your target.
4680Understand that you can use the files to build the target offline from
4681any machine and at any time.
4682
4683Follow these steps to build your target using the files in the downloads
4684directory:
4685
46861. *Using Local Files Only:* Inside your ``local.conf`` file, add the
4687 :term:`SOURCE_MIRROR_URL`
4688 variable, inherit the
4689 :ref:`own-mirrors <ref-classes-own-mirrors>`
4690 class, and use the
4691 :term:`bitbake:BB_NO_NETWORK`
4692 variable to your ``local.conf``.
4693 ::
4694
4695 SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/"
4696 INHERIT += "own-mirrors"
4697 BB_NO_NETWORK = "1"
4698
4699 The ``SOURCE_MIRROR_URL`` and ``own-mirror``
4700 class set up the system to use the downloads directory as your "own
4701 mirror". Using the ``BB_NO_NETWORK`` variable makes sure that
4702 BitBake's fetching process in step 3 stays local, which means files
4703 from your "own-mirror" are used.
4704
47052. *Start With a Clean Build:* You can start with a clean build by
4706 removing the
4707 ``${``\ :term:`TMPDIR`\ ``}``
4708 directory or using a new :term:`Build Directory`.
4709
47103. *Build Your Target:* Use BitBake to build your target:
4711 ::
4712
4713 $ bitbake target
4714
4715 The build completes using the known local "snapshot" of source
4716 files from your mirror. The resulting tarballs for your "snapshot" of
4717 source files are in the downloads directory.
4718
4719 .. note::
4720
4721 The offline build does not work if recipes attempt to find the
4722 latest version of software by setting
4723 :term:`SRCREV` to
4724 ``${``\ :term:`AUTOREV`\ ``}``:
4725 SRCREV = "${AUTOREV}" When a recipe sets ``SRCREV`` to
4726 ``${AUTOREV}``, the build system accesses the network in an
4727 attempt to determine the latest version of software from the SCM.
4728 Typically, recipes that use ``AUTOREV`` are custom or modified
4729 recipes. Recipes that reside in public repositories usually do not
4730 use ``AUTOREV``.
4731
4732 If you do have recipes that use ``AUTOREV``, you can take steps to
4733 still use the recipes in an offline build. Do the following:
4734
4735 1. Use a configuration generated by enabling `build
4736 history <#maintaining-build-output-quality>`__.
4737
4738 2. Use the ``buildhistory-collect-srcrevs`` command to collect the
4739 stored ``SRCREV`` values from the build's history. For more
4740 information on collecting these values, see the "`Build History
4741 Package Information <#build-history-package-information>`__"
4742 section.
4743
4744 3. Once you have the correct source revisions, you can modify
4745 those recipes to to set ``SRCREV`` to specific versions of the
4746 software.
4747
4748Speeding Up a Build
4749===================
4750
4751Build time can be an issue. By default, the build system uses simple
4752controls to try and maximize build efficiency. In general, the default
4753settings for all the following variables result in the most efficient
4754build times when dealing with single socket systems (i.e. a single CPU).
4755If you have multiple CPUs, you might try increasing the default values
4756to gain more speed. See the descriptions in the glossary for each
4757variable for more information:
4758
4759- :term:`BB_NUMBER_THREADS`:
4760 The maximum number of threads BitBake simultaneously executes.
4761
4762- :term:`bitbake:BB_NUMBER_PARSE_THREADS`:
4763 The number of threads BitBake uses during parsing.
4764
4765- :term:`PARALLEL_MAKE`: Extra
4766 options passed to the ``make`` command during the
4767 :ref:`ref-tasks-compile` task in
4768 order to specify parallel compilation on the local build host.
4769
4770- :term:`PARALLEL_MAKEINST`:
4771 Extra options passed to the ``make`` command during the
4772 :ref:`ref-tasks-install` task in
4773 order to specify parallel installation on the local build host.
4774
4775As mentioned, these variables all scale to the number of processor cores
4776available on the build system. For single socket systems, this
4777auto-scaling ensures that the build system fundamentally takes advantage
4778of potential parallel operations during the build based on the build
4779machine's capabilities.
4780
4781Following are additional factors that can affect build speed:
4782
4783- File system type: The file system type that the build is being
4784 performed on can also influence performance. Using ``ext4`` is
4785 recommended as compared to ``ext2`` and ``ext3`` due to ``ext4``
4786 improved features such as extents.
4787
4788- Disabling the updating of access time using ``noatime``: The
4789 ``noatime`` mount option prevents the build system from updating file
4790 and directory access times.
4791
4792- Setting a longer commit: Using the "commit=" mount option increases
4793 the interval in seconds between disk cache writes. Changing this
4794 interval from the five second default to something longer increases
4795 the risk of data loss but decreases the need to write to the disk,
4796 thus increasing the build performance.
4797
4798- Choosing the packaging backend: Of the available packaging backends,
4799 IPK is the fastest. Additionally, selecting a singular packaging
4800 backend also helps.
4801
4802- Using ``tmpfs`` for :term:`TMPDIR`
4803 as a temporary file system: While this can help speed up the build,
4804 the benefits are limited due to the compiler using ``-pipe``. The
4805 build system goes to some lengths to avoid ``sync()`` calls into the
4806 file system on the principle that if there was a significant failure,
4807 the :term:`Build Directory`
4808 contents could easily be rebuilt.
4809
4810- Inheriting the
4811 :ref:`rm_work <ref-classes-rm-work>` class:
4812 Inheriting this class has shown to speed up builds due to
4813 significantly lower amounts of data stored in the data cache as well
4814 as on disk. Inheriting this class also makes cleanup of
4815 :term:`TMPDIR` faster, at the
4816 expense of being easily able to dive into the source code. File
4817 system maintainers have recommended that the fastest way to clean up
4818 large numbers of files is to reformat partitions rather than delete
4819 files due to the linear nature of partitions. This, of course,
4820 assumes you structure the disk partitions and file systems in a way
4821 that this is practical.
4822
4823Aside from the previous list, you should keep some trade offs in mind
4824that can help you speed up the build:
4825
4826- Remove items from
4827 :term:`DISTRO_FEATURES`
4828 that you might not need.
4829
4830- Exclude debug symbols and other debug information: If you do not need
4831 these symbols and other debug information, disabling the ``*-dbg``
4832 package generation can speed up the build. You can disable this
4833 generation by setting the
4834 :term:`INHIBIT_PACKAGE_DEBUG_SPLIT`
4835 variable to "1".
4836
4837- Disable static library generation for recipes derived from
4838 ``autoconf`` or ``libtool``: Following is an example showing how to
4839 disable static libraries and still provide an override to handle
4840 exceptions:
4841 ::
4842
4843 STATICLIBCONF = "--disable-static"
4844 STATICLIBCONF_sqlite3-native = ""
4845 EXTRA_OECONF += "${STATICLIBCONF}"
4846
4847 .. note::
4848
4849 - Some recipes need static libraries in order to work correctly
4850 (e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides,
4851 as in the previous example, account for these kinds of
4852 exceptions.
4853
4854 - Some packages have packaging code that assumes the presence of
4855 the static libraries. If so, you might need to exclude them as
4856 well.
4857
4858.. _platdev-working-with-libraries:
4859
4860Working With Libraries
4861======================
4862
4863Libraries are an integral part of your system. This section describes
4864some common practices you might find helpful when working with libraries
4865to build your system:
4866
4867- `How to include static library
4868 files <#including-static-library-files>`__
4869
4870- `How to use the Multilib feature to combine multiple versions of
4871 library files into a single
4872 image <#combining-multiple-versions-library-files-into-one-image>`__
4873
4874- `How to install multiple versions of the same library in parallel on
4875 the same
4876 system <#installing-multiple-versions-of-the-same-library>`__
4877
4878Including Static Library Files
4879------------------------------
4880
4881If you are building a library and the library offers static linking, you
4882can control which static library files (``*.a`` files) get included in
4883the built library.
4884
4885The :term:`PACKAGES` and
4886:term:`FILES_* <FILES>` variables in the
4887``meta/conf/bitbake.conf`` configuration file define how files installed
4888by the ``do_install`` task are packaged. By default, the ``PACKAGES``
4889variable includes ``${PN}-staticdev``, which represents all static
4890library files.
4891
4892.. note::
4893
4894 Some previously released versions of the Yocto Project defined the
4895 static library files through
4896 ${PN}-dev
4897 .
4898
4899Following is part of the BitBake configuration file, where you can see
4900how the static library files are defined:
4901::
4902
4903 PACKAGE_BEFORE_PN ?= ""
4904 PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
4905 PACKAGES_DYNAMIC = "^${PN}-locale-.*"
4906 FILES = ""
4907
4908 FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
4909 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
4910 ${base_bindir}/* ${base_sbindir}/* \
4911 ${base_libdir}/*${SOLIBS} \
4912 ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \
4913 ${datadir}/${BPN} ${libdir}/${BPN}/* \
4914 ${datadir}/pixmaps ${datadir}/applications \
4915 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
4916 ${libdir}/bonobo/servers"
4917
4918 FILES_${PN}-bin = "${bindir}/* ${sbindir}/*"
4919
4920 FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
4921 ${datadir}/gnome/help"
4922 SECTION_${PN}-doc = "doc"
4923
4924 FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
4925 FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
4926 ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
4927 ${datadir}/aclocal ${base_libdir}/*.o \
4928 ${libdir}/${BPN}/*.la ${base_libdir}/*.la"
4929 SECTION_${PN}-dev = "devel"
4930 ALLOW_EMPTY_${PN}-dev = "1"
4931 RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"
4932
4933 FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
4934 SECTION_${PN}-staticdev = "devel"
4935 RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
4936
4937.. _combining-multiple-versions-library-files-into-one-image:
4938
4939Combining Multiple Versions of Library Files into One Image
4940-----------------------------------------------------------
4941
4942The build system offers the ability to build libraries with different
4943target optimizations or architecture formats and combine these together
4944into one system image. You can link different binaries in the image
4945against the different libraries as needed for specific use cases. This
4946feature is called "Multilib."
4947
4948An example would be where you have most of a system compiled in 32-bit
4949mode using 32-bit libraries, but you have something large, like a
4950database engine, that needs to be a 64-bit application and uses 64-bit
4951libraries. Multilib allows you to get the best of both 32-bit and 64-bit
4952libraries.
4953
4954While the Multilib feature is most commonly used for 32 and 64-bit
4955differences, the approach the build system uses facilitates different
4956target optimizations. You could compile some binaries to use one set of
4957libraries and other binaries to use a different set of libraries. The
4958libraries could differ in architecture, compiler options, or other
4959optimizations.
4960
4961Several examples exist in the ``meta-skeleton`` layer found in the
4962:term:`Source Directory`:
4963
4964- ``conf/multilib-example.conf`` configuration file
4965
4966- ``conf/multilib-example2.conf`` configuration file
4967
4968- ``recipes-multilib/images/core-image-multilib-example.bb`` recipe
4969
4970Preparing to Use Multilib
4971~~~~~~~~~~~~~~~~~~~~~~~~~
4972
4973User-specific requirements drive the Multilib feature. Consequently,
4974there is no one "out-of-the-box" configuration that likely exists to
4975meet your needs.
4976
4977In order to enable Multilib, you first need to ensure your recipe is
4978extended to support multiple libraries. Many standard recipes are
4979already extended and support multiple libraries. You can check in the
4980``meta/conf/multilib.conf`` configuration file in the
4981:term:`Source Directory` to see how this is
4982done using the
4983:term:`BBCLASSEXTEND` variable.
4984Eventually, all recipes will be covered and this list will not be
4985needed.
4986
4987For the most part, the Multilib class extension works automatically to
4988extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where
4989``MLPREFIX`` is the particular multilib (e.g. "lib32-" or "lib64-").
4990Standard variables such as
4991:term:`DEPENDS`,
4992:term:`RDEPENDS`,
4993:term:`RPROVIDES`,
4994:term:`RRECOMMENDS`,
4995:term:`PACKAGES`, and
4996:term:`PACKAGES_DYNAMIC` are
4997automatically extended by the system. If you are extending any manual
4998code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure
4999those names are extended correctly. This automatic extension code
5000resides in ``multilib.bbclass``.
5001
5002Using Multilib
5003~~~~~~~~~~~~~~
5004
5005After you have set up the recipes, you need to define the actual
5006combination of multiple libraries you want to build. You accomplish this
5007through your ``local.conf`` configuration file in the
5008:term:`Build Directory`. An example
5009configuration would be as follows:
5010::
5011
5012 MACHINE = "qemux86-64"
5013 require conf/multilib.conf
5014 MULTILIBS = "multilib:lib32"
5015 DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
5016 IMAGE_INSTALL_append = "lib32-glib-2.0"
5017
5018This example enables an additional library named
5019``lib32`` alongside the normal target packages. When combining these
5020"lib32" alternatives, the example uses "x86" for tuning. For information
5021on this particular tuning, see
5022``meta/conf/machine/include/ia32/arch-ia32.inc``.
5023
5024The example then includes ``lib32-glib-2.0`` in all the images, which
5025illustrates one method of including a multiple library dependency. You
5026can use a normal image build to include this dependency, for example:
5027::
5028
5029 $ bitbake core-image-sato
5030
5031You can also build Multilib packages
5032specifically with a command like this:
5033::
5034
5035 $ bitbake lib32-glib-2.0
5036
5037Additional Implementation Details
5038~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5039
5040Generic implementation details as well as details that are specific to
5041package management systems exist. Following are implementation details
5042that exist regardless of the package management system:
5043
5044- The typical convention used for the class extension code as used by
5045 Multilib assumes that all package names specified in
5046 :term:`PACKAGES` that contain
5047 ``${PN}`` have ``${PN}`` at the start of the name. When that
5048 convention is not followed and ``${PN}`` appears at the middle or the
5049 end of a name, problems occur.
5050
5051- The :term:`TARGET_VENDOR`
5052 value under Multilib will be extended to "-vendormlmultilib" (e.g.
5053 "-pokymllib32" for a "lib32" Multilib with Poky). The reason for this
5054 slightly unwieldy contraction is that any "-" characters in the
5055 vendor string presently break Autoconf's ``config.sub``, and other
5056 separators are problematic for different reasons.
5057
5058For the RPM Package Management System, the following implementation
5059details exist:
5060
5061- A unique architecture is defined for the Multilib packages, along
5062 with creating a unique deploy folder under ``tmp/deploy/rpm`` in the
5063 :term:`Build Directory`. For
5064 example, consider ``lib32`` in a ``qemux86-64`` image. The possible
5065 architectures in the system are "all", "qemux86_64",
5066 "lib32_qemux86_64", and "lib32_x86".
5067
5068- The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM
5069 packaging. The naming for a normal RPM package and a Multilib RPM
5070 package in a ``qemux86-64`` system resolves to something similar to
5071 ``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``,
5072 respectively.
5073
5074- When installing a Multilib image, the RPM backend first installs the
5075 base image and then installs the Multilib libraries.
5076
5077- The build system relies on RPM to resolve the identical files in the
5078 two (or more) Multilib packages.
5079
5080For the IPK Package Management System, the following implementation
5081details exist:
5082
5083- The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK
5084 packaging. The naming for a normal RPM package and a Multilib IPK
5085 package in a ``qemux86-64`` system resolves to something like
5086 ``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw_x86.ipk``,
5087 respectively.
5088
5089- The IPK deploy folder is not modified with ``${MLPREFIX}`` because
5090 packages with and without the Multilib feature can exist in the same
5091 folder due to the ``${PN}`` differences.
5092
5093- IPK defines a sanity check for Multilib installation using certain
5094 rules for file comparison, overridden, etc.
5095
5096Installing Multiple Versions of the Same Library
5097------------------------------------------------
5098
5099Situations can exist where you need to install and use multiple versions
5100of the same library on the same system at the same time. These
5101situations almost always exist when a library API changes and you have
5102multiple pieces of software that depend on the separate versions of the
5103library. To accommodate these situations, you can install multiple
5104versions of the same library in parallel on the same system.
5105
5106The process is straightforward as long as the libraries use proper
5107versioning. With properly versioned libraries, all you need to do to
5108individually specify the libraries is create separate, appropriately
5109named recipes where the :term:`PN` part of
5110the name includes a portion that differentiates each library version
5111(e.g.the major part of the version number). Thus, instead of having a
5112single recipe that loads one version of a library (e.g. ``clutter``),
5113you provide multiple recipes that result in different versions of the
5114libraries you want. As an example, the following two recipes would allow
5115the two separate versions of the ``clutter`` library to co-exist on the
5116same system:
5117::
5118
5119 clutter-1.6_1.6.20.bb
5120 clutter-1.8_1.8.4.bb
5121
5122Additionally, if
5123you have other recipes that depend on a given library, you need to use
5124the :term:`DEPENDS` variable to
5125create the dependency. Continuing with the same example, if you want to
5126have a recipe depend on the 1.8 version of the ``clutter`` library, use
5127the following in your recipe:
5128::
5129
5130 DEPENDS = "clutter-1.8"
5131
5132Using x32 psABI
5133===============
5134
5135x32 processor-specific Application Binary Interface (`x32
5136psABI <https://software.intel.com/en-us/node/628948>`__) is a native
513732-bit processor-specific ABI for Intel 64 (x86-64) architectures. An
5138ABI defines the calling conventions between functions in a processing
5139environment. The interface determines what registers are used and what
5140the sizes are for various C data types.
5141
5142Some processing environments prefer using 32-bit applications even when
5143running on Intel 64-bit platforms. Consider the i386 psABI, which is a
5144very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not
5145provide efficient use and access of the Intel 64-bit processor
5146resources, leaving the system underutilized. Now consider the x86_64
5147psABI. This ABI is newer and uses 64-bits for data sizes and program
5148pointers. The extra bits increase the footprint size of the programs,
5149libraries, and also increases the memory and file system size
5150requirements. Executing under the x32 psABI enables user programs to
5151utilize CPU and system resources more efficiently while keeping the
5152memory footprint of the applications low. Extra bits are used for
5153registers but not for addressing mechanisms.
5154
5155The Yocto Project supports the final specifications of x32 psABI as
5156follows:
5157
5158- You can create packages and images in x32 psABI format on x86_64
5159 architecture targets.
5160
5161- You can successfully build recipes with the x32 toolchain.
5162
5163- You can create and boot ``core-image-minimal`` and
5164 ``core-image-sato`` images.
5165
5166- RPM Package Manager (RPM) support exists for x32 binaries.
5167
5168- Support for large images exists.
5169
5170To use the x32 psABI, you need to edit your ``conf/local.conf``
5171configuration file as follows:
5172::
5173
5174 MACHINE = "qemux86-64"
5175 DEFAULTTUNE = "x86-64-x32"
5176 baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE') \
5177 or 'INVALID')) or 'lib'}"
5178
5179Once you have set
5180up your configuration file, use BitBake to build an image that supports
5181the x32 psABI. Here is an example:
5182::
5183
5184 $ bitbake core-image-sato
5185
5186Enabling GObject Introspection Support
5187======================================
5188
5189`GObject
5190introspection <https://wiki.gnome.org/Projects/GObjectIntrospection>`__
5191is the standard mechanism for accessing GObject-based software from
5192runtime environments. GObject is a feature of the GLib library that
5193provides an object framework for the GNOME desktop and related software.
5194GObject Introspection adds information to GObject that allows objects
5195created within it to be represented across different programming
5196languages. If you want to construct GStreamer pipelines using Python, or
5197control UPnP infrastructure using Javascript and GUPnP, GObject
5198introspection is the only way to do it.
5199
5200This section describes the Yocto Project support for generating and
5201packaging GObject introspection data. GObject introspection data is a
5202description of the API provided by libraries built on top of GLib
5203framework, and, in particular, that framework's GObject mechanism.
5204GObject Introspection Repository (GIR) files go to ``-dev`` packages,
5205``typelib`` files go to main packages as they are packaged together with
5206libraries that are introspected.
5207
5208The data is generated when building such a library, by linking the
5209library with a small executable binary that asks the library to describe
5210itself, and then executing the binary and processing its output.
5211
5212Generating this data in a cross-compilation environment is difficult
5213because the library is produced for the target architecture, but its
5214code needs to be executed on the build host. This problem is solved with
5215the OpenEmbedded build system by running the code through QEMU, which
5216allows precisely that. Unfortunately, QEMU does not always work
5217perfectly as mentioned in the "`Known Issues <#known-issues>`__"
5218section.
5219
5220Enabling the Generation of Introspection Data
5221---------------------------------------------
5222
5223Enabling the generation of introspection data (GIR files) in your
5224library package involves the following:
5225
52261. Inherit the
5227 :ref:`gobject-introspection <ref-classes-gobject-introspection>`
5228 class.
5229
52302. Make sure introspection is not disabled anywhere in the recipe or
5231 from anything the recipe includes. Also, make sure that
5232 "gobject-introspection-data" is not in
5233 :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
5234 and that "qemu-usermode" is not in
5235 :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
5236 If either of these conditions exist, nothing will happen.
5237
52383. Try to build the recipe. If you encounter build errors that look like
5239 something is unable to find ``.so`` libraries, check where these
5240 libraries are located in the source tree and add the following to the
5241 recipe:
5242 ::
5243
5244 GIR_EXTRA_LIBS_PATH = "${B}/something/.libs"
5245
5246 .. note::
5247
5248 See recipes in the
5249 oe-core
5250 repository that use that
5251 GIR_EXTRA_LIBS_PATH
5252 variable as an example.
5253
52544. Look for any other errors, which probably mean that introspection
5255 support in a package is not entirely standard, and thus breaks down
5256 in a cross-compilation environment. For such cases, custom-made fixes
5257 are needed. A good place to ask and receive help in these cases is
5258 the :ref:`Yocto Project mailing
5259 lists <resources-mailinglist>`.
5260
5261.. note::
5262
5263 Using a library that no longer builds against the latest Yocto
5264 Project release and prints introspection related errors is a good
5265 candidate for the previous procedure.
5266
5267Disabling the Generation of Introspection Data
5268----------------------------------------------
5269
5270You might find that you do not want to generate introspection data. Or,
5271perhaps QEMU does not work on your build host and target architecture
5272combination. If so, you can use either of the following methods to
5273disable GIR file generations:
5274
5275- Add the following to your distro configuration:
5276 ::
5277
5278 DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
5279
5280 Adding this statement disables generating introspection data using
5281 QEMU but will still enable building introspection tools and libraries
5282 (i.e. building them does not require the use of QEMU).
5283
5284- Add the following to your machine configuration:
5285 ::
5286
5287 MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode"
5288
5289 Adding this statement disables the use of QEMU when building packages for your
5290 machine. Currently, this feature is used only by introspection
5291 recipes and has the same effect as the previously described option.
5292
5293 .. note::
5294
5295 Future releases of the Yocto Project might have other features
5296 affected by this option.
5297
5298If you disable introspection data, you can still obtain it through other
5299means such as copying the data from a suitable sysroot, or by generating
5300it on the target hardware. The OpenEmbedded build system does not
5301currently provide specific support for these techniques.
5302
5303Testing that Introspection Works in an Image
5304--------------------------------------------
5305
5306Use the following procedure to test if generating introspection data is
5307working in an image:
5308
53091. Make sure that "gobject-introspection-data" is not in
5310 :term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
5311 and that "qemu-usermode" is not in
5312 :term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
5313
53142. Build ``core-image-sato``.
5315
53163. Launch a Terminal and then start Python in the terminal.
5317
53184. Enter the following in the terminal:
5319 ::
5320
5321 >>> from gi.repository import GLib
5322 >>> GLib.get_host_name()
5323
53245. For something a little more advanced, enter the following see:
5325 http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html
5326
5327Known Issues
5328------------
5329
5330The following know issues exist for GObject Introspection Support:
5331
5332- ``qemu-ppc64`` immediately crashes. Consequently, you cannot build
5333 introspection data on that architecture.
5334
5335- x32 is not supported by QEMU. Consequently, introspection data is
5336 disabled.
5337
5338- musl causes transient GLib binaries to crash on assertion failures.
5339 Consequently, generating introspection data is disabled.
5340
5341- Because QEMU is not able to run the binaries correctly, introspection
5342 is disabled for some specific packages under specific architectures
5343 (e.g. ``gcr``, ``libsecret``, and ``webkit``).
5344
5345- QEMU usermode might not work properly when running 64-bit binaries
5346 under 32-bit host machines. In particular, "qemumips64" is known to
5347 not work under i686.
5348
5349.. _dev-optionally-using-an-external-toolchain:
5350
5351Optionally Using an External Toolchain
5352======================================
5353
5354You might want to use an external toolchain as part of your development.
5355If this is the case, the fundamental steps you need to accomplish are as
5356follows:
5357
5358- Understand where the installed toolchain resides. For cases where you
5359 need to build the external toolchain, you would need to take separate
5360 steps to build and install the toolchain.
5361
5362- Make sure you add the layer that contains the toolchain to your
5363 ``bblayers.conf`` file through the
5364 :term:`BBLAYERS` variable.
5365
5366- Set the ``EXTERNAL_TOOLCHAIN`` variable in your ``local.conf`` file
5367 to the location in which you installed the toolchain.
5368
5369A good example of an external toolchain used with the Yocto Project is
5370Mentor Graphics Sourcery G++ Toolchain. You can see information on how
5371to use that particular layer in the ``README`` file at
5372http://github.com/MentorEmbedded/meta-sourcery/. You can find
5373further information by reading about the
5374:term:`TCMODE` variable in the Yocto
5375Project Reference Manual's variable glossary.
5376
5377Creating Partitioned Images Using Wic
5378=====================================
5379
5380Creating an image for a particular hardware target using the
5381OpenEmbedded build system does not necessarily mean you can boot that
5382image as is on your device. Physical devices accept and boot images in
5383various ways depending on the specifics of the device. Usually,
5384information about the hardware can tell you what image format the device
5385requires. Should your device require multiple partitions on an SD card,
5386flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to
5387create the properly partitioned image.
5388
5389The ``wic`` command generates partitioned images from existing
5390OpenEmbedded build artifacts. Image generation is driven by partitioning
5391commands contained in an Openembedded kickstart file (``.wks``)
5392specified either directly on the command line or as one of a selection
5393of canned kickstart files as shown with the ``wic list images`` command
5394in the "`Using an Existing Kickstart
5395File <#using-a-provided-kickstart-file>`__" section. When you apply the
5396command to a given set of build artifacts, the result is an image or set
5397of images that can be directly written onto media and used on a
5398particular system.
5399
5400.. note::
5401
5402 For a kickstart file reference, see the "
5403 OpenEmbedded Kickstart (
5404 .wks
5405 ) Reference
5406 " Chapter in the Yocto Project Reference Manual.
5407
5408The ``wic`` command and the infrastructure it is based on is by
5409definition incomplete. The purpose of the command is to allow the
5410generation of customized images, and as such, was designed to be
5411completely extensible through a plugin interface. See the "`Using the
5412Wic PlugIn Interface <#wic-using-the-wic-plugin-interface>`__" section
5413for information on these plugins.
5414
5415This section provides some background information on Wic, describes what
5416you need to have in place to run the tool, provides instruction on how
5417to use the Wic utility, provides information on using the Wic plugins
5418interface, and provides several examples that show how to use Wic.
5419
5420.. _wic-background:
5421
5422Background
5423----------
5424
5425This section provides some background on the Wic utility. While none of
5426this information is required to use Wic, you might find it interesting.
5427
5428- The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The
5429 "oe" diphthong in "oeic" was promoted to the letter "w", because
5430 "oeic" is both difficult to remember and to pronounce.
5431
5432- Wic is loosely based on the Meego Image Creator (``mic``) framework.
5433 The Wic implementation has been heavily modified to make direct use
5434 of OpenEmbedded build artifacts instead of package installation and
5435 configuration, which are already incorporated within the OpenEmbedded
5436 artifacts.
5437
5438- Wic is a completely independent standalone utility that initially
5439 provides easier-to-use and more flexible replacements for an existing
5440 functionality in OE-Core's
5441 :ref:`image-live <ref-classes-image-live>`
5442 class. The difference between Wic and those examples is that with Wic
5443 the functionality of those scripts is implemented by a
5444 general-purpose partitioning language, which is based on Redhat
5445 kickstart syntax.
5446
5447.. _wic-requirements:
5448
5449Requirements
5450------------
5451
5452In order to use the Wic utility with the OpenEmbedded Build system, your
5453system needs to meet the following requirements:
5454
5455- The Linux distribution on your development host must support the
5456 Yocto Project. See the ":ref:`detailed-supported-distros`"
5457 section in the Yocto Project Reference Manual for the list of
5458 distributions that support the Yocto Project.
5459
5460- The standard system utilities, such as ``cp``, must be installed on
5461 your development host system.
5462
5463- You must have sourced the build environment setup script (i.e.
5464 :ref:`structure-core-script`) found in the
5465 :term:`Build Directory`.
5466
5467- You need to have the build artifacts already available, which
5468 typically means that you must have already created an image using the
5469 Openembedded build system (e.g. ``core-image-minimal``). While it
5470 might seem redundant to generate an image in order to create an image
5471 using Wic, the current version of Wic requires the artifacts in the
5472 form generated by the OpenEmbedded build system.
5473
5474- You must build several native tools, which are built to run on the
5475 build system: $ bitbake parted-native dosfstools-native mtools-native
5476
5477- Include "wic" as part of the
5478 :term:`IMAGE_FSTYPES`
5479 variable.
5480
5481- Include the name of the :ref:`wic kickstart file <openembedded-kickstart-wks-reference>`
5482 as part of the :term:`WKS_FILE` variable
5483
5484.. _wic-getting-help:
5485
5486Getting Help
5487------------
5488
5489You can get general help for the ``wic`` command by entering the ``wic``
5490command by itself or by entering the command with a help argument as
5491follows:
5492::
5493
5494 $ wic -h
5495 $ wic --help
5496 $ wic help
5497
5498Currently, Wic supports seven commands: ``cp``, ``create``, ``help``,
5499``list``, ``ls``, ``rm``, and ``write``. You can get help for all these
5500commands except "help" by using the following form:
5501::
5502
5503 $ wic help command
5504
5505For example, the following command returns help for the ``write``
5506command:
5507::
5508
5509 $ wic help write
5510
5511Wic supports help for three topics: ``overview``, ``plugins``, and
5512``kickstart``. You can get help for any topic using the following form:
5513::
5514
5515 $ wic help topic
5516
5517For example, the following returns overview help for Wic:
5518::
5519
5520 $ wic help overview
5521
5522One additional level of help exists for Wic. You can get help on
5523individual images through the ``list`` command. You can use the ``list``
5524command to return the available Wic images as follows:
5525::
5526
5527 $ wic list images
5528 genericx86 Create an EFI disk image for genericx86*
5529 beaglebone-yocto Create SD card image for Beaglebone
5530 edgerouter Create SD card image for Edgerouter
5531 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
5532 directdisk-gpt Create a 'pcbios' direct disk image
5533 mkefidisk Create an EFI disk image
5534 directdisk Create a 'pcbios' direct disk image
5535 systemd-bootdisk Create an EFI disk image with systemd-boot
5536 mkhybridiso Create a hybrid ISO image
5537 sdimage-bootpart Create SD card image with a boot partition
5538 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
5539 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
5540
5541Once you know the list of available
5542Wic images, you can use ``help`` with the command to get help on a
5543particular image. For example, the following command returns help on the
5544"beaglebone-yocto" image:
5545::
5546
5547 $ wic list beaglebone-yocto help
5548
5549 Creates a partitioned SD card image for Beaglebone.
5550 Boot files are located in the first vfat partition.
5551
5552Operational Modes
5553-----------------
5554
5555You can use Wic in two different modes, depending on how much control
5556you need for specifying the Openembedded build artifacts that are used
5557for creating the image: Raw and Cooked:
5558
5559- *Raw Mode:* You explicitly specify build artifacts through Wic
5560 command-line arguments.
5561
5562- *Cooked Mode:* The current
5563 :term:`MACHINE` setting and image
5564 name are used to automatically locate and provide the build
5565 artifacts. You just supply a kickstart file and the name of the image
5566 from which to use artifacts.
5567
5568Regardless of the mode you use, you need to have the build artifacts
5569ready and available.
5570
5571Raw Mode
5572~~~~~~~~
5573
5574Running Wic in raw mode allows you to specify all the partitions through
5575the ``wic`` command line. The primary use for raw mode is if you have
5576built your kernel outside of the Yocto Project
5577:term:`Build Directory`. In other words, you
5578can point to arbitrary kernel, root filesystem locations, and so forth.
5579Contrast this behavior with cooked mode where Wic looks in the Build
5580Directory (e.g. ``tmp/deploy/images/``\ machine).
5581
5582The general form of the ``wic`` command in raw mode is:
5583::
5584
5585 $ wic create wks_file options ...
5586
5587 Where:
5588
5589 wks_file:
5590 An OpenEmbedded kickstart file. You can provide
5591 your own custom file or use a file from a set of
5592 existing files as described by further options.
5593
5594 optional arguments:
5595 -h, --help show this help message and exit
5596 -o OUTDIR, --outdir OUTDIR
5597 name of directory to create image in
5598 -e IMAGE_NAME, --image-name IMAGE_NAME
5599 name of the image to use the artifacts from e.g. core-
5600 image-sato
5601 -r ROOTFS_DIR, --rootfs-dir ROOTFS_DIR
5602 path to the /rootfs dir to use as the .wks rootfs
5603 source
5604 -b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR
5605 path to the dir containing the boot artifacts (e.g.
5606 /EFI or /syslinux dirs) to use as the .wks bootimg
5607 source
5608 -k KERNEL_DIR, --kernel-dir KERNEL_DIR
5609 path to the dir containing the kernel to use in the
5610 .wks bootimg
5611 -n NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT
5612 path to the native sysroot containing the tools to use
5613 to build the image
5614 -s, --skip-build-check
5615 skip the build check
5616 -f, --build-rootfs build rootfs
5617 -c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz}
5618 compress image with specified compressor
5619 -m, --bmap generate .bmap
5620 --no-fstab-update Do not change fstab file.
5621 -v VARS_DIR, --vars VARS_DIR
5622 directory with <image>.env files that store bitbake
5623 variables
5624 -D, --debug output debug information
5625
5626.. note::
5627
5628 You do not need root privileges to run Wic. In fact, you should not
5629 run as root when using the utility.
5630
5631Cooked Mode
5632~~~~~~~~~~~
5633
5634Running Wic in cooked mode leverages off artifacts in the Build
5635Directory. In other words, you do not have to specify kernel or root
5636filesystem locations as part of the command. All you need to provide is
5637a kickstart file and the name of the image from which to use artifacts
5638by using the "-e" option. Wic looks in the Build Directory (e.g.
5639``tmp/deploy/images/``\ machine) for artifacts.
5640
5641The general form of the ``wic`` command using Cooked Mode is as follows:
5642::
5643
5644 $ wic create wks_file -e IMAGE_NAME
5645
5646 Where:
5647
5648 wks_file:
5649 An OpenEmbedded kickstart file. You can provide
5650 your own custom file or use a file from a set of
5651 existing files provided with the Yocto Project
5652 release.
5653
5654 required argument:
5655 -e IMAGE_NAME, --image-name IMAGE_NAME
5656 name of the image to use the artifacts from e.g. core-
5657 image-sato
5658
5659.. _using-a-provided-kickstart-file:
5660
5661Using an Existing Kickstart File
5662--------------------------------
5663
5664If you do not want to create your own kickstart file, you can use an
5665existing file provided by the Wic installation. As shipped, kickstart
5666files can be found in the :ref:`overview-manual/overview-manual-development-environment:yocto project source repositories` in the
5667following two locations:
5668::
5669
5670 poky/meta-yocto-bsp/wic
5671 poky/scripts/lib/wic/canned-wks
5672
5673Use the following command to list the available kickstart files:
5674::
5675
5676 $ wic list images
5677 genericx86 Create an EFI disk image for genericx86*
5678 beaglebone-yocto Create SD card image for Beaglebone
5679 edgerouter Create SD card image for Edgerouter
5680 qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
5681 directdisk-gpt Create a 'pcbios' direct disk image
5682 mkefidisk Create an EFI disk image
5683 directdisk Create a 'pcbios' direct disk image
5684 systemd-bootdisk Create an EFI disk image with systemd-boot
5685 mkhybridiso Create a hybrid ISO image
5686 sdimage-bootpart Create SD card image with a boot partition
5687 directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
5688 directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
5689
5690When you use an existing file, you
5691do not have to use the ``.wks`` extension. Here is an example in Raw
5692Mode that uses the ``directdisk`` file:
5693::
5694
5695 $ wic create directdisk -r rootfs_dir -b bootimg_dir \
5696 -k kernel_dir -n native_sysroot
5697
5698Here are the actual partition language commands used in the
5699``genericx86.wks`` file to generate an image:
5700::
5701
5702 # short-description: Create an EFI disk image for genericx86*
5703 # long-description: Creates a partitioned EFI disk image for genericx86* machines
5704 part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
5705 part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
5706 part swap --ondisk sda --size 44 --label swap1 --fstype=swap
5707
5708 bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
5709
5710.. _wic-using-the-wic-plugin-interface:
5711
5712Using the Wic Plugin Interface
5713------------------------------
5714
5715You can extend and specialize Wic functionality by using Wic plugins.
5716This section explains the Wic plugin interface.
5717
5718.. note::
5719
5720 Wic plugins consist of "source" and "imager" plugins. Imager plugins
5721 are beyond the scope of this section.
5722
5723Source plugins provide a mechanism to customize partition content during
5724the Wic image generation process. You can use source plugins to map
5725values that you specify using ``--source`` commands in kickstart files
5726(i.e. ``*.wks``) to a plugin implementation used to populate a given
5727partition.
5728
5729.. note::
5730
5731 If you use plugins that have build-time dependencies (e.g. native
5732 tools, bootloaders, and so forth) when building a Wic image, you need
5733 to specify those dependencies using the
5734 WKS_FILE_DEPENDS
5735 variable.
5736
5737Source plugins are subclasses defined in plugin files. As shipped, the
5738Yocto Project provides several plugin files. You can see the source
5739plugin files that ship with the Yocto Project
5740:yocto_git:`here </cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source>`.
5741Each of these plugin files contains source plugins that are designed to
5742populate a specific Wic image partition.
5743
5744Source plugins are subclasses of the ``SourcePlugin`` class, which is
5745defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example,
5746the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py``
5747file is a subclass of the ``SourcePlugin`` class, which is found in the
5748``pluginbase.py`` file.
5749
5750You can also implement source plugins in a layer outside of the Source
5751Repositories (external layer). To do so, be sure that your plugin files
5752are located in a directory whose path is
5753``scripts/lib/wic/plugins/source/`` within your external layer. When the
5754plugin files are located there, the source plugins they contain are made
5755available to Wic.
5756
5757When the Wic implementation needs to invoke a partition-specific
5758implementation, it looks for the plugin with the same name as the
5759``--source`` parameter used in the kickstart file given to that
5760partition. For example, if the partition is set up using the following
5761command in a kickstart file:
5762::
5763
5764 part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
5765
5766The methods defined as class
5767members of the matching source plugin (i.e. ``bootimg-pcbios``) in the
5768``bootimg-pcbios.py`` plugin file are used.
5769
5770To be more concrete, here is the corresponding plugin definition from
5771the ``bootimg-pcbios.py`` file for the previous command along with an
5772example method called by the Wic implementation when it needs to prepare
5773a partition using an implementation-specific function:
5774::
5775
5776 .
5777 .
5778 .
5779 class BootimgPcbiosPlugin(SourcePlugin):
5780 """
5781 Create MBR boot partition and install syslinux on it.
5782 """
5783
5784 name = 'bootimg-pcbios'
5785 .
5786 .
5787 .
5788 @classmethod
5789 def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
5790 oe_builddir, bootimg_dir, kernel_dir,
5791 rootfs_dir, native_sysroot):
5792 """
5793 Called to do the actual content population for a partition i.e. it
5794 'prepares' the partition to be incorporated into the image.
5795 In this case, prepare content for legacy bios boot partition.
5796 """
5797 .
5798 .
5799 .
5800
5801If a
5802subclass (plugin) itself does not implement a particular function, Wic
5803locates and uses the default version in the superclass. It is for this
5804reason that all source plugins are derived from the ``SourcePlugin``
5805class.
5806
5807The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines
5808a set of methods that source plugins can implement or override. Any
5809plugins (subclass of ``SourcePlugin``) that do not implement a
5810particular method inherit the implementation of the method from the
5811``SourcePlugin`` class. For more information, see the ``SourcePlugin``
5812class in the ``pluginbase.py`` file for details:
5813
5814The following list describes the methods implemented in the
5815``SourcePlugin`` class:
5816
5817- ``do_prepare_partition()``: Called to populate a partition with
5818 actual content. In other words, the method prepares the final
5819 partition image that is incorporated into the disk image.
5820
5821- ``do_configure_partition()``: Called before
5822 ``do_prepare_partition()`` to create custom configuration files for a
5823 partition (e.g. syslinux or grub configuration files).
5824
5825- ``do_install_disk()``: Called after all partitions have been
5826 prepared and assembled into a disk image. This method provides a hook
5827 to allow finalization of a disk image (e.g. writing an MBR).
5828
5829- ``do_stage_partition()``: Special content-staging hook called
5830 before ``do_prepare_partition()``. This method is normally empty.
5831
5832 Typically, a partition just uses the passed-in parameters (e.g. the
5833 unmodified value of ``bootimg_dir``). However, in some cases, things
5834 might need to be more tailored. As an example, certain files might
5835 additionally need to be taken from ``bootimg_dir + /boot``. This hook
5836 allows those files to be staged in a customized fashion.
5837
5838 .. note::
5839
5840 get_bitbake_var()
5841 allows you to access non-standard variables that you might want to
5842 use for this behavior.
5843
5844You can extend the source plugin mechanism. To add more hooks, create
5845more source plugin methods within ``SourcePlugin`` and the corresponding
5846derived subclasses. The code that calls the plugin methods uses the
5847``plugin.get_source_plugin_methods()`` function to find the method or
5848methods needed by the call. Retrieval of those methods is accomplished
5849by filling up a dict with keys that contain the method names of
5850interest. On success, these will be filled in with the actual methods.
5851See the Wic implementation for examples and details.
5852
5853.. _wic-usage-examples:
5854
5855Wic Examples
5856------------
5857
5858This section provides several examples that show how to use the Wic
5859utility. All the examples assume the list of requirements in the
5860"`Requirements <#wic-requirements>`__" section have been met. The
5861examples assume the previously generated image is
5862``core-image-minimal``.
5863
5864.. _generate-an-image-using-a-provided-kickstart-file:
5865
5866Generate an Image using an Existing Kickstart File
5867~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5868
5869This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart
5870file:
5871::
5872
5873 $ wic create mkefidisk -e core-image-minimal
5874 INFO: Building wic-tools...
5875 .
5876 .
5877 .
5878 INFO: The new image(s) can be found here:
5879 ./mkefidisk-201804191017-sda.direct
5880
5881 The following build artifacts were used to create the image(s):
5882 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5883 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5884 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5885 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5886
5887 INFO: The image(s) were created using OE kickstart file:
5888 /home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
5889
5890The previous example shows the easiest way to create an image by running
5891in cooked mode and supplying a kickstart file and the "-e" option to
5892point to the existing build artifacts. Your ``local.conf`` file needs to
5893have the :term:`MACHINE` variable set
5894to the machine you are using, which is "qemux86" in this example.
5895
5896Once the image builds, the output provides image location, artifact use,
5897and kickstart file information.
5898
5899.. note::
5900
5901 You should always verify the details provided in the output to make
5902 sure that the image was indeed created exactly as expected.
5903
5904Continuing with the example, you can now write the image from the Build
5905Directory onto a USB stick, or whatever media for which you built your
5906image, and boot from the media. You can write the image by using
5907``bmaptool`` or ``dd``:
5908::
5909
5910 $ oe-run-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sdX
5911
5912or ::
5913
5914 $ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sdX
5915
5916.. note::
5917
5918 For more information on how to use the
5919 bmaptool
5920 to flash a device with an image, see the "
5921 Flashing Images Using
5922 bmaptool
5923 " section.
5924
5925Using a Modified Kickstart File
5926~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5927
5928Because partitioned image creation is driven by the kickstart file, it
5929is easy to affect image creation by changing the parameters in the file.
5930This next example demonstrates that through modification of the
5931``directdisk-gpt`` kickstart file.
5932
5933As mentioned earlier, you can use the command ``wic list images`` to
5934show the list of existing kickstart files. The directory in which the
5935``directdisk-gpt.wks`` file resides is
5936``scripts/lib/image/canned-wks/``, which is located in the
5937:term:`Source Directory` (e.g. ``poky``).
5938Because available files reside in this directory, you can create and add
5939your own custom files to the directory. Subsequent use of the
5940``wic list images`` command would then include your kickstart files.
5941
5942In this example, the existing ``directdisk-gpt`` file already does most
5943of what is needed. However, for the hardware in this example, the image
5944will need to boot from ``sdb`` instead of ``sda``, which is what the
5945``directdisk-gpt`` kickstart file uses.
5946
5947The example begins by making a copy of the ``directdisk-gpt.wks`` file
5948in the ``scripts/lib/image/canned-wks`` directory and then by changing
5949the lines that specify the target disk from which to boot.
5950::
5951
5952 $ cp /home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \
5953 /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5954
5955Next, the example modifies the ``directdisksdb-gpt.wks`` file and
5956changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The
5957example changes the following two lines and leaves the remaining lines
5958untouched:
5959::
5960
5961 part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024
5962 part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid
5963
5964Once the lines are changed, the
5965example generates the ``directdisksdb-gpt`` image. The command points
5966the process at the ``core-image-minimal`` artifacts for the Next Unit of
5967Computing (nuc) :term:`MACHINE` the
5968``local.conf``.
5969::
5970
5971 $ wic create directdisksdb-gpt -e core-image-minimal
5972 INFO: Building wic-tools...
5973 .
5974 .
5975 .
5976 Initialising tasks: 100% |#######################################| Time: 0:00:01
5977 NOTE: Executing SetScene Tasks
5978 NOTE: Executing RunQueue Tasks
5979 NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded.
5980 INFO: Creating image(s)...
5981
5982 INFO: The new image(s) can be found here:
5983 ./directdisksdb-gpt-201710090938-sdb.direct
5984
5985 The following build artifacts were used to create the image(s):
5986 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
5987 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
5988 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
5989 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
5990
5991 INFO: The image(s) were created using OE kickstart file:
5992 /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
5993
5994Continuing with the example, you can now directly ``dd`` the image to a
5995USB stick, or whatever media for which you built your image, and boot
5996the resulting media:
5997::
5998
5999 $ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb
6000 140966+0 records in
6001 140966+0 records out
6002 72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s
6003 $ sudo eject /dev/sdb
6004
6005Using a Modified Kickstart File and Running in Raw Mode
6006~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6007
6008This next example manually specifies each build artifact (runs in Raw
6009Mode) and uses a modified kickstart file. The example also uses the
6010``-o`` option to cause Wic to create the output somewhere other than the
6011default output directory, which is the current directory:
6012::
6013
6014 $ wic create /home/stephano/my_yocto/test.wks -o /home/stephano/testwic \
6015 --rootfs-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \
6016 --bootimg-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \
6017 --kernel-dir /home/stephano/build/master/build/tmp/deploy/images/qemux86 \
6018 --native-sysroot /home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
6019
6020 INFO: Creating image(s)...
6021
6022 INFO: The new image(s) can be found here:
6023 /home/stephano/testwic/test-201710091445-sdb.direct
6024
6025 The following build artifacts were used to create the image(s):
6026 ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
6027 BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
6028 KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86
6029 NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
6030
6031 INFO: The image(s) were created using OE kickstart file:
6032 /home/stephano/my_yocto/test.wks
6033
6034For this example,
6035:term:`MACHINE` did not have to be
6036specified in the ``local.conf`` file since the artifact is manually
6037specified.
6038
6039Using Wic to Manipulate an Image
6040~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6041
6042Wic image manipulation allows you to shorten turnaround time during
6043image development. For example, you can use Wic to delete the kernel
6044partition of a Wic image and then insert a newly built kernel. This
6045saves you time from having to rebuild the entire image each time you
6046modify the kernel.
6047
6048.. note::
6049
6050 In order to use Wic to manipulate a Wic image as in this example,
6051 your development machine must have the
6052 mtools
6053 package installed.
6054
6055The following example examines the contents of the Wic image, deletes
6056the existing kernel, and then inserts a new kernel:
6057
60581. *List the Partitions:* Use the ``wic ls`` command to list all the
6059 partitions in the Wic image:
6060 ::
6061
6062 $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic
6063 Num Start End Size Fstype
6064 1 1048576 25041919 23993344 fat16
6065 2 25165824 72157183 46991360 ext4
6066
6067 The previous output shows two partitions in the
6068 ``core-image-minimal-qemux86.wic`` image.
6069
60702. *Examine a Particular Partition:* Use the ``wic ls`` command again
6071 but in a different form to examine a particular partition.
6072
6073 .. note::
6074
6075 You can get command usage on any Wic command using the following
6076 form:
6077 ::
6078
6079 $ wic help command
6080
6081
6082 For example, the following command shows you the various ways to
6083 use the
6084 wic ls
6085 command:
6086 ::
6087
6088 $ wic help ls
6089
6090
6091 The following command shows what is in Partition one:
6092 ::
6093
6094 $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1
6095 Volume in drive : is boot
6096 Volume Serial Number is E894-1809
6097 Directory for ::/
6098
6099 libcom32 c32 186500 2017-10-09 16:06
6100 libutil c32 24148 2017-10-09 16:06
6101 syslinux cfg 220 2017-10-09 16:06
6102 vesamenu c32 27104 2017-10-09 16:06
6103 vmlinuz 6904608 2017-10-09 16:06
6104 5 files 7 142 580 bytes
6105 16 582 656 bytes free
6106
6107 The previous output shows five files, with the
6108 ``vmlinuz`` being the kernel.
6109
6110 .. note::
6111
6112 If you see the following error, you need to update or create a
6113 ~/.mtoolsrc
6114 file and be sure to have the line "mtools_skip_check=1" in the
6115 file. Then, run the Wic command again:
6116 ::
6117
6118 ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
6119 output: Total number of sectors (47824) not a multiple of sectors per track (32)!
6120 Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
6121
6122
61233. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the
6124 ``vmlinuz`` file (kernel):
6125 ::
6126
6127 $ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
6128
61294. *Add In the New Kernel:* Use the ``wic cp`` command to add the
6130 updated kernel to the Wic image. Depending on how you built your
6131 kernel, it could be in different places. If you used ``devtool`` and
6132 an SDK to build your kernel, it resides in the ``tmp/work`` directory
6133 of the extensible SDK. If you used ``make`` to build the kernel, the
6134 kernel will be in the ``workspace/sources`` area.
6135
6136 The following example assumes ``devtool`` was used to build the
6137 kernel:
6138 ::
6139
6140 cp ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \
6141 ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
6142
6143 Once the new kernel is added back into the image, you can use the
6144 ``dd`` command or ```bmaptool`` <#flashing-images-using-bmaptool>`__
6145 to flash your wic image onto an SD card or USB stick and test your
6146 target.
6147
6148 .. note::
6149
6150 Using
6151 bmaptool
6152 is generally 10 to 20 times faster than using
6153 dd
6154 .
6155
6156Flashing Images Using ``bmaptool``
6157==================================
6158
6159A fast and easy way to flash an image to a bootable device is to use
6160Bmaptool, which is integrated into the OpenEmbedded build system.
6161Bmaptool is a generic tool that creates a file's block map (bmap) and
6162then uses that map to copy the file. As compared to traditional tools
6163such as dd or cp, Bmaptool can copy (or flash) large files like raw
6164system image files much faster.
6165
6166.. note::
6167
6168 - If you are using Ubuntu or Debian distributions, you can install
6169 the ``bmap-tools`` package using the following command and then
6170 use the tool without specifying ``PATH`` even from the root
6171 account: $ sudo apt-get install bmap-tools
6172
6173 - If you are unable to install the ``bmap-tools`` package, you will
6174 need to build Bmaptool before using it. Use the following command:
6175 $ bitbake bmap-tools-native
6176
6177Following, is an example that shows how to flash a Wic image. Realize
6178that while this example uses a Wic image, you can use Bmaptool to flash
6179any type of image. Use these steps to flash an image using Bmaptool:
6180
61811. *Update your local.conf File:* You need to have the following set
6182 in your ``local.conf`` file before building your image:
6183 ::
6184
6185 IMAGE_FSTYPES += "wic wic.bmap"
6186
61872. *Get Your Image:* Either have your image ready (pre-built with the
6188 :term:`IMAGE_FSTYPES`
6189 setting previously mentioned) or take the step to build the image:
6190 ::
6191
6192 $ bitbake image
6193
61943. *Flash the Device:* Flash the device with the image by using Bmaptool
6195 depending on your particular setup. The following commands assume the
6196 image resides in the Build Directory's ``deploy/images/`` area:
6197
6198 - If you have write access to the media, use this command form:
6199 ::
6200
6201 $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
6202
6203 - If you do not have write access to the media, set your permissions
6204 first and then use the same command form:
6205 ::
6206
6207 $ sudo chmod 666 /dev/sdX
6208 $ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
6209
6210For help on the ``bmaptool`` command, use the following command:
6211::
6212
6213 $ bmaptool --help
6214
6215Making Images More Secure
6216=========================
6217
6218Security is of increasing concern for embedded devices. Consider the
6219issues and problems discussed in just this sampling of work found across
6220the Internet:
6221
6222- *"*\ `Security Risks of Embedded
6223 Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"*
6224 by Bruce Schneier
6225
6226- *"*\ `Internet Census
6227 2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna
6228 Botnet
6229
6230- *"*\ `Security Issues for Embedded
6231 Devices <http://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"*
6232 by Jake Edge
6233
6234When securing your image is of concern, there are steps, tools, and
6235variables that you can consider to help you reach the security goals you
6236need for your particular device. Not all situations are identical when
6237it comes to making an image secure. Consequently, this section provides
6238some guidance and suggestions for consideration when you want to make
6239your image more secure.
6240
6241.. note::
6242
6243 Because the security requirements and risks are different for every
6244 type of device, this section cannot provide a complete reference on
6245 securing your custom OS. It is strongly recommended that you also
6246 consult other sources of information on embedded Linux system
6247 hardening and on security.
6248
6249General Considerations
6250----------------------
6251
6252General considerations exist that help you create more secure images.
6253You should consider the following suggestions to help make your device
6254more secure:
6255
6256- Scan additional code you are adding to the system (e.g. application
6257 code) by using static analysis tools. Look for buffer overflows and
6258 other potential security problems.
6259
6260- Pay particular attention to the security for any web-based
6261 administration interface.
6262
6263 Web interfaces typically need to perform administrative functions and
6264 tend to need to run with elevated privileges. Thus, the consequences
6265 resulting from the interface's security becoming compromised can be
6266 serious. Look for common web vulnerabilities such as
6267 cross-site-scripting (XSS), unvalidated inputs, and so forth.
6268
6269 As with system passwords, the default credentials for accessing a
6270 web-based interface should not be the same across all devices. This
6271 is particularly true if the interface is enabled by default as it can
6272 be assumed that many end-users will not change the credentials.
6273
6274- Ensure you can update the software on the device to mitigate
6275 vulnerabilities discovered in the future. This consideration
6276 especially applies when your device is network-enabled.
6277
6278- Ensure you remove or disable debugging functionality before producing
6279 the final image. For information on how to do this, see the
6280 "`Considerations Specific to the OpenEmbedded Build
6281 System <#considerations-specific-to-the-openembedded-build-system>`__"
6282 section.
6283
6284- Ensure you have no network services listening that are not needed.
6285
6286- Remove any software from the image that is not needed.
6287
6288- Enable hardware support for secure boot functionality when your
6289 device supports this functionality.
6290
6291Security Flags
6292--------------
6293
6294The Yocto Project has security flags that you can enable that help make
6295your build output more secure. The security flags are in the
6296``meta/conf/distro/include/security_flags.inc`` file in your
6297:term:`Source Directory` (e.g. ``poky``).
6298
6299.. note::
6300
6301 Depending on the recipe, certain security flags are enabled and
6302 disabled by default.
6303
6304Use the following line in your ``local.conf`` file or in your custom
6305distribution configuration file to enable the security compiler and
6306linker flags for your build:
6307::
6308
6309 require conf/distro/include/security_flags.inc
6310
6311Considerations Specific to the OpenEmbedded Build System
6312--------------------------------------------------------
6313
6314You can take some steps that are specific to the OpenEmbedded build
6315system to make your images more secure:
6316
6317- Ensure "debug-tweaks" is not one of your selected
6318 :term:`IMAGE_FEATURES`.
6319 When creating a new project, the default is to provide you with an
6320 initial ``local.conf`` file that enables this feature using the
6321 :term:`EXTRA_IMAGE_FEATURES`
6322 variable with the line:
6323 ::
6324
6325 EXTRA_IMAGE_FEATURES = "debug-tweaks"
6326
6327 To disable that feature, simply comment out that line in your
6328 ``local.conf`` file, or make sure ``IMAGE_FEATURES`` does not contain
6329 "debug-tweaks" before producing your final image. Among other things,
6330 leaving this in place sets the root password as blank, which makes
6331 logging in for debugging or inspection easy during development but
6332 also means anyone can easily log in during production.
6333
6334- It is possible to set a root password for the image and also to set
6335 passwords for any extra users you might add (e.g. administrative or
6336 service type users). When you set up passwords for multiple images or
6337 users, you should not duplicate passwords.
6338
6339 To set up passwords, use the
6340 :ref:`extrausers <ref-classes-extrausers>`
6341 class, which is the preferred method. For an example on how to set up
6342 both root and user passwords, see the
6343 ":ref:`extrausers.bbclass <ref-classes-extrausers>`"
6344 section.
6345
6346 .. note::
6347
6348 When adding extra user accounts or setting a root password, be
6349 cautious about setting the same password on every device. If you
6350 do this, and the password you have set is exposed, then every
6351 device is now potentially compromised. If you need this access but
6352 want to ensure security, consider setting a different, random
6353 password for each device. Typically, you do this as a separate
6354 step after you deploy the image onto the device.
6355
6356- Consider enabling a Mandatory Access Control (MAC) framework such as
6357 SMACK or SELinux and tuning it appropriately for your device's usage.
6358 You can find more information in the
6359 `meta-selinux <http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/>`__
6360 layer.
6361
6362Tools for Hardening Your Image
6363------------------------------
6364
6365The Yocto Project provides tools for making your image more secure. You
6366can find these tools in the ``meta-security`` layer of the
6367:yocto_git:`Yocto Project Source Repositories <>`.
6368
6369Creating Your Own Distribution
6370==============================
6371
6372When you build an image using the Yocto Project and do not alter any
6373distribution :term:`Metadata`, you are
6374creating a Poky distribution. If you wish to gain more control over
6375package alternative selections, compile-time options, and other
6376low-level configurations, you can create your own distribution.
6377
6378To create your own distribution, the basic steps consist of creating
6379your own distribution layer, creating your own distribution
6380configuration file, and then adding any needed code and Metadata to the
6381layer. The following steps provide some more detail:
6382
6383- *Create a layer for your new distro:* Create your distribution layer
6384 so that you can keep your Metadata and code for the distribution
6385 separate. It is strongly recommended that you create and use your own
6386 layer for configuration and code. Using your own layer as compared to
6387 just placing configurations in a ``local.conf`` configuration file
6388 makes it easier to reproduce the same build configuration when using
6389 multiple build machines. See the
6390 ":ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the \`\`bitbake-layers\`\` script`"
6391 section for information on how to quickly set up a layer.
6392
6393- *Create the distribution configuration file:* The distribution
6394 configuration file needs to be created in the ``conf/distro``
6395 directory of your layer. You need to name it using your distribution
6396 name (e.g. ``mydistro.conf``).
6397
6398 .. note::
6399
6400 The
6401 DISTRO
6402 variable in your
6403 local.conf
6404 file determines the name of your distribution.
6405
6406 You can split out parts of your configuration file into include files
6407 and then "require" them from within your distribution configuration
6408 file. Be sure to place the include files in the
6409 ``conf/distro/include`` directory of your layer. A common example
6410 usage of include files would be to separate out the selection of
6411 desired version and revisions for individual recipes.
6412
6413 Your configuration file needs to set the following required
6414 variables:
6415
6416 - :term:`DISTRO_NAME`
6417
6418 - :term:`DISTRO_VERSION`
6419
6420 These following variables are optional and you typically set them
6421 from the distribution configuration file:
6422
6423 - :term:`DISTRO_FEATURES`
6424
6425 - :term:`DISTRO_EXTRA_RDEPENDS`
6426
6427 - :term:`DISTRO_EXTRA_RRECOMMENDS`
6428
6429 - :term:`TCLIBC`
6430
6431 .. tip::
6432
6433 If you want to base your distribution configuration file on the
6434 very basic configuration from OE-Core, you can use
6435 conf/distro/defaultsetup.conf
6436 as a reference and just include variables that differ as compared
6437 to
6438 defaultsetup.conf
6439 . Alternatively, you can create a distribution configuration file
6440 from scratch using the
6441 defaultsetup.conf
6442 file or configuration files from other distributions such as Poky
6443 or Angstrom as references.
6444
6445- *Provide miscellaneous variables:* Be sure to define any other
6446 variables for which you want to create a default or enforce as part
6447 of the distribution configuration. You can include nearly any
6448 variable from the ``local.conf`` file. The variables you use are not
6449 limited to the list in the previous bulleted item.
6450
6451- *Point to Your distribution configuration file:* In your
6452 ``local.conf`` file in the :term:`Build Directory`,
6453 set your
6454 :term:`DISTRO` variable to point to
6455 your distribution's configuration file. For example, if your
6456 distribution's configuration file is named ``mydistro.conf``, then
6457 you point to it as follows:
6458 ::
6459
6460 DISTRO = "mydistro"
6461
6462- *Add more to the layer if necessary:* Use your layer to hold other
6463 information needed for the distribution:
6464
6465 - Add recipes for installing distro-specific configuration files
6466 that are not already installed by another recipe. If you have
6467 distro-specific configuration files that are included by an
6468 existing recipe, you should add an append file (``.bbappend``) for
6469 those. For general information and recommendations on how to add
6470 recipes to your layer, see the "`Creating Your Own
6471 Layer <#creating-your-own-layer>`__" and "`Following Best
6472 Practices When Creating
6473 Layers <#best-practices-to-follow-when-creating-layers>`__"
6474 sections.
6475
6476 - Add any image recipes that are specific to your distribution.
6477
6478 - Add a ``psplash`` append file for a branded splash screen. For
6479 information on append files, see the "`Using .bbappend Files in
6480 Your Layer <#using-bbappend-files>`__" section.
6481
6482 - Add any other append files to make custom changes that are
6483 specific to individual recipes.
6484
6485Creating a Custom Template Configuration Directory
6486==================================================
6487
6488If you are producing your own customized version of the build system for
6489use by other users, you might want to customize the message shown by the
6490setup script or you might want to change the template configuration
6491files (i.e. ``local.conf`` and ``bblayers.conf``) that are created in a
6492new build directory.
6493
6494The OpenEmbedded build system uses the environment variable
6495``TEMPLATECONF`` to locate the directory from which it gathers
6496configuration information that ultimately ends up in the
6497:term:`Build Directory` ``conf`` directory.
6498By default, ``TEMPLATECONF`` is set as follows in the ``poky``
6499repository:
6500::
6501
6502 TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf}
6503
6504This is the
6505directory used by the build system to find templates from which to build
6506some key configuration files. If you look at this directory, you will
6507see the ``bblayers.conf.sample``, ``local.conf.sample``, and
6508``conf-notes.txt`` files. The build system uses these files to form the
6509respective ``bblayers.conf`` file, ``local.conf`` file, and display the
6510list of BitBake targets when running the setup script.
6511
6512To override these default configuration files with configurations you
6513want used within every new Build Directory, simply set the
6514``TEMPLATECONF`` variable to your directory. The ``TEMPLATECONF``
6515variable is set in the ``.templateconf`` file, which is in the top-level
6516:term:`Source Directory` folder
6517(e.g. ``poky``). Edit the ``.templateconf`` so that it can locate your
6518directory.
6519
6520Best practices dictate that you should keep your template configuration
6521directory in your custom distribution layer. For example, suppose you
6522have a layer named ``meta-mylayer`` located in your home directory and
6523you want your template configuration directory named ``myconf``.
6524Changing the ``.templateconf`` as follows causes the OpenEmbedded build
6525system to look in your directory and base its configuration files on the
6526``*.sample`` configuration files it finds. The final configuration files
6527(i.e. ``local.conf`` and ``bblayers.conf`` ultimately still end up in
6528your Build Directory, but they are based on your ``*.sample`` files.
6529::
6530
6531 TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf}
6532
6533Aside from the ``*.sample`` configuration files, the ``conf-notes.txt``
6534also resides in the default ``meta-poky/conf`` directory. The script
6535that sets up the build environment (i.e.
6536:ref:`structure-core-script`) uses this file to
6537display BitBake targets as part of the script output. Customizing this
6538``conf-notes.txt`` file is a good way to make sure your list of custom
6539targets appears as part of the script's output.
6540
6541Here is the default list of targets displayed as a result of running
6542either of the setup scripts:
6543::
6544
6545 You can now run 'bitbake <target>'
6546
6547 Common targets are:
6548 core-image-minimal
6549 core-image-sato
6550 meta-toolchain
6551 meta-ide-support
6552
6553Changing the listed common targets is as easy as editing your version of
6554``conf-notes.txt`` in your custom template configuration directory and
6555making sure you have ``TEMPLATECONF`` set to your directory.
6556
6557.. _dev-saving-memory-during-a-build:
6558
6559Conserving Disk Space During Builds
6560===================================
6561
6562To help conserve disk space during builds, you can add the following
6563statement to your project's ``local.conf`` configuration file found in
6564the :term:`Build Directory`:
6565::
6566
6567 INHERIT += "rm_work"
6568
6569Adding this statement deletes the work directory used for
6570building a recipe once the recipe is built. For more information on
6571"rm_work", see the
6572:ref:`rm_work <ref-classes-rm-work>` class in the
6573Yocto Project Reference Manual.
6574
6575Working with Packages
6576=====================
6577
6578This section describes a few tasks that involve packages:
6579
6580- `Excluding packages from an
6581 image <#excluding-packages-from-an-image>`__
6582
6583- `Incrementing a binary package
6584 version <#incrementing-a-binary-package-version>`__
6585
6586- `Handling optional module
6587 packaging <#handling-optional-module-packaging>`__
6588
6589- `Using runtime package
6590 management <#using-runtime-package-management>`__
6591
6592- `Generating and using signed
6593 packages <#generating-and-using-signed-packages>`__
6594
6595- `Setting up and running package test
6596 (ptest) <#testing-packages-with-ptest>`__
6597
6598- `Creating node package manager (NPM)
6599 packages <#creating-node-package-manager-npm-packages>`__
6600
6601- `Adding custom metadata to
6602 packages <#adding-custom-metadata-to-packages>`__
6603
6604Excluding Packages from an Image
6605--------------------------------
6606
6607You might find it necessary to prevent specific packages from being
6608installed into an image. If so, you can use several variables to direct
6609the build system to essentially ignore installing recommended packages
6610or to not install a package at all.
6611
6612The following list introduces variables you can use to prevent packages
6613from being installed into your image. Each of these variables only works
6614with IPK and RPM package types. Support for Debian packages does not
6615exist. Also, you can use these variables from your ``local.conf`` file
6616or attach them to a specific image recipe by using a recipe name
6617override. For more detail on the variables, see the descriptions in the
6618Yocto Project Reference Manual's glossary chapter.
6619
6620- :term:`BAD_RECOMMENDATIONS`:
6621 Use this variable to specify "recommended-only" packages that you do
6622 not want installed.
6623
6624- :term:`NO_RECOMMENDATIONS`:
6625 Use this variable to prevent all "recommended-only" packages from
6626 being installed.
6627
6628- :term:`PACKAGE_EXCLUDE`:
6629 Use this variable to prevent specific packages from being installed
6630 regardless of whether they are "recommended-only" or not. You need to
6631 realize that the build process could fail with an error when you
6632 prevent the installation of a package whose presence is required by
6633 an installed package.
6634
6635.. _incrementing-a-binary-package-version:
6636
6637Incrementing a Package Version
6638------------------------------
6639
6640This section provides some background on how binary package versioning
6641is accomplished and presents some of the services, variables, and
6642terminology involved.
6643
6644In order to understand binary package versioning, you need to consider
6645the following:
6646
6647- Binary Package: The binary package that is eventually built and
6648 installed into an image.
6649
6650- Binary Package Version: The binary package version is composed of two
6651 components - a version and a revision.
6652
6653 .. note::
6654
6655 Technically, a third component, the "epoch" (i.e.
6656 PE
6657 ) is involved but this discussion for the most part ignores
6658 PE
6659 .
6660
6661 The version and revision are taken from the
6662 :term:`PV` and
6663 :term:`PR` variables, respectively.
6664
6665- ``PV``: The recipe version. ``PV`` represents the version of the
6666 software being packaged. Do not confuse ``PV`` with the binary
6667 package version.
6668
6669- ``PR``: The recipe revision.
6670
6671- :term:`SRCPV`: The OpenEmbedded
6672 build system uses this string to help define the value of ``PV`` when
6673 the source code revision needs to be included in it.
6674
6675- :yocto_wiki:`PR Service </wiki/PR_Service>`: A
6676 network-based service that helps automate keeping package feeds
6677 compatible with existing package manager applications such as RPM,
6678 APT, and OPKG.
6679
6680Whenever the binary package content changes, the binary package version
6681must change. Changing the binary package version is accomplished by
6682changing or "bumping" the ``PR`` and/or ``PV`` values. Increasing these
6683values occurs one of two ways:
6684
6685- Automatically using a Package Revision Service (PR Service).
6686
6687- Manually incrementing the ``PR`` and/or ``PV`` variables.
6688
6689Given a primary challenge of any build system and its users is how to
6690maintain a package feed that is compatible with existing package manager
6691applications such as RPM, APT, and OPKG, using an automated system is
6692much preferred over a manual system. In either system, the main
6693requirement is that binary package version numbering increases in a
6694linear fashion and that a number of version components exist that
6695support that linear progression. For information on how to ensure
6696package revisioning remains linear, see the "`Automatically Incrementing
6697a Binary Package Revision
6698Number <#automatically-incrementing-a-binary-package-revision-number>`__"
6699section.
6700
6701The following three sections provide related information on the PR
6702Service, the manual method for "bumping" ``PR`` and/or ``PV``, and on
6703how to ensure binary package revisioning remains linear.
6704
6705Working With a PR Service
6706~~~~~~~~~~~~~~~~~~~~~~~~~
6707
6708As mentioned, attempting to maintain revision numbers in the
6709:term:`Metadata` is error prone, inaccurate,
6710and causes problems for people submitting recipes. Conversely, the PR
6711Service automatically generates increasing numbers, particularly the
6712revision field, which removes the human element.
6713
6714.. note::
6715
6716 For additional information on using a PR Service, you can see the
6717 PR Service
6718 wiki page.
6719
6720The Yocto Project uses variables in order of decreasing priority to
6721facilitate revision numbering (i.e.
6722:term:`PE`,
6723:term:`PV`, and
6724:term:`PR` for epoch, version, and
6725revision, respectively). The values are highly dependent on the policies
6726and procedures of a given distribution and package feed.
6727
6728Because the OpenEmbedded build system uses
6729":ref:`signatures <overview-checksums>`", which are
6730unique to a given build, the build system knows when to rebuild
6731packages. All the inputs into a given task are represented by a
6732signature, which can trigger a rebuild when different. Thus, the build
6733system itself does not rely on the ``PR``, ``PV``, and ``PE`` numbers to
6734trigger a rebuild. The signatures, however, can be used to generate
6735these values.
6736
6737The PR Service works with both ``OEBasic`` and ``OEBasicHash``
6738generators. The value of ``PR`` bumps when the checksum changes and the
6739different generator mechanisms change signatures under different
6740circumstances.
6741
6742As implemented, the build system includes values from the PR Service
6743into the ``PR`` field as an addition using the form "``.x``" so ``r0``
6744becomes ``r0.1``, ``r0.2`` and so forth. This scheme allows existing
6745``PR`` values to be used for whatever reasons, which include manual
6746``PR`` bumps, should it be necessary.
6747
6748By default, the PR Service is not enabled or running. Thus, the packages
6749generated are just "self consistent". The build system adds and removes
6750packages and there are no guarantees about upgrade paths but images will
6751be consistent and correct with the latest changes.
6752
6753The simplest form for a PR Service is for it to exist for a single host
6754development system that builds the package feed (building system). For
6755this scenario, you can enable a local PR Service by setting
6756:term:`PRSERV_HOST` in your
6757``local.conf`` file in the :term:`Build Directory`:
6758::
6759
6760 PRSERV_HOST = "localhost:0"
6761
6762Once the service is started, packages will automatically
6763get increasing ``PR`` values and BitBake takes care of starting and
6764stopping the server.
6765
6766If you have a more complex setup where multiple host development systems
6767work against a common, shared package feed, you have a single PR Service
6768running and it is connected to each building system. For this scenario,
6769you need to start the PR Service using the ``bitbake-prserv`` command:
6770::
6771
6772 bitbake-prserv --host ip --port port --start
6773
6774In addition to
6775hand-starting the service, you need to update the ``local.conf`` file of
6776each building system as described earlier so each system points to the
6777server and port.
6778
6779It is also recommended you use build history, which adds some sanity
6780checks to binary package versions, in conjunction with the server that
6781is running the PR Service. To enable build history, add the following to
6782each building system's ``local.conf`` file:
6783::
6784
6785 # It is recommended to activate "buildhistory" for testing the PR service
6786 INHERIT += "buildhistory"
6787 BUILDHISTORY_COMMIT = "1"
6788
6789For information on build
6790history, see the "`Maintaining Build Output
6791Quality <#maintaining-build-output-quality>`__" section.
6792
6793.. note::
6794
6795 The OpenEmbedded build system does not maintain ``PR`` information as
6796 part of the shared state (sstate) packages. If you maintain an sstate
6797 feed, its expected that either all your building systems that
6798 contribute to the sstate feed use a shared PR Service, or you do not
6799 run a PR Service on any of your building systems. Having some systems
6800 use a PR Service while others do not leads to obvious problems.
6801
6802 For more information on shared state, see the
6803 ":ref:`overview-manual/overview-manual-concepts:shared state cache`"
6804 section in the Yocto Project Overview and Concepts Manual.
6805
6806Manually Bumping PR
6807~~~~~~~~~~~~~~~~~~~
6808
6809The alternative to setting up a PR Service is to manually "bump" the
6810:term:`PR` variable.
6811
6812If a committed change results in changing the package output, then the
6813value of the PR variable needs to be increased (or "bumped") as part of
6814that commit. For new recipes you should add the ``PR`` variable and set
6815its initial value equal to "r0", which is the default. Even though the
6816default value is "r0", the practice of adding it to a new recipe makes
6817it harder to forget to bump the variable when you make changes to the
6818recipe in future.
6819
6820If you are sharing a common ``.inc`` file with multiple recipes, you can
6821also use the ``INC_PR`` variable to ensure that the recipes sharing the
6822``.inc`` file are rebuilt when the ``.inc`` file itself is changed. The
6823``.inc`` file must set ``INC_PR`` (initially to "r0"), and all recipes
6824referring to it should set ``PR`` to "${INC_PR}.0" initially,
6825incrementing the last number when the recipe is changed. If the ``.inc``
6826file is changed then its ``INC_PR`` should be incremented.
6827
6828When upgrading the version of a binary package, assuming the ``PV``
6829changes, the ``PR`` variable should be reset to "r0" (or "${INC_PR}.0"
6830if you are using ``INC_PR``).
6831
6832Usually, version increases occur only to binary packages. However, if
6833for some reason ``PV`` changes but does not increase, you can increase
6834the ``PE`` variable (Package Epoch). The ``PE`` variable defaults to
6835"0".
6836
6837Binary package version numbering strives to follow the `Debian Version
6838Field Policy
6839Guidelines <http://www.debian.org/doc/debian-policy/ch-controlfields.html>`__.
6840These guidelines define how versions are compared and what "increasing"
6841a version means.
6842
6843.. _automatically-incrementing-a-binary-package-revision-number:
6844
6845Automatically Incrementing a Package Version Number
6846~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6847
6848When fetching a repository, BitBake uses the
6849:term:`SRCREV` variable to determine
6850the specific source code revision from which to build. You set the
6851``SRCREV`` variable to
6852:term:`AUTOREV` to cause the
6853OpenEmbedded build system to automatically use the latest revision of
6854the software:
6855::
6856
6857 SRCREV = "${AUTOREV}"
6858
6859Furthermore, you need to reference ``SRCPV`` in ``PV`` in order to
6860automatically update the version whenever the revision of the source
6861code changes. Here is an example:
6862::
6863
6864 PV = "1.0+git${SRCPV}"
6865
6866The OpenEmbedded build system substitutes ``SRCPV`` with the following:
6867::
6868
6869 AUTOINC+source_code_revision
6870
6871The build system replaces the ``AUTOINC``
6872with a number. The number used depends on the state of the PR Service:
6873
6874- If PR Service is enabled, the build system increments the number,
6875 which is similar to the behavior of
6876 :term:`PR`. This behavior results in
6877 linearly increasing package versions, which is desirable. Here is an
6878 example:
6879 ::
6880
6881 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6882 hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk
6883
6884- If PR Service is not enabled, the build system replaces the
6885 ``AUTOINC`` placeholder with zero (i.e. "0"). This results in
6886 changing the package version since the source revision is included.
6887 However, package versions are not increased linearly. Here is an
6888 example:
6889 ::
6890
6891 hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk
6892 hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk
6893
6894In summary, the OpenEmbedded build system does not track the history of
6895binary package versions for this purpose. ``AUTOINC``, in this case, is
6896comparable to ``PR``. If PR server is not enabled, ``AUTOINC`` in the
6897package version is simply replaced by "0". If PR server is enabled, the
6898build system keeps track of the package versions and bumps the number
6899when the package revision changes.
6900
6901Handling Optional Module Packaging
6902----------------------------------
6903
6904Many pieces of software split functionality into optional modules (or
6905plugins) and the plugins that are built might depend on configuration
6906options. To avoid having to duplicate the logic that determines what
6907modules are available in your recipe or to avoid having to package each
6908module by hand, the OpenEmbedded build system provides functionality to
6909handle module packaging dynamically.
6910
6911To handle optional module packaging, you need to do two things:
6912
6913- Ensure the module packaging is actually done.
6914
6915- Ensure that any dependencies on optional modules from other recipes
6916 are satisfied by your recipe.
6917
6918Making Sure the Packaging is Done
6919~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6920
6921To ensure the module packaging actually gets done, you use the
6922``do_split_packages`` function within the ``populate_packages`` Python
6923function in your recipe. The ``do_split_packages`` function searches for
6924a pattern of files or directories under a specified path and creates a
6925package for each one it finds by appending to the
6926:term:`PACKAGES` variable and
6927setting the appropriate values for ``FILES_packagename``,
6928``RDEPENDS_packagename``, ``DESCRIPTION_packagename``, and so forth.
6929Here is an example from the ``lighttpd`` recipe:
6930::
6931
6932 python populate_packages_prepend () {
6933 lighttpd_libdir = d.expand('${libdir}')
6934 do_split_packages(d, lighttpd_libdir, '^mod_(.*).so$',
6935 'lighttpd-module-%s', 'Lighttpd module for %s',
6936 extra_depends='')
6937 }
6938
6939The previous example specifies a number of things in the call to
6940``do_split_packages``.
6941
6942- A directory within the files installed by your recipe through
6943 ``do_install`` in which to search.
6944
6945- A regular expression used to match module files in that directory. In
6946 the example, note the parentheses () that mark the part of the
6947 expression from which the module name should be derived.
6948
6949- A pattern to use for the package names.
6950
6951- A description for each package.
6952
6953- An empty string for ``extra_depends``, which disables the default
6954 dependency on the main ``lighttpd`` package. Thus, if a file in
6955 ``${libdir}`` called ``mod_alias.so`` is found, a package called
6956 ``lighttpd-module-alias`` is created for it and the
6957 :term:`DESCRIPTION` is set to
6958 "Lighttpd module for alias".
6959
6960Often, packaging modules is as simple as the previous example. However,
6961more advanced options exist that you can use within
6962``do_split_packages`` to modify its behavior. And, if you need to, you
6963can add more logic by specifying a hook function that is called for each
6964package. It is also perfectly acceptable to call ``do_split_packages``
6965multiple times if you have more than one set of modules to package.
6966
6967For more examples that show how to use ``do_split_packages``, see the
6968``connman.inc`` file in the ``meta/recipes-connectivity/connman/``
6969directory of the ``poky`` :ref:`source repository <yocto-project-repositories>`. You can
6970also find examples in ``meta/classes/kernel.bbclass``.
6971
6972Following is a reference that shows ``do_split_packages`` mandatory and
6973optional arguments:
6974::
6975
6976 Mandatory arguments
6977
6978 root
6979 The path in which to search
6980 file_regex
6981 Regular expression to match searched files.
6982 Use parentheses () to mark the part of this
6983 expression that should be used to derive the
6984 module name (to be substituted where %s is
6985 used in other function arguments as noted below)
6986 output_pattern
6987 Pattern to use for the package names. Must
6988 include %s.
6989 description
6990 Description to set for each package. Must
6991 include %s.
6992
6993 Optional arguments
6994
6995 postinst
6996 Postinstall script to use for all packages
6997 (as a string)
6998 recursive
6999 True to perform a recursive search - default
7000 False
7001 hook
7002 A hook function to be called for every match.
7003 The function will be called with the following
7004 arguments (in the order listed):
7005
7006 f
7007 Full path to the file/directory match
7008 pkg
7009 The package name
7010 file_regex
7011 As above
7012 output_pattern
7013 As above
7014 modulename
7015 The module name derived using file_regex
7016 extra_depends
7017 Extra runtime dependencies (RDEPENDS) to be
7018 set for all packages. The default value of None
7019 causes a dependency on the main package
7020 (${PN}) - if you do not want this, pass empty
7021 string '' for this parameter.
7022 aux_files_pattern
7023 Extra item(s) to be added to FILES for each
7024 package. Can be a single string item or a list
7025 of strings for multiple items. Must include %s.
7026 postrm
7027 postrm script to use for all packages (as a
7028 string)
7029 allow_dirs
7030 True to allow directories to be matched -
7031 default False
7032 prepend
7033 If True, prepend created packages to PACKAGES
7034 instead of the default False which appends them
7035 match_path
7036 match file_regex on the whole relative path to
7037 the root rather than just the file name
7038 aux_files_pattern_verbatim
7039 Extra item(s) to be added to FILES for each
7040 package, using the actual derived module name
7041 rather than converting it to something legal
7042 for a package name. Can be a single string item
7043 or a list of strings for multiple items. Must
7044 include %s.
7045 allow_links
7046 True to allow symlinks to be matched - default
7047 False
7048 summary
7049 Summary to set for each package. Must include %s;
7050 defaults to description if not set.
7051
7052
7053
7054Satisfying Dependencies
7055~~~~~~~~~~~~~~~~~~~~~~~
7056
7057The second part for handling optional module packaging is to ensure that
7058any dependencies on optional modules from other recipes are satisfied by
7059your recipe. You can be sure these dependencies are satisfied by using
7060the :term:`PACKAGES_DYNAMIC`
7061variable. Here is an example that continues with the ``lighttpd`` recipe
7062shown earlier:
7063::
7064
7065 PACKAGES_DYNAMIC = "lighttpd-module-.*"
7066
7067The name
7068specified in the regular expression can of course be anything. In this
7069example, it is ``lighttpd-module-`` and is specified as the prefix to
7070ensure that any :term:`RDEPENDS` and
7071:term:`RRECOMMENDS` on a package
7072name starting with the prefix are satisfied during build time. If you
7073are using ``do_split_packages`` as described in the previous section,
7074the value you put in ``PACKAGES_DYNAMIC`` should correspond to the name
7075pattern specified in the call to ``do_split_packages``.
7076
7077Using Runtime Package Management
7078--------------------------------
7079
7080During a build, BitBake always transforms a recipe into one or more
7081packages. For example, BitBake takes the ``bash`` recipe and produces a
7082number of packages (e.g. ``bash``, ``bash-bashbug``,
7083``bash-completion``, ``bash-completion-dbg``, ``bash-completion-dev``,
7084``bash-completion-extra``, ``bash-dbg``, and so forth). Not all
7085generated packages are included in an image.
7086
7087In several situations, you might need to update, add, remove, or query
7088the packages on a target device at runtime (i.e. without having to
7089generate a new image). Examples of such situations include:
7090
7091- You want to provide in-the-field updates to deployed devices (e.g.
7092 security updates).
7093
7094- You want to have a fast turn-around development cycle for one or more
7095 applications that run on your device.
7096
7097- You want to temporarily install the "debug" packages of various
7098 applications on your device so that debugging can be greatly improved
7099 by allowing access to symbols and source debugging.
7100
7101- You want to deploy a more minimal package selection of your device
7102 but allow in-the-field updates to add a larger selection for
7103 customization.
7104
7105In all these situations, you have something similar to a more
7106traditional Linux distribution in that in-field devices are able to
7107receive pre-compiled packages from a server for installation or update.
7108Being able to install these packages on a running, in-field device is
7109what is termed "runtime package management".
7110
7111In order to use runtime package management, you need a host or server
7112machine that serves up the pre-compiled packages plus the required
7113metadata. You also need package manipulation tools on the target. The
7114build machine is a likely candidate to act as the server. However, that
7115machine does not necessarily have to be the package server. The build
7116machine could push its artifacts to another machine that acts as the
7117server (e.g. Internet-facing). In fact, doing so is advantageous for a
7118production environment as getting the packages away from the development
7119system's build directory prevents accidental overwrites.
7120
7121A simple build that targets just one device produces more than one
7122package database. In other words, the packages produced by a build are
7123separated out into a couple of different package groupings based on
7124criteria such as the target's CPU architecture, the target board, or the
7125C library used on the target. For example, a build targeting the
7126``qemux86`` device produces the following three package databases:
7127``noarch``, ``i586``, and ``qemux86``. If you wanted your ``qemux86``
7128device to be aware of all the packages that were available to it, you
7129would need to point it to each of these databases individually. In a
7130similar way, a traditional Linux distribution usually is configured to
7131be aware of a number of software repositories from which it retrieves
7132packages.
7133
7134Using runtime package management is completely optional and not required
7135for a successful build or deployment in any way. But if you want to make
7136use of runtime package management, you need to do a couple things above
7137and beyond the basics. The remainder of this section describes what you
7138need to do.
7139
7140.. _runtime-package-management-build:
7141
7142Build Considerations
7143~~~~~~~~~~~~~~~~~~~~
7144
7145This section describes build considerations of which you need to be
7146aware in order to provide support for runtime package management.
7147
7148When BitBake generates packages, it needs to know what format or formats
7149to use. In your configuration, you use the
7150:term:`PACKAGE_CLASSES`
7151variable to specify the format:
7152
71531. Open the ``local.conf`` file inside your
7154 :term:`Build Directory` (e.g.
7155 ``~/poky/build/conf/local.conf``).
7156
71572. Select the desired package format as follows:
7158 ::
7159
7160 PACKAGE_CLASSES ?= "package_packageformat"
7161
7162 where packageformat can be "ipk", "rpm",
7163 "deb", or "tar" which are the supported package formats.
7164
7165 .. note::
7166
7167 Because the Yocto Project supports four different package formats,
7168 you can set the variable with more than one argument. However, the
7169 OpenEmbedded build system only uses the first argument when
7170 creating an image or Software Development Kit (SDK).
7171
7172If you would like your image to start off with a basic package database
7173containing the packages in your current build as well as to have the
7174relevant tools available on the target for runtime package management,
7175you can include "package-management" in the
7176:term:`IMAGE_FEATURES`
7177variable. Including "package-management" in this configuration variable
7178ensures that when the image is assembled for your target, the image
7179includes the currently-known package databases as well as the
7180target-specific tools required for runtime package management to be
7181performed on the target. However, this is not strictly necessary. You
7182could start your image off without any databases but only include the
7183required on-target package tool(s). As an example, you could include
7184"opkg" in your
7185:term:`IMAGE_INSTALL` variable
7186if you are using the IPK package format. You can then initialize your
7187target's package database(s) later once your image is up and running.
7188
7189Whenever you perform any sort of build step that can potentially
7190generate a package or modify existing package, it is always a good idea
7191to re-generate the package index after the build by using the following
7192command:
7193::
7194
7195 $ bitbake package-index
7196
7197It might be tempting to build the
7198package and the package index at the same time with a command such as
7199the following:
7200::
7201
7202 $ bitbake some-package package-index
7203
7204Do not do this as
7205BitBake does not schedule the package index for after the completion of
7206the package you are building. Consequently, you cannot be sure of the
7207package index including information for the package you just built.
7208Thus, be sure to run the package update step separately after building
7209any packages.
7210
7211You can use the
7212:term:`PACKAGE_FEED_ARCHS`,
7213:term:`PACKAGE_FEED_BASE_PATHS`,
7214and
7215:term:`PACKAGE_FEED_URIS`
7216variables to pre-configure target images to use a package feed. If you
7217do not define these variables, then manual steps as described in the
7218subsequent sections are necessary to configure the target. You should
7219set these variables before building the image in order to produce a
7220correctly configured image.
7221
7222When your build is complete, your packages reside in the
7223``${TMPDIR}/deploy/packageformat`` directory. For example, if
7224``${``\ :term:`TMPDIR`\ ``}`` is
7225``tmp`` and your selected package type is RPM, then your RPM packages
7226are available in ``tmp/deploy/rpm``.
7227
7228.. _runtime-package-management-server:
7229
7230Host or Server Machine Setup
7231~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7232
7233Although other protocols are possible, a server using HTTP typically
7234serves packages. If you want to use HTTP, then set up and configure a
7235web server such as Apache 2, lighttpd, or SimpleHTTPServer on the
7236machine serving the packages.
7237
7238To keep things simple, this section describes how to set up a
7239SimpleHTTPServer web server to share package feeds from the developer's
7240machine. Although this server might not be the best for a production
7241environment, the setup is simple and straight forward. Should you want
7242to use a different server more suited for production (e.g. Apache 2,
7243Lighttpd, or Nginx), take the appropriate steps to do so.
7244
7245From within the build directory where you have built an image based on
7246your packaging choice (i.e. the
7247:term:`PACKAGE_CLASSES`
7248setting), simply start the server. The following example assumes a build
7249directory of ``~/poky/build/tmp/deploy/rpm`` and a ``PACKAGE_CLASSES``
7250setting of "package_rpm":
7251::
7252
7253 $ cd ~/poky/build/tmp/deploy/rpm
7254 $ python -m SimpleHTTPServer
7255
7256.. _runtime-package-management-target:
7257
7258Target Setup
7259~~~~~~~~~~~~
7260
7261Setting up the target differs depending on the package management
7262system. This section provides information for RPM, IPK, and DEB.
7263
7264.. _runtime-package-management-target-rpm:
7265
7266Using RPM
7267^^^^^^^^^
7268
7269The `Dandified Packaging
7270Tool <https://en.wikipedia.org/wiki/DNF_(software)>`__ (DNF) performs
7271runtime package management of RPM packages. In order to use DNF for
7272runtime package management, you must perform an initial setup on the
7273target machine for cases where the ``PACKAGE_FEED_*`` variables were not
7274set as part of the image that is running on the target. This means if
7275you built your image and did not not use these variables as part of the
7276build and your image is now running on the target, you need to perform
7277the steps in this section if you want to use runtime package management.
7278
7279.. note::
7280
7281 For information on the PACKAGE_FEED_* variables, see
7282 PACKAGE_FEED_ARCHS
7283 ,
7284 PACKAGE_FEED_BASE_PATHS
7285 , and
7286 PACKAGE_FEED_URIS
7287 in the Yocto Project Reference Manual variables glossary.
7288
7289On the target, you must inform DNF that package databases are available.
7290You do this by creating a file named
7291``/etc/yum.repos.d/oe-packages.repo`` and defining the ``oe-packages``.
7292
7293As an example, assume the target is able to use the following package
7294databases: ``all``, ``i586``, and ``qemux86`` from a server named
7295``my.server``. The specifics for setting up the web server are up to
7296you. The critical requirement is that the URIs in the target repository
7297configuration point to the correct remote location for the feeds.
7298
7299.. note::
7300
7301 For development purposes, you can point the web server to the build
7302 system's
7303 deploy
7304 directory. However, for production use, it is better to copy the
7305 package directories to a location outside of the build area and use
7306 that location. Doing so avoids situations where the build system
7307 overwrites or changes the
7308 deploy
7309 directory.
7310
7311When telling DNF where to look for the package databases, you must
7312declare individual locations per architecture or a single location used
7313for all architectures. You cannot do both:
7314
7315- *Create an Explicit List of Architectures:* Define individual base
7316 URLs to identify where each package database is located:
7317 ::
7318
7319 [oe-packages]
7320 baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all
7321
7322 This example
7323 informs DNF about individual package databases for all three
7324 architectures.
7325
7326- *Create a Single (Full) Package Index:* Define a single base URL that
7327 identifies where a full package database is located:
7328 ::
7329
7330 [oe-packages]
7331 baseurl=http://my.server/rpm
7332
7333 This example informs DNF about a single
7334 package database that contains all the package index information for
7335 all supported architectures.
7336
7337Once you have informed DNF where to find the package databases, you need
7338to fetch them:
7339::
7340
7341 # dnf makecache
7342
7343DNF is now able to find, install, and
7344upgrade packages from the specified repository or repositories.
7345
7346.. note::
7347
7348 See the
7349 DNF documentation
7350 for additional information.
7351
7352.. _runtime-package-management-target-ipk:
7353
7354Using IPK
7355^^^^^^^^^
7356
7357The ``opkg`` application performs runtime package management of IPK
7358packages. You must perform an initial setup for ``opkg`` on the target
7359machine if the
7360:term:`PACKAGE_FEED_ARCHS`,
7361:term:`PACKAGE_FEED_BASE_PATHS`,
7362and
7363:term:`PACKAGE_FEED_URIS`
7364variables have not been set or the target image was built before the
7365variables were set.
7366
7367The ``opkg`` application uses configuration files to find available
7368package databases. Thus, you need to create a configuration file inside
7369the ``/etc/opkg/`` direction, which informs ``opkg`` of any repository
7370you want to use.
7371
7372As an example, suppose you are serving packages from a ``ipk/``
7373directory containing the ``i586``, ``all``, and ``qemux86`` databases
7374through an HTTP server named ``my.server``. On the target, create a
7375configuration file (e.g. ``my_repo.conf``) inside the ``/etc/opkg/``
7376directory containing the following:
7377::
7378
7379 src/gz all http://my.server/ipk/all
7380 src/gz i586 http://my.server/ipk/i586
7381 src/gz qemux86 http://my.server/ipk/qemux86
7382
7383Next, instruct ``opkg`` to fetch the
7384repository information: # opkg update The ``opkg`` application is now
7385able to find, install, and upgrade packages from the specified
7386repository.
7387
7388.. _runtime-package-management-target-deb:
7389
7390Using DEB
7391^^^^^^^^^
7392
7393The ``apt`` application performs runtime package management of DEB
7394packages. This application uses a source list file to find available
7395package databases. You must perform an initial setup for ``apt`` on the
7396target machine if the
7397:term:`PACKAGE_FEED_ARCHS`,
7398:term:`PACKAGE_FEED_BASE_PATHS`,
7399and
7400:term:`PACKAGE_FEED_URIS`
7401variables have not been set or the target image was built before the
7402variables were set.
7403
7404To inform ``apt`` of the repository you want to use, you might create a
7405list file (e.g. ``my_repo.list``) inside the
7406``/etc/apt/sources.list.d/`` directory. As an example, suppose you are
7407serving packages from a ``deb/`` directory containing the ``i586``,
7408``all``, and ``qemux86`` databases through an HTTP server named
7409``my.server``. The list file should contain:
7410::
7411
7412 deb http://my.server/deb/all ./
7413 deb http://my.server/deb/i586 ./
7414 deb http://my.server/deb/qemux86 ./
7415
7416Next, instruct the ``apt`` application
7417to fetch the repository information:
7418::
7419
7420 # apt-get update
7421
7422After this step,
7423``apt`` is able to find, install, and upgrade packages from the
7424specified repository.
7425
7426Generating and Using Signed Packages
7427------------------------------------
7428
7429In order to add security to RPM packages used during a build, you can
7430take steps to securely sign them. Once a signature is verified, the
7431OpenEmbedded build system can use the package in the build. If security
7432fails for a signed package, the build system aborts the build.
7433
7434This section describes how to sign RPM packages during a build and how
7435to use signed package feeds (repositories) when doing a build.
7436
7437Signing RPM Packages
7438~~~~~~~~~~~~~~~~~~~~
7439
7440To enable signing RPM packages, you must set up the following
7441configurations in either your ``local.config`` or ``distro.config``
7442file:
7443::
7444
7445 # Inherit sign_rpm.bbclass to enable signing functionality
7446 INHERIT += " sign_rpm"
7447 # Define the GPG key that will be used for signing.
7448 RPM_GPG_NAME = "key_name"
7449 # Provide passphrase for the key
7450 RPM_GPG_PASSPHRASE = "passphrase"
7451
7452.. note::
7453
7454 Be sure to supply appropriate values for both
7455 key_name
7456 and
7457 passphrase
7458
7459Aside from the ``RPM_GPG_NAME`` and ``RPM_GPG_PASSPHRASE`` variables in
7460the previous example, two optional variables related to signing exist:
7461
7462- *GPG_BIN:* Specifies a ``gpg`` binary/wrapper that is executed
7463 when the package is signed.
7464
7465- *GPG_PATH:* Specifies the ``gpg`` home directory used when the
7466 package is signed.
7467
7468Processing Package Feeds
7469~~~~~~~~~~~~~~~~~~~~~~~~
7470
7471In addition to being able to sign RPM packages, you can also enable
7472signed package feeds for IPK and RPM packages.
7473
7474The steps you need to take to enable signed package feed use are similar
7475to the steps used to sign RPM packages. You must define the following in
7476your ``local.config`` or ``distro.config`` file:
7477::
7478
7479 INHERIT += "sign_package_feed"
7480 PACKAGE_FEED_GPG_NAME = "key_name"
7481 PACKAGE_FEED_GPG_PASSPHRASE_FILE = "path_to_file_containing_passphrase"
7482
7483For signed package feeds, the passphrase must exist in a separate file,
7484which is pointed to by the ``PACKAGE_FEED_GPG_PASSPHRASE_FILE``
7485variable. Regarding security, keeping a plain text passphrase out of the
7486configuration is more secure.
7487
7488Aside from the ``PACKAGE_FEED_GPG_NAME`` and
7489``PACKAGE_FEED_GPG_PASSPHRASE_FILE`` variables, three optional variables
7490related to signed package feeds exist:
7491
7492- *GPG_BIN* Specifies a ``gpg`` binary/wrapper that is executed
7493 when the package is signed.
7494
7495- *GPG_PATH:* Specifies the ``gpg`` home directory used when the
7496 package is signed.
7497
7498- *PACKAGE_FEED_GPG_SIGNATURE_TYPE:* Specifies the type of ``gpg``
7499 signature. This variable applies only to RPM and IPK package feeds.
7500 Allowable values for the ``PACKAGE_FEED_GPG_SIGNATURE_TYPE`` are
7501 "ASC", which is the default and specifies ascii armored, and "BIN",
7502 which specifies binary.
7503
7504Testing Packages With ptest
7505---------------------------
7506
7507A Package Test (ptest) runs tests against packages built by the
7508OpenEmbedded build system on the target machine. A ptest contains at
7509least two items: the actual test, and a shell script (``run-ptest``)
7510that starts the test. The shell script that starts the test must not
7511contain the actual test - the script only starts the test. On the other
7512hand, the test can be anything from a simple shell script that runs a
7513binary and checks the output to an elaborate system of test binaries and
7514data files.
7515
7516The test generates output in the format used by Automake:
7517::
7518
7519 result: testname
7520
7521where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
7522the testname can be any identifying string.
7523
7524For a list of Yocto Project recipes that are already enabled with ptest,
7525see the :yocto_wiki:`Ptest </wiki/Ptest>` wiki page.
7526
7527.. note::
7528
7529 A recipe is "ptest-enabled" if it inherits the
7530 ptest
7531 class.
7532
7533Adding ptest to Your Build
7534~~~~~~~~~~~~~~~~~~~~~~~~~~
7535
7536To add package testing to your build, add the
7537:term:`DISTRO_FEATURES` and
7538:term:`EXTRA_IMAGE_FEATURES`
7539variables to your ``local.conf`` file, which is found in the
7540:term:`Build Directory`:
7541::
7542
7543 DISTRO_FEATURES_append = " ptest"
7544 EXTRA_IMAGE_FEATURES += "ptest-pkgs"
7545
7546Once your build is complete, the ptest files are installed into the
7547``/usr/lib/package/ptest`` directory within the image, where ``package``
7548is the name of the package.
7549
7550Running ptest
7551~~~~~~~~~~~~~
7552
7553The ``ptest-runner`` package installs a shell script that loops through
7554all installed ptest test suites and runs them in sequence. Consequently,
7555you might want to add this package to your image.
7556
7557Getting Your Package Ready
7558~~~~~~~~~~~~~~~~~~~~~~~~~~
7559
7560In order to enable a recipe to run installed ptests on target hardware,
7561you need to prepare the recipes that build the packages you want to
7562test. Here is what you have to do for each recipe:
7563
7564- *Be sure the recipe inherits
7565 the*\ :ref:`ptest <ref-classes-ptest>`\ *class:*
7566 Include the following line in each recipe:
7567 ::
7568
7569 inherit ptest
7570
7571- *Create run-ptest:* This script starts your test. Locate the
7572 script where you will refer to it using
7573 :term:`SRC_URI`. Here is an
7574 example that starts a test for ``dbus``:
7575 ::
7576
7577 #!/bin/sh
7578 cd test
7579 make -k runtest-TESTS
7580
7581- *Ensure dependencies are met:* If the test adds build or runtime
7582 dependencies that normally do not exist for the package (such as
7583 requiring "make" to run the test suite), use the
7584 :term:`DEPENDS` and
7585 :term:`RDEPENDS` variables in
7586 your recipe in order for the package to meet the dependencies. Here
7587 is an example where the package has a runtime dependency on "make":
7588 ::
7589
7590 RDEPENDS_${PN}-ptest += "make"
7591
7592- *Add a function to build the test suite:* Not many packages support
7593 cross-compilation of their test suites. Consequently, you usually
7594 need to add a cross-compilation function to the package.
7595
7596 Many packages based on Automake compile and run the test suite by
7597 using a single command such as ``make check``. However, the host
7598 ``make check`` builds and runs on the same computer, while
7599 cross-compiling requires that the package is built on the host but
7600 executed for the target architecture (though often, as in the case
7601 for ptest, the execution occurs on the host). The built version of
7602 Automake that ships with the Yocto Project includes a patch that
7603 separates building and execution. Consequently, packages that use the
7604 unaltered, patched version of ``make check`` automatically
7605 cross-compiles.
7606
7607 Regardless, you still must add a ``do_compile_ptest`` function to
7608 build the test suite. Add a function similar to the following to your
7609 recipe:
7610 ::
7611
7612 do_compile_ptest() {
7613 oe_runmake buildtest-TESTS
7614 }
7615
7616- *Ensure special configurations are set:* If the package requires
7617 special configurations prior to compiling the test code, you must
7618 insert a ``do_configure_ptest`` function into the recipe.
7619
7620- *Install the test suite:* The ``ptest`` class automatically copies
7621 the file ``run-ptest`` to the target and then runs make
7622 ``install-ptest`` to run the tests. If this is not enough, you need
7623 to create a ``do_install_ptest`` function and make sure it gets
7624 called after the "make install-ptest" completes.
7625
7626Creating Node Package Manager (NPM) Packages
7627--------------------------------------------
7628
7629`NPM <https://en.wikipedia.org/wiki/Npm_(software)>`__ is a package
7630manager for the JavaScript programming language. The Yocto Project
7631supports the NPM :ref:`fetcher <bitbake:bb-fetchers>`. You can
7632use this fetcher in combination with
7633:doc:```devtool`` <../ref-manual/ref-devtool-reference>` to create
7634recipes that produce NPM packages.
7635
7636Two workflows exist that allow you to create NPM packages using
7637``devtool``: the NPM registry modules method and the NPM project code
7638method.
7639
7640.. note::
7641
7642 While it is possible to create NPM recipes manually, using
7643 devtool
7644 is far simpler.
7645
7646Additionally, some requirements and caveats exist.
7647
7648.. _npm-package-creation-requirements:
7649
7650Requirements and Caveats
7651~~~~~~~~~~~~~~~~~~~~~~~~
7652
7653You need to be aware of the following before using ``devtool`` to create
7654NPM packages:
7655
7656- Of the two methods that you can use ``devtool`` to create NPM
7657 packages, the registry approach is slightly simpler. However, you
7658 might consider the project approach because you do not have to
7659 publish your module in the NPM registry
7660 (`npm-registry <https://docs.npmjs.com/misc/registry>`_), which
7661 is NPM's public registry.
7662
7663- Be familiar with
7664 :doc:```devtool`` <../ref-manual/ref-devtool-reference>`.
7665
7666- The NPM host tools need the native ``nodejs-npm`` package, which is
7667 part of the OpenEmbedded environment. You need to get the package by
7668 cloning the https://github.com/openembedded/meta-openembedded
7669 repository out of GitHub. Be sure to add the path to your local copy
7670 to your ``bblayers.conf`` file.
7671
7672- ``devtool`` cannot detect native libraries in module dependencies.
7673 Consequently, you must manually add packages to your recipe.
7674
7675- While deploying NPM packages, ``devtool`` cannot determine which
7676 dependent packages are missing on the target (e.g. the node runtime
7677 ``nodejs``). Consequently, you need to find out what files are
7678 missing and be sure they are on the target.
7679
7680- Although you might not need NPM to run your node package, it is
7681 useful to have NPM on your target. The NPM package name is
7682 ``nodejs-npm``.
7683
7684.. _npm-using-the-registry-modules-method:
7685
7686Using the Registry Modules Method
7687~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7688
7689This section presents an example that uses the ``cute-files`` module,
7690which is a file browser web application.
7691
7692.. note::
7693
7694 You must know the
7695 cute-files
7696 module version.
7697
7698The first thing you need to do is use ``devtool`` and the NPM fetcher to
7699create the recipe:
7700::
7701
7702 $ devtool add "npm://registry.npmjs.org;package=cute-files;version=1.0.2"
7703
7704The
7705``devtool add`` command runs ``recipetool create`` and uses the same
7706fetch URI to download each dependency and capture license details where
7707possible. The result is a generated recipe.
7708
7709The recipe file is fairly simple and contains every license that
7710``recipetool`` finds and includes the licenses in the recipe's
7711:term:`LIC_FILES_CHKSUM`
7712variables. You need to examine the variables and look for those with
7713"unknown" in the :term:`LICENSE`
7714field. You need to track down the license information for "unknown"
7715modules and manually add the information to the recipe.
7716
7717``recipetool`` creates a "shrinkwrap" file for your recipe. Shrinkwrap
7718files capture the version of all dependent modules. Many packages do not
7719provide shrinkwrap files. ``recipetool`` create a shrinkwrap file as it
7720runs.
7721
7722.. note::
7723
7724 A package is created for each sub-module. This policy is the only
7725 practical way to have the licenses for all of the dependencies
7726 represented in the license manifest of the image.
7727
7728The ``devtool edit-recipe`` command lets you take a look at the recipe:
7729::
7730
7731 $ devtool edit-recipe cute-files
7732 SUMMARY = "Turn any folder on your computer into a cute file browser, available on the local network."
7733 LICENSE = "MIT & ISC & Unknown"
7734 LIC_FILES_CHKSUM = "file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \
7735 file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502 \
7736 file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99 \
7737 ...
7738 SRC_URI = " \
7739 npm://registry.npmjs.org/;package=cute-files;version=${PV} \
7740 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
7741 "
7742 S = "${WORKDIR}/npm"
7743 inherit npm LICENSE_${PN} = "MIT"
7744 LICENSE_${PN}-accepts = "MIT"
7745 LICENSE_${PN}-array-flatten = "MIT"
7746 ...
7747 LICENSE_${PN}-vary = "MIT"
7748
7749Three key points exist in the previous example:
7750
7751- :term:`SRC_URI` uses the NPM
7752 scheme so that the NPM fetcher is used.
7753
7754- ``recipetool`` collects all the license information. If a
7755 sub-module's license is unavailable, the sub-module's name appears in
7756 the comments.
7757
7758- The ``inherit npm`` statement causes the
7759 :ref:`npm <ref-classes-npm>` class to package
7760 up all the modules.
7761
7762You can run the following command to build the ``cute-files`` package:
7763::
7764
7765 $ devtool build cute-files
7766
7767Remember that ``nodejs`` must be installed on
7768the target before your package.
7769
7770Assuming 192.168.7.2 for the target's IP address, use the following
7771command to deploy your package:
7772::
7773
7774 $ devtool deploy-target -s cute-files root@192.168.7.2
7775
7776Once the package is installed on the target, you can
7777test the application:
7778
7779.. note::
7780
7781 Because of a know issue, you cannot simply run
7782 cute-files
7783 as you would if you had run
7784 npm install
7785 .
7786
7787::
7788
7789 $ cd /usr/lib/node_modules/cute-files
7790 $ node cute-files.js
7791
7792On a browser,
7793go to ``http://192.168.7.2:3000`` and you see the following:
7794
7795.. image:: figures/cute-files-npm-example.png
7796 :align: center
7797
7798You can find the recipe in ``workspace/recipes/cute-files``. You can use
7799the recipe in any layer you choose.
7800
7801.. _npm-using-the-npm-projects-method:
7802
7803Using the NPM Projects Code Method
7804~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7805
7806Although it is useful to package modules already in the NPM registry,
7807adding ``node.js`` projects under development is a more common developer
7808use case.
7809
7810This section covers the NPM projects code method, which is very similar
7811to the "registry" approach described in the previous section. In the NPM
7812projects method, you provide ``devtool`` with an URL that points to the
7813source files.
7814
7815Replicating the same example, (i.e. ``cute-files``) use the following
7816command:
7817::
7818
7819 $ devtool add https://github.com/martinaglv/cute-files.git
7820
7821The
7822recipe this command generates is very similar to the recipe created in
7823the previous section. However, the ``SRC_URI`` looks like the following:
7824::
7825
7826 SRC_URI = " \
7827 git://github.com/martinaglv/cute-files.git;protocol=https \
7828 npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \
7829 "
7830
7831In this example,
7832the main module is taken from the Git repository and dependents are
7833taken from the NPM registry. Other than those differences, the recipe is
7834basically the same between the two methods. You can build and deploy the
7835package exactly as described in the previous section that uses the
7836registry modules method.
7837
7838Adding custom metadata to packages
7839----------------------------------
7840
7841The variable
7842:term:`PACKAGE_ADD_METADATA`
7843can be used to add additional metadata to packages. This is reflected in
7844the package control/spec file. To take the ipk format for example, the
7845CONTROL file stored inside would contain the additional metadata as
7846additional lines.
7847
7848The variable can be used in multiple ways, including using suffixes to
7849set it for a specific package type and/or package. Note that the order
7850of precedence is the same as this list:
7851
7852- ``PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>``
7853
7854- ``PACKAGE_ADD_METADATA_<PKGTYPE>``
7855
7856- ``PACKAGE_ADD_METADATA_<PN>``
7857
7858- ``PACKAGE_ADD_METADATA``
7859
7860<PKGTYPE> is a parameter and expected to be a distinct name of specific
7861package type:
7862
7863- IPK for .ipk packages
7864
7865- DEB for .deb packages
7866
7867- RPM for .rpm packages
7868
7869<PN> is a parameter and expected to be a package name.
7870
7871The variable can contain multiple [one-line] metadata fields separated
7872by the literal sequence '\n'. The separator can be redefined using the
7873variable flag ``separator``.
7874
7875The following is an example that adds two custom fields for ipk
7876packages: PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup:
7877Applications/Spreadsheets"
7878
7879Efficiently Fetching Source Files During a Build
7880================================================
7881
7882The OpenEmbedded build system works with source files located through
7883the :term:`SRC_URI` variable. When
7884you build something using BitBake, a big part of the operation is
7885locating and downloading all the source tarballs. For images,
7886downloading all the source for various packages can take a significant
7887amount of time.
7888
7889This section shows you how you can use mirrors to speed up fetching
7890source files and how you can pre-fetch files all of which leads to more
7891efficient use of resources and time.
7892
7893Setting up Effective Mirrors
7894----------------------------
7895
7896A good deal that goes into a Yocto Project build is simply downloading
7897all of the source tarballs. Maybe you have been working with another
7898build system (OpenEmbedded or Angstrom) for which you have built up a
7899sizable directory of source tarballs. Or, perhaps someone else has such
7900a directory for which you have read access. If so, you can save time by
7901adding statements to your configuration file so that the build process
7902checks local directories first for existing tarballs before checking the
7903Internet.
7904
7905Here is an efficient way to set it up in your ``local.conf`` file:
7906::
7907
7908 SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/"
7909 INHERIT += "own-mirrors"
7910 BB_GENERATE_MIRROR_TARBALLS = "1"
7911 # BB_NO_NETWORK = "1"
7912
7913In the previous example, the
7914:term:`BB_GENERATE_MIRROR_TARBALLS`
7915variable causes the OpenEmbedded build system to generate tarballs of
7916the Git repositories and store them in the
7917:term:`DL_DIR` directory. Due to
7918performance reasons, generating and storing these tarballs is not the
7919build system's default behavior.
7920
7921You can also use the
7922:term:`PREMIRRORS` variable. For
7923an example, see the variable's glossary entry in the Yocto Project
7924Reference Manual.
7925
7926Getting Source Files and Suppressing the Build
7927----------------------------------------------
7928
7929Another technique you can use to ready yourself for a successive string
7930of build operations, is to pre-fetch all the source files without
7931actually starting a build. This technique lets you work through any
7932download issues and ultimately gathers all the source files into your
7933download directory :ref:`structure-build-downloads`,
7934which is located with :term:`DL_DIR`.
7935
7936Use the following BitBake command form to fetch all the necessary
7937sources without starting the build:
7938::
7939
7940 $ bitbake target --runall=fetch
7941
7942This
7943variation of the BitBake command guarantees that you have all the
7944sources for that BitBake target should you disconnect from the Internet
7945and want to do the build later offline.
7946
7947Selecting an Initialization Manager
7948===================================
7949
7950By default, the Yocto Project uses SysVinit as the initialization
7951manager. However, support also exists for systemd, which is a full
7952replacement for init with parallel starting of services, reduced shell
7953overhead and other features that are used by many distributions.
7954
7955Within the system, SysVinit treats system components as services. These
7956services are maintained as shell scripts stored in the ``/etc/init.d/``
7957directory. Services organize into different run levels. This
7958organization is maintained by putting links to the services in the
7959``/etc/rcN.d/`` directories, where N/ is one of the following options:
7960"S", "0", "1", "2", "3", "4", "5", or "6".
7961
7962.. note::
7963
7964 Each runlevel has a dependency on the previous runlevel. This
7965 dependency allows the services to work properly.
7966
7967In comparison, systemd treats components as units. Using units is a
7968broader concept as compared to using a service. A unit includes several
7969different types of entities. Service is one of the types of entities.
7970The runlevel concept in SysVinit corresponds to the concept of a target
7971in systemd, where target is also a type of supported unit.
7972
7973In a SysVinit-based system, services load sequentially (i.e. one by one)
7974during and parallelization is not supported. With systemd, services
7975start in parallel. Needless to say, the method can have an impact on
7976system startup performance.
7977
7978If you want to use SysVinit, you do not have to do anything. But, if you
7979want to use systemd, you must take some steps as described in the
7980following sections.
7981
7982Using systemd Exclusively
7983-------------------------
7984
7985Set these variables in your distribution configuration file as follows:
7986::
7987
7988 DISTRO_FEATURES_append = " systemd"
7989 VIRTUAL-RUNTIME_init_manager = "systemd"
7990
7991You can also prevent the SysVinit distribution feature from
7992being automatically enabled as follows:
7993::
7994
7995 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
7996
7997Doing so removes any
7998redundant SysVinit scripts.
7999
8000To remove initscripts from your image altogether, set this variable
8001also:
8002::
8003
8004 VIRTUAL-RUNTIME_initscripts = ""
8005
8006For information on the backfill variable, see
8007:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`.
8008
8009Using systemd for the Main Image and Using SysVinit for the Rescue Image
8010------------------------------------------------------------------------
8011
8012Set these variables in your distribution configuration file as follows:
8013::
8014
8015 DISTRO_FEATURES_append = " systemd"
8016 VIRTUAL-RUNTIME_init_manager = "systemd"
8017
8018Doing so causes your main image to use the
8019``packagegroup-core-boot.bb`` recipe and systemd. The rescue/minimal
8020image cannot use this package group. However, it can install SysVinit
8021and the appropriate packages will have support for both systemd and
8022SysVinit.
8023
8024.. _selecting-dev-manager:
8025
8026Selecting a Device Manager
8027==========================
8028
8029The Yocto Project provides multiple ways to manage the device manager
8030(``/dev``):
8031
8032- Persistent and Pre-Populated\ ``/dev``: For this case, the ``/dev``
8033 directory is persistent and the required device nodes are created
8034 during the build.
8035
8036- Use ``devtmpfs`` with a Device Manager: For this case, the ``/dev``
8037 directory is provided by the kernel as an in-memory file system and
8038 is automatically populated by the kernel at runtime. Additional
8039 configuration of device nodes is done in user space by a device
8040 manager like ``udev`` or ``busybox-mdev``.
8041
8042.. _static-dev-management:
8043
8044Using Persistent and Pre-Populated\ ``/dev``
8045--------------------------------------------
8046
8047To use the static method for device population, you need to set the
8048:term:`USE_DEVFS` variable to "0"
8049as follows:
8050::
8051
8052 USE_DEVFS = "0"
8053
8054The content of the resulting ``/dev`` directory is defined in a Device
8055Table file. The
8056:term:`IMAGE_DEVICE_TABLES`
8057variable defines the Device Table to use and should be set in the
8058machine or distro configuration file. Alternatively, you can set this
8059variable in your ``local.conf`` configuration file.
8060
8061If you do not define the ``IMAGE_DEVICE_TABLES`` variable, the default
8062``device_table-minimal.txt`` is used:
8063::
8064
8065 IMAGE_DEVICE_TABLES = "device_table-mymachine.txt"
8066
8067The population is handled by the ``makedevs`` utility during image
8068creation:
8069
8070.. _devtmpfs-dev-management:
8071
8072Using ``devtmpfs`` and a Device Manager
8073---------------------------------------
8074
8075To use the dynamic method for device population, you need to use (or be
8076sure to set) the :term:`USE_DEVFS`
8077variable to "1", which is the default:
8078::
8079
8080 USE_DEVFS = "1"
8081
8082With this
8083setting, the resulting ``/dev`` directory is populated by the kernel
8084using ``devtmpfs``. Make sure the corresponding kernel configuration
8085variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux
8086kernel.
8087
8088All devices created by ``devtmpfs`` will be owned by ``root`` and have
8089permissions ``0600``.
8090
8091To have more control over the device nodes, you can use a device manager
8092like ``udev`` or ``busybox-mdev``. You choose the device manager by
8093defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or
8094distro configuration file. Alternatively, you can set this variable in
8095your ``local.conf`` configuration file:
8096::
8097
8098 VIRTUAL-RUNTIME_dev_manager = "udev"
8099
8100 # Some alternative values
8101 # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev"
8102 # VIRTUAL-RUNTIME_dev_manager = "systemd"
8103
8104.. _platdev-appdev-srcrev:
8105
8106Using an External SCM
8107=====================
8108
8109If you're working on a recipe that pulls from an external Source Code
8110Manager (SCM), it is possible to have the OpenEmbedded build system
8111notice new recipe changes added to the SCM and then build the resulting
8112packages that depend on the new recipes by using the latest versions.
8113This only works for SCMs from which it is possible to get a sensible
8114revision number for changes. Currently, you can do this with Apache
8115Subversion (SVN), Git, and Bazaar (BZR) repositories.
8116
8117To enable this behavior, the :term:`PV` of
8118the recipe needs to reference
8119:term:`SRCPV`. Here is an example:
8120::
8121
8122 PV = "1.2.3+git${SRCPV}"
8123
8124Then, you can add the following to your
8125``local.conf``:
8126::
8127
8128 SRCREV_pn-PN = "${AUTOREV}"
8129
8130:term:`PN` is the name of the recipe for
8131which you want to enable automatic source revision updating.
8132
8133If you do not want to update your local configuration file, you can add
8134the following directly to the recipe to finish enabling the feature:
8135::
8136
8137 SRCREV = "${AUTOREV}"
8138
8139The Yocto Project provides a distribution named ``poky-bleeding``, whose
8140configuration file contains the line:
8141::
8142
8143 require conf/distro/include/poky-floating-revisions.inc
8144
8145This line pulls in the
8146listed include file that contains numerous lines of exactly that form:
8147::
8148
8149 #SRCREV_pn-opkg-native ?= "${AUTOREV}"
8150 #SRCREV_pn-opkg-sdk ?= "${AUTOREV}"
8151 #SRCREV_pn-opkg ?= "${AUTOREV}"
8152 #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}"
8153 #SRCREV_pn-opkg-utils ?= "${AUTOREV}"
8154 SRCREV_pn-gconf-dbus ?= "${AUTOREV}"
8155 SRCREV_pn-matchbox-common ?= "${AUTOREV}"
8156 SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}"
8157 SRCREV_pn-matchbox-desktop ?= "${AUTOREV}"
8158 SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}"
8159 SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}"
8160 SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}"
8161 SRCREV_pn-matchbox-terminal ?= "${AUTOREV}"
8162 SRCREV_pn-matchbox-wm ?= "${AUTOREV}"
8163 SRCREV_pn-settings-daemon ?= "${AUTOREV}"
8164 SRCREV_pn-screenshot ?= "${AUTOREV}"
8165 . . .
8166
8167These lines allow you to
8168experiment with building a distribution that tracks the latest
8169development source for numerous packages.
8170
8171.. note::
8172
8173 The
8174 poky-bleeding
8175 distribution is not tested on a regular basis. Keep this in mind if
8176 you use it.
8177
8178Creating a Read-Only Root Filesystem
8179====================================
8180
8181Suppose, for security reasons, you need to disable your target device's
8182root filesystem's write permissions (i.e. you need a read-only root
8183filesystem). Or, perhaps you are running the device's operating system
8184from a read-only storage device. For either case, you can customize your
8185image for that behavior.
8186
8187.. note::
8188
8189 Supporting a read-only root filesystem requires that the system and
8190 applications do not try to write to the root filesystem. You must
8191 configure all parts of the target system to write elsewhere, or to
8192 gracefully fail in the event of attempting to write to the root
8193 filesystem.
8194
8195Creating the Root Filesystem
8196----------------------------
8197
8198To create the read-only root filesystem, simply add the
8199"read-only-rootfs" feature to your image, normally in one of two ways.
8200The first way is to add the "read-only-rootfs" image feature in the
8201image's recipe file via the ``IMAGE_FEATURES`` variable:
8202::
8203
8204 IMAGE_FEATURES += "read-only-rootfs"
8205
8206As an alternative, you can add the same feature
8207from within your build directory's ``local.conf`` file with the
8208associated ``EXTRA_IMAGE_FEATURES`` variable, as in:
8209::
8210
8211 EXTRA_IMAGE_FEATURES = "read-only-rootfs"
8212
8213For more information on how to use these variables, see the
8214":ref:`usingpoky-extend-customimage-imagefeatures`"
8215section. For information on the variables, see
8216:term:`IMAGE_FEATURES` and
8217:term:`EXTRA_IMAGE_FEATURES`.
8218
8219Post-Installation Scripts and Read-Only Root Filesystem
8220-------------------------------------------------------
8221
8222It is very important that you make sure all post-Installation
8223(``pkg_postinst``) scripts for packages that are installed into the
8224image can be run at the time when the root filesystem is created during
8225the build on the host system. These scripts cannot attempt to run during
8226first-boot on the target device. With the "read-only-rootfs" feature
8227enabled, the build system checks during root filesystem creation to make
8228sure all post-installation scripts succeed. If any of these scripts
8229still need to be run after the root filesystem is created, the build
8230immediately fails. These build-time checks ensure that the build fails
8231rather than the target device fails later during its initial boot
8232operation.
8233
8234Most of the common post-installation scripts generated by the build
8235system for the out-of-the-box Yocto Project are engineered so that they
8236can run during root filesystem creation (e.g. post-installation scripts
8237for caching fonts). However, if you create and add custom scripts, you
8238need to be sure they can be run during this file system creation.
8239
8240Here are some common problems that prevent post-installation scripts
8241from running during root filesystem creation:
8242
8243- *Not using $D in front of absolute paths:* The build system defines
8244 ``$``\ :term:`D` when the root
8245 filesystem is created. Furthermore, ``$D`` is blank when the script
8246 is run on the target device. This implies two purposes for ``$D``:
8247 ensuring paths are valid in both the host and target environments,
8248 and checking to determine which environment is being used as a method
8249 for taking appropriate actions.
8250
8251- *Attempting to run processes that are specific to or dependent on the
8252 target architecture:* You can work around these attempts by using
8253 native tools, which run on the host system, to accomplish the same
8254 tasks, or by alternatively running the processes under QEMU, which
8255 has the ``qemu_run_binary`` function. For more information, see the
8256 :ref:`qemu <ref-classes-qemu>` class.
8257
8258Areas With Write Access
8259-----------------------
8260
8261With the "read-only-rootfs" feature enabled, any attempt by the target
8262to write to the root filesystem at runtime fails. Consequently, you must
8263make sure that you configure processes and applications that attempt
8264these types of writes do so to directories with write access (e.g.
8265``/tmp`` or ``/var/run``).
8266
8267Maintaining Build Output Quality
8268================================
8269
8270Many factors can influence the quality of a build. For example, if you
8271upgrade a recipe to use a new version of an upstream software package or
8272you experiment with some new configuration options, subtle changes can
8273occur that you might not detect until later. Consider the case where
8274your recipe is using a newer version of an upstream package. In this
8275case, a new version of a piece of software might introduce an optional
8276dependency on another library, which is auto-detected. If that library
8277has already been built when the software is building, the software will
8278link to the built library and that library will be pulled into your
8279image along with the new software even if you did not want the library.
8280
8281The :ref:`buildhistory <ref-classes-buildhistory>`
8282class exists to help you maintain the quality of your build output. You
8283can use the class to highlight unexpected and possibly unwanted changes
8284in the build output. When you enable build history, it records
8285information about the contents of each package and image and then
8286commits that information to a local Git repository where you can examine
8287the information.
8288
8289The remainder of this section describes the following:
8290
8291- How you can enable and disable build history
8292
8293- How to understand what the build history contains
8294
8295- How to limit the information used for build history
8296
8297- How to examine the build history from both a command-line and web
8298 interface
8299
8300Enabling and Disabling Build History
8301------------------------------------
8302
8303Build history is disabled by default. To enable it, add the following
8304``INHERIT`` statement and set the
8305:term:`BUILDHISTORY_COMMIT`
8306variable to "1" at the end of your ``conf/local.conf`` file found in the
8307:term:`Build Directory`:
8308::
8309
8310 INHERIT += "buildhistory"
8311 BUILDHISTORY_COMMIT = "1"
8312
8313Enabling build history as
8314previously described causes the OpenEmbedded build system to collect
8315build output information and commit it as a single commit to a local
8316:ref:`overview-manual/overview-manual-development-environment:git` repository.
8317
8318.. note::
8319
8320 Enabling build history increases your build times slightly,
8321 particularly for images, and increases the amount of disk space used
8322 during the build.
8323
8324You can disable build history by removing the previous statements from
8325your ``conf/local.conf`` file.
8326
8327Understanding What the Build History Contains
8328---------------------------------------------
8329
8330Build history information is kept in
8331``${``\ :term:`TOPDIR`\ ``}/buildhistory``
8332in the Build Directory as defined by the
8333:term:`BUILDHISTORY_DIR`
8334variable. The following is an example abbreviated listing:
8335
8336.. image:: figures/buildhistory.png
8337 :align: center
8338
8339At the top level, a ``metadata-revs`` file exists that lists the
8340revisions of the repositories for the enabled layers when the build was
8341produced. The rest of the data splits into separate ``packages``,
8342``images`` and ``sdk`` directories, the contents of which are described
8343as follows.
8344
8345Build History Package Information
8346~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8347
8348The history for each package contains a text file that has name-value
8349pairs with information about the package. For example,
8350``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
8351contains the following:
8352::
8353
8354 PV = 1.22.1
8355 PR = r32
8356 RPROVIDES =
8357 RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
8358 RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
8359 PKGSIZE = 540168
8360 FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
8361 /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
8362 /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
8363 /usr/share/pixmaps /usr/share/applications /usr/share/idl \
8364 /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
8365 FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
8366 /etc/busybox.links.nosuid /etc/busybox.links.suid
8367
8368Most of these
8369name-value pairs correspond to variables used to produce the package.
8370The exceptions are ``FILELIST``, which is the actual list of files in
8371the package, and ``PKGSIZE``, which is the total size of files in the
8372package in bytes.
8373
8374A file also exists that corresponds to the recipe from which the package
8375came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``):
8376::
8377
8378 PV = 1.22.1
8379 PR = r32
8380 DEPENDS = initscripts kern-tools-native update-rc.d-native \
8381 virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
8382 virtual/libc virtual/update-alternatives
8383 PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
8384 busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
8385 busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
8386
8387Finally, for those recipes fetched from a version control system (e.g.,
8388Git), a file exists that lists source revisions that are specified in
8389the recipe and lists the actual revisions used during the build. Listed
8390and actual revisions might differ when
8391:term:`SRCREV` is set to
8392${:term:`AUTOREV`}. Here is an
8393example assuming
8394``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``):
8395::
8396
8397 # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
8398 SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
8399 # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
8400 SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f"
8401
8402You can use the
8403``buildhistory-collect-srcrevs`` command with the ``-a`` option to
8404collect the stored ``SRCREV`` values from build history and report them
8405in a format suitable for use in global configuration (e.g.,
8406``local.conf`` or a distro include file) to override floating
8407``AUTOREV`` values to a fixed set of revisions. Here is some example
8408output from this command:
8409::
8410
8411 $ buildhistory-collect-srcrevs -a
8412 # i586-poky-linux
8413 SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072"
8414 SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072"
8415 SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a"
8416 SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
8417 # x86_64-linux
8418 SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa"
8419 SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf"
8420 SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11"
8421 SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072"
8422 SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3"
8423 SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca"
8424 SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a"
8425 SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff"
8426 SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
8427 # qemux86-poky-linux
8428 SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
8429 SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f"
8430 # all-poky-linux
8431 SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
8432
8433.. note::
8434
8435 Here are some notes on using the
8436 buildhistory-collect-srcrevs
8437 command:
8438
8439 - By default, only values where the ``SRCREV`` was not hardcoded
8440 (usually when ``AUTOREV`` is used) are reported. Use the ``-a``
8441 option to see all ``SRCREV`` values.
8442
8443 - The output statements might not have any effect if overrides are
8444 applied elsewhere in the build system configuration. Use the
8445 ``-f`` option to add the ``forcevariable`` override to each output
8446 line if you need to work around this restriction.
8447
8448 - The script does apply special handling when building for multiple
8449 machines. However, the script does place a comment before each set
8450 of values that specifies which triplet to which they belong as
8451 previously shown (e.g., ``i586-poky-linux``).
8452
8453Build History Image Information
8454~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8455
8456The files produced for each image are as follows:
8457
8458- ``image-files:`` A directory containing selected files from the root
8459 filesystem. The files are defined by
8460 :term:`BUILDHISTORY_IMAGE_FILES`.
8461
8462- ``build-id.txt:`` Human-readable information about the build
8463 configuration and metadata source revisions. This file contains the
8464 full build header as printed by BitBake.
8465
8466- ``*.dot:`` Dependency graphs for the image that are compatible with
8467 ``graphviz``.
8468
8469- ``files-in-image.txt:`` A list of files in the image with
8470 permissions, owner, group, size, and symlink information.
8471
8472- ``image-info.txt:`` A text file containing name-value pairs with
8473 information about the image. See the following listing example for
8474 more information.
8475
8476- ``installed-package-names.txt:`` A list of installed packages by name
8477 only.
8478
8479- ``installed-package-sizes.txt:`` A list of installed packages ordered
8480 by size.
8481
8482- ``installed-packages.txt:`` A list of installed packages with full
8483 package filenames.
8484
8485.. note::
8486
8487 Installed package information is able to be gathered and produced
8488 even if package management is disabled for the final image.
8489
8490Here is an example of ``image-info.txt``:
8491::
8492
8493 DISTRO = poky
8494 DISTRO_VERSION = 1.7
8495 USER_CLASSES = buildstats image-mklibs image-prelink
8496 IMAGE_CLASSES = image_types
8497 IMAGE_FEATURES = debug-tweaks
8498 IMAGE_LINGUAS =
8499 IMAGE_INSTALL = packagegroup-core-boot run-postinsts
8500 BAD_RECOMMENDATIONS =
8501 NO_RECOMMENDATIONS =
8502 PACKAGE_EXCLUDE =
8503 ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \
8504 write_image_manifest ; buildhistory_list_installed_image ; \
8505 buildhistory_get_image_installed ; ssh_allow_empty_password; \
8506 postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ;
8507 IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
8508 IMAGESIZE = 6900
8509
8510Other than ``IMAGESIZE``,
8511which is the total size of the files in the image in Kbytes, the
8512name-value pairs are variables that may have influenced the content of
8513the image. This information is often useful when you are trying to
8514determine why a change in the package or file listings has occurred.
8515
8516Using Build History to Gather Image Information Only
8517~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8518
8519As you can see, build history produces image information, including
8520dependency graphs, so you can see why something was pulled into the
8521image. If you are just interested in this information and not interested
8522in collecting specific package or SDK information, you can enable
8523writing only image information without any history by adding the
8524following to your ``conf/local.conf`` file found in the
8525:term:`Build Directory`:
8526::
8527
8528 INHERIT += "buildhistory"
8529 BUILDHISTORY_COMMIT = "0"
8530 BUILDHISTORY_FEATURES = "image"
8531
8532Here, you set the
8533:term:`BUILDHISTORY_FEATURES`
8534variable to use the image feature only.
8535
8536Build History SDK Information
8537~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8538
8539Build history collects similar information on the contents of SDKs (e.g.
8540``bitbake -c populate_sdk imagename``) as compared to information it
8541collects for images. Furthermore, this information differs depending on
8542whether an extensible or standard SDK is being produced.
8543
8544The following list shows the files produced for SDKs:
8545
8546- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
8547 owner, group, size, and symlink information. This list includes both
8548 the host and target parts of the SDK.
8549
8550- ``sdk-info.txt:`` A text file containing name-value pairs with
8551 information about the SDK. See the following listing example for more
8552 information.
8553
8554- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
8555 with information about task group sizes (e.g. ``do_populate_sysroot``
8556 tasks have a total size). The ``sstate-task-sizes.txt`` file exists
8557 only when an extensible SDK is created.
8558
8559- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
8560 with information for the shared-state packages and sizes in the SDK.
8561 The ``sstate-package-sizes.txt`` file exists only when an extensible
8562 SDK is created.
8563
8564- ``sdk-files:`` A folder that contains copies of the files mentioned
8565 in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
8566 Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
8567 specific to the extensible SDK although you can set it differently if
8568 you would like to pull in specific files from the standard SDK.
8569
8570 The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
8571 ``conf/auto.conf``, ``conf/locked-sigs.inc``, and
8572 ``conf/devtool.conf``. Thus, for an extensible SDK, these files get
8573 copied into the ``sdk-files`` directory.
8574
8575- The following information appears under each of the ``host`` and
8576 ``target`` directories for the portions of the SDK that run on the
8577 host and on the target, respectively:
8578
8579 .. note::
8580
8581 The following files for the most part are empty when producing an
8582 extensible SDK because this type of SDK is not constructed from
8583 packages as is the standard SDK.
8584
8585 - ``depends.dot:`` Dependency graph for the SDK that is compatible
8586 with ``graphviz``.
8587
8588 - ``installed-package-names.txt:`` A list of installed packages by
8589 name only.
8590
8591 - ``installed-package-sizes.txt:`` A list of installed packages
8592 ordered by size.
8593
8594 - ``installed-packages.txt:`` A list of installed packages with full
8595 package filenames.
8596
8597Here is an example of ``sdk-info.txt``:
8598::
8599
8600 DISTRO = poky
8601 DISTRO_VERSION = 1.3+snapshot-20130327
8602 SDK_NAME = poky-glibc-i686-arm
8603 SDK_VERSION = 1.3+snapshot
8604 SDKMACHINE =
8605 SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
8606 BAD_RECOMMENDATIONS =
8607 SDKSIZE = 352712
8608
8609Other than ``SDKSIZE``, which is
8610the total size of the files in the SDK in Kbytes, the name-value pairs
8611are variables that might have influenced the content of the SDK. This
8612information is often useful when you are trying to determine why a
8613change in the package or file listings has occurred.
8614
8615Examining Build History Information
8616~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8617
8618You can examine build history output from the command line or from a web
8619interface.
8620
8621To see any changes that have occurred (assuming you have
8622:term:`BUILDHISTORY_COMMIT` = "1"),
8623you can simply use any Git command that allows you to view the history
8624of a repository. Here is one method:
8625::
8626
8627 $ git log -p
8628
8629You need to realize,
8630however, that this method does show changes that are not significant
8631(e.g. a package's size changing by a few bytes).
8632
8633A command-line tool called ``buildhistory-diff`` does exist, though,
8634that queries the Git repository and prints just the differences that
8635might be significant in human-readable form. Here is an example:
8636::
8637
8638 $ ~/poky/poky/scripts/buildhistory-diff . HEAD^
8639 Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
8640 /etc/anotherpkg.conf was added
8641 /sbin/anotherpkg was added
8642 * (installed-package-names.txt):
8643 * anotherpkg was added
8644 Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
8645 anotherpkg was added
8646 packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
8647 * PR changed from "r0" to "r1"
8648 * PV changed from "0.1.10" to "0.1.12"
8649 packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
8650 * PR changed from "r0" to "r1"
8651 * PV changed from "0.1.10" to "0.1.12"
8652
8653.. note::
8654
8655 The
8656 buildhistory-diff
8657 tool requires the
8658 GitPython
8659 package. Be sure to install it using Pip3 as follows:
8660 ::
8661
8662 $ pip3 install GitPython --user
8663
8664
8665 Alternatively, you can install
8666 python3-git
8667 using the appropriate distribution package manager (e.g.
8668 apt-get
8669 ,
8670 dnf
8671 , or
8672 zipper
8673 ).
8674
8675To see changes to the build history using a web interface, follow the
8676instruction in the ``README`` file here.
8677http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/.
8678
8679Here is a sample screenshot of the interface:
8680
8681.. image:: figures/buildhistory-web.png
8682 :align: center
8683
8684Performing Automated Runtime Testing
8685====================================
8686
8687The OpenEmbedded build system makes available a series of automated
8688tests for images to verify runtime functionality. You can run these
8689tests on either QEMU or actual target hardware. Tests are written in
8690Python making use of the ``unittest`` module, and the majority of them
8691run commands on the target system over SSH. This section describes how
8692you set up the environment to use these tests, run available tests, and
8693write and add your own tests.
8694
8695For information on the test and QA infrastructure available within the
8696Yocto Project, see the ":ref:`ref-manual/ref-release-process:testing and quality assurance`"
8697section in the Yocto Project Reference Manual.
8698
8699Enabling Tests
8700--------------
8701
8702Depending on whether you are planning to run tests using QEMU or on the
8703hardware, you have to take different steps to enable the tests. See the
8704following subsections for information on how to enable both types of
8705tests.
8706
8707.. _qemu-image-enabling-tests:
8708
8709Enabling Runtime Tests on QEMU
8710~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8711
8712In order to run tests, you need to do the following:
8713
8714- *Set up to avoid interaction with sudo for networking:* To
8715 accomplish this, you must do one of the following:
8716
8717 - Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all
8718 commands or just for ``runqemu-ifup``. You must provide the full
8719 path as that can change if you are using multiple clones of the
8720 source repository.
8721
8722 .. note::
8723
8724 On some distributions, you also need to comment out "Defaults
8725 requiretty" in
8726 /etc/sudoers
8727 .
8728
8729 - Manually configure a tap interface for your system.
8730
8731 - Run as root the script in ``scripts/runqemu-gen-tapdevs``, which
8732 should generate a list of tap devices. This is the option
8733 typically chosen for Autobuilder-type environments.
8734
8735 .. note::
8736
8737 - Be sure to use an absolute path when calling this script
8738 with sudo.
8739
8740 - The package recipe ``qemu-helper-native`` is required to run
8741 this script. Build the package using the following command:
8742 $ bitbake qemu-helper-native
8743
8744- *Set the DISPLAY variable:* You need to set this variable so that
8745 you have an X server available (e.g. start ``vncserver`` for a
8746 headless machine).
8747
8748- *Be sure your host's firewall accepts incoming connections from
8749 192.168.7.0/24:* Some of the tests (in particular DNF tests) start an
8750 HTTP server on a random high number port, which is used to serve
8751 files to the target. The DNF module serves
8752 ``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands.
8753 That means your host's firewall must accept incoming connections from
8754 192.168.7.0/24, which is the default IP range used for tap devices by
8755 ``runqemu``.
8756
8757- *Be sure your host has the correct packages installed:* Depending
8758 your host's distribution, you need to have the following packages
8759 installed:
8760
8761 - Ubuntu and Debian: ``sysstat`` and ``iproute2``
8762
8763 - OpenSUSE: ``sysstat`` and ``iproute2``
8764
8765 - Fedora: ``sysstat`` and ``iproute``
8766
8767 - CentOS: ``sysstat`` and ``iproute``
8768
8769Once you start running the tests, the following happens:
8770
87711. A copy of the root filesystem is written to ``${WORKDIR}/testimage``.
8772
87732. The image is booted under QEMU using the standard ``runqemu`` script.
8774
87753. A default timeout of 500 seconds occurs to allow for the boot process
8776 to reach the login prompt. You can change the timeout period by
8777 setting
8778 :term:`TEST_QEMUBOOT_TIMEOUT`
8779 in the ``local.conf`` file.
8780
87814. Once the boot process is reached and the login prompt appears, the
8782 tests run. The full boot log is written to
8783 ``${WORKDIR}/testimage/qemu_boot_log``.
8784
87855. Each test module loads in the order found in ``TEST_SUITES``. You can
8786 find the full output of the commands run over SSH in
8787 ``${WORKDIR}/testimgage/ssh_target_log``.
8788
87896. If no failures occur, the task running the tests ends successfully.
8790 You can find the output from the ``unittest`` in the task log at
8791 ``${WORKDIR}/temp/log.do_testimage``.
8792
8793.. _hardware-image-enabling-tests:
8794
8795Enabling Runtime Tests on Hardware
8796~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8797
8798The OpenEmbedded build system can run tests on real hardware, and for
8799certain devices it can also deploy the image to be tested onto the
8800device beforehand.
8801
8802For automated deployment, a "master image" is installed onto the
8803hardware once as part of setup. Then, each time tests are to be run, the
8804following occurs:
8805
88061. The master image is booted into and used to write the image to be
8807 tested to a second partition.
8808
88092. The device is then rebooted using an external script that you need to
8810 provide.
8811
88123. The device boots into the image to be tested.
8813
8814When running tests (independent of whether the image has been deployed
8815automatically or not), the device is expected to be connected to a
8816network on a pre-determined IP address. You can either use static IP
8817addresses written into the image, or set the image to use DHCP and have
8818your DHCP server on the test network assign a known IP address based on
8819the MAC address of the device.
8820
8821In order to run tests on hardware, you need to set ``TEST_TARGET`` to an
8822appropriate value. For QEMU, you do not have to change anything, the
8823default value is "qemu". For running tests on hardware, the following
8824options exist:
8825
8826- *"simpleremote":* Choose "simpleremote" if you are going to run tests
8827 on a target system that is already running the image to be tested and
8828 is available on the network. You can use "simpleremote" in
8829 conjunction with either real hardware or an image running within a
8830 separately started QEMU or any other virtual machine manager.
8831
8832- *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is
8833 an EFI-based machine with ``systemd-boot`` as bootloader and
8834 ``core-image-testmaster`` (or something similar) is installed. Also,
8835 your hardware under test must be in a DHCP-enabled network that gives
8836 it the same IP address for each reboot.
8837
8838 If you choose "SystemdbootTarget", there are additional requirements
8839 and considerations. See the "`Selecting
8840 SystemdbootTarget <#selecting-systemdboottarget>`__" section, which
8841 follows, for more information.
8842
8843- *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying
8844 images and running tests on the BeagleBone "Black" or original
8845 "White" hardware. For information on how to use these tests, see the
8846 comments at the top of the BeagleBoneTarget
8847 ``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file.
8848
8849- *"EdgeRouterTarget":* Choose "EdgeRouterTarget" is you are deploying
8850 images and running tests on the Ubiquiti Networks EdgeRouter Lite.
8851 For information on how to use these tests, see the comments at the
8852 top of the EdgeRouterTarget
8853 ``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file.
8854
8855- *"GrubTarget":* Choose the "supports deploying images and running
8856 tests on any generic PC that boots using GRUB. For information on how
8857 to use these tests, see the comments at the top of the GrubTarget
8858 ``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file.
8859
8860- *"your-target":* Create your own custom target if you want to run
8861 tests when you are deploying images and running tests on a custom
8862 machine within your BSP layer. To do this, you need to add a Python
8863 unit that defines the target class under ``lib/oeqa/controllers/``
8864 within your layer. You must also provide an empty ``__init__.py``.
8865 For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``.
8866
8867Selecting SystemdbootTarget
8868~~~~~~~~~~~~~~~~~~~~~~~~~~~
8869
8870If you did not set ``TEST_TARGET`` to "SystemdbootTarget", then you do
8871not need any information in this section. You can skip down to the
8872"`Running Tests <#qemu-image-running-tests>`__" section.
8873
8874If you did set ``TEST_TARGET`` to "SystemdbootTarget", you also need to
8875perform a one-time setup of your master image by doing the following:
8876
88771. *Set EFI_PROVIDER:* Be sure that ``EFI_PROVIDER`` is as follows:
8878 ::
8879
8880 EFI_PROVIDER = "systemd-boot"
8881
88822. *Build the master image:* Build the ``core-image-testmaster`` image.
8883 The ``core-image-testmaster`` recipe is provided as an example for a
8884 "master" image and you can customize the image recipe as you would
8885 any other recipe.
8886
8887 Here are the image recipe requirements:
8888
8889 - Inherits ``core-image`` so that kernel modules are installed.
8890
8891 - Installs normal linux utilities not busybox ones (e.g. ``bash``,
8892 ``coreutils``, ``tar``, ``gzip``, and ``kmod``).
8893
8894 - Uses a custom Initial RAM Disk (initramfs) image with a custom
8895 installer. A normal image that you can install usually creates a
8896 single rootfs partition. This image uses another installer that
8897 creates a specific partition layout. Not all Board Support
8898 Packages (BSPs) can use an installer. For such cases, you need to
8899 manually create the following partition layout on the target:
8900
8901 - First partition mounted under ``/boot``, labeled "boot".
8902
8903 - The main rootfs partition where this image gets installed,
8904 which is mounted under ``/``.
8905
8906 - Another partition labeled "testrootfs" where test images get
8907 deployed.
8908
89093. *Install image:* Install the image that you just built on the target
8910 system.
8911
8912The final thing you need to do when setting ``TEST_TARGET`` to
8913"SystemdbootTarget" is to set up the test image:
8914
89151. *Set up your local.conf file:* Make sure you have the following
8916 statements in your ``local.conf`` file:
8917 ::
8918
8919 IMAGE_FSTYPES += "tar.gz"
8920 INHERIT += "testimage"
8921 TEST_TARGET = "SystemdbootTarget"
8922 TEST_TARGET_IP = "192.168.2.3"
8923
89242. *Build your test image:* Use BitBake to build the image:
8925 ::
8926
8927 $ bitbake core-image-sato
8928
8929Power Control
8930~~~~~~~~~~~~~
8931
8932For most hardware targets other than "simpleremote", you can control
8933power:
8934
8935- You can use ``TEST_POWERCONTROL_CMD`` together with
8936 ``TEST_POWERCONTROL_EXTRA_ARGS`` as a command that runs on the host
8937 and does power cycling. The test code passes one argument to that
8938 command: off, on or cycle (off then on). Here is an example that
8939 could appear in your ``local.conf`` file:
8940 ::
8941
8942 TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1"
8943
8944 In this example, the expect
8945 script does the following:
8946 ::
8947
8948 ssh test@10.11.12.1 "pyctl nuc1 arg"
8949
8950 It then runs a Python script that controls power for a label called
8951 ``nuc1``.
8952
8953 .. note::
8954
8955 You need to customize
8956 TEST_POWERCONTROL_CMD
8957 and
8958 TEST_POWERCONTROL_EXTRA_ARGS
8959 for your own setup. The one requirement is that it accepts "on",
8960 "off", and "cycle" as the last argument.
8961
8962- When no command is defined, it connects to the device over SSH and
8963 uses the classic reboot command to reboot the device. Classic reboot
8964 is fine as long as the machine actually reboots (i.e. the SSH test
8965 has not failed). It is useful for scenarios where you have a simple
8966 setup, typically with a single board, and where some manual
8967 interaction is okay from time to time.
8968
8969If you have no hardware to automatically perform power control but still
8970wish to experiment with automated hardware testing, you can use the
8971dialog-power-control script that shows a dialog prompting you to perform
8972the required power action. This script requires either KDialog or Zenity
8973to be installed. To use this script, set the
8974:term:`TEST_POWERCONTROL_CMD`
8975variable as follows:
8976::
8977
8978 TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control"
8979
8980Serial Console Connection
8981~~~~~~~~~~~~~~~~~~~~~~~~~
8982
8983For test target classes requiring a serial console to interact with the
8984bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget),
8985you need to specify a command to use to connect to the serial console of
8986the target machine by using the
8987:term:`TEST_SERIALCONTROL_CMD`
8988variable and optionally the
8989:term:`TEST_SERIALCONTROL_EXTRA_ARGS`
8990variable.
8991
8992These cases could be a serial terminal program if the machine is
8993connected to a local serial port, or a ``telnet`` or ``ssh`` command
8994connecting to a remote console server. Regardless of the case, the
8995command simply needs to connect to the serial console and forward that
8996connection to standard input and output as any normal terminal program
8997does. For example, to use the picocom terminal program on serial device
8998``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:
8999::
9000
9001 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
9002
9003For local
9004devices where the serial port device disappears when the device reboots,
9005an additional "serdevtry" wrapper script is provided. To use this
9006wrapper, simply prefix the terminal command with
9007``${COREBASE}/scripts/contrib/serdevtry``:
9008::
9009
9010 TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0"
9011
9012.. _qemu-image-running-tests:
9013
9014Running Tests
9015-------------
9016
9017You can start the tests automatically or manually:
9018
9019- *Automatically running tests:* To run the tests automatically after
9020 the OpenEmbedded build system successfully creates an image, first
9021 set the
9022 :term:`TESTIMAGE_AUTO`
9023 variable to "1" in your ``local.conf`` file in the
9024 :term:`Build Directory`:
9025 ::
9026
9027 TESTIMAGE_AUTO = "1"
9028
9029 Next, build your image. If the image successfully builds, the
9030 tests run:
9031 ::
9032
9033 bitbake core-image-sato
9034
9035- *Manually running tests:* To manually run the tests, first globally
9036 inherit the
9037 :ref:`testimage <ref-classes-testimage*>` class
9038 by editing your ``local.conf`` file:
9039 ::
9040
9041 INHERIT += "testimage"
9042
9043 Next, use BitBake to run the tests:
9044 ::
9045
9046 bitbake -c testimage image
9047
9048All test files reside in ``meta/lib/oeqa/runtime`` in the
9049:term:`Source Directory`. A test name maps
9050directly to a Python module. Each test module may contain a number of
9051individual tests. Tests are usually grouped together by the area tested
9052(e.g tests for systemd reside in ``meta/lib/oeqa/runtime/systemd.py``).
9053
9054You can add tests to any layer provided you place them in the proper
9055area and you extend :term:`BBPATH` in
9056the ``local.conf`` file as normal. Be sure that tests reside in
9057``layer/lib/oeqa/runtime``.
9058
9059.. note::
9060
9061 Be sure that module names do not collide with module names used in
9062 the default set of test modules in
9063 meta/lib/oeqa/runtime
9064 .
9065
9066You can change the set of tests run by appending or overriding
9067:term:`TEST_SUITES` variable in
9068``local.conf``. Each name in ``TEST_SUITES`` represents a required test
9069for the image. Test modules named within ``TEST_SUITES`` cannot be
9070skipped even if a test is not suitable for an image (e.g. running the
9071RPM tests on an image without ``rpm``). Appending "auto" to
9072``TEST_SUITES`` causes the build system to try to run all tests that are
9073suitable for the image (i.e. each test module may elect to skip itself).
9074
9075The order you list tests in ``TEST_SUITES`` is important and influences
9076test dependencies. Consequently, tests that depend on other tests should
9077be added after the test on which they depend. For example, since the
9078``ssh`` test depends on the ``ping`` test, "ssh" needs to come after
9079"ping" in the list. The test class provides no re-ordering or dependency
9080handling.
9081
9082.. note::
9083
9084 Each module can have multiple classes with multiple test methods.
9085 And, Python
9086 unittest
9087 rules apply.
9088
9089Here are some things to keep in mind when running tests:
9090
9091- The default tests for the image are defined as:
9092 ::
9093
9094 DEFAULT_TEST_SUITES_pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg"
9095
9096- Add your own test to the list of the by using the following:
9097 ::
9098
9099 TEST_SUITES_append = " mytest"
9100
9101- Run a specific list of tests as follows: TEST_SUITES = "test1 test2
9102 test3" Remember, order is important. Be sure to place a test that is
9103 dependent on another test later in the order.
9104
9105Exporting Tests
9106---------------
9107
9108You can export tests so that they can run independently of the build
9109system. Exporting tests is required if you want to be able to hand the
9110test execution off to a scheduler. You can only export tests that are
9111defined in :term:`TEST_SUITES`.
9112
9113If your image is already built, make sure the following are set in your
9114``local.conf`` file:
9115::
9116
9117 INHERIT +="testexport"
9118 TEST_TARGET_IP = "IP-address-for-the-test-target"
9119 TEST_SERVER_IP = "IP-address-for-the-test-server"
9120
9121You can then export the tests with the
9122following BitBake command form:
9123::
9124
9125 $ bitbake image -c testexport
9126
9127Exporting the tests places them in the
9128:term:`Build Directory` in
9129``tmp/testexport/``\ image, which is controlled by the
9130``TEST_EXPORT_DIR`` variable.
9131
9132You can now run the tests outside of the build environment:
9133::
9134
9135 $ cd tmp/testexport/image
9136 $ ./runexported.py testdata.json
9137
9138Here is a complete example that shows IP addresses and uses the
9139``core-image-sato`` image:
9140::
9141
9142 INHERIT +="testexport"
9143 TEST_TARGET_IP = "192.168.7.2"
9144 TEST_SERVER_IP = "192.168.7.1"
9145
9146Use BitBake to export the tests:
9147::
9148
9149 $ bitbake core-image-sato -c testexport
9150
9151Run the tests outside of
9152the build environment using the following:
9153::
9154
9155 $ cd tmp/testexport/core-image-sato
9156 $ ./runexported.py testdata.json
9157
9158.. _qemu-image-writing-new-tests:
9159
9160Writing New Tests
9161-----------------
9162
9163As mentioned previously, all new test files need to be in the proper
9164place for the build system to find them. New tests for additional
9165functionality outside of the core should be added to the layer that adds
9166the functionality, in ``layer/lib/oeqa/runtime`` (as long as
9167:term:`BBPATH` is extended in the
9168layer's ``layer.conf`` file as normal). Just remember the following:
9169
9170- Filenames need to map directly to test (module) names.
9171
9172- Do not use module names that collide with existing core tests.
9173
9174- Minimally, an empty ``__init__.py`` file must exist in the runtime
9175 directory.
9176
9177To create a new test, start by copying an existing module (e.g.
9178``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use
9179code from ``meta/lib/oeqa/utils``, which are helper classes.
9180
9181.. note::
9182
9183 Structure shell commands such that you rely on them and they return a
9184 single code for success. Be aware that sometimes you will need to
9185 parse the output. See the
9186 df.py
9187 and
9188 date.py
9189 modules for examples.
9190
9191You will notice that all test classes inherit ``oeRuntimeTest``, which
9192is found in ``meta/lib/oetest.py``. This base class offers some helper
9193attributes, which are described in the following sections:
9194
9195.. _qemu-image-writing-tests-class-methods:
9196
9197Class Methods
9198~~~~~~~~~~~~~
9199
9200Class methods are as follows:
9201
9202- *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed
9203 package list of the image, which is based on the manifest file that
9204 is generated during the ``do_rootfs`` task.
9205
9206- *hasFeature(feature):* Returns "True" if the feature is in
9207 :term:`IMAGE_FEATURES` or
9208 :term:`DISTRO_FEATURES`.
9209
9210.. _qemu-image-writing-tests-class-attributes:
9211
9212Class Attributes
9213~~~~~~~~~~~~~~~~
9214
9215Class attributes are as follows:
9216
9217- *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image.
9218 Otherwise, ``pscmd`` equals "ps" (busybox).
9219
9220- *tc:* The called test context, which gives access to the
9221 following attributes:
9222
9223 - *d:* The BitBake datastore, which allows you to use stuff such
9224 as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``.
9225
9226 - *testslist and testsrequired:* Used internally. The tests
9227 do not need these.
9228
9229 - *filesdir:* The absolute path to
9230 ``meta/lib/oeqa/runtime/files``, which contains helper files for
9231 tests meant for copying on the target such as small files written
9232 in C for compilation.
9233
9234 - *target:* The target controller object used to deploy and
9235 start an image on a particular target (e.g. Qemu, SimpleRemote,
9236 and SystemdbootTarget). Tests usually use the following:
9237
9238 - *ip:* The target's IP address.
9239
9240 - *server_ip:* The host's IP address, which is usually used
9241 by the DNF test suite.
9242
9243 - *run(cmd, timeout=None):* The single, most used method.
9244 This command is a wrapper for: ``ssh root@host "cmd"``. The
9245 command returns a tuple: (status, output), which are what their
9246 names imply - the return code of "cmd" and whatever output it
9247 produces. The optional timeout argument represents the number
9248 of seconds the test should wait for "cmd" to return. If the
9249 argument is "None", the test uses the default instance's
9250 timeout period, which is 300 seconds. If the argument is "0",
9251 the test runs until the command returns.
9252
9253 - *copy_to(localpath, remotepath):*
9254 ``scp localpath root@ip:remotepath``.
9255
9256 - *copy_from(remotepath, localpath):*
9257 ``scp root@host:remotepath localpath``.
9258
9259.. _qemu-image-writing-tests-instance-attributes:
9260
9261Instance Attributes
9262~~~~~~~~~~~~~~~~~~~
9263
9264A single instance attribute exists, which is ``target``. The ``target``
9265instance attribute is identical to the class attribute of the same name,
9266which is described in the previous section. This attribute exists as
9267both an instance and class attribute so tests can use
9268``self.target.run(cmd)`` in instance methods instead of
9269``oeRuntimeTest.tc.target.run(cmd)``.
9270
9271Installing Packages in the DUT Without the Package Manager
9272----------------------------------------------------------
9273
9274When a test requires a package built by BitBake, it is possible to
9275install that package. Installing the package does not require a package
9276manager be installed in the device under test (DUT). It does, however,
9277require an SSH connection and the target must be using the
9278``sshcontrol`` class.
9279
9280.. note::
9281
9282 This method uses
9283 scp
9284 to copy files from the host to the target, which causes permissions
9285 and special attributes to be lost.
9286
9287A JSON file is used to define the packages needed by a test. This file
9288must be in the same path as the file used to define the tests.
9289Furthermore, the filename must map directly to the test module name with
9290a ``.json`` extension.
9291
9292The JSON file must include an object with the test name as keys of an
9293object or an array. This object (or array of objects) uses the following
9294data:
9295
9296- "pkg" - A mandatory string that is the name of the package to be
9297 installed.
9298
9299- "rm" - An optional boolean, which defaults to "false", that specifies
9300 to remove the package after the test.
9301
9302- "extract" - An optional boolean, which defaults to "false", that
9303 specifies if the package must be extracted from the package format.
9304 When set to "true", the package is not automatically installed into
9305 the DUT.
9306
9307Following is an example JSON file that handles test "foo" installing
9308package "bar" and test "foobar" installing packages "foo" and "bar".
9309Once the test is complete, the packages are removed from the DUT.
9310::
9311
9312 {
9313 "foo": {
9314 "pkg": "bar"
9315 },
9316 "foobar": [
9317 {
9318 "pkg": "foo",
9319 "rm": true
9320 },
9321 {
9322 "pkg": "bar",
9323 "rm": true
9324 }
9325 ]
9326 }
9327
9328.. _usingpoky-debugging-tools-and-techniques:
9329
9330Debugging Tools and Techniques
9331==============================
9332
9333The exact method for debugging build failures depends on the nature of
9334the problem and on the system's area from which the bug originates.
9335Standard debugging practices such as comparison against the last known
9336working version with examination of the changes and the re-application
9337of steps to identify the one causing the problem are valid for the Yocto
9338Project just as they are for any other system. Even though it is
9339impossible to detail every possible potential failure, this section
9340provides some general tips to aid in debugging given a variety of
9341situations.
9342
9343.. note::
9344
9345 A useful feature for debugging is the error reporting tool.
9346 Configuring the Yocto Project to use this tool causes the
9347 OpenEmbedded build system to produce error reporting commands as part
9348 of the console output. You can enter the commands after the build
9349 completes to log error information into a common database, that can
9350 help you figure out what might be going wrong. For information on how
9351 to enable and use this feature, see the "
9352 Using the Error Reporting Tool
9353 " section.
9354
9355The following list shows the debugging topics in the remainder of this
9356section:
9357
9358- "`Viewing Logs from Failed
9359 Tasks <#dev-debugging-viewing-logs-from-failed-tasks>`__" describes
9360 how to find and view logs from tasks that failed during the build
9361 process.
9362
9363- "`Viewing Variable
9364 Values <#dev-debugging-viewing-variable-values>`__" describes how to
9365 use the BitBake ``-e`` option to examine variable values after a
9366 recipe has been parsed.
9367
9368- ":ref:`dev-manual/dev-manual-common-tasks:viewing package information with \`\`oe-pkgdata-util\`\``"
9369 describes how to use the ``oe-pkgdata-util`` utility to query
9370 :term:`PKGDATA_DIR` and
9371 display package-related information for built packages.
9372
9373- "`Viewing Dependencies Between Recipes and
9374 Tasks <#dev-viewing-dependencies-between-recipes-and-tasks>`__"
9375 describes how to use the BitBake ``-g`` option to display recipe
9376 dependency information used during the build.
9377
9378- "`Viewing Task Variable
9379 Dependencies <#dev-viewing-task-variable-dependencies>`__" describes
9380 how to use the ``bitbake-dumpsig`` command in conjunction with key
9381 subdirectories in the
9382 :term:`Build Directory` to determine
9383 variable dependencies.
9384
9385- "`Running Specific Tasks <#dev-debugging-taskrunning>`__" describes
9386 how to use several BitBake options (e.g. ``-c``, ``-C``, and ``-f``)
9387 to run specific tasks in the build chain. It can be useful to run
9388 tasks "out-of-order" when trying isolate build issues.
9389
9390- "`General BitBake Problems <#dev-debugging-bitbake>`__" describes how
9391 to use BitBake's ``-D`` debug output option to reveal more about what
9392 BitBake is doing during the build.
9393
9394- "`Building with No Dependencies <#dev-debugging-buildfile>`__"
9395 describes how to use the BitBake ``-b`` option to build a recipe
9396 while ignoring dependencies.
9397
9398- "`Recipe Logging Mechanisms <#recipe-logging-mechanisms>`__"
9399 describes how to use the many recipe logging functions to produce
9400 debugging output and report errors and warnings.
9401
9402- "`Debugging Parallel Make Races <#debugging-parallel-make-races>`__"
9403 describes how to debug situations where the build consists of several
9404 parts that are run simultaneously and when the output or result of
9405 one part is not ready for use with a different part of the build that
9406 depends on that output.
9407
9408- "`Debugging With the GNU Project Debugger (GDB)
9409 Remotely <#platdev-gdb-remotedebug>`__" describes how to use GDB to
9410 allow you to examine running programs, which can help you fix
9411 problems.
9412
9413- "`Debugging with the GNU Project Debugger (GDB) on the
9414 Target <#debugging-with-the-gnu-project-debugger-gdb-on-the-target>`__"
9415 describes how to use GDB directly on target hardware for debugging.
9416
9417- "`Other Debugging Tips <#dev-other-debugging-others>`__" describes
9418 miscellaneous debugging tips that can be useful.
9419
9420.. _dev-debugging-viewing-logs-from-failed-tasks:
9421
9422Viewing Logs from Failed Tasks
9423------------------------------
9424
9425You can find the log for a task in the file
9426``${``\ :term:`WORKDIR`\ ``}/temp/log.do_``\ taskname.
9427For example, the log for the
9428:ref:`ref-tasks-compile` task of the
9429QEMU minimal image for the x86 machine (``qemux86``) might be in
9430``tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile``.
9431To see the commands :term:`BitBake` ran
9432to generate a log, look at the corresponding ``run.do_``\ taskname file
9433in the same directory.
9434
9435``log.do_``\ taskname and ``run.do_``\ taskname are actually symbolic
9436links to ``log.do_``\ taskname\ ``.``\ pid and
9437``log.run_``\ taskname\ ``.``\ pid, where pid is the PID the task had
9438when it ran. The symlinks always point to the files corresponding to the
9439most recent run.
9440
9441.. _dev-debugging-viewing-variable-values:
9442
9443Viewing Variable Values
9444-----------------------
9445
9446Sometimes you need to know the value of a variable as a result of
9447BitBake's parsing step. This could be because some unexpected behavior
9448occurred in your project. Perhaps an attempt to :ref:`modify a variable
9449<bitbake:bitbake-user-manual/bitbake-user-manual-metadata:modifying existing
9450variables>` did not work out as expected.
9451
9452BitBake's ``-e`` option is used to display variable values after
9453parsing. The following command displays the variable values after the
9454configuration files (i.e. ``local.conf``, ``bblayers.conf``,
9455``bitbake.conf`` and so forth) have been parsed:
9456::
9457
9458 $ bitbake -e
9459
9460The following command displays variable values after a specific recipe has
9461been parsed. The variables include those from the configuration as well:
9462::
9463
9464 $ bitbake -e recipename
9465
9466.. note::
9467
9468 Each recipe has its own private set of variables (datastore).
9469 Internally, after parsing the configuration, a copy of the resulting
9470 datastore is made prior to parsing each recipe. This copying implies
9471 that variables set in one recipe will not be visible to other
9472 recipes.
9473
9474 Likewise, each task within a recipe gets a private datastore based on
9475 the recipe datastore, which means that variables set within one task
9476 will not be visible to other tasks.
9477
9478In the output of ``bitbake -e``, each variable is preceded by a
9479description of how the variable got its value, including temporary
9480values that were later overriden. This description also includes
9481variable flags (varflags) set on the variable. The output can be very
9482helpful during debugging.
9483
9484Variables that are exported to the environment are preceded by
9485``export`` in the output of ``bitbake -e``. See the following example:
9486::
9487
9488 export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86"
9489
9490In addition to variable values, the output of the ``bitbake -e`` and
9491``bitbake -e`` recipe commands includes the following information:
9492
9493- The output starts with a tree listing all configuration files and
9494 classes included globally, recursively listing the files they include
9495 or inherit in turn. Much of the behavior of the OpenEmbedded build
9496 system (including the behavior of the :ref:`ref-manual/ref-tasks:normal recipe build tasks`) is
9497 implemented in the
9498 :ref:`base <ref-classes-base>` class and the
9499 classes it inherits, rather than being built into BitBake itself.
9500
9501- After the variable values, all functions appear in the output. For
9502 shell functions, variables referenced within the function body are
9503 expanded. If a function has been modified using overrides or using
9504 override-style operators like ``_append`` and ``_prepend``, then the
9505 final assembled function body appears in the output.
9506
9507Viewing Package Information with ``oe-pkgdata-util``
9508----------------------------------------------------
9509
9510You can use the ``oe-pkgdata-util`` command-line utility to query
9511:term:`PKGDATA_DIR` and display
9512various package-related information. When you use the utility, you must
9513use it to view information on packages that have already been built.
9514
9515Following are a few of the available ``oe-pkgdata-util`` subcommands.
9516
9517.. note::
9518
9519 You can use the standard \* and ? globbing wildcards as part of
9520 package names and paths.
9521
9522- ``oe-pkgdata-util list-pkgs [pattern]``: Lists all packages
9523 that have been built, optionally limiting the match to packages that
9524 match pattern.
9525
9526- ``oe-pkgdata-util list-pkg-files package ...``: Lists the
9527 files and directories contained in the given packages.
9528
9529 .. note::
9530
9531 A different way to view the contents of a package is to look at
9532 the
9533 ``${``\ :term:`WORKDIR`\ ``}/packages-split``
9534 directory of the recipe that generates the package. This directory
9535 is created by the
9536 :ref:`ref-tasks-package` task
9537 and has one subdirectory for each package the recipe generates,
9538 which contains the files stored in that package.
9539
9540 If you want to inspect the ``${WORKDIR}/packages-split``
9541 directory, make sure that
9542 :ref:`rm_work <ref-classes-rm-work>` is not
9543 enabled when you build the recipe.
9544
9545- ``oe-pkgdata-util find-path path ...``: Lists the names of
9546 the packages that contain the given paths. For example, the following
9547 tells us that ``/usr/share/man/man1/make.1`` is contained in the
9548 ``make-doc`` package:
9549 ::
9550
9551 $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 make-doc: /usr/share/man/man1/make.1
9552
9553- ``oe-pkgdata-util lookup-recipe package ...``: Lists the name
9554 of the recipes that produce the given packages.
9555
9556For more information on the ``oe-pkgdata-util`` command, use the help
9557facility:
9558::
9559
9560 $ oe-pkgdata-util DASHDASHhelp
9561 $ oe-pkgdata-util subcommand --help
9562
9563.. _dev-viewing-dependencies-between-recipes-and-tasks:
9564
9565Viewing Dependencies Between Recipes and Tasks
9566----------------------------------------------
9567
9568Sometimes it can be hard to see why BitBake wants to build other recipes
9569before the one you have specified. Dependency information can help you
9570understand why a recipe is built.
9571
9572To generate dependency information for a recipe, run the following
9573command:
9574::
9575
9576 $ bitbake -g recipename
9577
9578This command writes the following files in the current directory:
9579
9580- ``pn-buildlist``: A list of recipes/targets involved in building
9581 recipename. "Involved" here means that at least one task from the
9582 recipe needs to run when building recipename from scratch. Targets
9583 that are in
9584 :term:`ASSUME_PROVIDED`
9585 are not listed.
9586
9587- ``task-depends.dot``: A graph showing dependencies between tasks.
9588
9589The graphs are in
9590`DOT <https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29>`__
9591format and can be converted to images (e.g. using the ``dot`` tool from
9592`Graphviz <http://www.graphviz.org/>`__).
9593
9594.. note::
9595
9596 - DOT files use a plain text format. The graphs generated using the
9597 ``bitbake -g`` command are often so large as to be difficult to
9598 read without special pruning (e.g. with Bitbake's ``-I`` option)
9599 and processing. Despite the form and size of the graphs, the
9600 corresponding ``.dot`` files can still be possible to read and
9601 provide useful information.
9602
9603 As an example, the ``task-depends.dot`` file contains lines such
9604 as the following:
9605 ::
9606
9607 "libxslt.do_configure" -> "libxml2.do_populate_sysroot"
9608
9609 The above example line reveals that the
9610 :ref:`ref-tasks-configure`
9611 task in ``libxslt`` depends on the
9612 :ref:`ref-tasks-populate_sysroot`
9613 task in ``libxml2``, which is a normal
9614 :term:`DEPENDS` dependency
9615 between the two recipes.
9616
9617 - For an example of how ``.dot`` files can be processed, see the
9618 ``scripts/contrib/graph-tool`` Python script, which finds and
9619 displays paths between graph nodes.
9620
9621You can use a different method to view dependency information by using
9622the following command:
9623::
9624
9625 $ bitbake -g -u taskexp recipename
9626
9627This command
9628displays a GUI window from which you can view build-time and runtime
9629dependencies for the recipes involved in building recipename.
9630
9631.. _dev-viewing-task-variable-dependencies:
9632
9633Viewing Task Variable Dependencies
9634----------------------------------
9635
9636As mentioned in the
9637":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-execution:checksums (signatures)`" section of the BitBake
9638User Manual, BitBake tries to automatically determine what variables a
9639task depends on so that it can rerun the task if any values of the
9640variables change. This determination is usually reliable. However, if
9641you do things like construct variable names at runtime, then you might
9642have to manually declare dependencies on those variables using
9643``vardeps`` as described in the
9644":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags`" section of the BitBake
9645User Manual.
9646
9647If you are unsure whether a variable dependency is being picked up
9648automatically for a given task, you can list the variable dependencies
9649BitBake has determined by doing the following:
9650
96511. Build the recipe containing the task:
9652::
9653
9654 $ bitbake recipename
9655
96562. Inside the :term:`STAMPS_DIR`
9657 directory, find the signature data (``sigdata``) file that
9658 corresponds to the task. The ``sigdata`` files contain a pickled
9659 Python database of all the metadata that went into creating the input
9660 checksum for the task. As an example, for the
9661 :ref:`ref-tasks-fetch` task of the
9662 ``db`` recipe, the ``sigdata`` file might be found in the following
9663 location:
9664 ::
9665
9666 ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
9667
9668 For tasks that are accelerated through the shared state
9669 (:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>`) cache, an
9670 additional ``siginfo`` file is written into
9671 :term:`SSTATE_DIR` along with
9672 the cached task output. The ``siginfo`` files contain exactly the
9673 same information as ``sigdata`` files.
9674
96753. Run ``bitbake-dumpsig`` on the ``sigdata`` or ``siginfo`` file. Here
9676 is an example:
9677 ::
9678
9679 $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
9680
9681 In the output of the above command, you will find a line like the
9682 following, which lists all the (inferred) variable dependencies for
9683 the task. This list also includes indirect dependencies from
9684 variables depending on other variables, recursively.
9685 ::
9686
9687 Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
9688
9689 .. note::
9690
9691 Functions (e.g.
9692 base_do_fetch
9693 ) also count as variable dependencies. These functions in turn
9694 depend on the variables they reference.
9695
9696 The output of ``bitbake-dumpsig`` also includes the value each
9697 variable had, a list of dependencies for each variable, and
9698 :term:`bitbake:BB_HASHBASE_WHITELIST`
9699 information.
9700
9701There is also a ``bitbake-diffsigs`` command for comparing two
9702``siginfo`` or ``sigdata`` files. This command can be helpful when
9703trying to figure out what changed between two versions of a task. If you
9704call ``bitbake-diffsigs`` with just one file, the command behaves like
9705``bitbake-dumpsig``.
9706
9707You can also use BitBake to dump out the signature construction
9708information without executing tasks by using either of the following
9709BitBake command-line options:
9710::
9711
9712 ‐‐dump-signatures=SIGNATURE_HANDLER
9713 -S SIGNATURE_HANDLER
9714
9715
9716.. note::
9717
9718 Two common values for
9719 SIGNATURE_HANDLER
9720 are "none" and "printdiff", which dump only the signature or compare
9721 the dumped signature with the cached one, respectively.
9722
9723Using BitBake with either of these options causes BitBake to dump out
9724``sigdata`` files in the ``stamps`` directory for every task it would
9725have executed instead of building the specified target package.
9726
9727.. _dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task:
9728
9729Viewing Metadata Used to Create the Input Signature of a Shared State Task
9730--------------------------------------------------------------------------
9731
9732Seeing what metadata went into creating the input signature of a shared
9733state (sstate) task can be a useful debugging aid. This information is
9734available in signature information (``siginfo``) files in
9735:term:`SSTATE_DIR`. For
9736information on how to view and interpret information in ``siginfo``
9737files, see the "`Viewing Task Variable
9738Dependencies <#dev-viewing-task-variable-dependencies>`__" section.
9739
9740For conceptual information on shared state, see the
9741":ref:`overview-manual/overview-manual-concepts:shared state`"
9742section in the Yocto Project Overview and Concepts Manual.
9743
9744.. _dev-invalidating-shared-state-to-force-a-task-to-run:
9745
9746Invalidating Shared State to Force a Task to Run
9747------------------------------------------------
9748
9749The OpenEmbedded build system uses
9750:ref:`checksums <overview-checksums>` and
9751:ref:`overview-manual/overview-manual-concepts:shared state` cache to avoid unnecessarily
9752rebuilding tasks. Collectively, this scheme is known as "shared state
9753code."
9754
9755As with all schemes, this one has some drawbacks. It is possible that
9756you could make implicit changes to your code that the checksum
9757calculations do not take into account. These implicit changes affect a
9758task's output but do not trigger the shared state code into rebuilding a
9759recipe. Consider an example during which a tool changes its output.
9760Assume that the output of ``rpmdeps`` changes. The result of the change
9761should be that all the ``package`` and ``package_write_rpm`` shared
9762state cache items become invalid. However, because the change to the
9763output is external to the code and therefore implicit, the associated
9764shared state cache items do not become invalidated. In this case, the
9765build process uses the cached items rather than running the task again.
9766Obviously, these types of implicit changes can cause problems.
9767
9768To avoid these problems during the build, you need to understand the
9769effects of any changes you make. Realize that changes you make directly
9770to a function are automatically factored into the checksum calculation.
9771Thus, these explicit changes invalidate the associated area of shared
9772state cache. However, you need to be aware of any implicit changes that
9773are not obvious changes to the code and could affect the output of a
9774given task.
9775
9776When you identify an implicit change, you can easily take steps to
9777invalidate the cache and force the tasks to run. The steps you can take
9778are as simple as changing a function's comments in the source code. For
9779example, to invalidate package shared state files, change the comment
9780statements of
9781:ref:`ref-tasks-package` or the
9782comments of one of the functions it calls. Even though the change is
9783purely cosmetic, it causes the checksum to be recalculated and forces
9784the build system to run the task again.
9785
9786.. note::
9787
9788 For an example of a commit that makes a cosmetic change to invalidate
9789 shared state, see this
9790 commit
9791 .
9792
9793.. _dev-debugging-taskrunning:
9794
9795Running Specific Tasks
9796----------------------
9797
9798Any given recipe consists of a set of tasks. The standard BitBake
9799behavior in most cases is: ``do_fetch``, ``do_unpack``, ``do_patch``,
9800``do_configure``, ``do_compile``, ``do_install``, ``do_package``,
9801``do_package_write_*``, and ``do_build``. The default task is
9802``do_build`` and any tasks on which it depends build first. Some tasks,
9803such as ``do_devshell``, are not part of the default build chain. If you
9804wish to run a task that is not part of the default build chain, you can
9805use the ``-c`` option in BitBake. Here is an example:
9806::
9807
9808 $ bitbake matchbox-desktop -c devshell
9809
9810The ``-c`` option respects task dependencies, which means that all other
9811tasks (including tasks from other recipes) that the specified task
9812depends on will be run before the task. Even when you manually specify a
9813task to run with ``-c``, BitBake will only run the task if it considers
9814it "out of date". See the
9815":ref:`overview-manual/overview-manual-concepts:stamp files and the rerunning of tasks`"
9816section in the Yocto Project Overview and Concepts Manual for how
9817BitBake determines whether a task is "out of date".
9818
9819If you want to force an up-to-date task to be rerun (e.g. because you
9820made manual modifications to the recipe's
9821:term:`WORKDIR` that you want to try
9822out), then you can use the ``-f`` option.
9823
9824.. note::
9825
9826 The reason
9827 -f
9828 is never required when running the
9829 do_devshell
9830 task is because the
9831 [
9832 nostamp
9833 ]
9834 variable flag is already set for the task.
9835
9836The following example shows one way you can use the ``-f`` option:
9837::
9838
9839 $ bitbake matchbox-desktop
9840 .
9841 .
9842 make some changes to the source code in the work directory
9843 .
9844 .
9845 $ bitbake matchbox-desktop -c compile -f
9846 $ bitbake matchbox-desktop
9847
9848This sequence first builds and then recompiles ``matchbox-desktop``. The
9849last command reruns all tasks (basically the packaging tasks) after the
9850compile. BitBake recognizes that the ``do_compile`` task was rerun and
9851therefore understands that the other tasks also need to be run again.
9852
9853Another, shorter way to rerun a task and all
9854:ref:`ref-manual/ref-tasks:normal recipe build tasks`
9855that depend on it is to use the ``-C`` option.
9856
9857.. note::
9858
9859 This option is upper-cased and is separate from the
9860 -c
9861 option, which is lower-cased.
9862
9863Using this option invalidates the given task and then runs the
9864:ref:`ref-tasks-build` task, which is
9865the default task if no task is given, and the tasks on which it depends.
9866You could replace the final two commands in the previous example with
9867the following single command:
9868::
9869
9870 $ bitbake matchbox-desktop -C compile
9871
9872Internally, the ``-f`` and ``-C`` options work by tainting (modifying)
9873the input checksum of the specified task. This tainting indirectly
9874causes the task and its dependent tasks to be rerun through the normal
9875task dependency mechanisms.
9876
9877.. note::
9878
9879 BitBake explicitly keeps track of which tasks have been tainted in
9880 this fashion, and will print warnings such as the following for
9881 builds involving such tasks:
9882 ::
9883
9884 WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run
9885
9886
9887 The purpose of the warning is to let you know that the work directory
9888 and build output might not be in the clean state they would be in for
9889 a "normal" build, depending on what actions you took. To get rid of
9890 such warnings, you can remove the work directory and rebuild the
9891 recipe, as follows:
9892 ::
9893
9894 $ bitbake matchbox-desktop -c clean
9895 $ bitbake matchbox-desktop
9896
9897
9898You can view a list of tasks in a given package by running the
9899``do_listtasks`` task as follows:
9900::
9901
9902 $ bitbake matchbox-desktop -c listtasks
9903
9904The results appear as output to the console and are also in
9905the file ``${WORKDIR}/temp/log.do_listtasks``.
9906
9907.. _dev-debugging-bitbake:
9908
9909General BitBake Problems
9910------------------------
9911
9912You can see debug output from BitBake by using the ``-D`` option. The
9913debug output gives more information about what BitBake is doing and the
9914reason behind it. Each ``-D`` option you use increases the logging
9915level. The most common usage is ``-DDD``.
9916
9917The output from ``bitbake -DDD -v`` targetname can reveal why BitBake
9918chose a certain version of a package or why BitBake picked a certain
9919provider. This command could also help you in a situation where you
9920think BitBake did something unexpected.
9921
9922.. _dev-debugging-buildfile:
9923
9924Building with No Dependencies
9925-----------------------------
9926
9927To build a specific recipe (``.bb`` file), you can use the following
9928command form:
9929::
9930
9931 $ bitbake -b somepath/somerecipe.bb
9932
9933This command form does
9934not check for dependencies. Consequently, you should use it only when
9935you know existing dependencies have been met.
9936
9937.. note::
9938
9939 You can also specify fragments of the filename. In this case, BitBake
9940 checks for a unique match.
9941
9942Recipe Logging Mechanisms
9943-------------------------
9944
9945The Yocto Project provides several logging functions for producing
9946debugging output and reporting errors and warnings. For Python
9947functions, the following logging functions exist. All of these functions
9948log to ``${T}/log.do_``\ task, and can also log to standard output
9949(stdout) with the right settings:
9950
9951- ``bb.plain(msg)``: Writes msg as is to the log while also
9952 logging to stdout.
9953
9954- ``bb.note(msg)``: Writes "NOTE: msg" to the log. Also logs to
9955 stdout if BitBake is called with "-v".
9956
9957- ``bb.debug(level, msg)``: Writes "DEBUG: msg" to the
9958 log. Also logs to stdout if the log level is greater than or equal to
9959 level. See the ":ref:`-D <bitbake:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax>`" option
9960 in the BitBake User Manual for more information.
9961
9962- ``bb.warn(msg)``: Writes "WARNING: msg" to the log while also
9963 logging to stdout.
9964
9965- ``bb.error(msg)``: Writes "ERROR: msg" to the log while also
9966 logging to standard out (stdout).
9967
9968 .. note::
9969
9970 Calling this function does not cause the task to fail.
9971
9972- ``bb.fatal(``\ msg\ ``)``: This logging function is similar to
9973 ``bb.error(``\ msg\ ``)`` but also causes the calling task to fail.
9974
9975 .. note::
9976
9977 bb.fatal()
9978 raises an exception, which means you do not need to put a "return"
9979 statement after the function.
9980
9981The same logging functions are also available in shell functions, under
9982the names ``bbplain``, ``bbnote``, ``bbdebug``, ``bbwarn``, ``bberror``,
9983and ``bbfatal``. The
9984:ref:`logging <ref-classes-logging>` class
9985implements these functions. See that class in the ``meta/classes``
9986folder of the :term:`Source Directory` for information.
9987
9988Logging With Python
9989~~~~~~~~~~~~~~~~~~~
9990
9991When creating recipes using Python and inserting code that handles build
9992logs, keep in mind the goal is to have informative logs while keeping
9993the console as "silent" as possible. Also, if you want status messages
9994in the log, use the "debug" loglevel.
9995
9996Following is an example written in Python. The code handles logging for
9997a function that determines the number of tasks needed to be run. See the
9998":ref:`ref-tasks-listtasks`"
9999section for additional information:
10000::
10001
10002 python do_listtasks() {
10003 bb.debug(2, "Starting to figure out the task list")
10004 if noteworthy_condition:
10005 bb.note("There are 47 tasks to run")
10006 bb.debug(2, "Got to point xyz")
10007 if warning_trigger:
10008 bb.warn("Detected warning_trigger, this might be a problem later.")
10009 if recoverable_error:
10010 bb.error("Hit recoverable_error, you really need to fix this!")
10011 if fatal_error:
10012 bb.fatal("fatal_error detected, unable to print the task list")
10013 bb.plain("The tasks present are abc")
10014 bb.debug(2, "Finished figuring out the tasklist")
10015 }
10016
10017Logging With Bash
10018~~~~~~~~~~~~~~~~~
10019
10020When creating recipes using Bash and inserting code that handles build
10021logs, you have the same goals - informative with minimal console output.
10022The syntax you use for recipes written in Bash is similar to that of
10023recipes written in Python described in the previous section.
10024
10025Following is an example written in Bash. The code logs the progress of
10026the ``do_my_function`` function.
10027::
10028
10029 do_my_function() {
10030 bbdebug 2 "Running do_my_function"
10031 if [ exceptional_condition ]; then
10032 bbnote "Hit exceptional_condition"
10033 fi
10034 bbdebug 2 "Got to point xyz"
10035 if [ warning_trigger ]; then
10036 bbwarn "Detected warning_trigger, this might cause a problem later."
10037 fi
10038 if [ recoverable_error ]; then
10039 bberror "Hit recoverable_error, correcting"
10040 fi
10041 if [ fatal_error ]; then
10042 bbfatal "fatal_error detected"
10043 fi
10044 bbdebug 2 "Completed do_my_function"
10045 }
10046
10047
10048Debugging Parallel Make Races
10049-----------------------------
10050
10051A parallel ``make`` race occurs when the build consists of several parts
10052that are run simultaneously and a situation occurs when the output or
10053result of one part is not ready for use with a different part of the
10054build that depends on that output. Parallel make races are annoying and
10055can sometimes be difficult to reproduce and fix. However, some simple
10056tips and tricks exist that can help you debug and fix them. This section
10057presents a real-world example of an error encountered on the Yocto
10058Project autobuilder and the process used to fix it.
10059
10060.. note::
10061
10062 If you cannot properly fix a
10063 make
10064 race condition, you can work around it by clearing either the
10065 PARALLEL_MAKE
10066 or
10067 PARALLEL_MAKEINST
10068 variables.
10069
10070The Failure
10071~~~~~~~~~~~
10072
10073For this example, assume that you are building an image that depends on
10074the "neard" package. And, during the build, BitBake runs into problems
10075and creates the following output.
10076
10077.. note::
10078
10079 This example log file has longer lines artificially broken to make
10080 the listing easier to read.
10081
10082If you examine the output or the log file, you see the failure during
10083``make``:
10084::
10085
10086 | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
10087 | DEBUG: Executing shell function do_compile
10088 | NOTE: make -j 16
10089 | make --no-print-directory all-am
10090 | /bin/mkdir -p include/near
10091 | /bin/mkdir -p include/near
10092 | /bin/mkdir -p include/near
10093 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10094 0.14-r0/neard-0.14/include/types.h include/near/types.h
10095 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10096 0.14-r0/neard-0.14/include/log.h include/near/log.h
10097 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10098 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h
10099 | /bin/mkdir -p include/near
10100 | /bin/mkdir -p include/near
10101 | /bin/mkdir -p include/near
10102 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10103 0.14-r0/neard-0.14/include/tag.h include/near/tag.h
10104 | /bin/mkdir -p include/near
10105 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10106 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h
10107 | /bin/mkdir -p include/near
10108 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10109 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h
10110 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10111 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h
10112 | /bin/mkdir -p include/near
10113 | /bin/mkdir -p include/near
10114 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10115 0.14-r0/neard-0.14/include/setting.h include/near/setting.h
10116 | /bin/mkdir -p include/near
10117 | /bin/mkdir -p include/near
10118 | /bin/mkdir -p include/near
10119 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10120 0.14-r0/neard-0.14/include/device.h include/near/device.h
10121 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10122 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h
10123 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10124 0.14-r0/neard-0.14/include/snep.h include/near/snep.h
10125 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10126 0.14-r0/neard-0.14/include/version.h include/near/version.h
10127 | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/
10128 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h
10129 | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h
10130 | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/
10131 build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/
10132 yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0
10133 -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/
10134 lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/
10135 tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/
10136 nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/
10137 yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3
10138 -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\"
10139 -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c
10140 -o tools/snep-send.o tools/snep-send.c
10141 | In file included from tools/snep-send.c:16:0:
10142 | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
10143 | #include <near/dbus.h>
10144 | ^
10145 | compilation terminated.
10146 | make[1]: *** [tools/snep-send.o] Error 1
10147 | make[1]: *** Waiting for unfinished jobs....
10148 | make: *** [all] Error 2
10149 | ERROR: oe_runmake failed
10150
10151Reproducing the Error
10152~~~~~~~~~~~~~~~~~~~~~
10153
10154Because race conditions are intermittent, they do not manifest
10155themselves every time you do the build. In fact, most times the build
10156will complete without problems even though the potential race condition
10157exists. Thus, once the error surfaces, you need a way to reproduce it.
10158
10159In this example, compiling the "neard" package is causing the problem.
10160So the first thing to do is build "neard" locally. Before you start the
10161build, set the
10162:term:`PARALLEL_MAKE` variable
10163in your ``local.conf`` file to a high number (e.g. "-j 20"). Using a
10164high value for ``PARALLEL_MAKE`` increases the chances of the race
10165condition showing up:
10166::
10167
10168 $ bitbake neard
10169
10170Once the local build for "neard" completes, start a ``devshell`` build:
10171::
10172
10173 $ bitbake neard -c devshell
10174
10175For information on how to use a
10176``devshell``, see the "`Using a Development
10177Shell <#platdev-appdev-devshell>`__" section.
10178
10179In the ``devshell``, do the following:
10180::
10181
10182 $ make clean
10183 $ make tools/snep-send.o
10184
10185The ``devshell`` commands cause the failure to clearly
10186be visible. In this case, a missing dependency exists for the "neard"
10187Makefile target. Here is some abbreviated, sample output with the
10188missing dependency clearly visible at the end:
10189::
10190
10191 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/......
10192 .
10193 .
10194 .
10195 tools/snep-send.c
10196 In file included from tools/snep-send.c:16:0:
10197 tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory
10198 #include <near/dbus.h>
10199 ^
10200 compilation terminated.
10201 make: *** [tools/snep-send.o] Error 1
10202 $
10203
10204
10205Creating a Patch for the Fix
10206~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10207
10208Because there is a missing dependency for the Makefile target, you need
10209to patch the ``Makefile.am`` file, which is generated from
10210``Makefile.in``. You can use Quilt to create the patch:
10211::
10212
10213 $ quilt new parallelmake.patch
10214 Patch patches/parallelmake.patch is now on top
10215 $ quilt add Makefile.am
10216 File Makefile.am added to patch patches/parallelmake.patch
10217
10218For more information on using Quilt, see the
10219"`Using Quilt in Your Workflow <#using-a-quilt-workflow>`__" section.
10220
10221At this point you need to make the edits to ``Makefile.am`` to add the
10222missing dependency. For our example, you have to add the following line
10223to the file:
10224::
10225
10226 tools/snep-send.$(OBJEXT): include/near/dbus.h
10227
10228Once you have edited the file, use the ``refresh`` command to create the
10229patch:
10230::
10231
10232 $ quilt refresh
10233 Refreshed patch patches/parallelmake.patch
10234
10235Once
10236the patch file exists, you need to add it back to the originating recipe
10237folder. Here is an example assuming a top-level
10238:term:`Source Directory` named ``poky``:
10239::
10240
10241 $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard
10242
10243The final thing you need to do to implement the fix in the build is to
10244update the "neard" recipe (i.e. ``neard-0.14.bb``) so that the
10245:term:`SRC_URI` statement includes
10246the patch file. The recipe file is in the folder above the patch. Here
10247is what the edited ``SRC_URI`` statement would look like:
10248::
10249
10250 SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \
10251 file://neard.in \
10252 file://neard.service.in \
10253 file://parallelmake.patch \
10254 "
10255
10256With the patch complete and moved to the correct folder and the
10257``SRC_URI`` statement updated, you can exit the ``devshell``:
10258::
10259
10260 $ exit
10261
10262Testing the Build
10263~~~~~~~~~~~~~~~~~
10264
10265With everything in place, you can get back to trying the build again
10266locally:
10267::
10268
10269 $ bitbake neard This build should succeed.
10270
10271Now you can open up a ``devshell`` again and repeat the clean and make
10272operations as follows:
10273::
10274
10275 $ bitbake neard -c devshell
10276 $ make clean
10277 $ make tools/snep-send.o
10278
10279The build should work without issue.
10280
10281As with all solved problems, if they originated upstream, you need to
10282submit the fix for the recipe in OE-Core and upstream so that the
10283problem is taken care of at its source. See the "`Submitting a Change to
10284the Yocto Project <#how-to-submit-a-change>`__" section for more
10285information.
10286
10287.. _platdev-gdb-remotedebug:
10288
10289Debugging With the GNU Project Debugger (GDB) Remotely
10290------------------------------------------------------
10291
10292GDB allows you to examine running programs, which in turn helps you to
10293understand and fix problems. It also allows you to perform post-mortem
10294style analysis of program crashes. GDB is available as a package within
10295the Yocto Project and is installed in SDK images by default. See the
10296":ref:`ref-manual/ref-images:Images`" chapter in the Yocto
10297Project Reference Manual for a description of these images. You can find
10298information on GDB at http://sourceware.org/gdb/.
10299
10300.. note::
10301
10302 For best results, install debug (
10303 -dbg
10304 ) packages for the applications you are going to debug. Doing so
10305 makes extra debug symbols available that give you more meaningful
10306 output.
10307
10308Sometimes, due to memory or disk space constraints, it is not possible
10309to use GDB directly on the remote target to debug applications. These
10310constraints arise because GDB needs to load the debugging information
10311and the binaries of the process being debugged. Additionally, GDB needs
10312to perform many computations to locate information such as function
10313names, variable names and values, stack traces and so forth - even
10314before starting the debugging process. These extra computations place
10315more load on the target system and can alter the characteristics of the
10316program being debugged.
10317
10318To help get past the previously mentioned constraints, you can use
10319gdbserver, which runs on the remote target and does not load any
10320debugging information from the debugged process. Instead, a GDB instance
10321processes the debugging information that is run on a remote computer -
10322the host GDB. The host GDB then sends control commands to gdbserver to
10323make it stop or start the debugged program, as well as read or write
10324memory regions of that debugged program. All the debugging information
10325loaded and processed as well as all the heavy debugging is done by the
10326host GDB. Offloading these processes gives the gdbserver running on the
10327target a chance to remain small and fast.
10328
10329Because the host GDB is responsible for loading the debugging
10330information and for doing the necessary processing to make actual
10331debugging happen, you have to make sure the host can access the
10332unstripped binaries complete with their debugging information and also
10333be sure the target is compiled with no optimizations. The host GDB must
10334also have local access to all the libraries used by the debugged
10335program. Because gdbserver does not need any local debugging
10336information, the binaries on the remote target can remain stripped.
10337However, the binaries must also be compiled without optimization so they
10338match the host's binaries.
10339
10340To remain consistent with GDB documentation and terminology, the binary
10341being debugged on the remote target machine is referred to as the
10342"inferior" binary. For documentation on GDB see the `GDB
10343site <http://sourceware.org/gdb/documentation/>`__.
10344
10345The following steps show you how to debug using the GNU project
10346debugger.
10347
103481. *Configure your build system to construct the companion debug
10349 filesystem:*
10350
10351 In your ``local.conf`` file, set the following:
10352 ::
10353
10354 IMAGE_GEN_DEBUGFS = "1"
10355 IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
10356
10357 These options cause the
10358 OpenEmbedded build system to generate a special companion filesystem
10359 fragment, which contains the matching source and debug symbols to
10360 your deployable filesystem. The build system does this by looking at
10361 what is in the deployed filesystem, and pulling the corresponding
10362 ``-dbg`` packages.
10363
10364 The companion debug filesystem is not a complete filesystem, but only
10365 contains the debug fragments. This filesystem must be combined with
10366 the full filesystem for debugging. Subsequent steps in this procedure
10367 show how to combine the partial filesystem with the full filesystem.
10368
103692. *Configure the system to include gdbserver in the target filesystem:*
10370
10371 Make the following addition in either your ``local.conf`` file or in
10372 an image recipe:
10373 ::
10374
10375 IMAGE_INSTALL_append = " gdbserver"
10376
10377 The change makes
10378 sure the ``gdbserver`` package is included.
10379
103803. *Build the environment:*
10381
10382 Use the following command to construct the image and the companion
10383 Debug Filesystem:
10384 ::
10385
10386 $ bitbake image
10387
10388 Build the cross GDB component and
10389 make it available for debugging. Build the SDK that matches the
10390 image. Building the SDK is best for a production build that can be
10391 used later for debugging, especially during long term maintenance:
10392 ::
10393
10394 $ bitbake -c populate_sdk image
10395
10396 Alternatively, you can build the minimal toolchain components that
10397 match the target. Doing so creates a smaller than typical SDK and
10398 only contains a minimal set of components with which to build simple
10399 test applications, as well as run the debugger:
10400 ::
10401
10402 $ bitbake meta-toolchain
10403
10404 A final method is to build Gdb itself within the build system:
10405 ::
10406
10407 $ bitbake gdb-cross-<architecture>
10408
10409 Doing so produces a temporary copy of
10410 ``cross-gdb`` you can use for debugging during development. While
10411 this is the quickest approach, the two previous methods in this step
10412 are better when considering long-term maintenance strategies.
10413
10414 .. note::
10415
10416 If you run
10417 bitbake gdb-cross
10418 , the OpenEmbedded build system suggests the actual image (e.g.
10419 gdb-cross-i586
10420 ). The suggestion is usually the actual name you want to use.
10421
104224. *Set up the* ``debugfs``
10423
10424 Run the following commands to set up the ``debugfs``:
10425 ::
10426
10427 $ mkdir debugfs
10428 $ cd debugfs
10429 $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2
10430 $ tar xvfj build-dir/tmp-glibc/deploy/images/machine/image-dbg.rootfs.tar.bz2
10431
104325. *Set up GDB*
10433
10434 Install the SDK (if you built one) and then source the correct
10435 environment file. Sourcing the environment file puts the SDK in your
10436 ``PATH`` environment variable.
10437
10438 If you are using the build system, Gdb is located in
10439 build-dir/tmp/sysroots/host/usr/bin/architecture/architecture-gdb
10440
104416. *Boot the target:*
10442
10443 For information on how to run QEMU, see the `QEMU
10444 Documentation <http://wiki.qemu.org/Documentation/GettingStartedDevelopers>`__.
10445
10446 .. note::
10447
10448 Be sure to verify that your host can access the target via TCP.
10449
104507. *Debug a program:*
10451
10452 Debugging a program involves running gdbserver on the target and then
10453 running Gdb on the host. The example in this step debugs ``gzip``:
10454 ::
10455
10456 root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help
10457
10458 For
10459 additional gdbserver options, see the `GDB Server
10460 Documentation <https://www.gnu.org/software/gdb/documentation/>`__.
10461
10462 After running gdbserver on the target, you need to run Gdb on the
10463 host and configure it and connect to the target. Use these commands:
10464 ::
10465
10466 $ cd directory-holding-the-debugfs-directory
10467 $ arch-gdb
10468 (gdb) set sysroot debugfs
10469 (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug
10470 (gdb) target remote IP-of-target:1234
10471
10472 At this
10473 point, everything should automatically load (i.e. matching binaries,
10474 symbols and headers).
10475
10476 .. note::
10477
10478 The Gdb
10479 set
10480 commands in the previous example can be placed into the users
10481 ~/.gdbinit
10482 file. Upon starting, Gdb automatically runs whatever commands are
10483 in that file.
10484
104858. *Deploying without a full image rebuild:*
10486
10487 In many cases, during development you want a quick method to deploy a
10488 new binary to the target and debug it, without waiting for a full
10489 image build.
10490
10491 One approach to solving this situation is to just build the component
10492 you want to debug. Once you have built the component, copy the
10493 executable directly to both the target and the host ``debugfs``.
10494
10495 If the binary is processed through the debug splitting in
10496 OpenEmbedded, you should also copy the debug items (i.e. ``.debug``
10497 contents and corresponding ``/usr/src/debug`` files) from the work
10498 directory. Here is an example:
10499 ::
10500
10501 $ bitbake bash
10502 $ bitbake -c devshell bash
10503 $ cd ..
10504 $ scp packages-split/bash/bin/bash target:/bin/bash
10505 $ cp -a packages-split/bash-dbg/\* path/debugfs
10506
10507Debugging with the GNU Project Debugger (GDB) on the Target
10508-----------------------------------------------------------
10509
10510The previous section addressed using GDB remotely for debugging
10511purposes, which is the most usual case due to the inherent hardware
10512limitations on many embedded devices. However, debugging in the target
10513hardware itself is also possible with more powerful devices. This
10514section describes what you need to do in order to support using GDB to
10515debug on the target hardware.
10516
10517To support this kind of debugging, you need do the following:
10518
10519- Ensure that GDB is on the target. You can do this by adding "gdb" to
10520 :term:`IMAGE_INSTALL`:
10521 IMAGE_INSTALL_append = " gdb" Alternatively, you can add
10522 "tools-debug" to
10523 :term:`IMAGE_FEATURES`:
10524 ::
10525
10526 IMAGE_FEATURES_append = " tools-debug"
10527
10528- Ensure that debug symbols are present. You can make sure these
10529 symbols are present by installing ``-dbg``:
10530 ::
10531
10532 IMAGE_INSTALL_append = "packagename-dbg"
10533
10534 Alternatively, you can do the following to include
10535 all the debug symbols:
10536 ::
10537
10538 IMAGE_FEATURES_append = " dbg-pkgs"
10539
10540.. note::
10541
10542 To improve the debug information accuracy, you can reduce the level
10543 of optimization used by the compiler. For example, when adding the
10544 following line to your
10545 local.conf
10546 file, you will reduce optimization from
10547 FULL_OPTIMIZATION
10548 of "-O2" to
10549 DEBUG_OPTIMIZATION
10550 of "-O -fno-omit-frame-pointer":
10551 ::
10552
10553 DEBUG_BUILD = "1"
10554
10555
10556 Consider that this will reduce the application's performance and is
10557 recommended only for debugging purposes.
10558
10559.. _dev-other-debugging-others:
10560
10561Other Debugging Tips
10562--------------------
10563
10564Here are some other tips that you might find useful:
10565
10566- When adding new packages, it is worth watching for undesirable items
10567 making their way into compiler command lines. For example, you do not
10568 want references to local system files like ``/usr/lib/`` or
10569 ``/usr/include/``.
10570
10571- If you want to remove the ``psplash`` boot splashscreen, add
10572 ``psplash=false`` to the kernel command line. Doing so prevents
10573 ``psplash`` from loading and thus allows you to see the console. It
10574 is also possible to switch out of the splashscreen by switching the
10575 virtual console (e.g. Fn+Left or Fn+Right on a Zaurus).
10576
10577- Removing :term:`TMPDIR` (usually
10578 ``tmp/``, within the
10579 :term:`Build Directory`) can often fix
10580 temporary build issues. Removing ``TMPDIR`` is usually a relatively
10581 cheap operation, because task output will be cached in
10582 :term:`SSTATE_DIR` (usually
10583 ``sstate-cache/``, which is also in the Build Directory).
10584
10585 .. note::
10586
10587 Removing
10588 TMPDIR
10589 might be a workaround rather than a fix. Consequently, trying to
10590 determine the underlying cause of an issue before removing the
10591 directory is a good idea.
10592
10593- Understanding how a feature is used in practice within existing
10594 recipes can be very helpful. It is recommended that you configure
10595 some method that allows you to quickly search through files.
10596
10597 Using GNU Grep, you can use the following shell function to
10598 recursively search through common recipe-related files, skipping
10599 binary files, ``.git`` directories, and the Build Directory (assuming
10600 its name starts with "build"):
10601 ::
10602
10603 g() {
10604 grep -Ir \
10605 --exclude-dir=.git \
10606 --exclude-dir='build*' \
10607 --include='*.bb*' \
10608 --include='*.inc*' \
10609 --include='*.conf*' \
10610 --include='*.py*' \
10611 "$@"
10612 }
10613
10614 Following are some usage examples:
10615 ::
10616
10617 $ g FOO # Search recursively for "FOO"
10618 $ g -i foo # Search recursively for "foo", ignoring case
10619 $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR"
10620
10621 If figuring
10622 out how some feature works requires a lot of searching, it might
10623 indicate that the documentation should be extended or improved. In
10624 such cases, consider filing a documentation bug using the Yocto
10625 Project implementation of
10626 :yocto_bugs:`Bugzilla <>`. For information on
10627 how to submit a bug against the Yocto Project, see the Yocto Project
10628 Bugzilla :yocto_wiki:`wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`
10629 and the "`Submitting a Defect Against the Yocto
10630 Project <#submitting-a-defect-against-the-yocto-project>`__" section.
10631
10632 .. note::
10633
10634 The manuals might not be the right place to document variables
10635 that are purely internal and have a limited scope (e.g. internal
10636 variables used to implement a single
10637 .bbclass
10638 file).
10639
10640Making Changes to the Yocto Project
10641===================================
10642
10643Because the Yocto Project is an open-source, community-based project,
10644you can effect changes to the project. This section presents procedures
10645that show you how to submit a defect against the project and how to
10646submit a change.
10647
10648Submitting a Defect Against the Yocto Project
10649---------------------------------------------
10650
10651Use the Yocto Project implementation of
10652`Bugzilla <http://www.bugzilla.org/about/>`__ to submit a defect (bug)
10653against the Yocto Project. For additional information on this
10654implementation of Bugzilla see the :ref:"`Yocto Project
10655Bugzilla <resources-bugtracker>`" section in the
10656Yocto Project Reference Manual. For more detail on any of the following
10657steps, see the Yocto Project
10658:yocto_wiki:`Bugzilla wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`.
10659
10660Use the following general steps to submit a bug"
10661
106621. Open the Yocto Project implementation of :yocto_bugs:`Bugzilla <>`.
10663
106642. Click "File a Bug" to enter a new bug.
10665
106663. Choose the appropriate "Classification", "Product", and "Component"
10667 for which the bug was found. Bugs for the Yocto Project fall into
10668 one of several classifications, which in turn break down into
10669 several products and components. For example, for a bug against the
10670 ``meta-intel`` layer, you would choose "Build System, Metadata &
10671 Runtime", "BSPs", and "bsps-meta-intel", respectively.
10672
106734. Choose the "Version" of the Yocto Project for which you found the
10674 bug (e.g. DISTRO).
10675
106765. Determine and select the "Severity" of the bug. The severity
10677 indicates how the bug impacted your work.
10678
106796. Choose the "Hardware" that the bug impacts.
10680
106817. Choose the "Architecture" that the bug impacts.
10682
106838. Choose a "Documentation change" item for the bug. Fixing a bug might
10684 or might not affect the Yocto Project documentation. If you are
10685 unsure of the impact to the documentation, select "Don't Know".
10686
106879. Provide a brief "Summary" of the bug. Try to limit your summary to
10688 just a line or two and be sure to capture the essence of the bug.
10689
1069010. Provide a detailed "Description" of the bug. You should provide as
10691 much detail as you can about the context, behavior, output, and so
10692 forth that surrounds the bug. You can even attach supporting files
10693 for output from logs by using the "Add an attachment" button.
10694
1069511. Click the "Submit Bug" button submit the bug. A new Bugzilla number
10696 is assigned to the bug and the defect is logged in the bug tracking
10697 system.
10698
10699Once you file a bug, the bug is processed by the Yocto Project Bug
10700Triage Team and further details concerning the bug are assigned (e.g.
10701priority and owner). You are the "Submitter" of the bug and any further
10702categorization, progress, or comments on the bug result in Bugzilla
10703sending you an automated email concerning the particular change or
10704progress to the bug.
10705
10706.. _how-to-submit-a-change:
10707
10708Submitting a Change to the Yocto Project
10709----------------------------------------
10710
10711Contributions to the Yocto Project and OpenEmbedded are very welcome.
10712Because the system is extremely configurable and flexible, we recognize
10713that developers will want to extend, configure or optimize it for their
10714specific uses.
10715
10716The Yocto Project uses a mailing list and a patch-based workflow that is
10717similar to the Linux kernel but contains important differences. In
10718general, a mailing list exists through which you can submit patches. You
10719should send patches to the appropriate mailing list so that they can be
10720reviewed and merged by the appropriate maintainer. The specific mailing
10721list you need to use depends on the location of the code you are
10722changing. Each component (e.g. layer) should have a ``README`` file that
10723indicates where to send the changes and which process to follow.
10724
10725You can send the patch to the mailing list using whichever approach you
10726feel comfortable with to generate the patch. Once sent, the patch is
10727usually reviewed by the community at large. If somebody has concerns
10728with the patch, they will usually voice their concern over the mailing
10729list. If a patch does not receive any negative reviews, the maintainer
10730of the affected layer typically takes the patch, tests it, and then
10731based on successful testing, merges the patch.
10732
10733The "poky" repository, which is the Yocto Project's reference build
10734environment, is a hybrid repository that contains several individual
10735pieces (e.g. BitBake, Metadata, documentation, and so forth) built using
10736the combo-layer tool. The upstream location used for submitting changes
10737varies by component:
10738
10739- *Core Metadata:* Send your patch to the
10740 `openembedded-core <http://lists.openembedded.org/mailman/listinfo/openembedded-core>`__
10741 mailing list. For example, a change to anything under the ``meta`` or
10742 ``scripts`` directories should be sent to this mailing list.
10743
10744- *BitBake:* For changes to BitBake (i.e. anything under the
10745 ``bitbake`` directory), send your patch to the
10746 `bitbake-devel <http://lists.openembedded.org/mailman/listinfo/bitbake-devel>`__
10747 mailing list.
10748
10749- *"meta-\*" trees:* These trees contain Metadata. Use the
10750 `poky <https://lists.yoctoproject.org/g/poky>`__ mailing list.
10751
10752- *Documentation*: For changes to the Yocto Project documentation, use the `docs
10753 <https://lists.yoctoproject.org/g/docs>`__ mailing list.
10754
10755For changes to other layers hosted in the Yocto Project source
10756repositories (i.e. ``yoctoproject.org``) and tools use the `Yocto Project
10757<https://lists.yoctoproject.org/g/yocto/>`__ general mailing list.
10758
10759.. note::
10760
10761 Sometimes a layer's documentation specifies to use a particular
10762 mailing list. If so, use that list.
10763
10764For additional recipes that do not fit into the core Metadata, you
10765should determine which layer the recipe should go into and submit the
10766change in the manner recommended by the documentation (e.g. the
10767``README`` file) supplied with the layer. If in doubt, please ask on the
10768Yocto general mailing list or on the openembedded-devel mailing list.
10769
10770You can also push a change upstream and request a maintainer to pull the
10771change into the component's upstream repository. You do this by pushing
10772to a contribution repository that is upstream. See the ":ref:`gs-git-workflows-and-the-yocto-project`"
10773section in the Yocto Project Overview and Concepts Manual for additional
10774concepts on working in the Yocto Project development environment.
10775
10776Two commonly used testing repositories exist for OpenEmbedded-Core:
10777
10778- *"ross/mut" branch:* The "mut" (master-under-test) tree exists in the
10779 ``poky-contrib`` repository in the
10780 :yocto_git:`Yocto Project source repositories <>`.
10781
10782- *"master-next" branch:* This branch is part of the main "poky"
10783 repository in the Yocto Project source repositories.
10784
10785Maintainers use these branches to test submissions prior to merging
10786patches. Thus, you can get an idea of the status of a patch based on
10787whether the patch has been merged into one of these branches.
10788
10789.. note::
10790
10791 This system is imperfect and changes can sometimes get lost in the
10792 flow. Asking about the status of a patch or change is reasonable if
10793 the change has been idle for a while with no feedback. The Yocto
10794 Project does have plans to use
10795 Patchwork
10796 to track the status of patches and also to automatically preview
10797 patches.
10798
10799The following sections provide procedures for submitting a change.
10800
10801.. _pushing-a-change-upstream:
10802
10803Using Scripts to Push a Change Upstream and Request a Pull
10804~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10805
10806Follow this procedure to push a change to an upstream "contrib" Git
10807repository:
10808
10809.. note::
10810
10811 You can find general Git information on how to push a change upstream
10812 in the
10813 Git Community Book
10814 .
10815
108161. *Make Your Changes Locally:* Make your changes in your local Git
10817 repository. You should make small, controlled, isolated changes.
10818 Keeping changes small and isolated aids review, makes
10819 merging/rebasing easier and keeps the change history clean should
10820 anyone need to refer to it in future.
10821
108222. *Stage Your Changes:* Stage your changes by using the ``git add``
10823 command on each file you changed.
10824
108253. *Commit Your Changes:* Commit the change by using the ``git commit``
10826 command. Make sure your commit information follows standards by
10827 following these accepted conventions:
10828
10829 - Be sure to include a "Signed-off-by:" line in the same style as
10830 required by the Linux kernel. Adding this line signifies that you,
10831 the submitter, have agreed to the Developer's Certificate of
10832 Origin 1.1 as follows:
10833 ::
10834
10835 Developer's Certificate of Origin 1.1
10836
10837 By making a contribution to this project, I certify that:
10838
10839 (a) The contribution was created in whole or in part by me and I
10840 have the right to submit it under the open source license
10841 indicated in the file; or
10842
10843 (b) The contribution is based upon previous work that, to the best
10844 of my knowledge, is covered under an appropriate open source
10845 license and I have the right under that license to submit that
10846 work with modifications, whether created in whole or in part
10847 by me, under the same open source license (unless I am
10848 permitted to submit under a different license), as indicated
10849 in the file; or
10850
10851 (c) The contribution was provided directly to me by some other
10852 person who certified (a), (b) or (c) and I have not modified
10853 it.
10854
10855 (d) I understand and agree that this project and the contribution
10856 are public and that a record of the contribution (including all
10857 personal information I submit with it, including my sign-off) is
10858 maintained indefinitely and may be redistributed consistent with
10859 this project or the open source license(s) involved.
10860
10861 - Provide a single-line summary of the change. and, if more
10862 explanation is needed, provide more detail in the body of the
10863 commit. This summary is typically viewable in the "shortlist" of
10864 changes. Thus, providing something short and descriptive that
10865 gives the reader a summary of the change is useful when viewing a
10866 list of many commits. You should prefix this short description
10867 with the recipe name (if changing a recipe), or else with the
10868 short form path to the file being changed.
10869
10870 - For the body of the commit message, provide detailed information
10871 that describes what you changed, why you made the change, and the
10872 approach you used. It might also be helpful if you mention how you
10873 tested the change. Provide as much detail as you can in the body
10874 of the commit message.
10875
10876 .. note::
10877
10878 You do not need to provide a more detailed explanation of a
10879 change if the change is minor to the point of the single line
10880 summary providing all the information.
10881
10882 - If the change addresses a specific bug or issue that is associated
10883 with a bug-tracking ID, include a reference to that ID in your
10884 detailed description. For example, the Yocto Project uses a
10885 specific convention for bug references - any commit that addresses
10886 a specific bug should use the following form for the detailed
10887 description. Be sure to use the actual bug-tracking ID from
10888 Bugzilla for bug-id:
10889 ::
10890
10891 Fixes [YOCTO #bug-id]
10892
10893 detailed description of change
10894
108954. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for
10896 permissions to push to an upstream contrib repository, push the
10897 change to that repository:
10898 ::
10899
10900 $ git push upstream_remote_repo local_branch_name
10901
10902 For example, suppose you have permissions to push
10903 into the upstream ``meta-intel-contrib`` repository and you are
10904 working in a local branch named your_name\ ``/README``. The following
10905 command pushes your local commits to the ``meta-intel-contrib``
10906 upstream repository and puts the commit in a branch named
10907 your_name\ ``/README``:
10908 ::
10909
10910 $ git push meta-intel-contrib your_name/README
10911
109125. *Determine Who to Notify:* Determine the maintainer or the mailing
10913 list that you need to notify for the change.
10914
10915 Before submitting any change, you need to be sure who the maintainer
10916 is or what mailing list that you need to notify. Use either these
10917 methods to find out:
10918
10919 - *Maintenance File:* Examine the ``maintainers.inc`` file, which is
10920 located in the :term:`Source Directory` at
10921 ``meta/conf/distro/include``, to see who is responsible for code.
10922
10923 - *Search by File:* Using :ref:`overview-manual/overview-manual-development-environment:git`, you can
10924 enter the following command to bring up a short list of all
10925 commits against a specific file:
10926 ::
10927
10928 git shortlog -- filename
10929
10930 Just provide the name of the file for which you are interested. The
10931 information returned is not ordered by history but does include a
10932 list of everyone who has committed grouped by name. From the list,
10933 you can see who is responsible for the bulk of the changes against
10934 the file.
10935
10936 - *Examine the List of Mailing Lists:* For a list of the Yocto
10937 Project and related mailing lists, see the ":ref:`Mailing
10938 lists <resources-mailinglist>`" section in
10939 the Yocto Project Reference Manual.
10940
109416. *Make a Pull Request:* Notify the maintainer or the mailing list that
10942 you have pushed a change by making a pull request.
10943
10944 The Yocto Project provides two scripts that conveniently let you
10945 generate and send pull requests to the Yocto Project. These scripts
10946 are ``create-pull-request`` and ``send-pull-request``. You can find
10947 these scripts in the ``scripts`` directory within the
10948 :term:`Source Directory` (e.g.
10949 ``~/poky/scripts``).
10950
10951 Using these scripts correctly formats the requests without
10952 introducing any whitespace or HTML formatting. The maintainer that
10953 receives your patches either directly or through the mailing list
10954 needs to be able to save and apply them directly from your emails.
10955 Using these scripts is the preferred method for sending patches.
10956
10957 First, create the pull request. For example, the following command
10958 runs the script, specifies the upstream repository in the contrib
10959 directory into which you pushed the change, and provides a subject
10960 line in the created patch files:
10961 ::
10962
10963 $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README"
10964
10965 Running this script forms ``*.patch`` files in a folder named
10966 ``pull-``\ PID in the current directory. One of the patch files is a
10967 cover letter.
10968
10969 Before running the ``send-pull-request`` script, you must edit the
10970 cover letter patch to insert information about your change. After
10971 editing the cover letter, send the pull request. For example, the
10972 following command runs the script and specifies the patch directory
10973 and email address. In this example, the email address is a mailing
10974 list:
10975 ::
10976
10977 $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org
10978
10979 You need to follow the prompts as the script is interactive.
10980
10981 .. note::
10982
10983 For help on using these scripts, simply provide the
10984 -h
10985 argument as follows:
10986 ::
10987
10988 $ poky/scripts/create-pull-request -h
10989 $ poky/scripts/send-pull-request -h
10990
10991
10992.. _submitting-a-patch:
10993
10994Using Email to Submit a Patch
10995~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10996
10997You can submit patches without using the ``create-pull-request`` and
10998``send-pull-request`` scripts described in the previous section.
10999However, keep in mind, the preferred method is to use the scripts.
11000
11001Depending on the components changed, you need to submit the email to a
11002specific mailing list. For some guidance on which mailing list to use,
11003see the `list <#figuring-out-the-mailing-list-to-use>`__ at the
11004beginning of this section. For a description of all the available
11005mailing lists, see the ":ref:`Mailing Lists <resources-mailinglist>`" section in the
11006Yocto Project Reference Manual.
11007
11008Here is the general procedure on how to submit a patch through email
11009without using the scripts:
11010
110111. *Make Your Changes Locally:* Make your changes in your local Git
11012 repository. You should make small, controlled, isolated changes.
11013 Keeping changes small and isolated aids review, makes
11014 merging/rebasing easier and keeps the change history clean should
11015 anyone need to refer to it in future.
11016
110172. *Stage Your Changes:* Stage your changes by using the ``git add``
11018 command on each file you changed.
11019
110203. *Commit Your Changes:* Commit the change by using the
11021 ``git commit --signoff`` command. Using the ``--signoff`` option
11022 identifies you as the person making the change and also satisfies the
11023 Developer's Certificate of Origin (DCO) shown earlier.
11024
11025 When you form a commit, you must follow certain standards established
11026 by the Yocto Project development team. See `Step
11027 3 <#making-sure-you-have-correct-commit-information>`__ in the
11028 previous section for information on how to provide commit information
11029 that meets Yocto Project commit message standards.
11030
110314. *Format the Commit:* Format the commit into an email message. To
11032 format commits, use the ``git format-patch`` command. When you
11033 provide the command, you must include a revision list or a number of
11034 patches as part of the command. For example, either of these two
11035 commands takes your most recent single commit and formats it as an
11036 email message in the current directory:
11037 ::
11038
11039 $ git format-patch -1
11040
11041 or ::
11042
11043 $ git format-patch HEAD~
11044
11045 After the command is run, the current directory contains a numbered
11046 ``.patch`` file for the commit.
11047
11048 If you provide several commits as part of the command, the
11049 ``git format-patch`` command produces a series of numbered files in
11050 the current directory – one for each commit. If you have more than
11051 one patch, you should also use the ``--cover`` option with the
11052 command, which generates a cover letter as the first "patch" in the
11053 series. You can then edit the cover letter to provide a description
11054 for the series of patches. For information on the
11055 ``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed
11056 using the ``man git-format-patch`` command.
11057
11058 .. note::
11059
11060 If you are or will be a frequent contributor to the Yocto Project
11061 or to OpenEmbedded, you might consider requesting a contrib area
11062 and the necessary associated rights.
11063
110645. *Import the Files Into Your Mail Client:* Import the files into your
11065 mail client by using the ``git send-email`` command.
11066
11067 .. note::
11068
11069 In order to use
11070 git send-email
11071 , you must have the proper Git packages installed on your host.
11072 For Ubuntu, Debian, and Fedora the package is
11073 git-email
11074 .
11075
11076 The ``git send-email`` command sends email by using a local or remote
11077 Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or
11078 through a direct ``smtp`` configuration in your Git ``~/.gitconfig``
11079 file. If you are submitting patches through email only, it is very
11080 important that you submit them without any whitespace or HTML
11081 formatting that either you or your mailer introduces. The maintainer
11082 that receives your patches needs to be able to save and apply them
11083 directly from your emails. A good way to verify that what you are
11084 sending will be applicable by the maintainer is to do a dry run and
11085 send them to yourself and then save and apply them as the maintainer
11086 would.
11087
11088 The ``git send-email`` command is the preferred method for sending
11089 your patches using email since there is no risk of compromising
11090 whitespace in the body of the message, which can occur when you use
11091 your own mail client. The command also has several options that let
11092 you specify recipients and perform further editing of the email
11093 message. For information on how to use the ``git send-email``
11094 command, see ``GIT-SEND-EMAIL(1)`` displayed using the
11095 ``man git-send-email`` command.
11096
11097Working With Licenses
11098=====================
11099
11100As mentioned in the ":ref:`overview-manual/overview-manual-development-environment:licensing`"
11101section in the Yocto Project Overview and Concepts Manual, open source
11102projects are open to the public and they consequently have different
11103licensing structures in place. This section describes the mechanism by
11104which the :term:`OpenEmbedded Build System`
11105tracks changes to
11106licensing text and covers how to maintain open source license compliance
11107during your project's lifecycle. The section also describes how to
11108enable commercially licensed recipes, which by default are disabled.
11109
11110.. _usingpoky-configuring-LIC_FILES_CHKSUM:
11111
11112Tracking License Changes
11113------------------------
11114
11115The license of an upstream project might change in the future. In order
11116to prevent these changes going unnoticed, the
11117:term:`LIC_FILES_CHKSUM`
11118variable tracks changes to the license text. The checksums are validated
11119at the end of the configure step, and if the checksums do not match, the
11120build will fail.
11121
11122.. _usingpoky-specifying-LIC_FILES_CHKSUM:
11123
11124Specifying the ``LIC_FILES_CHKSUM`` Variable
11125~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11126
11127The ``LIC_FILES_CHKSUM`` variable contains checksums of the license text
11128in the source code for the recipe. Following is an example of how to
11129specify ``LIC_FILES_CHKSUM``:
11130::
11131
11132 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
11133 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
11134 file://licfile2.txt;endline=50;md5=zzzz \
11135 ..."
11136
11137.. note::
11138
11139 - When using "beginline" and "endline", realize that line numbering
11140 begins with one and not zero. Also, the included lines are
11141 inclusive (i.e. lines five through and including 29 in the
11142 previous example for ``licfile1.txt``).
11143
11144 - When a license check fails, the selected license text is included
11145 as part of the QA message. Using this output, you can determine
11146 the exact start and finish for the needed license text.
11147
11148The build system uses the :term:`S`
11149variable as the default directory when searching files listed in
11150``LIC_FILES_CHKSUM``. The previous example employs the default
11151directory.
11152
11153Consider this next example:
11154::
11155
11156 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
11157 md5=bb14ed3c4cda583abc85401304b5cd4e"
11158 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
11159
11160The first line locates a file in ``${S}/src/ls.c`` and isolates lines
11161five through 16 as license text. The second line refers to a file in
11162:term:`WORKDIR`.
11163
11164Note that ``LIC_FILES_CHKSUM`` variable is mandatory for all recipes,
11165unless the ``LICENSE`` variable is set to "CLOSED".
11166
11167.. _usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax:
11168
11169Explanation of Syntax
11170~~~~~~~~~~~~~~~~~~~~~
11171
11172As mentioned in the previous section, the ``LIC_FILES_CHKSUM`` variable
11173lists all the important files that contain the license text for the
11174source code. It is possible to specify a checksum for an entire file, or
11175a specific section of a file (specified by beginning and ending line
11176numbers with the "beginline" and "endline" parameters, respectively).
11177The latter is useful for source files with a license notice header,
11178README documents, and so forth. If you do not use the "beginline"
11179parameter, then it is assumed that the text begins on the first line of
11180the file. Similarly, if you do not use the "endline" parameter, it is
11181assumed that the license text ends with the last line of the file.
11182
11183The "md5" parameter stores the md5 checksum of the license text. If the
11184license text changes in any way as compared to this parameter then a
11185mismatch occurs. This mismatch triggers a build failure and notifies the
11186developer. Notification allows the developer to review and address the
11187license text changes. Also note that if a mismatch occurs during the
11188build, the correct md5 checksum is placed in the build log and can be
11189easily copied to the recipe.
11190
11191There is no limit to how many files you can specify using the
11192``LIC_FILES_CHKSUM`` variable. Generally, however, every project
11193requires a few specifications for license tracking. Many projects have a
11194"COPYING" file that stores the license information for all the source
11195code files. This practice allows you to just track the "COPYING" file as
11196long as it is kept up to date.
11197
11198.. note::
11199
11200 - If you specify an empty or invalid "md5" parameter,
11201 :term:`BitBake` returns an md5
11202 mis-match error and displays the correct "md5" parameter value
11203 during the build. The correct parameter is also captured in the
11204 build log.
11205
11206 - If the whole file contains only license text, you do not need to
11207 use the "beginline" and "endline" parameters.
11208
11209Enabling Commercially Licensed Recipes
11210--------------------------------------
11211
11212By default, the OpenEmbedded build system disables components that have
11213commercial or other special licensing requirements. Such requirements
11214are defined on a recipe-by-recipe basis through the
11215:term:`LICENSE_FLAGS` variable
11216definition in the affected recipe. For instance, the
11217``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
11218contains the following statement:
11219::
11220
11221 LICENSE_FLAGS = "commercial"
11222
11223Here is a
11224slightly more complicated example that contains both an explicit recipe
11225name and version (after variable expansion):
11226::
11227
11228 LICENSE_FLAGS = "license_${PN}_${PV}"
11229
11230In order for a component restricted by a
11231``LICENSE_FLAGS`` definition to be enabled and included in an image, it
11232needs to have a matching entry in the global
11233:term:`LICENSE_FLAGS_WHITELIST`
11234variable, which is a variable typically defined in your ``local.conf``
11235file. For example, to enable the
11236``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
11237could add either the string "commercial_gst-plugins-ugly" or the more
11238general string "commercial" to ``LICENSE_FLAGS_WHITELIST``. See the
11239"`License Flag Matching <#license-flag-matching>`__" section for a full
11240explanation of how ``LICENSE_FLAGS`` matching works. Here is the
11241example:
11242::
11243
11244 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
11245
11246Likewise, to additionally enable the package built from the recipe
11247containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
11248the actual recipe name was ``emgd_1.10.bb``, the following string would
11249enable that package as well as the original ``gst-plugins-ugly``
11250package:
11251::
11252
11253 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
11254
11255As a convenience, you do not need to specify the
11256complete license string in the whitelist for every package. You can use
11257an abbreviated form, which consists of just the first portion or
11258portions of the license string before the initial underscore character
11259or characters. A partial string will match any license that contains the
11260given string as the first portion of its license. For example, the
11261following whitelist string will also match both of the packages
11262previously mentioned as well as any other packages that have licenses
11263starting with "commercial" or "license".
11264::
11265
11266 LICENSE_FLAGS_WHITELIST = "commercial license"
11267
11268License Flag Matching
11269~~~~~~~~~~~~~~~~~~~~~
11270
11271License flag matching allows you to control what recipes the
11272OpenEmbedded build system includes in the build. Fundamentally, the
11273build system attempts to match ``LICENSE_FLAGS`` strings found in
11274recipes against ``LICENSE_FLAGS_WHITELIST`` strings found in the
11275whitelist. A match causes the build system to include a recipe in the
11276build, while failure to find a match causes the build system to exclude
11277a recipe.
11278
11279In general, license flag matching is simple. However, understanding some
11280concepts will help you correctly and effectively use matching.
11281
11282Before a flag defined by a particular recipe is tested against the
11283contents of the whitelist, the expanded string ``_${PN}`` is appended to
11284the flag. This expansion makes each ``LICENSE_FLAGS`` value
11285recipe-specific. After expansion, the string is then matched against the
11286whitelist. Thus, specifying ``LICENSE_FLAGS = "commercial"`` in recipe
11287"foo", for example, results in the string ``"commercial_foo"``. And, to
11288create a match, that string must appear in the whitelist.
11289
11290Judicious use of the ``LICENSE_FLAGS`` strings and the contents of the
11291``LICENSE_FLAGS_WHITELIST`` variable allows you a lot of flexibility for
11292including or excluding recipes based on licensing. For example, you can
11293broaden the matching capabilities by using license flags string subsets
11294in the whitelist.
11295
11296.. note::
11297
11298 When using a string subset, be sure to use the part of the expanded
11299 string that precedes the appended underscore character (e.g.
11300 usethispart_1.3
11301 ,
11302 usethispart_1.4
11303 , and so forth).
11304
11305For example, simply specifying the string "commercial" in the whitelist
11306matches any expanded ``LICENSE_FLAGS`` definition that starts with the
11307string "commercial" such as "commercial_foo" and "commercial_bar", which
11308are the strings the build system automatically generates for
11309hypothetical recipes named "foo" and "bar" assuming those recipes simply
11310specify the following:
11311::
11312
11313 LICENSE_FLAGS = "commercial"
11314
11315Thus, you can choose
11316to exhaustively enumerate each license flag in the whitelist and allow
11317only specific recipes into the image, or you can use a string subset
11318that causes a broader range of matches to allow a range of recipes into
11319the image.
11320
11321This scheme works even if the ``LICENSE_FLAGS`` string already has
11322``_${PN}`` appended. For example, the build system turns the license
11323flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
11324both the general "commercial" and the specific "commercial_1.2_foo"
11325strings found in the whitelist, as expected.
11326
11327Here are some other scenarios:
11328
11329- You can specify a versioned string in the recipe such as
11330 "commercial_foo_1.2" in a "foo" recipe. The build system expands this
11331 string to "commercial_foo_1.2_foo". Combine this license flag with a
11332 whitelist that has the string "commercial" and you match the flag
11333 along with any other flag that starts with the string "commercial".
11334
11335- Under the same circumstances, you can use "commercial_foo" in the
11336 whitelist and the build system not only matches "commercial_foo_1.2"
11337 but also matches any license flag with the string "commercial_foo",
11338 regardless of the version.
11339
11340- You can be very specific and use both the package and version parts
11341 in the whitelist (e.g. "commercial_foo_1.2") to specifically match a
11342 versioned recipe.
11343
11344Other Variables Related to Commercial Licenses
11345~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11346
11347Other helpful variables related to commercial license handling exist and
11348are defined in the
11349``poky/meta/conf/distro/include/default-distrovars.inc`` file:
11350::
11351
11352 COMMERCIAL_AUDIO_PLUGINS ?= ""
11353 COMMERCIAL_VIDEO_PLUGINS ?= ""
11354
11355If you
11356want to enable these components, you can do so by making sure you have
11357statements similar to the following in your ``local.conf`` configuration
11358file:
11359::
11360
11361 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
11362 gst-plugins-ugly-mpegaudioparse"
11363 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
11364 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
11365 LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
11366
11367
11368Of course, you could also create a matching whitelist for those
11369components using the more general "commercial" in the whitelist, but
11370that would also enable all the other packages with ``LICENSE_FLAGS``
11371containing "commercial", which you may or may not want:
11372::
11373
11374 LICENSE_FLAGS_WHITELIST = "commercial"
11375
11376Specifying audio and video plugins as part of the
11377``COMMERCIAL_AUDIO_PLUGINS`` and ``COMMERCIAL_VIDEO_PLUGINS`` statements
11378(along with the enabling ``LICENSE_FLAGS_WHITELIST``) includes the
11379plugins or components into built images, thus adding support for media
11380formats or components.
11381
11382Maintaining Open Source License Compliance During Your Product's Lifecycle
11383--------------------------------------------------------------------------
11384
11385One of the concerns for a development organization using open source
11386software is how to maintain compliance with various open source
11387licensing during the lifecycle of the product. While this section does
11388not provide legal advice or comprehensively cover all scenarios, it does
11389present methods that you can use to assist you in meeting the compliance
11390requirements during a software release.
11391
11392With hundreds of different open source licenses that the Yocto Project
11393tracks, it is difficult to know the requirements of each and every
11394license. However, the requirements of the major FLOSS licenses can begin
11395to be covered by assuming that three main areas of concern exist:
11396
11397- Source code must be provided.
11398
11399- License text for the software must be provided.
11400
11401- Compilation scripts and modifications to the source code must be
11402 provided.
11403
11404There are other requirements beyond the scope of these three and the
11405methods described in this section (e.g. the mechanism through which
11406source code is distributed).
11407
11408As different organizations have different methods of complying with open
11409source licensing, this section is not meant to imply that there is only
11410one single way to meet your compliance obligations, but rather to
11411describe one method of achieving compliance. The remainder of this
11412section describes methods supported to meet the previously mentioned
11413three requirements. Once you take steps to meet these requirements, and
11414prior to releasing images, sources, and the build system, you should
11415audit all artifacts to ensure completeness.
11416
11417.. note::
11418
11419 The Yocto Project generates a license manifest during image creation
11420 that is located in
11421 ${DEPLOY_DIR}/licenses/
11422 image_name-datestamp
11423 to assist with any audits.
11424
11425Providing the Source Code
11426~~~~~~~~~~~~~~~~~~~~~~~~~
11427
11428Compliance activities should begin before you generate the final image.
11429The first thing you should look at is the requirement that tops the list
11430for most compliance groups - providing the source. The Yocto Project has
11431a few ways of meeting this requirement.
11432
11433One of the easiest ways to meet this requirement is to provide the
11434entire :term:`DL_DIR` used by the
11435build. This method, however, has a few issues. The most obvious is the
11436size of the directory since it includes all sources used in the build
11437and not just the source used in the released image. It will include
11438toolchain source, and other artifacts, which you would not generally
11439release. However, the more serious issue for most companies is
11440accidental release of proprietary software. The Yocto Project provides
11441an :ref:`archiver <ref-classes-archiver>` class to
11442help avoid some of these concerns.
11443
11444Before you employ ``DL_DIR`` or the ``archiver`` class, you need to
11445decide how you choose to provide source. The source ``archiver`` class
11446can generate tarballs and SRPMs and can create them with various levels
11447of compliance in mind.
11448
11449One way of doing this (but certainly not the only way) is to release
11450just the source as a tarball. You can do this by adding the following to
11451the ``local.conf`` file found in the
11452:term:`Build Directory`:
11453::
11454
11455 INHERIT += "archiver"
11456 ARCHIVER_MODE[src] = "original"
11457
11458During the creation of your
11459image, the source from all recipes that deploy packages to the image is
11460placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
11461:term:`LICENSE` for each recipe.
11462Releasing the entire directory enables you to comply with requirements
11463concerning providing the unmodified source. It is important to note that
11464the size of the directory can get large.
11465
11466A way to help mitigate the size issue is to only release tarballs for
11467licenses that require the release of source. Let us assume you are only
11468concerned with GPL code as identified by running the following script:
11469::
11470
11471 # Script to archive a subset of packages matching specific license(s)
11472 # Source and license files are copied into sub folders of package folder
11473 # Must be run from build folder
11474 #!/bin/bash
11475 src_release_dir="source-release"
11476 mkdir -p $src_release_dir
11477 for a in tmp/deploy/sources/*; do
11478 for d in $a/*; do
11479 # Get package name from path
11480 p=`basename $d`
11481 p=${p%-*}
11482 p=${p%-*}
11483 # Only archive GPL packages (update *GPL* regex for your license check)
11484 numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
11485 if [ $numfiles -gt 1 ]; then
11486 echo Archiving $p
11487 mkdir -p $src_release_dir/$p/source
11488 cp $d/* $src_release_dir/$p/source 2> /dev/null
11489 mkdir -p $src_release_dir/$p/license
11490 cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
11491 fi
11492 done
11493 done
11494
11495At this point, you
11496could create a tarball from the ``gpl_source_release`` directory and
11497provide that to the end user. This method would be a step toward
11498achieving compliance with section 3a of GPLv2 and with section 6 of
11499GPLv3.
11500
11501Providing License Text
11502~~~~~~~~~~~~~~~~~~~~~~
11503
11504One requirement that is often overlooked is inclusion of license text.
11505This requirement also needs to be dealt with prior to generating the
11506final image. Some licenses require the license text to accompany the
11507binary. You can achieve this by adding the following to your
11508``local.conf`` file:
11509::
11510
11511 COPY_LIC_MANIFEST = "1"
11512 COPY_LIC_DIRS = "1"
11513 LICENSE_CREATE_PACKAGE = "1"
11514
11515Adding these statements to the
11516configuration file ensures that the licenses collected during package
11517generation are included on your image.
11518
11519.. note::
11520
11521 Setting all three variables to "1" results in the image having two
11522 copies of the same license file. One copy resides in
11523 ``/usr/share/common-licenses`` and the other resides in
11524 ``/usr/share/license``.
11525
11526 The reason for this behavior is because
11527 :term:`COPY_LIC_DIRS` and
11528 :term:`COPY_LIC_MANIFEST`
11529 add a copy of the license when the image is built but do not offer a
11530 path for adding licenses for newly installed packages to an image.
11531 :term:`LICENSE_CREATE_PACKAGE`
11532 adds a separate package and an upgrade path for adding licenses to an
11533 image.
11534
11535As the source ``archiver`` class has already archived the original
11536unmodified source that contains the license files, you would have
11537already met the requirements for inclusion of the license information
11538with source as defined by the GPL and other open source licenses.
11539
11540Providing Compilation Scripts and Source Code Modifications
11541~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11542
11543At this point, we have addressed all we need to prior to generating the
11544image. The next two requirements are addressed during the final
11545packaging of the release.
11546
11547By releasing the version of the OpenEmbedded build system and the layers
11548used during the build, you will be providing both compilation scripts
11549and the source code modifications in one step.
11550
11551If the deployment team has a :ref:`overview-manual/overview-manual-concepts:bsp layer`
11552and a distro layer, and those
11553those layers are used to patch, compile, package, or modify (in any way)
11554any open source software included in your released images, you might be
11555required to release those layers under section 3 of GPLv2 or section 1
11556of GPLv3. One way of doing that is with a clean checkout of the version
11557of the Yocto Project and layers used during your build. Here is an
11558example:
11559::
11560
11561 # We built using the dunfell branch of the poky repo
11562 $ git clone -b dunfell git://git.yoctoproject.org/poky
11563 $ cd poky
11564 # We built using the release_branch for our layers
11565 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
11566 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
11567 # clean up the .git repos
11568 $ find . -name ".git" -type d -exec rm -rf {} \;
11569
11570One
11571thing a development organization might want to consider for end-user
11572convenience is to modify ``meta-poky/conf/bblayers.conf.sample`` to
11573ensure that when the end user utilizes the released build system to
11574build an image, the development organization's layers are included in
11575the ``bblayers.conf`` file automatically:
11576::
11577
11578 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
11579 # changes incompatibly
11580 POKY_BBLAYERS_CONF_VERSION = "2"
11581
11582 BBPATH = "${TOPDIR}"
11583 BBFILES ?= ""
11584
11585 BBLAYERS ?= " \
11586 ##OEROOT##/meta \
11587 ##OEROOT##/meta-poky \
11588 ##OEROOT##/meta-yocto-bsp \
11589 ##OEROOT##/meta-mylayer \
11590 "
11591
11592Creating and
11593providing an archive of the :term:`Metadata`
11594layers (recipes, configuration files, and so forth) enables you to meet
11595your requirements to include the scripts to control compilation as well
11596as any modifications to the original source.
11597
11598Copying Licenses that Do Not Exist
11599----------------------------------
11600
11601Some packages, such as the linux-firmware package, have many licenses
11602that are not in any way common. You can avoid adding a lot of these
11603types of common license files, which are only applicable to a specific
11604package, by using the
11605:term:`NO_GENERIC_LICENSE`
11606variable. Using this variable also avoids QA errors when you use a
11607non-common, non-CLOSED license in a recipe.
11608
11609The following is an example that uses the ``LICENSE.Abilis.txt`` file as
11610the license from the fetched source:
11611::
11612
11613 NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
11614
11615Using the Error Reporting Tool
11616==============================
11617
11618The error reporting tool allows you to submit errors encountered during
11619builds to a central database. Outside of the build environment, you can
11620use a web interface to browse errors, view statistics, and query for
11621errors. The tool works using a client-server system where the client
11622portion is integrated with the installed Yocto Project
11623:term:`Source Directory` (e.g. ``poky``).
11624The server receives the information collected and saves it in a
11625database.
11626
11627A live instance of the error reporting server exists at
11628http://errors.yoctoproject.org. This server exists so that when
11629you want to get help with build failures, you can submit all of the
11630information on the failure easily and then point to the URL in your bug
11631report or send an email to the mailing list.
11632
11633.. note::
11634
11635 If you send error reports to this server, the reports become publicly
11636 visible.
11637
11638Enabling and Using the Tool
11639---------------------------
11640
11641By default, the error reporting tool is disabled. You can enable it by
11642inheriting the
11643:ref:`report-error <ref-classes-report-error>`
11644class by adding the following statement to the end of your
11645``local.conf`` file in your
11646:term:`Build Directory`.
11647::
11648
11649 INHERIT += "report-error"
11650
11651By default, the error reporting feature stores information in
11652``${``\ :term:`LOG_DIR`\ ``}/error-report``.
11653However, you can specify a directory to use by adding the following to
11654your ``local.conf`` file:
11655::
11656
11657 ERR_REPORT_DIR = "path"
11658
11659Enabling error
11660reporting causes the build process to collect the errors and store them
11661in a file as previously described. When the build system encounters an
11662error, it includes a command as part of the console output. You can run
11663the command to send the error file to the server. For example, the
11664following command sends the errors to an upstream server:
11665::
11666
11667 $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
11668
11669In the previous example, the errors are sent to a public database
11670available at http://errors.yoctoproject.org, which is used by the
11671entire community. If you specify a particular server, you can send the
11672errors to a different database. Use the following command for more
11673information on available options:
11674::
11675
11676 $ send-error-report --help
11677
11678When sending the error file, you are prompted to review the data being
11679sent as well as to provide a name and optional email address. Once you
11680satisfy these prompts, the command returns a link from the server that
11681corresponds to your entry in the database. For example, here is a
11682typical link: http://errors.yoctoproject.org/Errors/Details/9522/
11683
11684Following the link takes you to a web interface where you can browse,
11685query the errors, and view statistics.
11686
11687Disabling the Tool
11688------------------
11689
11690To disable the error reporting feature, simply remove or comment out the
11691following statement from the end of your ``local.conf`` file in your
11692:term:`Build Directory`.
11693::
11694
11695 INHERIT += "report-error"
11696
11697Setting Up Your Own Error Reporting Server
11698------------------------------------------
11699
11700If you want to set up your own error reporting server, you can obtain
11701the code from the Git repository at
11702http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/.
11703Instructions on how to set it up are in the README document.
11704
11705.. _dev-using-wayland-and-weston:
11706
11707Using Wayland and Weston
11708========================
11709
11710`Wayland <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)>`__
11711is a computer display server protocol that provides a method for
11712compositing window managers to communicate directly with applications
11713and video hardware and expects them to communicate with input hardware
11714using other libraries. Using Wayland with supporting targets can result
11715in better control over graphics frame rendering than an application
11716might otherwise achieve.
11717
11718The Yocto Project provides the Wayland protocol libraries and the
11719reference
11720`Weston <http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston>`__
11721compositor as part of its release. You can find the integrated packages
11722in the ``meta`` layer of the :term:`Source Directory`.
11723Specifically, you
11724can find the recipes that build both Wayland and Weston at
11725``meta/recipes-graphics/wayland``.
11726
11727You can build both the Wayland and Weston packages for use only with
11728targets that accept the `Mesa 3D and Direct Rendering
11729Infrastructure <https://en.wikipedia.org/wiki/Mesa_(computer_graphics)>`__,
11730which is also known as Mesa DRI. This implies that you cannot build and
11731use the packages if your target uses, for example, the Intel Embedded
11732Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI.
11733
11734.. note::
11735
11736 Due to lack of EGL support, Weston 1.0.3 will not run directly on the
11737 emulated QEMU hardware. However, this version of Weston will run
11738 under X emulation without issues.
11739
11740This section describes what you need to do to implement Wayland and use
11741the Weston compositor when building an image for a supporting target.
11742
11743Enabling Wayland in an Image
11744----------------------------
11745
11746To enable Wayland, you need to enable it to be built and enable it to be
11747included (installed) in the image.
11748
11749.. _enable-building:
11750
11751Building Wayland
11752~~~~~~~~~~~~~~~~
11753
11754To cause Mesa to build the ``wayland-egl`` platform and Weston to build
11755Wayland with Kernel Mode Setting
11756(`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__)
11757support, include the "wayland" flag in the
11758:term:`DISTRO_FEATURES`
11759statement in your ``local.conf`` file:
11760::
11761
11762 DISTRO_FEATURES_append = " wayland"
11763
11764.. note::
11765
11766 If X11 has been enabled elsewhere, Weston will build Wayland with X11
11767 support
11768
11769.. _enable-installation-in-an-image:
11770
11771Installing Wayland and Weston
11772~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11773
11774To install the Wayland feature into an image, you must include the
11775following
11776:term:`CORE_IMAGE_EXTRA_INSTALL`
11777statement in your ``local.conf`` file:
11778::
11779
11780 CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
11781
11782Running Weston
11783--------------
11784
11785To run Weston inside X11, enabling it as described earlier and building
11786a Sato image is sufficient. If you are running your image under Sato, a
11787Weston Launcher appears in the "Utility" category.
11788
11789Alternatively, you can run Weston through the command-line interpretor
11790(CLI), which is better suited for development work. To run Weston under
11791the CLI, you need to do the following after your image is built:
11792
117931. Run these commands to export ``XDG_RUNTIME_DIR``:
11794 ::
11795
11796 mkdir -p /tmp/$USER-weston
11797 chmod 0700 /tmp/$USER-weston
11798 export XDG_RUNTIME_DIR=/tmp/$USER-weston
11799
118002. Launch Weston in the shell:
11801 ::
11802
11803 weston
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index e9ce182a59..247f6abfd4 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='extendpoky'> 6<chapter id='extendpoky'>
6 7
@@ -3189,7 +3190,7 @@
3189 building an image. 3190 building an image.
3190 </para></listitem> 3191 </para></listitem>
3191 <listitem><para> 3192 <listitem><para>
3192 <filename>virtual/mesa</filename>: 3193 <filename>virtual/libgbm</filename>:
3193 Provides <filename>gbm.pc</filename>. 3194 Provides <filename>gbm.pc</filename>.
3194 </para></listitem> 3195 </para></listitem>
3195 <listitem><para> 3196 <listitem><para>
@@ -8383,7 +8384,7 @@
8383 If you see the following error, you need to 8384 If you see the following error, you need to
8384 update or create a 8385 update or create a
8385 <filename>~/.mtoolsrc</filename> file and 8386 <filename>~/.mtoolsrc</filename> file and
8386 be sure to have the line mtools_skip_check=1 8387 be sure to have the line "mtools_skip_check=1"
8387 in the file. 8388 in the file.
8388 Then, run the Wic command again: 8389 Then, run the Wic command again:
8389 <literallayout class='monospaced'> 8390 <literallayout class='monospaced'>
@@ -9057,6 +9058,9 @@
9057 <listitem><para> 9058 <listitem><para>
9058 <link linkend='creating-node-package-manager-npm-packages'>Creating node package manager (NPM) packages</link> 9059 <link linkend='creating-node-package-manager-npm-packages'>Creating node package manager (NPM) packages</link>
9059 </para></listitem> 9060 </para></listitem>
9061 <listitem><para>
9062 <link linkend='adding-custom-metadata-to-packages'>Adding custom metadata to packages</link>
9063 </para></listitem>
9060 </itemizedlist> 9064 </itemizedlist>
9061 </para> 9065 </para>
9062 9066
@@ -9833,7 +9837,7 @@
9833 <listitem><para> 9837 <listitem><para>
9834 Select the desired package format as follows: 9838 Select the desired package format as follows:
9835 <literallayout class='monospaced'> 9839 <literallayout class='monospaced'>
9836 PACKAGE_CLASSES ?= package_<replaceable>packageformat</replaceable> 9840 PACKAGE_CLASSES ?= "package_<replaceable>packageformat</replaceable>"
9837 </literallayout> 9841 </literallayout>
9838 where <replaceable>packageformat</replaceable> 9842 where <replaceable>packageformat</replaceable>
9839 can be "ipk", "rpm", "deb", or "tar" which are the 9843 can be "ipk", "rpm", "deb", or "tar" which are the
@@ -10761,6 +10765,61 @@
10761 </para> 10765 </para>
10762 </section> 10766 </section>
10763 </section> 10767 </section>
10768
10769 <section id='adding-custom-metadata-to-packages'>
10770 <title>Adding custom metadata to packages</title>
10771
10772 <para>
10773 The variable <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ADD_METADATA'><filename>PACKAGE_ADD_METADATA</filename></ulink>
10774 can be used to add additional metadata to packages. This is
10775 reflected in the package control/spec file. To take the ipk
10776 format for example, the CONTROL file stored inside would
10777 contain the additional metadata as additional lines.
10778 </para>
10779
10780 <para>
10781 The variable can be used in multiple ways, including using
10782 suffixes to set it for a specific package type and/or package.
10783 Note that the order of precedence is the same as this list:
10784 <itemizedlist>
10785 <listitem><para>
10786 <filename>PACKAGE_ADD_METADATA_&lt;PKGTYPE&gt;_&lt;PN&gt;</filename>
10787 </para></listitem>
10788 <listitem><para>
10789 <filename>PACKAGE_ADD_METADATA_&lt;PKGTYPE&gt;</filename>
10790 </para></listitem>
10791 <listitem><para>
10792 <filename>PACKAGE_ADD_METADATA_&lt;PN&gt;</filename>
10793 </para></listitem>
10794 <listitem><para>
10795 <filename>PACKAGE_ADD_METADATA</filename>
10796 </para></listitem>
10797 </itemizedlist>
10798 &lt;PKGTYPE&gt; is a parameter and expected to be a
10799 distinct name of specific package type:
10800 <itemizedlist>
10801 <listitem><para>IPK for .ipk packages</para></listitem>
10802 <listitem><para>DEB for .deb packages</para></listitem>
10803 <listitem><para>RPM for .rpm packages</para></listitem>
10804 </itemizedlist>
10805 &lt;PN&gt; is a parameter and expected to be a package name.
10806 </para>
10807
10808 <para>
10809 The variable can contain multiple [one-line] metadata fields
10810 separated by the literal sequence '\n'. The separator can be
10811 redefined using the variable flag <filename>separator</filename>.
10812 </para>
10813
10814 <para>
10815 The following is an example that adds two custom fields for
10816 ipk packages:
10817 <literallayout class='monospaced'>
10818 PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup: Applications/Spreadsheets"
10819 </literallayout>
10820 </para>
10821 </section>
10822
10764 </section> 10823 </section>
10765 10824
10766 <section id='efficiently-fetching-source-files-during-a-build'> 10825 <section id='efficiently-fetching-source-files-during-a-build'>
@@ -14134,7 +14193,7 @@
14134 <filename>local.conf</filename> file or in an image 14193 <filename>local.conf</filename> file or in an image
14135 recipe: 14194 recipe:
14136 <literallayout class='monospaced'> 14195 <literallayout class='monospaced'>
14137 IMAGE_INSTALL_append = gdbserver" 14196 IMAGE_INSTALL_append = " gdbserver"
14138 </literallayout> 14197 </literallayout>
14139 The change makes sure the <filename>gdbserver</filename> 14198 The change makes sure the <filename>gdbserver</filename>
14140 package is included. 14199 package is included.
diff --git a/documentation/dev-manual/dev-manual-customization.xsl b/documentation/dev-manual/dev-manual-customization.xsl
index 523ea3c5ed..6b16bcabf9 100644
--- a/documentation/dev-manual/dev-manual-customization.xsl
+++ b/documentation/dev-manual/dev-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/dev-manual/dev-manual-intro.rst b/documentation/dev-manual/dev-manual-intro.rst
new file mode 100644
index 0000000000..3225c6ca45
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-intro.rst
@@ -0,0 +1,61 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************************************
4The Yocto Project Development Tasks Manual
5******************************************
6
7.. _dev-welcome:
8
9Welcome
10=======
11
12Welcome to the Yocto Project Development Tasks Manual! This manual
13provides relevant procedures necessary for developing in the Yocto
14Project environment (i.e. developing embedded Linux images and
15user-space applications that run on targeted devices). The manual groups
16related procedures into higher-level sections. Procedures can consist of
17high-level steps or low-level steps depending on the topic.
18
19This manual provides the following:
20
21- Procedures that help you get going with the Yocto Project. For
22 example, procedures that show you how to set up a build host and work
23 with the Yocto Project source repositories.
24
25- Procedures that show you how to submit changes to the Yocto Project.
26 Changes can be improvements, new features, or bug fixes.
27
28- Procedures related to "everyday" tasks you perform while developing
29 images and applications using the Yocto Project. For example,
30 procedures to create a layer, customize an image, write a new recipe,
31 and so forth.
32
33This manual does not provide the following:
34
35- Redundant Step-by-step Instructions: For example, the
36 :doc:`../sdk-manual/sdk-manual` manual contains detailed
37 instructions on how to install an SDK, which is used to develop
38 applications for target hardware.
39
40- Reference or Conceptual Material: This type of material resides in an
41 appropriate reference manual. For example, system variables are
42 documented in the :doc`../ref-manual/ref-manual`.
43
44- Detailed Public Information Not Specific to the Yocto Project: For
45 example, exhaustive information on how to use the Source Control
46 Manager Git is better covered with Internet searches and official Git
47 Documentation than through the Yocto Project documentation.
48
49Other Information
50=================
51
52Because this manual presents information for many different topics,
53supplemental information is recommended for full comprehension. For
54introductory information on the Yocto Project, see the
55:yocto_home:`Yocto Project Website <>`. If you want to build an image with no
56knowledge of Yocto Project as a way of quickly testing it out, see the
57:doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document.
58
59For a comprehensive list of links and other documentation, see the
60":ref:`ref-manual/resources:links and related documentation`"
61section in the Yocto Project Reference Manual.
diff --git a/documentation/dev-manual/dev-manual-intro.xml b/documentation/dev-manual/dev-manual-intro.xml
index 3a34094b8c..38de5e4f53 100644
--- a/documentation/dev-manual/dev-manual-intro.xml
+++ b/documentation/dev-manual/dev-manual-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='dev-manual-intro'> 6<chapter id='dev-manual-intro'>
6 7
diff --git a/documentation/dev-manual/dev-manual-qemu.rst b/documentation/dev-manual/dev-manual-qemu.rst
new file mode 100644
index 0000000000..2833689d5f
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-qemu.rst
@@ -0,0 +1,470 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************************
4Using the Quick EMUlator (QEMU)
5*******************************
6
7The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
8Open Source project as part of the Yocto Project development "tool set".
9This chapter provides both procedures that show you how to use the Quick
10EMUlator (QEMU) and other QEMU information helpful for development
11purposes.
12
13.. _qemu-dev-overview:
14
15Overview
16========
17
18Within the context of the Yocto Project, QEMU is an emulator and
19virtualization machine that allows you to run a complete image you have
20built using the Yocto Project as just another task on your build system.
21QEMU is useful for running and testing images and applications on
22supported Yocto Project architectures without having actual hardware.
23Among other things, the Yocto Project uses QEMU to run automated Quality
24Assurance (QA) tests on final images shipped with each release.
25
26.. note::
27
28 This implementation is not the same as QEMU in general.
29
30This section provides a brief reference for the Yocto Project
31implementation of QEMU.
32
33For official information and documentation on QEMU in general, see the
34following references:
35
36- `QEMU Website <http://wiki.qemu.org/Main_Page>`__\ *:* The official
37 website for the QEMU Open Source project.
38
39- `Documentation <http://wiki.qemu.org/Manual>`__\ *:* The QEMU user
40 manual.
41
42.. _qemu-running-qemu:
43
44Running QEMU
45============
46
47To use QEMU, you need to have QEMU installed and initialized as well as
48have the proper artifacts (i.e. image files and root filesystems)
49available. Follow these general steps to run QEMU:
50
511. *Install QEMU:* QEMU is made available with the Yocto Project a
52 number of ways. One method is to install a Software Development Kit
53 (SDK). See ":ref:`sdk-manual/sdk-intro:the qemu emulator`" section in the
54 Yocto Project Application Development and the Extensible Software
55 Development Kit (eSDK) manual for information on how to install QEMU.
56
572. *Setting Up the Environment:* How you set up the QEMU environment
58 depends on how you installed QEMU:
59
60 - If you cloned the ``poky`` repository or you downloaded and
61 unpacked a Yocto Project release tarball, you can source the build
62 environment script (i.e. :ref:`structure-core-script`):
63 ::
64
65 $ cd ~/poky
66 $ source oe-init-build-env
67
68 - If you installed a cross-toolchain, you can run the script that
69 initializes the toolchain. For example, the following commands run
70 the initialization script from the default ``poky_sdk`` directory:
71 ::
72
73 . ~/poky_sdk/environment-setup-core2-64-poky-linux
74
753. *Ensure the Artifacts are in Place:* You need to be sure you have a
76 pre-built kernel that will boot in QEMU. You also need the target
77 root filesystem for your target machine's architecture:
78
79 - If you have previously built an image for QEMU (e.g. ``qemux86``,
80 ``qemuarm``, and so forth), then the artifacts are in place in
81 your :term:`Build Directory`.
82
83 - If you have not built an image, you can go to the
84 :yocto_dl:`machines/qemu </releases/yocto/yocto-3.1.2/machines/qemu/>` area and download a
85 pre-built image that matches your architecture and can be run on
86 QEMU.
87
88 See the ":ref:`sdk-manual/sdk-appendix-obtain:extracting the root filesystem`"
89 section in the Yocto Project Application Development and the
90 Extensible Software Development Kit (eSDK) manual for information on
91 how to extract a root filesystem.
92
934. *Run QEMU:* The basic ``runqemu`` command syntax is as follows:
94 ::
95
96 $ runqemu [option ] [...]
97
98 Based on what you provide on the command
99 line, ``runqemu`` does a good job of figuring out what you are trying
100 to do. For example, by default, QEMU looks for the most recently
101 built image according to the timestamp when it needs to look for an
102 image. Minimally, through the use of options, you must provide either
103 a machine name, a virtual machine image (``*wic.vmdk``), or a kernel
104 image (``*.bin``).
105
106 Here are some additional examples to help illustrate further QEMU:
107
108 - This example starts QEMU with MACHINE set to "qemux86-64".
109 Assuming a standard
110 :term:`Build Directory`, ``runqemu``
111 automatically finds the ``bzImage-qemux86-64.bin`` image file and
112 the ``core-image-minimal-qemux86-64-20200218002850.rootfs.ext4``
113 (assuming the current build created a ``core-image-minimal``
114 image).
115
116 .. note::
117
118 When more than one image with the same name exists, QEMU finds
119 and uses the most recently built image according to the
120 timestamp.
121
122 ::
123
124 $ runqemu qemux86-64
125
126 - This example produces the exact same results as the previous
127 example. This command, however, specifically provides the image
128 and root filesystem type.
129 ::
130
131 $ runqemu qemux86-64 core-image-minimal ext4
132
133 - This example specifies to boot an initial RAM disk image and to
134 enable audio in QEMU. For this case, ``runqemu`` set the internal
135 variable ``FSTYPE`` to "cpio.gz". Also, for audio to be enabled,
136 an appropriate driver must be installed (see the previous
137 description for the ``audio`` option for more information).
138 ::
139
140 $ runqemu qemux86-64 ramfs audio
141
142 - This example does not provide enough information for QEMU to
143 launch. While the command does provide a root filesystem type, it
144 must also minimally provide a MACHINE, KERNEL, or VM option.
145 ::
146
147 $ runqemu ext4
148
149 - This example specifies to boot a virtual machine image
150 (``.wic.vmdk`` file). From the ``.wic.vmdk``, ``runqemu``
151 determines the QEMU architecture (MACHINE) to be "qemux86-64" and
152 the root filesystem type to be "vmdk".
153 ::
154
155 $ runqemu /home/scott-lenovo/vm/core-image-minimal-qemux86-64.wic.vmdk
156
157Switching Between Consoles
158==========================
159
160When booting or running QEMU, you can switch between supported consoles
161by using Ctrl+Alt+number. For example, Ctrl+Alt+3 switches you to the
162serial console as long as that console is enabled. Being able to switch
163consoles is helpful, for example, if the main QEMU console breaks for
164some reason.
165
166.. note::
167
168 Usually, "2" gets you to the main console and "3" gets you to the
169 serial console.
170
171Removing the Splash Screen
172==========================
173
174You can remove the splash screen when QEMU is booting by using Alt+left.
175Removing the splash screen allows you to see what is happening in the
176background.
177
178Disabling the Cursor Grab
179=========================
180
181The default QEMU integration captures the cursor within the main window.
182It does this since standard mouse devices only provide relative input
183and not absolute coordinates. You then have to break out of the grab
184using the "Ctrl+Alt" key combination. However, the Yocto Project's
185integration of QEMU enables the wacom USB touch pad driver by default to
186allow input of absolute coordinates. This default means that the mouse
187can enter and leave the main window without the grab taking effect
188leading to a better user experience.
189
190.. _qemu-running-under-a-network-file-system-nfs-server:
191
192Running Under a Network File System (NFS) Server
193================================================
194
195One method for running QEMU is to run it on an NFS server. This is
196useful when you need to access the same file system from both the build
197and the emulated system at the same time. It is also worth noting that
198the system does not need root privileges to run. It uses a user space
199NFS server to avoid that. Follow these steps to set up for running QEMU
200using an NFS server.
201
2021. *Extract a Root Filesystem:* Once you are able to run QEMU in your
203 environment, you can use the ``runqemu-extract-sdk`` script, which is
204 located in the ``scripts`` directory along with the ``runqemu``
205 script.
206
207 The ``runqemu-extract-sdk`` takes a root filesystem tarball and
208 extracts it into a location that you specify. Here is an example that
209 takes a file system and extracts it to a directory named
210 ``test-nfs``:
211 ::
212
213 runqemu-extract-sdk ./tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.tar.bz2 test-nfs
214
2152. *Start QEMU:* Once you have extracted the file system, you can run
216 ``runqemu`` normally with the additional location of the file system.
217 You can then also make changes to the files within ``./test-nfs`` and
218 see those changes appear in the image in real time. Here is an
219 example using the ``qemux86`` image:
220 ::
221
222 runqemu qemux86-64 ./test-nfs
223
224.. note::
225
226 Should you need to start, stop, or restart the NFS share, you can use
227 the following commands:
228
229 - The following command starts the NFS share: runqemu-export-rootfs
230 start file-system-location
231
232 - The following command stops the NFS share: runqemu-export-rootfs
233 stop file-system-location
234
235 - The following command restarts the NFS share:
236 runqemu-export-rootfs restart file-system-location
237
238.. _qemu-kvm-cpu-compatibility:
239
240QEMU CPU Compatibility Under KVM
241================================
242
243By default, the QEMU build compiles for and targets 64-bit and x86 Intel
244Core2 Duo processors and 32-bit x86 Intel Pentium II processors. QEMU
245builds for and targets these CPU types because they display a broad
246range of CPU feature compatibility with many commonly used CPUs.
247
248Despite this broad range of compatibility, the CPUs could support a
249feature that your host CPU does not support. Although this situation is
250not a problem when QEMU uses software emulation of the feature, it can
251be a problem when QEMU is running with KVM enabled. Specifically,
252software compiled with a certain CPU feature crashes when run on a CPU
253under KVM that does not support that feature. To work around this
254problem, you can override QEMU's runtime CPU setting by changing the
255``QB_CPU_KVM`` variable in ``qemuboot.conf`` in the
256:term:`Build Directory` ``deploy/image``
257directory. This setting specifies a ``-cpu`` option passed into QEMU in
258the ``runqemu`` script. Running ``qemu -cpu help`` returns a list of
259available supported CPU types.
260
261.. _qemu-dev-performance:
262
263QEMU Performance
264================
265
266Using QEMU to emulate your hardware can result in speed issues depending
267on the target and host architecture mix. For example, using the
268``qemux86`` image in the emulator on an Intel-based 32-bit (x86) host
269machine is fast because the target and host architectures match. On the
270other hand, using the ``qemuarm`` image on the same Intel-based host can
271be slower. But, you still achieve faithful emulation of ARM-specific
272issues.
273
274To speed things up, the QEMU images support using ``distcc`` to call a
275cross-compiler outside the emulated system. If you used ``runqemu`` to
276start QEMU, and the ``distccd`` application is present on the host
277system, any BitBake cross-compiling toolchain available from the build
278system is automatically used from within QEMU simply by calling
279``distcc``. You can accomplish this by defining the cross-compiler
280variable (e.g. ``export CC="distcc"``). Alternatively, if you are using
281a suitable SDK image or the appropriate stand-alone toolchain is
282present, the toolchain is also automatically used.
283
284.. note::
285
286 Several mechanisms exist that let you connect to the system running
287 on the QEMU emulator:
288
289 - QEMU provides a framebuffer interface that makes standard consoles
290 available.
291
292 - Generally, headless embedded devices have a serial port. If so,
293 you can configure the operating system of the running image to use
294 that port to run a console. The connection uses standard IP
295 networking.
296
297 - SSH servers exist in some QEMU images. The ``core-image-sato``
298 QEMU image has a Dropbear secure shell (SSH) server that runs with
299 the root password disabled. The ``core-image-full-cmdline`` and
300 ``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear.
301 Including these SSH servers allow you to use standard ``ssh`` and
302 ``scp`` commands. The ``core-image-minimal`` QEMU image, however,
303 contains no SSH server.
304
305 - You can use a provided, user-space NFS server to boot the QEMU
306 session using a local copy of the root filesystem on the host. In
307 order to make this connection, you must extract a root filesystem
308 tarball by using the ``runqemu-extract-sdk`` command. After
309 running the command, you must then point the ``runqemu`` script to
310 the extracted directory instead of a root filesystem image file.
311 See the "`Running Under a Network File System (NFS)
312 Server <#qemu-running-under-a-network-file-system-nfs-server>`__"
313 section for more information.
314
315.. _qemu-dev-command-line-syntax:
316
317QEMU Command-Line Syntax
318========================
319
320The basic ``runqemu`` command syntax is as follows:
321::
322
323 $ runqemu [option ] [...]
324
325Based on what you provide on the command line, ``runqemu`` does a
326good job of figuring out what you are trying to do. For example, by
327default, QEMU looks for the most recently built image according to the
328timestamp when it needs to look for an image. Minimally, through the use
329of options, you must provide either a machine name, a virtual machine
330image (``*wic.vmdk``), or a kernel image (``*.bin``).
331
332Following is the command-line help output for the ``runqemu`` command:
333::
334
335 $ runqemu --help
336
337 Usage: you can run this script with any valid combination
338 of the following environment variables (in any order):
339 KERNEL - the kernel image file to use
340 ROOTFS - the rootfs image file or nfsroot directory to use
341 MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
342 Simplified QEMU command-line options can be passed with:
343 nographic - disable video console
344 serial - enable a serial console on /dev/ttyS0
345 slirp - enable user networking, no root privileges is required
346 kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
347 kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
348 publicvnc - enable a VNC server open to all hosts
349 audio - enable audio
350 [*/]ovmf* - OVMF firmware file or base name for booting with UEFI
351 tcpserial=<port> - specify tcp serial port number
352 biosdir=<dir> - specify custom bios dir
353 biosfilename=<filename> - specify bios filename
354 qemuparams=<xyz> - specify custom parameters to QEMU
355 bootparams=<xyz> - specify custom kernel parameters during boot
356 help, -h, --help: print this text
357
358 Examples:
359 runqemu
360 runqemu qemuarm
361 runqemu tmp/deploy/images/qemuarm
362 runqemu tmp/deploy/images/qemux86/<qemuboot.conf>
363 runqemu qemux86-64 core-image-sato ext4
364 runqemu qemux86-64 wic-image-minimal wic
365 runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
366 runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
367 runqemu qemux86 qemuparams="-m 256"
368 runqemu qemux86 bootparams="psplash=false"
369 runqemu path/to/<image>-<machine>.wic
370 runqemu path/to/<image>-<machine>.wic.vmdk
371
372.. _qemu-dev-runqemu-command-line-options:
373
374``runqemu`` Command-Line Options
375================================
376
377Following is a description of ``runqemu`` options you can provide on the
378command line:
379
380.. note::
381
382 If you do provide some "illegal" option combination or perhaps you do
383 not provide enough in the way of options,
384 runqemu
385 provides appropriate error messaging to help you correct the problem.
386
387- QEMUARCH: The QEMU machine architecture, which must be "qemuarm",
388 "qemuarm64", "qemumips", "qemumips64", "qemuppc", "qemux86", or
389 "qemux86-64".
390
391- ``VM``: The virtual machine image, which must be a ``.wic.vmdk``
392 file. Use this option when you want to boot a ``.wic.vmdk`` image.
393 The image filename you provide must contain one of the following
394 strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64",
395 "qemumips", "qemuppc", or "qemush4".
396
397- ROOTFS: A root filesystem that has one of the following filetype
398 extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If
399 the filename you provide for this option uses "nfs", it must provide
400 an explicit root filesystem path.
401
402- KERNEL: A kernel image, which is a ``.bin`` file. When you provide a
403 ``.bin`` file, ``runqemu`` detects it and assumes the file is a
404 kernel image.
405
406- MACHINE: The architecture of the QEMU machine, which must be one of
407 the following: "qemux86", "qemux86-64", "qemuarm", "qemuarm64",
408 "qemumips", "qemumips64", or "qemuppc". The MACHINE and QEMUARCH
409 options are basically identical. If you do not provide a MACHINE
410 option, ``runqemu`` tries to determine it based on other options.
411
412- ``ramfs``: Indicates you are booting an initial RAM disk (initramfs)
413 image, which means the ``FSTYPE`` is ``cpio.gz``.
414
415- ``iso``: Indicates you are booting an ISO image, which means the
416 ``FSTYPE`` is ``.iso``.
417
418- ``nographic``: Disables the video console, which sets the console to
419 "ttys0". This option is useful when you have logged into a server and
420 you do not want to disable forwarding from the X Window System (X11)
421 to your workstation or laptop.
422
423- ``serial``: Enables a serial console on ``/dev/ttyS0``.
424
425- ``biosdir``: Establishes a custom directory for BIOS, VGA BIOS and
426 keymaps.
427
428- ``biosfilename``: Establishes a custom BIOS name.
429
430- ``qemuparams=\"xyz\"``: Specifies custom QEMU parameters. Use this
431 option to pass options other than the simple "kvm" and "serial"
432 options.
433
434- ``bootparams=\"xyz\"``: Specifies custom boot parameters for the
435 kernel.
436
437- ``audio``: Enables audio in QEMU. The MACHINE option must be either
438 "qemux86" or "qemux86-64" in order for audio to be enabled.
439 Additionally, the ``snd_intel8x0`` or ``snd_ens1370`` driver must be
440 installed in linux guest.
441
442- ``slirp``: Enables "slirp" networking, which is a different way of
443 networking that does not need root access but also is not as easy to
444 use or comprehensive as the default.
445
446- ``kvm``: Enables KVM when running "qemux86" or "qemux86-64" QEMU
447 architectures. For KVM to work, all the following conditions must be
448 met:
449
450 - Your MACHINE must be either qemux86" or "qemux86-64".
451
452 - Your build host has to have the KVM modules installed, which are
453 ``/dev/kvm``.
454
455 - The build host ``/dev/kvm`` directory has to be both writable and
456 readable.
457
458- ``kvm-vhost``: Enables KVM with VHOST support when running "qemux86"
459 or "qemux86-64" QEMU architectures. For KVM with VHOST to work, the
460 following conditions must be met:
461
462 - `kvm <#kvm-cond>`__ option conditions must be met.
463
464 - Your build host has to have virtio net device, which are
465 ``/dev/vhost-net``.
466
467 - The build host ``/dev/vhost-net`` directory has to be either
468 readable or writable and "slirp-enabled".
469
470- ``publicvnc``: Enables a VNC server open to all hosts.
diff --git a/documentation/dev-manual/dev-manual-qemu.xml b/documentation/dev-manual/dev-manual-qemu.xml
index 5ccc0dfe83..1a526dd2f5 100644
--- a/documentation/dev-manual/dev-manual-qemu.xml
+++ b/documentation/dev-manual/dev-manual-qemu.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='dev-manual-qemu'> 6<chapter id='dev-manual-qemu'>
6 7
@@ -105,7 +106,7 @@
105 You need to be sure you have a pre-built kernel that 106 You need to be sure you have a pre-built kernel that
106 will boot in QEMU. 107 will boot in QEMU.
107 You also need the target root filesystem for your target 108 You also need the target root filesystem for your target
108 machines architecture: 109 machine's architecture:
109 <itemizedlist> 110 <itemizedlist>
110 <listitem><para> 111 <listitem><para>
111 If you have previously built an image for QEMU 112 If you have previously built an image for QEMU
@@ -552,7 +553,7 @@
552 A root filesystem that has one of the following 553 A root filesystem that has one of the following
553 filetype extensions: "ext2", "ext3", "ext4", "jffs2", 554 filetype extensions: "ext2", "ext3", "ext4", "jffs2",
554 "nfs", or "btrfs". 555 "nfs", or "btrfs".
555 If the filename you provide for this option uses nfs, it 556 If the filename you provide for this option uses "nfs", it
556 must provide an explicit root filesystem path. 557 must provide an explicit root filesystem path.
557 </para></listitem> 558 </para></listitem>
558 <listitem><para> 559 <listitem><para>
@@ -566,7 +567,7 @@
566 <replaceable>MACHINE</replaceable>: 567 <replaceable>MACHINE</replaceable>:
567 The architecture of the QEMU machine, which must be one 568 The architecture of the QEMU machine, which must be one
568 of the following: "qemux86", "qemux86-64", "qemuarm", 569 of the following: "qemux86", "qemux86-64", "qemuarm",
569 "qemuarm64", "qemumips", qemumips64", or "qemuppc". 570 "qemuarm64", "qemumips", "qemumips64", or "qemuppc".
570 The <replaceable>MACHINE</replaceable> and 571 The <replaceable>MACHINE</replaceable> and
571 <replaceable>QEMUARCH</replaceable> options are basically 572 <replaceable>QEMUARCH</replaceable> options are basically
572 identical. 573 identical.
@@ -673,7 +674,7 @@ qemux86" or "qemux86-64".
673 <listitem><para> 674 <listitem><para>
674 The build host <filename>/dev/vhost-net</filename> 675 The build host <filename>/dev/vhost-net</filename>
675 directory has to be either readable or writable 676 directory has to be either readable or writable
676 and slirp-enabled. 677 and "slirp-enabled".
677 </para></listitem> 678 </para></listitem>
678 </itemizedlist> 679 </itemizedlist>
679 </para></listitem> 680 </para></listitem>
diff --git a/documentation/dev-manual/dev-manual-start.rst b/documentation/dev-manual/dev-manual-start.rst
new file mode 100644
index 0000000000..d9c1e4de00
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-start.rst
@@ -0,0 +1,940 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************************
4Setting Up to Use the Yocto Project
5***********************************
6
7This chapter provides guidance on how to prepare to use the Yocto
8Project. You can learn about creating a team environment to develop
9using the Yocto Project, how to set up a :ref:`build
10host <dev-manual/dev-manual-start:preparing the build host>`, how to locate
11Yocto Project source repositories, and how to create local Git
12repositories.
13
14.. _usingpoky-changes-collaborate:
15
16Creating a Team Development Environment
17=======================================
18
19It might not be immediately clear how you can use the Yocto Project in a
20team development environment, or how to scale it for a large team of
21developers. You can adapt the Yocto Project to many different use cases
22and scenarios; however, this flexibility could cause difficulties if you
23are trying to create a working setup that scales effectively.
24
25To help you understand how to set up this type of environment, this
26section presents a procedure that gives you information that can help
27you get the results you want. The procedure is high-level and presents
28some of the project's most successful experiences, practices, solutions,
29and available technologies that have proved to work well in the past;
30however, keep in mind, the procedure here is simply a starting point.
31You can build off these steps and customize the procedure to fit any
32particular working environment and set of practices.
33
341. *Determine Who is Going to be Developing:* You first need to
35 understand who is going to be doing anything related to the Yocto
36 Project and determine their roles. Making this determination is
37 essential to completing subsequent steps, which are to get your
38 equipment together and set up your development environment's
39 hardware topology.
40
41 The following roles exist:
42
43 - *Application Developer:* This type of developer does application
44 level work on top of an existing software stack.
45
46 - *Core System Developer:* This type of developer works on the
47 contents of the operating system image itself.
48
49 - *Build Engineer:* This type of developer manages Autobuilders and
50 releases. Depending on the specifics of the environment, not all
51 situations might need a Build Engineer.
52
53 - *Test Engineer:* This type of developer creates and manages
54 automated tests that are used to ensure all application and core
55 system development meets desired quality standards.
56
572. *Gather the Hardware:* Based on the size and make-up of the team,
58 get the hardware together. Ideally, any development, build, or test
59 engineer uses a system that runs a supported Linux distribution.
60 These systems, in general, should be high performance (e.g. dual,
61 six-core Xeons with 24 Gbytes of RAM and plenty of disk space). You
62 can help ensure efficiency by having any machines used for testing
63 or that run Autobuilders be as high performance as possible.
64
65 .. note::
66
67 Given sufficient processing power, you might also consider
68 building Yocto Project development containers to be run under
69 Docker, which is described later.
70
713. *Understand the Hardware Topology of the Environment:* Once you
72 understand the hardware involved and the make-up of the team, you
73 can understand the hardware topology of the development environment.
74 You can get a visual idea of the machines and their roles across the
75 development environment.
76
774. *Use Git as Your Source Control Manager (SCM):* Keeping your
78 :term:`Metadata` (i.e. recipes,
79 configuration files, classes, and so forth) and any software you are
80 developing under the control of an SCM system that is compatible
81 with the OpenEmbedded build system is advisable. Of all of the SCMs
82 supported by BitBake, the Yocto Project team strongly recommends using
83 :ref:`overview-manual/overview-manual-development-environment:git`.
84 Git is a distributed system
85 that is easy to back up, allows you to work remotely, and then
86 connects back to the infrastructure.
87
88 .. note::
89
90 For information about BitBake, see the
91 BitBake User Manual
92 .
93
94 It is relatively easy to set up Git services and create
95 infrastructure like
96 :yocto_git:`http://git.yoctoproject.org <>`, which is based on
97 server software called ``gitolite`` with ``cgit`` being used to
98 generate the web interface that lets you view the repositories. The
99 ``gitolite`` software identifies users using SSH keys and allows
100 branch-based access controls to repositories that you can control as
101 little or as much as necessary.
102
103 .. note::
104
105 The setup of these services is beyond the scope of this manual.
106 However, sites such as the following exist that describe how to
107 perform setup:
108
109 - `Git documentation <http://git-scm.com/book/ch4-8.html>`__:
110 Describes how to install ``gitolite`` on the server.
111
112 - `Gitolite <http://gitolite.com>`__: Information for
113 ``gitolite``.
114
115 - `Interfaces, frontends, and
116 tools <https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools>`__:
117 Documentation on how to create interfaces and frontends for
118 Git.
119
1205. *Set up the Application Development Machines:* As mentioned earlier,
121 application developers are creating applications on top of existing
122 software stacks. Following are some best practices for setting up
123 machines used for application development:
124
125 - Use a pre-built toolchain that contains the software stack
126 itself. Then, develop the application code on top of the stack.
127 This method works well for small numbers of relatively isolated
128 applications.
129
130 - Keep your cross-development toolchains updated. You can do this
131 through provisioning either as new toolchain downloads or as
132 updates through a package update mechanism using ``opkg`` to
133 provide updates to an existing toolchain. The exact mechanics of
134 how and when to do this depend on local policy.
135
136 - Use multiple toolchains installed locally into different
137 locations to allow development across versions.
138
1396. *Set up the Core Development Machines:* As mentioned earlier, core
140 developers work on the contents of the operating system itself.
141 Following are some best practices for setting up machines used for
142 developing images:
143
144 - Have the :term:`OpenEmbedded Build System` available on
145 the developer workstations so developers can run their own builds
146 and directly rebuild the software stack.
147
148 - Keep the core system unchanged as much as possible and do your
149 work in layers on top of the core system. Doing so gives you a
150 greater level of portability when upgrading to new versions of
151 the core system or Board Support Packages (BSPs).
152
153 - Share layers amongst the developers of a particular project and
154 contain the policy configuration that defines the project.
155
1567. *Set up an Autobuilder:* Autobuilders are often the core of the
157 development environment. It is here that changes from individual
158 developers are brought together and centrally tested. Based on this
159 automated build and test environment, subsequent decisions about
160 releases can be made. Autobuilders also allow for "continuous
161 integration" style testing of software components and regression
162 identification and tracking.
163
164 See "`Yocto Project
165 Autobuilder <http://autobuilder.yoctoproject.org>`__" for more
166 information and links to buildbot. The Yocto Project team has found
167 this implementation works well in this role. A public example of
168 this is the Yocto Project Autobuilders, which the Yocto Project team
169 uses to test the overall health of the project.
170
171 The features of this system are:
172
173 - Highlights when commits break the build.
174
175 - Populates an :ref:`sstate
176 cache <overview-manual/overview-manual-concepts:shared state cache>` from which
177 developers can pull rather than requiring local builds.
178
179 - Allows commit hook triggers, which trigger builds when commits
180 are made.
181
182 - Allows triggering of automated image booting and testing under
183 the QuickEMUlator (QEMU).
184
185 - Supports incremental build testing and from-scratch builds.
186
187 - Shares output that allows developer testing and historical
188 regression investigation.
189
190 - Creates output that can be used for releases.
191
192 - Allows scheduling of builds so that resources can be used
193 efficiently.
194
1958. *Set up Test Machines:* Use a small number of shared, high
196 performance systems for testing purposes. Developers can use these
197 systems for wider, more extensive testing while they continue to
198 develop locally using their primary development system.
199
2009. *Document Policies and Change Flow:* The Yocto Project uses a
201 hierarchical structure and a pull model. Scripts exist to create and
202 send pull requests (i.e. ``create-pull-request`` and
203 ``send-pull-request``). This model is in line with other open source
204 projects where maintainers are responsible for specific areas of the
205 project and a single maintainer handles the final "top-of-tree"
206 merges.
207
208 .. note::
209
210 You can also use a more collective push model. The
211 gitolite
212 software supports both the push and pull models quite easily.
213
214 As with any development environment, it is important to document the
215 policy used as well as any main project guidelines so they are
216 understood by everyone. It is also a good idea to have
217 well-structured commit messages, which are usually a part of a
218 project's guidelines. Good commit messages are essential when
219 looking back in time and trying to understand why changes were made.
220
221 If you discover that changes are needed to the core layer of the
222 project, it is worth sharing those with the community as soon as
223 possible. Chances are if you have discovered the need for changes,
224 someone else in the community needs them also.
225
22610. *Development Environment Summary:* Aside from the previous steps,
227 some best practices exist within the Yocto Project development
228 environment. Consider the following:
229
230 - Use :ref:`overview-manual/overview-manual-development-environment:git` as the source control
231 system.
232
233 - Maintain your Metadata in layers that make sense for your
234 situation. See the ":ref:`overview-manual/overview-manual-yp-intro:the yocto project layer model`"
235 section in the Yocto Project Overview and Concepts Manual and the
236 ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
237 section for more information on layers.
238
239 - Separate the project's Metadata and code by using separate Git
240 repositories. See the ":ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`"
241 section in the Yocto Project Overview and Concepts Manual for
242 information on these repositories. See the "`Locating Yocto
243 Project Source Files <#locating-yocto-project-source-files>`__"
244 section for information on how to set up local Git repositories
245 for related upstream Yocto Project Git repositories.
246
247 - Set up the directory for the shared state cache
248 (:term:`SSTATE_DIR`) where
249 it makes sense. For example, set up the sstate cache on a system
250 used by developers in the same organization and share the same
251 source directories on their machines.
252
253 - Set up an Autobuilder and have it populate the sstate cache and
254 source directories.
255
256 - The Yocto Project community encourages you to send patches to the
257 project to fix bugs or add features. If you do submit patches,
258 follow the project commit guidelines for writing good commit
259 messages. See the
260 ":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
261 section.
262
263 - Send changes to the core sooner than later as others are likely
264 to run into the same issues. For some guidance on mailing lists
265 to use, see the list in the
266 ":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
267 section. For a description
268 of the available mailing lists, see the ":ref:`resources-mailinglist`" section in
269 the Yocto Project Reference Manual.
270
271.. _dev-preparing-the-build-host:
272
273Preparing the Build Host
274========================
275
276This section provides procedures to set up a system to be used as your
277:term:`Build Host` for
278development using the Yocto Project. Your build host can be a native
279Linux machine (recommended), it can be a machine (Linux, Mac, or
280Windows) that uses `CROPS <https://github.com/crops/poky-container>`__,
281which leverages `Docker Containers <https://www.docker.com/>`__ or it
282can be a Windows machine capable of running Windows Subsystem For Linux
283v2 (WSL).
284
285.. note::
286
287 The Yocto Project is not compatible with
288 Windows Subsystem for Linux v1
289 . It is compatible but not officially supported nor validated with
290 WSLv2. If you still decide to use WSL please upgrade to
291 WSLv2
292 .
293
294Once your build host is set up to use the Yocto Project, further steps
295are necessary depending on what you want to accomplish. See the
296following references for information on how to prepare for Board Support
297Package (BSP) development and kernel development:
298
299- *BSP Development:* See the ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
300 section in the Yocto Project Board Support Package (BSP) Developer's
301 Guide.
302
303- *Kernel Development:* See the ":ref:`kernel-dev/kernel-dev-common:preparing the build host to work on the kernel`"
304 section in the Yocto Project Linux Kernel Development Manual.
305
306Setting Up a Native Linux Host
307------------------------------
308
309Follow these steps to prepare a native Linux machine as your Yocto
310Project Build Host:
311
3121. *Use a Supported Linux Distribution:* You should have a reasonably
313 current Linux-based host system. You will have the best results with
314 a recent release of Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS
315 as these releases are frequently tested against the Yocto Project and
316 officially supported. For a list of the distributions under
317 validation and their status, see the ":ref:`Supported Linux
318 Distributions <detailed-supported-distros>`"
319 section in the Yocto Project Reference Manual and the wiki page at
320 :yocto_wiki:`Distribution Support </wiki/Distribution_Support>`.
321
3222. *Have Enough Free Memory:* Your system should have at least 50 Gbytes
323 of free disk space for building images.
324
3253. *Meet Minimal Version Requirements:* The OpenEmbedded build system
326 should be able to run on any modern distribution that has the
327 following versions for Git, tar, Python and gcc.
328
329 - Git 1.8.3.1 or greater
330
331 - tar 1.28 or greater
332
333 - Python 3.5.0 or greater.
334
335 - gcc 5.0 or greater.
336
337 If your build host does not meet any of these three listed version
338 requirements, you can take steps to prepare the system so that you
339 can still use the Yocto Project. See the
340 ":ref:`ref-manual/ref-system-requirements:required git, tar, python and gcc versions`"
341 section in the Yocto Project Reference Manual for information.
342
3434. *Install Development Host Packages:* Required development host
344 packages vary depending on your build host and what you want to do
345 with the Yocto Project. Collectively, the number of required packages
346 is large if you want to be able to cover all cases.
347
348 For lists of required packages for all scenarios, see the
349 ":ref:`ref-manual/ref-system-requirements:required packages for the build host`"
350 section in the Yocto Project Reference Manual.
351
352Once you have completed the previous steps, you are ready to continue
353using a given development path on your native Linux machine. If you are
354going to use BitBake, see the
355":ref:`dev-manual/dev-manual-start:cloning the \`\`poky\`\` repository`"
356section. If you are going
357to use the Extensible SDK, see the ":doc:`../sdk-manual/sdk-extensible`" Chapter in the Yocto
358Project Application Development and the Extensible Software Development
359Kit (eSDK) manual. If you want to work on the kernel, see the :doc:`../kernel-dev/kernel-dev`. If you are going to use
360Toaster, see the ":doc:`../toaster-manual/toaster-manual-setup-and-use`"
361section in the Toaster User Manual.
362
363.. _setting-up-to-use-crops:
364
365Setting Up to Use CROss PlatformS (CROPS)
366-----------------------------------------
367
368With `CROPS <https://github.com/crops/poky-container>`__, which
369leverages `Docker Containers <https://www.docker.com/>`__, you can
370create a Yocto Project development environment that is operating system
371agnostic. You can set up a container in which you can develop using the
372Yocto Project on a Windows, Mac, or Linux machine.
373
374Follow these general steps to prepare a Windows, Mac, or Linux machine
375as your Yocto Project build host:
376
3771. *Determine What Your Build Host Needs:*
378 `Docker <https://www.docker.com/what-docker>`__ is a software
379 container platform that you need to install on the build host.
380 Depending on your build host, you might have to install different
381 software to support Docker containers. Go to the Docker installation
382 page and read about the platform requirements in "`Supported
383 Platforms <https://docs.docker.com/engine/install/#supported-platforms>`__"
384 your build host needs to run containers.
385
3862. *Choose What To Install:* Depending on whether or not your build host
387 meets system requirements, you need to install "Docker CE Stable" or
388 the "Docker Toolbox". Most situations call for Docker CE. However, if
389 you have a build host that does not meet requirements (e.g.
390 Pre-Windows 10 or Windows 10 "Home" version), you must install Docker
391 Toolbox instead.
392
3933. *Go to the Install Site for Your Platform:* Click the link for the
394 Docker edition associated with your build host's native software. For
395 example, if your build host is running Microsoft Windows Version 10
396 and you want the Docker CE Stable edition, click that link under
397 "Supported Platforms".
398
3994. *Install the Software:* Once you have understood all the
400 pre-requisites, you can download and install the appropriate
401 software. Follow the instructions for your specific machine and the
402 type of the software you need to install:
403
404 - Install `Docker CE for
405 Windows <https://docs.docker.com/docker-for-windows/install/#install-docker-desktop-on-windows>`__
406 for Windows build hosts that meet requirements.
407
408 - Install `Docker CE for
409 MacOs <https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-desktop-on-mac>`__
410 for Mac build hosts that meet requirements.
411
412 - Install `Docker Toolbox for
413 Windows <https://docs.docker.com/toolbox/toolbox_install_windows/>`__
414 for Windows build hosts that do not meet Docker requirements.
415
416 - Install `Docker Toolbox for
417 MacOS <https://docs.docker.com/toolbox/toolbox_install_mac/>`__
418 for Mac build hosts that do not meet Docker requirements.
419
420 - Install `Docker CE for
421 CentOS <https://docs.docker.com/install/linux/docker-ce/centos/>`__
422 for Linux build hosts running the CentOS distribution.
423
424 - Install `Docker CE for
425 Debian <https://docs.docker.com/install/linux/docker-ce/debian/>`__
426 for Linux build hosts running the Debian distribution.
427
428 - Install `Docker CE for
429 Fedora <https://docs.docker.com/install/linux/docker-ce/fedora/>`__
430 for Linux build hosts running the Fedora distribution.
431
432 - Install `Docker CE for
433 Ubuntu <https://docs.docker.com/install/linux/docker-ce/ubuntu/>`__
434 for Linux build hosts running the Ubuntu distribution.
435
4365. *Optionally Orient Yourself With Docker:* If you are unfamiliar with
437 Docker and the container concept, you can learn more here -
438 https://docs.docker.com/get-started/.
439
4406. *Launch Docker or Docker Toolbox:* You should be able to launch
441 Docker or the Docker Toolbox and have a terminal shell on your
442 development host.
443
4447. *Set Up the Containers to Use the Yocto Project:* Go to
445 https://github.com/crops/docker-win-mac-docs/wiki and follow
446 the directions for your particular build host (i.e. Linux, Mac, or
447 Windows).
448
449 Once you complete the setup instructions for your machine, you have
450 the Poky, Extensible SDK, and Toaster containers available. You can
451 click those links from the page and learn more about using each of
452 those containers.
453
454Once you have a container set up, everything is in place to develop just
455as if you were running on a native Linux machine. If you are going to
456use the Poky container, see the "`Cloning the ``poky``
457Repository <#cloning-the-poky-repository>`__" section. If you are going
458to use the Extensible SDK container, see the
459":doc:`../sdk-manual/sdk-extensible`" Chapter in the Yocto
460Project Application Development and the Extensible Software Development
461Kit (eSDK) manual. If you are going to use the Toaster container, see
462the ":doc:`../toaster-manual/toaster-manual-setup-and-use`"
463section in the Toaster User Manual.
464
465.. _setting-up-to-use-wsl:
466
467Setting Up to Use Windows Subsystem For Linux (WSLv2)
468-----------------------------------------------------
469
470With `Windows Subsystem for Linux
471(WSLv2) <https://docs.microsoft.com/en-us/windows/wsl/wsl2-about>`__,
472you can create a Yocto Project development environment that allows you
473to build on Windows. You can set up a Linux distribution inside Windows
474in which you can develop using the Yocto Project.
475
476Follow these general steps to prepare a Windows machine using WSLv2 as
477your Yocto Project build host:
478
4791. *Make sure your Windows 10 machine is capable of running WSLv2:*
480 WSLv2 is only available for Windows 10 builds > 18917. To check which
481 build version you are running, you may open a command prompt on
482 Windows and execute the command "ver".
483 ::
484
485 C:\Users\myuser> ver
486
487 Microsoft Windows [Version 10.0.19041.153]
488
489 If your build is capable of running
490 WSLv2 you may continue, for more information on this subject or
491 instructions on how to upgrade to WSLv2 visit `Windows 10
492 WSLv2 <https://docs.microsoft.com/en-us/windows/wsl/wsl2-install>`__
493
4942. *Install the Linux distribution of your choice inside Windows 10:*
495 Once you know your version of Windows 10 supports WSLv2, you can
496 install the distribution of your choice from the Microsoft Store.
497 Open the Microsoft Store and search for Linux. While there are
498 several Linux distributions available, the assumption is that your
499 pick will be one of the distributions supported by the Yocto Project
500 as stated on the instructions for using a native Linux host. After
501 making your selection, simply click "Get" to download and install the
502 distribution.
503
5043. *Check your Linux distribution is using WSLv2:* Open a Windows
505 PowerShell and run:
506 ::
507
508 C:\WINDOWS\system32> wsl -l -v
509 NAME STATE VERSION
510 *Ubuntu Running 2
511
512 Note the version column which says the WSL version
513 being used by your distribution, on compatible systems, this can be
514 changed back at any point in time.
515
5164. *Optionally Orient Yourself on WSL:* If you are unfamiliar with WSL,
517 you can learn more here -
518 https://docs.microsoft.com/en-us/windows/wsl/wsl2-about.
519
5205. *Launch your WSL Distibution:* From the Windows start menu simply
521 launch your WSL distribution just like any other application.
522
5236. *Optimize your WSLv2 storage often:* Due to the way storage is
524 handled on WSLv2, the storage space used by the undelying Linux
525 distribution is not reflected immedately, and since bitbake heavily
526 uses storage, after several builds, you may be unaware you are
527 running out of space. WSLv2 uses a VHDX file for storage, this issue
528 can be easily avoided by manually optimizing this file often, this
529 can be done in the following way:
530
531 1. *Find the location of your VHDX file:* First you need to find the
532 distro app package directory, to achieve this open a Windows
533 Powershell as Administrator and run:
534 ::
535
536 C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
537 PackageFamilyName
538 -----------------
539 CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
540
541
542 You should now
543 replace the PackageFamilyName and your user on the following path
544 to find your VHDX file:
545 ::
546
547 ls C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
548 Mode LastWriteTime Length Name
549 -a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
550
551 Your VHDX file path is:
552 ``C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx``
553
554 2. *Optimize your VHDX file:* Open a Windows Powershell as
555 Administrator to optimize your VHDX file, shutting down WSL first:
556 ::
557
558 C:\WINDOWS\system32> wsl --shutdown
559 C:\WINDOWS\system32> optimize-vhd -Path C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
560
561 A progress bar should be shown while optimizing the
562 VHDX file, and storage should now be reflected correctly on the
563 Windows Explorer.
564
565.. note::
566
567 The current implementation of WSLv2 does not have out-of-the-box
568 access to external devices such as those connected through a USB
569 port, but it automatically mounts your
570 C:
571 drive on
572 /mnt/c/
573 (and others), which you can use to share deploy artifacts to be later
574 flashed on hardware through Windows, but your build directory should
575 not reside inside this mountpoint.
576
577Once you have WSLv2 set up, everything is in place to develop just as if
578you were running on a native Linux machine. If you are going to use the
579Extensible SDK container, see the ":doc:`../sdk-manual/sdk-extensible`" Chapter in the Yocto
580Project Application Development and the Extensible Software Development
581Kit (eSDK) manual. If you are going to use the Toaster container, see
582the ":doc:`../toaster-manual/toaster-manual-setup-and-use`"
583section in the Toaster User Manual.
584
585Locating Yocto Project Source Files
586===================================
587
588This section shows you how to locate, fetch and configure the source
589files you'll need to work with the Yocto Project.
590
591.. note::
592
593 - For concepts and introductory information about Git as it is used
594 in the Yocto Project, see the ":ref:`overview-manual/overview-manual-development-environment:git`"
595 section in the Yocto Project Overview and Concepts Manual.
596
597 - For concepts on Yocto Project source repositories, see the
598 ":ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`"
599 section in the Yocto Project Overview and Concepts Manual."
600
601Accessing Source Repositories
602-----------------------------
603
604Working from a copy of the upstream :ref:`dev-manual/dev-manual-start:accessing source repositories` is the
605preferred method for obtaining and using a Yocto Project release. You
606can view the Yocto Project Source Repositories at
607:yocto_git:`/`. In particular, you can find the ``poky``
608repository at :yocto_git:`/cgit.cgi/poky`.
609
610Use the following procedure to locate the latest upstream copy of the
611``poky`` Git repository:
612
6131. *Access Repositories:* Open a browser and go to
614 :yocto_git:`/` to access the GUI-based interface into the
615 Yocto Project source repositories.
616
6172. *Select the Repository:* Click on the repository in which you are
618 interested (e.g. ``poky``).
619
6203. *Find the URL Used to Clone the Repository:* At the bottom of the
621 page, note the URL used to clone that repository
622 (e.g. :yocto_git:`/cgit.cgi/poky`).
623
624 .. note::
625
626 For information on cloning a repository, see the "
627 Cloning the
628 poky
629 Repository
630 " section.
631
632Accessing Index of Releases
633---------------------------
634
635Yocto Project maintains an Index of Releases area that contains related
636files that contribute to the Yocto Project. Rather than Git
637repositories, these files are tarballs that represent snapshots in time
638of a given component.
639
640.. note::
641
642 The recommended method for accessing Yocto Project components is to
643 use Git to clone the upstream repository and work from within that
644 locally cloned repository. The procedure in this section exists
645 should you desire a tarball snapshot of any given component.
646
647Follow these steps to locate and download a particular tarball:
648
6491. *Access the Index of Releases:* Open a browser and go to
650 :yocto_dl:`Index of Releases </releases>`. The
651 list represents released components (e.g. ``bitbake``, ``sato``, and
652 so on).
653
654 .. note::
655
656 The
657 yocto
658 directory contains the full array of released Poky tarballs. The
659 poky
660 directory in the Index of Releases was historically used for very
661 early releases and exists now only for retroactive completeness.
662
6632. *Select a Component:* Click on any released component in which you
664 are interested (e.g. ``yocto``).
665
6663. *Find the Tarball:* Drill down to find the associated tarball. For
667 example, click on ``yocto-&DISTRO;`` to view files associated with the
668 Yocto Project &DISTRO; release (e.g.
669 ``&YOCTO_POKY;.tar.bz2``, which is the
670 released Poky tarball).
671
6724. *Download the Tarball:* Click the tarball to download and save a
673 snapshot of the given component.
674
675Using the Downloads Page
676------------------------
677
678The :yocto_home:`Yocto Project Website <>` uses a "DOWNLOADS" page
679from which you can locate and download tarballs of any Yocto Project
680release. Rather than Git repositories, these files represent snapshot
681tarballs similar to the tarballs located in the Index of Releases
682described in the "`Accessing Index of
683Releases <#accessing-index-of-releases>`__" section.
684
685.. note::
686
687 The recommended method for accessing Yocto Project components is to
688 use Git to clone a repository and work from within that local
689 repository. The procedure in this section exists should you desire a
690 tarball snapshot of any given component.
691
6921. *Go to the Yocto Project Website:* Open The
693 :yocto_home:`Yocto Project Website <>` in your browser.
694
6952. *Get to the Downloads Area:* Select the "DOWNLOADS" item from the
696 pull-down "SOFTWARE" tab menu near the top of the page.
697
6983. *Select a Yocto Project Release:* Use the menu next to "RELEASE" to
699 display and choose a recent or past supported Yocto Project release
700 (e.g. &DISTRO_NAME_NO_CAP;, &DISTRO_NAME_NO_CAP_MINUS_ONE;, and so forth).
701
702 .. note::
703
704 For a "map" of Yocto Project releases to version numbers, see the
705 Releases
706 wiki page.
707
708 You can use the "RELEASE ARCHIVE" link to reveal a menu of all Yocto
709 Project releases.
710
7114. *Download Tools or Board Support Packages (BSPs):* From the
712 "DOWNLOADS" page, you can download tools or BSPs as well. Just scroll
713 down the page and look for what you need.
714
715Accessing Nightly Builds
716------------------------
717
718Yocto Project maintains an area for nightly builds that contains tarball
719releases at https://autobuilder.yocto.io//pub/nightly/. These builds include Yocto
720Project releases ("poky"), toolchains, and builds for supported
721machines.
722
723Should you ever want to access a nightly build of a particular Yocto
724Project component, use the following procedure:
725
7261. *Locate the Index of Nightly Builds:* Open a browser and go to
727 https://autobuilder.yocto.io//pub/nightly/ to access the Nightly Builds.
728
7292. *Select a Date:* Click on the date in which you are interested. If
730 you want the latest builds, use "CURRENT".
731
7323. *Select a Build:* Choose the area in which you are interested. For
733 example, if you are looking for the most recent toolchains, select
734 the "toolchain" link.
735
7364. *Find the Tarball:* Drill down to find the associated tarball.
737
7385. *Download the Tarball:* Click the tarball to download and save a
739 snapshot of the given component.
740
741Cloning and Checking Out Branches
742=================================
743
744To use the Yocto Project for development, you need a release locally
745installed on your development system. This locally installed set of
746files is referred to as the :term:`Source Directory`
747in the Yocto Project documentation.
748
749The preferred method of creating your Source Directory is by using
750:ref:`overview-manual/overview-manual-development-environment:git` to clone a local copy of the upstream
751``poky`` repository. Working from a cloned copy of the upstream
752repository allows you to contribute back into the Yocto Project or to
753simply work with the latest software on a development branch. Because
754Git maintains and creates an upstream repository with a complete history
755of changes and you are working with a local clone of that repository,
756you have access to all the Yocto Project development branches and tag
757names used in the upstream repository.
758
759Cloning the ``poky`` Repository
760-------------------------------
761
762Follow these steps to create a local version of the upstream
763:term:`Poky` Git repository.
764
7651. *Set Your Directory:* Change your working directory to where you want
766 to create your local copy of ``poky``.
767
7682. *Clone the Repository:* The following example command clones the
769 ``poky`` repository and uses the default name "poky" for your local
770 repository:
771 ::
772
773 $ git clone git://git.yoctoproject.org/poky
774 Cloning into 'poky'...
775 remote: Counting objects: 432160, done.
776 remote: Compressing objects: 100% (102056/102056), done.
777 remote: Total 432160 (delta 323116), reused 432037 (delta 323000)
778 Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
779 Resolving deltas: 100% (323116/323116), done.
780 Checking connectivity... done.
781
782 Unless you
783 specify a specific development branch or tag name, Git clones the
784 "master" branch, which results in a snapshot of the latest
785 development changes for "master". For information on how to check out
786 a specific development branch or on how to check out a local branch
787 based on a tag name, see the "`Checking Out By Branch in
788 Poky <#checking-out-by-branch-in-poky>`__" and `Checking Out By Tag
789 in Poky <#checkout-out-by-tag-in-poky>`__" sections, respectively.
790
791 Once the local repository is created, you can change to that
792 directory and check its status. Here, the single "master" branch
793 exists on your system and by default, it is checked out:
794 ::
795
796 $ cd ~/poky
797 $ git status
798 On branch master
799 Your branch is up-to-date with 'origin/master'.
800 nothing to commit, working directory clean
801 $ git branch
802 * master
803
804 Your local repository of poky is identical to the
805 upstream poky repository at the time from which it was cloned. As you
806 work with the local branch, you can periodically use the
807 ``git pull --rebase`` command to be sure you are up-to-date
808 with the upstream branch.
809
810Checking Out by Branch in Poky
811------------------------------
812
813When you clone the upstream poky repository, you have access to all its
814development branches. Each development branch in a repository is unique
815as it forks off the "master" branch. To see and use the files of a
816particular development branch locally, you need to know the branch name
817and then specifically check out that development branch.
818
819.. note::
820
821 Checking out an active development branch by branch name gives you a
822 snapshot of that particular branch at the time you check it out.
823 Further development on top of the branch that occurs after check it
824 out can occur.
825
8261. *Switch to the Poky Directory:* If you have a local poky Git
827 repository, switch to that directory. If you do not have the local
828 copy of poky, see the "`Cloning the ``poky``
829 Repository <#cloning-the-poky-repository>`__" section.
830
8312. *Determine Existing Branch Names:*
832 ::
833
834 $ git branch -a
835 * master
836 remotes/origin/1.1_M1
837 remotes/origin/1.1_M2
838 remotes/origin/1.1_M3
839 remotes/origin/1.1_M4
840 remotes/origin/1.2_M1
841 remotes/origin/1.2_M2
842 remotes/origin/1.2_M3
843 . . .
844 remotes/origin/thud
845 remotes/origin/thud-next
846 remotes/origin/warrior
847 remotes/origin/warrior-next
848 remotes/origin/zeus
849 remotes/origin/zeus-next
850 ... and so on ...
851
8523. *Check out the Branch:* Check out the development branch in which you
853 want to work. For example, to access the files for the Yocto Project
854 &DISTRO; Release (&DISTRO_NAME;), use the following command:
855 ::
856
857 $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
858 Branch &DISTRO_NAME; set up to track remote branch &DISTRO_NAME; from origin.
859 Switched to a new branch '&DISTRO_NAME;'
860
861 The previous command checks out the "&DISTRO_NAME;" development
862 branch and reports that the branch is tracking the upstream
863 "origin/&DISTRO_NAME;" branch.
864
865 The following command displays the branches that are now part of your
866 local poky repository. The asterisk character indicates the branch
867 that is currently checked out for work:
868 ::
869
870 $ git branch
871 master *
872 &DISTRO_NAME;
873
874.. _checkout-out-by-tag-in-poky:
875
876Checking Out by Tag in Poky
877---------------------------
878
879Similar to branches, the upstream repository uses tags to mark specific
880commits associated with significant points in a development branch (i.e.
881a release point or stage of a release). You might want to set up a local
882branch based on one of those points in the repository. The process is
883similar to checking out by branch name except you use tag names.
884
885.. note::
886
887 Checking out a branch based on a tag gives you a stable set of files
888 not affected by development on the branch above the tag.
889
8901. *Switch to the Poky Directory:* If you have a local poky Git
891 repository, switch to that directory. If you do not have the local
892 copy of poky, see the "`Cloning the ``poky``
893 Repository <#cloning-the-poky-repository>`__" section.
894
8952. *Fetch the Tag Names:* To checkout the branch based on a tag name,
896 you need to fetch the upstream tags into your local repository:
897 ::
898
899 $ git fetch --tags
900 $
901
9023. *List the Tag Names:* You can list the tag names now:
903 ::
904
905 $ git tag
906 1.1_M1.final
907 1.1_M1.rc1
908 1.1_M1.rc2
909 1.1_M2.final
910 1.1_M2.rc1
911 .
912 .
913 .
914 yocto-2.5
915 yocto-2.5.1
916 yocto-2.5.2
917 yocto-2.5.3
918 yocto-2.6
919 yocto-2.6.1
920 yocto-2.6.2
921 yocto-2.7
922 yocto_1.5_M5.rc8
923
924
9254. *Check out the Branch:*
926 ::
927
928 $ git checkout tags/yocto-&DISTRO; -b my_yocto_&DISTRO;
929 Switched to a new branch 'my_yocto_&DISTRO;'
930 $ git branch
931 master
932 * my_yocto_&DISTRO;
933
934 The previous command creates and
935 checks out a local branch named "my_yocto_&DISTRO;", which is based on
936 the commit in the upstream poky repository that has the same tag. In
937 this example, the files you have available locally as a result of the
938 ``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;"
939 development branch at the point where Yocto Project &DISTRO; was
940 released.
diff --git a/documentation/dev-manual/dev-manual-start.xml b/documentation/dev-manual/dev-manual-start.xml
index 8cb5631f0d..9ff9ac4c8f 100644
--- a/documentation/dev-manual/dev-manual-start.xml
+++ b/documentation/dev-manual/dev-manual-start.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='dev-manual-start'> 6<chapter id='dev-manual-start'>
6 7
diff --git a/documentation/dev-manual/dev-manual.rst b/documentation/dev-manual/dev-manual.rst
new file mode 100644
index 0000000000..c629067153
--- /dev/null
+++ b/documentation/dev-manual/dev-manual.rst
@@ -0,0 +1,19 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3======================================
4Yocto Project Development Tasks Manual
5======================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 dev-manual-intro
14 dev-manual-start
15 dev-manual-common-tasks
16 dev-manual-qemu
17 history
18
19.. include:: /boilerplate.rst
diff --git a/documentation/dev-manual/dev-manual.xml b/documentation/dev-manual/dev-manual.xml
index 26d37da354..66439930e4 100755
--- a/documentation/dev-manual/dev-manual.xml
+++ b/documentation/dev-manual/dev-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='dev-manual' lang='en' 6<book id='dev-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -117,28 +118,8 @@
117 </revision> 118 </revision>
118 <revision> 119 <revision>
119 <revnumber>3.1</revnumber> 120 <revnumber>3.1</revnumber>
120 <date>April 2020</date>
121 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
122 </revision>
123 <revision>
124 <revnumber>3.1.1</revnumber>
125 <date>June 2020</date>
126 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
127 </revision>
128 <revision>
129 <revnumber>3.1.2</revnumber>
130 <date>August 2020</date>
131 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
132 </revision>
133 <revision>
134 <revnumber>3.1.3</revnumber>
135 <date>October 2020</date>
136 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
137 </revision>
138 <revision>
139 <revnumber>3.1.4</revnumber>
140 <date>&REL_MONTH_YEAR;</date> 121 <date>&REL_MONTH_YEAR;</date>
141 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 122 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
142 </revision> 123 </revision>
143 </revhistory> 124 </revhistory>
144 125
diff --git a/documentation/dev-manual/dev-style.css b/documentation/dev-manual/dev-style.css
index 6d0aa8e9fa..331c7c54d4 100644
--- a/documentation/dev-manual/dev-style.css
+++ b/documentation/dev-manual/dev-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/dev-manual/history.rst b/documentation/dev-manual/history.rst
new file mode 100644
index 0000000000..8b149a6ef5
--- /dev/null
+++ b/documentation/dev-manual/history.rst
@@ -0,0 +1,67 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 1.1
15 - October 2011
16 - The initial document released with the Yocto Project 1.1 Release
17 * - 1.2
18 - April 2012
19 - Released with the Yocto Project 1.2 Release.
20 * - 1.3
21 - October 2012
22 - Released with the Yocto Project 1.3 Release.
23 * - 1.4
24 - April 2013
25 - Released with the Yocto Project 1.4 Release.
26 * - 1.5
27 - October 2013
28 - Released with the Yocto Project 1.5 Release.
29 * - 1.6
30 - April 2014
31 - Released with the Yocto Project 1.6 Release.
32 * - 1.7
33 - October 2014
34 - Released with the Yocto Project 1.7 Release.
35 * - 1.8
36 - April 2015
37 - Released with the Yocto Project 1.8 Release.
38 * - 2.0
39 - October 2015
40 - Released with the Yocto Project 2.0 Release.
41 * - 2.1
42 - April 2016
43 - Released with the Yocto Project 2.1 Release.
44 * - 2.2
45 - October 2016
46 - Released with the Yocto Project 2.2 Release.
47 * - 2.3
48 - May 2017
49 - Released with the Yocto Project 2.3 Release.
50 * - 2.4
51 - October 2017
52 - Released with the Yocto Project 2.4 Release.
53 * - 2.5
54 - May 2018
55 - Released with the Yocto Project 2.5 Release.
56 * - 2.6
57 - November 2018
58 - Released with the Yocto Project 2.6 Release.
59 * - 2.7
60 - May 2019
61 - Released with the Yocto Project 2.7 Release.
62 * - 3.0
63 - October 2019
64 - Released with the Yocto Project 3.0 Release.
65 * - 3.1
66 - April 2020
67 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/figures/yp-how-it-works-new-diagram.png b/documentation/figures/yp-how-it-works-new-diagram.png
new file mode 100644
index 0000000000..2ce076f3c3
--- /dev/null
+++ b/documentation/figures/yp-how-it-works-new-diagram.png
Binary files differ
diff --git a/documentation/genindex.rst b/documentation/genindex.rst
new file mode 100644
index 0000000000..a4af06f656
--- /dev/null
+++ b/documentation/genindex.rst
@@ -0,0 +1,3 @@
1=====
2Index
3=====
diff --git a/documentation/index.rst b/documentation/index.rst
new file mode 100644
index 0000000000..258ecb81a7
--- /dev/null
+++ b/documentation/index.rst
@@ -0,0 +1,53 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3.. The Yocto Project documentation master file, created by
4 sphinx-quickstart on Mon Apr 13 09:38:33 2020.
5 You can adapt this file completely to your liking, but it should at least
6 contain the root `toctree` directive.
7
8Welcome to The Yocto Project's documentation!
9=============================================
10
11|
12
13.. toctree::
14 :maxdepth: 1
15 :caption: Introduction and Overview
16
17 Quick Build <brief-yoctoprojectqs/brief-yoctoprojectqs>
18 what-i-wish-id-known
19 transitioning-to-a-custom-environment
20 Yocto Project Software Overview <https://www.yoctoproject.org/software-overview/>
21 Tips and Tricks Wiki <https://wiki.yoctoproject.org/wiki/TipsAndTricks>
22
23
24.. toctree::
25 :maxdepth: 1
26 :caption: Manuals
27
28 Overview and Concepts Manual <overview-manual/overview-manual>
29 Reference Manual <ref-manual/ref-manual>
30 Board Support Package (BSP) Developer's guide <bsp-guide/bsp-guide>
31 Development Tasks Manual <dev-manual/dev-manual>
32 Linux Kernel Development Manual <kernel-dev/kernel-dev>
33 Profile and Tracing Manual <profile-manual/profile-manual>
34 Application Development and the Extensible SDK (eSDK) <sdk-manual/sdk-manual>
35 Toaster Manual <toaster-manual/toaster-manual>
36 Test Environment Manual <test-manual/test-manual>
37 Bitbake User Manual <https://docs.yoctoproject.org/bitbake>
38
39.. toctree::
40 :maxdepth: 1
41 :caption: 'Mega' Manual
42
43 All-in-one 'Mega' Manual <https://docs.yoctoproject.org/singleindex.html>
44
45.. toctree::
46 :maxdepth: 1
47 :caption: Manuals/Variable Index
48
49 genindex
50 Current/Previous Version Specific Manuals <releases>
51
52
53
diff --git a/documentation/kernel-dev/history.rst b/documentation/kernel-dev/history.rst
new file mode 100644
index 0000000000..3ffb7eacb8
--- /dev/null
+++ b/documentation/kernel-dev/history.rst
@@ -0,0 +1,58 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 1.4
15 - April 2013
16 - The initial document released with the Yocto Project 1.4 Release
17 * - 1.5
18 - October 2013
19 - Released with the Yocto Project 1.5 Release.
20 * - 1.6
21 - April 2014
22 - Released with the Yocto Project 1.6 Release.
23 * - 1.7
24 - October 2014
25 - Released with the Yocto Project 1.7 Release.
26 * - 1.8
27 - April 2015
28 - Released with the Yocto Project 1.8 Release.
29 * - 2.0
30 - October 2015
31 - Released with the Yocto Project 2.0 Release.
32 * - 2.1
33 - April 2016
34 - Released with the Yocto Project 2.1 Release.
35 * - 2.2
36 - October 2016
37 - Released with the Yocto Project 2.2 Release.
38 * - 2.3
39 - May 2017
40 - Released with the Yocto Project 2.3 Release.
41 * - 2.4
42 - October 2017
43 - Released with the Yocto Project 2.4 Release.
44 * - 2.5
45 - May 2018
46 - Released with the Yocto Project 2.5 Release.
47 * - 2.6
48 - November 2018
49 - Released with the Yocto Project 2.6 Release.
50 * - 2.7
51 - May 2019
52 - Released with the Yocto Project 2.7 Release.
53 * - 3.0
54 - October 2019
55 - Released with the Yocto Project 3.0 Release.
56 * - 3.1
57 - April 2020
58 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/kernel-dev/kernel-dev-advanced.rst b/documentation/kernel-dev/kernel-dev-advanced.rst
new file mode 100644
index 0000000000..36133caae3
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-advanced.rst
@@ -0,0 +1,983 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************************************************
4Working with Advanced Metadata (``yocto-kernel-cache``)
5*******************************************************
6
7.. _kernel-dev-advanced-overview:
8
9Overview
10========
11
12In addition to supporting configuration fragments and patches, the Yocto
13Project kernel tools also support rich
14:term:`Metadata` that you can use to define
15complex policies and Board Support Package (BSP) support. The purpose of
16the Metadata and the tools that manage it is to help you manage the
17complexity of the configuration and sources used to support multiple
18BSPs and Linux kernel types.
19
20Kernel Metadata exists in many places. One area in the
21:ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`
22is the ``yocto-kernel-cache`` Git repository. You can find this repository
23grouped under the "Yocto Linux Kernel" heading in the
24:yocto_git:`Yocto Project Source Repositories <>`.
25
26Kernel development tools ("kern-tools") exist also in the Yocto Project
27Source Repositories under the "Yocto Linux Kernel" heading in the
28``yocto-kernel-tools`` Git repository. The recipe that builds these
29tools is ``meta/recipes-kernel/kern-tools/kern-tools-native_git.bb`` in
30the :term:`Source Directory` (e.g.
31``poky``).
32
33Using Kernel Metadata in a Recipe
34=================================
35
36As mentioned in the introduction, the Yocto Project contains kernel
37Metadata, which is located in the ``yocto-kernel-cache`` Git repository.
38This Metadata defines Board Support Packages (BSPs) that correspond to
39definitions in linux-yocto recipes for corresponding BSPs. A BSP
40consists of an aggregation of kernel policy and enabled
41hardware-specific features. The BSP can be influenced from within the
42linux-yocto recipe.
43
44.. note::
45
46 A Linux kernel recipe that contains kernel Metadata (e.g. inherits
47 from the
48 linux-yocto.inc
49 file) is said to be a "linux-yocto style" recipe.
50
51Every linux-yocto style recipe must define the
52:term:`KMACHINE` variable. This
53variable is typically set to the same value as the ``MACHINE`` variable,
54which is used by :term:`BitBake`.
55However, in some cases, the variable might instead refer to the
56underlying platform of the ``MACHINE``.
57
58Multiple BSPs can reuse the same ``KMACHINE`` name if they are built
59using the same BSP description. Multiple Corei7-based BSPs could share
60the same "intel-corei7-64" value for ``KMACHINE``. It is important to
61realize that ``KMACHINE`` is just for kernel mapping, while ``MACHINE``
62is the machine type within a BSP Layer. Even with this distinction,
63however, these two variables can hold the same value. See the `BSP
64Descriptions <#bsp-descriptions>`__ section for more information.
65
66Every linux-yocto style recipe must also indicate the Linux kernel
67source repository branch used to build the Linux kernel. The
68:term:`KBRANCH` variable must be set
69to indicate the branch.
70
71.. note::
72
73 You can use the
74 KBRANCH
75 value to define an alternate branch typically with a machine override
76 as shown here from the
77 meta-yocto-bsp
78 layer:
79 ::
80
81 KBRANCH_edgerouter = "standard/edgerouter"
82
83
84The linux-yocto style recipes can optionally define the following
85variables:
86
87 - :term:`KERNEL_FEATURES`
88
89 - :term:`LINUX_KERNEL_TYPE`
90
91:term:`LINUX_KERNEL_TYPE`
92defines the kernel type to be used in assembling the configuration. If
93you do not specify a ``LINUX_KERNEL_TYPE``, it defaults to "standard".
94Together with ``KMACHINE``, ``LINUX_KERNEL_TYPE`` defines the search
95arguments used by the kernel tools to find the appropriate description
96within the kernel Metadata with which to build out the sources and
97configuration. The linux-yocto recipes define "standard", "tiny", and
98"preempt-rt" kernel types. See the "`Kernel Types <#kernel-types>`__"
99section for more information on kernel types.
100
101During the build, the kern-tools search for the BSP description file
102that most closely matches the ``KMACHINE`` and ``LINUX_KERNEL_TYPE``
103variables passed in from the recipe. The tools use the first BSP
104description it finds that match both variables. If the tools cannot find
105a match, they issue a warning.
106
107The tools first search for the ``KMACHINE`` and then for the
108``LINUX_KERNEL_TYPE``. If the tools cannot find a partial match, they
109will use the sources from the ``KBRANCH`` and any configuration
110specified in the :term:`SRC_URI`.
111
112You can use the
113:term:`KERNEL_FEATURES`
114variable to include features (configuration fragments, patches, or both)
115that are not already included by the ``KMACHINE`` and
116``LINUX_KERNEL_TYPE`` variable combination. For example, to include a
117feature specified as "features/netfilter/netfilter.scc", specify:
118::
119
120 KERNEL_FEATURES += "features/netfilter/netfilter.scc"
121
122To include a
123feature called "cfg/sound.scc" just for the ``qemux86`` machine,
124specify:
125::
126
127 KERNEL_FEATURES_append_qemux86 = " cfg/sound.scc"
128
129The value of
130the entries in ``KERNEL_FEATURES`` are dependent on their location
131within the kernel Metadata itself. The examples here are taken from the
132``yocto-kernel-cache`` repository. Each branch of this repository
133contains "features" and "cfg" subdirectories at the top-level. For more
134information, see the "`Kernel Metadata
135Syntax <#kernel-metadata-syntax>`__" section.
136
137Kernel Metadata Syntax
138======================
139
140The kernel Metadata consists of three primary types of files: ``scc``
141[1]_ description files, configuration fragments, and patches. The
142``scc`` files define variables and include or otherwise reference any of
143the three file types. The description files are used to aggregate all
144types of kernel Metadata into what ultimately describes the sources and
145the configuration required to build a Linux kernel tailored to a
146specific machine.
147
148The ``scc`` description files are used to define two fundamental types
149of kernel Metadata:
150
151- Features
152
153- Board Support Packages (BSPs)
154
155Features aggregate sources in the form of patches and configuration
156fragments into a modular reusable unit. You can use features to
157implement conceptually separate kernel Metadata descriptions such as
158pure configuration fragments, simple patches, complex features, and
159kernel types. `Kernel types <#kernel-types>`__ define general kernel
160features and policy to be reused in the BSPs.
161
162BSPs define hardware-specific features and aggregate them with kernel
163types to form the final description of what will be assembled and built.
164
165While the kernel Metadata syntax does not enforce any logical separation
166of configuration fragments, patches, features or kernel types, best
167practices dictate a logical separation of these types of Metadata. The
168following Metadata file hierarchy is recommended:
169::
170
171 base/
172 bsp/
173 cfg/
174 features/
175 ktypes/
176 patches/
177
178The ``bsp`` directory contains the `BSP
179descriptions <#bsp-descriptions>`__. The remaining directories all
180contain "features". Separating ``bsp`` from the rest of the structure
181aids conceptualizing intended usage.
182
183Use these guidelines to help place your ``scc`` description files within
184the structure:
185
186- If your file contains only configuration fragments, place the file in
187 the ``cfg`` directory.
188
189- If your file contains only source-code fixes, place the file in the
190 ``patches`` directory.
191
192- If your file encapsulates a major feature, often combining sources
193 and configurations, place the file in ``features`` directory.
194
195- If your file aggregates non-hardware configuration and patches in
196 order to define a base kernel policy or major kernel type to be
197 reused across multiple BSPs, place the file in ``ktypes`` directory.
198
199These distinctions can easily become blurred - especially as out-of-tree
200features slowly merge upstream over time. Also, remember that how the
201description files are placed is a purely logical organization and has no
202impact on the functionality of the kernel Metadata. There is no impact
203because all of ``cfg``, ``features``, ``patches``, and ``ktypes``,
204contain "features" as far as the kernel tools are concerned.
205
206Paths used in kernel Metadata files are relative to base, which is
207either
208:term:`FILESEXTRAPATHS` if
209you are creating Metadata in `recipe-space <#recipe-space-metadata>`__,
210or the top level of
211:yocto_git:`yocto-kernel-cache </cgit/cgit.cgi/yocto-kernel-cache/tree/>`
212if you are creating `Metadata outside of the
213recipe-space <#metadata-outside-the-recipe-space>`__.
214
215.. [1]
216 ``scc`` stands for Series Configuration Control, but the naming has
217 less significance in the current implementation of the tooling than
218 it had in the past. Consider ``scc`` files to be description files.
219
220Configuration
221-------------
222
223The simplest unit of kernel Metadata is the configuration-only feature.
224This feature consists of one or more Linux kernel configuration
225parameters in a configuration fragment file (``.cfg``) and a ``.scc``
226file that describes the fragment.
227
228As an example, consider the Symmetric Multi-Processing (SMP) fragment
229used with the ``linux-yocto-4.12`` kernel as defined outside of the
230recipe space (i.e. ``yocto-kernel-cache``). This Metadata consists of
231two files: ``smp.scc`` and ``smp.cfg``. You can find these files in the
232``cfg`` directory of the ``yocto-4.12`` branch in the
233``yocto-kernel-cache`` Git repository:
234::
235
236 cfg/smp.scc:
237 define KFEATURE_DESCRIPTION "Enable SMP for 32 bit builds"
238 define KFEATURE_COMPATIBILITY all
239
240 kconf hardware smp.cfg
241
242 cfg/smp.cfg:
243 CONFIG_SMP=y
244 CONFIG_SCHED_SMT=y
245 # Increase default NR_CPUS from 8 to 64 so that platform with
246 # more than 8 processors can be all activated at boot time
247 CONFIG_NR_CPUS=64
248 # The following is needed when setting NR_CPUS to something
249 # greater than 8 on x86 architectures, it should be automatically
250 # disregarded by Kconfig when using a different arch
251 CONFIG_X86_BIGSMP=y
252
253You can find general information on configuration
254fragment files in the "`Creating Configuration
255Fragments <#creating-config-fragments>`__" section.
256
257Within the ``smp.scc`` file, the
258:term:`KFEATURE_DESCRIPTION`
259statement provides a short description of the fragment. Higher level
260kernel tools use this description.
261
262Also within the ``smp.scc`` file, the ``kconf`` command includes the
263actual configuration fragment in an ``.scc`` file, and the "hardware"
264keyword identifies the fragment as being hardware enabling, as opposed
265to general policy, which would use the "non-hardware" keyword. The
266distinction is made for the benefit of the configuration validation
267tools, which warn you if a hardware fragment overrides a policy set by a
268non-hardware fragment.
269
270.. note::
271
272 The description file can include multiple
273 kconf
274 statements, one per fragment.
275
276As described in the "`Validating
277Configuration <#validating-configuration>`__" section, you can use the
278following BitBake command to audit your configuration:
279::
280
281 $ bitbake linux-yocto -c kernel_configcheck -f
282
283Patches
284-------
285
286Patch descriptions are very similar to configuration fragment
287descriptions, which are described in the previous section. However,
288instead of a ``.cfg`` file, these descriptions work with source patches
289(i.e. ``.patch`` files).
290
291A typical patch includes a description file and the patch itself. As an
292example, consider the build patches used with the ``linux-yocto-4.12``
293kernel as defined outside of the recipe space (i.e.
294``yocto-kernel-cache``). This Metadata consists of several files:
295``build.scc`` and a set of ``*.patch`` files. You can find these files
296in the ``patches/build`` directory of the ``yocto-4.12`` branch in the
297``yocto-kernel-cache`` Git repository.
298
299The following listings show the ``build.scc`` file and part of the
300``modpost-mask-trivial-warnings.patch`` file:
301::
302
303 patches/build/build.scc:
304 patch arm-serialize-build-targets.patch
305 patch powerpc-serialize-image-targets.patch
306 patch kbuild-exclude-meta-directory-from-distclean-processi.patch
307
308 # applied by kgit
309 # patch kbuild-add-meta-files-to-the-ignore-li.patch
310
311 patch modpost-mask-trivial-warnings.patch
312 patch menuconfig-check-lxdiaglog.sh-Allow-specification-of.patch
313
314 patches/build/modpost-mask-trivial-warnings.patch:
315 From bd48931bc142bdd104668f3a062a1f22600aae61 Mon Sep 17 00:00:00 2001
316 From: Paul Gortmaker <paul.gortmaker@windriver.com>
317 Date: Sun, 25 Jan 2009 17:58:09 -0500
318 Subject: [PATCH] modpost: mask trivial warnings
319
320 Newer HOSTCC will complain about various stdio fcns because
321 .
322 .
323 .
324 char *dump_write = NULL, *files_source = NULL;
325 int opt;
326 --
327 2.10.1
328
329 generated by cgit v0.10.2 at 2017-09-28 15:23:23 (GMT)
330
331The description file can
332include multiple patch statements where each statement handles a single
333patch. In the example ``build.scc`` file, five patch statements exist
334for the five patches in the directory.
335
336You can create a typical ``.patch`` file using ``diff -Nurp`` or
337``git format-patch`` commands. For information on how to create patches,
338see the "`Using ``devtool`` to Patch the
339Kernel <#using-devtool-to-patch-the-kernel>`__" and "`Using Traditional
340Kernel Development to Patch the
341Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
342sections.
343
344Features
345--------
346
347Features are complex kernel Metadata types that consist of configuration
348fragments, patches, and possibly other feature description files. As an
349example, consider the following generic listing:
350::
351
352 features/myfeature.scc
353 define KFEATURE_DESCRIPTION "Enable myfeature"
354
355 patch 0001-myfeature-core.patch
356 patch 0002-myfeature-interface.patch
357
358 include cfg/myfeature_dependency.scc
359 kconf non-hardware myfeature.cfg
360
361This example shows how the ``patch`` and ``kconf`` commands are used as well
362as how an additional feature description file is included with the
363``include`` command.
364
365Typically, features are less granular than configuration fragments and
366are more likely than configuration fragments and patches to be the types
367of things you want to specify in the ``KERNEL_FEATURES`` variable of the
368Linux kernel recipe. See the "`Using Kernel Metadata in a
369Recipe <#using-kernel-metadata-in-a-recipe>`__" section earlier in the
370manual.
371
372Kernel Types
373------------
374
375A kernel type defines a high-level kernel policy by aggregating
376non-hardware configuration fragments with patches you want to use when
377building a Linux kernel of a specific type (e.g. a real-time kernel).
378Syntactically, kernel types are no different than features as described
379in the "`Features <#features>`__" section. The
380:term:`LINUX_KERNEL_TYPE`
381variable in the kernel recipe selects the kernel type. For example, in
382the ``linux-yocto_4.12.bb`` kernel recipe found in
383``poky/meta/recipes-kernel/linux``, a
384:ref:`require <bitbake:require-inclusion>` directive
385includes the ``poky/meta/recipes-kernel/linux/linux-yocto.inc`` file,
386which has the following statement that defines the default kernel type:
387::
388
389 LINUX_KERNEL_TYPE ??= "standard"
390
391Another example would be the real-time kernel (i.e.
392``linux-yocto-rt_4.12.bb``). This kernel recipe directly sets the kernel
393type as follows:
394::
395
396 LINUX_KERNEL_TYPE = "preempt-rt"
397
398.. note::
399
400 You can find kernel recipes in the
401 meta/recipes-kernel/linux
402 directory of the
403 Source Directory
404 (e.g.
405 poky/meta/recipes-kernel/linux/linux-yocto_4.12.bb
406 ). See the "
407 Using Kernel Metadata in a Recipe
408 " section for more information.
409
410Three kernel types ("standard", "tiny", and "preempt-rt") are supported
411for Linux Yocto kernels:
412
413- "standard": Includes the generic Linux kernel policy of the Yocto
414 Project linux-yocto kernel recipes. This policy includes, among other
415 things, which file systems, networking options, core kernel features,
416 and debugging and tracing options are supported.
417
418- "preempt-rt": Applies the ``PREEMPT_RT`` patches and the
419 configuration options required to build a real-time Linux kernel.
420 This kernel type inherits from the "standard" kernel type.
421
422- "tiny": Defines a bare minimum configuration meant to serve as a base
423 for very small Linux kernels. The "tiny" kernel type is independent
424 from the "standard" configuration. Although the "tiny" kernel type
425 does not currently include any source changes, it might in the
426 future.
427
428For any given kernel type, the Metadata is defined by the ``.scc`` (e.g.
429``standard.scc``). Here is a partial listing for the ``standard.scc``
430file, which is found in the ``ktypes/standard`` directory of the
431``yocto-kernel-cache`` Git repository:
432::
433
434 # Include this kernel type fragment to get the standard features and
435 # configuration values.
436
437 # Note: if only the features are desired, but not the configuration
438 # then this should be included as:
439 # include ktypes/standard/standard.scc nocfg
440 # if no chained configuration is desired, include it as:
441 # include ktypes/standard/standard.scc nocfg inherit
442
443
444
445 include ktypes/base/base.scc
446 branch standard
447
448 kconf non-hardware standard.cfg
449
450 include features/kgdb/kgdb.scc
451 .
452 .
453 .
454
455 include cfg/net/ip6_nf.scc
456 include cfg/net/bridge.scc
457
458 include cfg/systemd.scc
459
460 include features/rfkill/rfkill.scc
461
462As with any ``.scc`` file, a kernel type definition can aggregate other
463``.scc`` files with ``include`` commands. These definitions can also
464directly pull in configuration fragments and patches with the ``kconf``
465and ``patch`` commands, respectively.
466
467.. note::
468
469 It is not strictly necessary to create a kernel type
470 .scc
471 file. The Board Support Package (BSP) file can implicitly define the
472 kernel type using a
473 define
474 KTYPE
475 myktype
476 line. See the "
477 BSP Descriptions
478 " section for more information.
479
480BSP Descriptions
481----------------
482
483BSP descriptions (i.e. ``*.scc`` files) combine kernel types with
484hardware-specific features. The hardware-specific Metadata is typically
485defined independently in the BSP layer, and then aggregated with each
486supported kernel type.
487
488.. note::
489
490 For BSPs supported by the Yocto Project, the BSP description files
491 are located in the
492 bsp
493 directory of the
494 yocto-kernel-cache
495 repository organized under the "Yocto Linux Kernel" heading in the
496 Yocto Project Source Repositories
497 .
498
499This section overviews the BSP description structure, the aggregation
500concepts, and presents a detailed example using a BSP supported by the
501Yocto Project (i.e. BeagleBone Board). For complete information on BSP
502layer file hierarchy, see the :doc:`../bsp-guide/bsp-guide`.
503
504.. _bsp-description-file-overview:
505
506Description Overview
507~~~~~~~~~~~~~~~~~~~~
508
509For simplicity, consider the following root BSP layer description files
510for the BeagleBone board. These files employ both a structure and naming
511convention for consistency. The naming convention for the file is as
512follows:
513::
514
515 bsp_root_name-kernel_type.scc
516
517Here are some example root layer
518BSP filenames for the BeagleBone Board BSP, which is supported by the
519Yocto Project:
520::
521
522 beaglebone-standard.scc
523 beaglebone-preempt-rt.scc
524
525Each file uses the root name (i.e "beaglebone") BSP name followed by the
526kernel type.
527
528Examine the ``beaglebone-standard.scc`` file:
529::
530
531 define KMACHINE beaglebone
532 define KTYPE standard
533 define KARCH arm
534
535 include ktypes/standard/standard.scc
536 branch beaglebone
537
538 include beaglebone.scc
539
540 # default policy for standard kernels
541 include features/latencytop/latencytop.scc
542 include features/profiling/profiling.scc
543
544Every top-level BSP description file
545should define the :term:`KMACHINE`,
546:term:`KTYPE`, and
547:term:`KARCH` variables. These
548variables allow the OpenEmbedded build system to identify the
549description as meeting the criteria set by the recipe being built. This
550example supports the "beaglebone" machine for the "standard" kernel and
551the "arm" architecture.
552
553Be aware that a hard link between the ``KTYPE`` variable and a kernel
554type description file does not exist. Thus, if you do not have the
555kernel type defined in your kernel Metadata as it is here, you only need
556to ensure that the
557:term:`LINUX_KERNEL_TYPE`
558variable in the kernel recipe and the ``KTYPE`` variable in the BSP
559description file match.
560
561To separate your kernel policy from your hardware configuration, you
562include a kernel type (``ktype``), such as "standard". In the previous
563example, this is done using the following:
564::
565
566 include ktypes/standard/standard.scc
567
568This file aggregates all the configuration
569fragments, patches, and features that make up your standard kernel
570policy. See the "`Kernel Types <#kernel-types>`__" section for more
571information.
572
573To aggregate common configurations and features specific to the kernel
574for mybsp, use the following:
575::
576
577 include mybsp.scc
578
579You can see that in the BeagleBone example with the following:
580::
581
582 include beaglebone.scc
583
584For information on how to break a complete ``.config`` file into the various
585configuration fragments, see the "`Creating Configuration
586Fragments <#creating-config-fragments>`__" section.
587
588Finally, if you have any configurations specific to the hardware that
589are not in a ``*.scc`` file, you can include them as follows:
590::
591
592 kconf hardware mybsp-extra.cfg
593
594The BeagleBone example does not include these
595types of configurations. However, the Malta 32-bit board does
596("mti-malta32"). Here is the ``mti-malta32-le-standard.scc`` file:
597::
598
599 define KMACHINE mti-malta32-le
600 define KMACHINE qemumipsel
601 define KTYPE standard
602 define KARCH mips
603
604 include ktypes/standard/standard.scc
605 branch mti-malta32
606
607 include mti-malta32.scc
608 kconf hardware mti-malta32-le.cfg
609
610.. _bsp-description-file-example-minnow:
611
612Example
613~~~~~~~
614
615Many real-world examples are more complex. Like any other ``.scc`` file,
616BSP descriptions can aggregate features. Consider the Minnow BSP
617definition given the ``linux-yocto-4.4`` branch of the
618``yocto-kernel-cache`` (i.e.
619``yocto-kernel-cache/bsp/minnow/minnow.scc``):
620
621.. note::
622
623 Although the Minnow Board BSP is unused, the Metadata remains and is
624 being used here just as an example.
625
626::
627
628 include cfg/x86.scc
629 include features/eg20t/eg20t.scc
630 include cfg/dmaengine.scc
631 include features/power/intel.scc
632 include cfg/efi.scc
633 include features/usb/ehci-hcd.scc
634 include features/usb/ohci-hcd.scc
635 include features/usb/usb-gadgets.scc
636 include features/usb/touchscreen-composite.scc
637 include cfg/timer/hpet.scc
638 include features/leds/leds.scc
639 include features/spi/spidev.scc
640 include features/i2c/i2cdev.scc
641 include features/mei/mei-txe.scc
642
643 # Earlyprintk and port debug requires 8250
644 kconf hardware cfg/8250.cfg
645
646 kconf hardware minnow.cfg
647 kconf hardware minnow-dev.cfg
648
649The ``minnow.scc`` description file includes a hardware configuration
650fragment (``minnow.cfg``) specific to the Minnow BSP as well as several
651more general configuration fragments and features enabling hardware
652found on the machine. This ``minnow.scc`` description file is then
653included in each of the three "minnow" description files for the
654supported kernel types (i.e. "standard", "preempt-rt", and "tiny").
655Consider the "minnow" description for the "standard" kernel type (i.e.
656``minnow-standard.scc``:
657::
658
659 define KMACHINE minnow
660 define KTYPE standard
661 define KARCH i386
662
663 include ktypes/standard
664
665 include minnow.scc
666
667 # Extra minnow configs above the minimal defined in minnow.scc
668 include cfg/efi-ext.scc
669 include features/media/media-all.scc
670 include features/sound/snd_hda_intel.scc
671
672 # The following should really be in standard.scc
673 # USB live-image support
674 include cfg/usb-mass-storage.scc
675 include cfg/boot-live.scc
676
677 # Basic profiling
678 include features/latencytop/latencytop.scc
679 include features/profiling/profiling.scc
680
681 # Requested drivers that don't have an existing scc
682 kconf hardware minnow-drivers-extra.cfg
683
684The ``include`` command midway through the file includes the ``minnow.scc`` description
685that defines all enabled hardware for the BSP that is common to all
686kernel types. Using this command significantly reduces duplication.
687
688Now consider the "minnow" description for the "tiny" kernel type (i.e.
689``minnow-tiny.scc``):
690::
691
692 define KMACHINE minnow
693 define KTYPE tiny
694 define KARCH i386
695
696 include ktypes/tiny
697
698 include minnow.scc
699
700As you might expect,
701the "tiny" description includes quite a bit less. In fact, it includes
702only the minimal policy defined by the "tiny" kernel type and the
703hardware-specific configuration required for booting the machine along
704with the most basic functionality of the system as defined in the base
705"minnow" description file.
706
707Notice again the three critical variables:
708:term:`KMACHINE`,
709:term:`KTYPE`, and
710:term:`KARCH`. Of these variables, only
711``KTYPE`` has changed to specify the "tiny" kernel type.
712
713Kernel Metadata Location
714========================
715
716Kernel Metadata always exists outside of the kernel tree either defined
717in a kernel recipe (recipe-space) or outside of the recipe. Where you
718choose to define the Metadata depends on what you want to do and how you
719intend to work. Regardless of where you define the kernel Metadata, the
720syntax used applies equally.
721
722If you are unfamiliar with the Linux kernel and only wish to apply a
723configuration and possibly a couple of patches provided to you by
724others, the recipe-space method is recommended. This method is also a
725good approach if you are working with Linux kernel sources you do not
726control or if you just do not want to maintain a Linux kernel Git
727repository on your own. For partial information on how you can define
728kernel Metadata in the recipe-space, see the "`Modifying an Existing
729Recipe <#modifying-an-existing-recipe>`__" section.
730
731Conversely, if you are actively developing a kernel and are already
732maintaining a Linux kernel Git repository of your own, you might find it
733more convenient to work with kernel Metadata kept outside the
734recipe-space. Working with Metadata in this area can make iterative
735development of the Linux kernel more efficient outside of the BitBake
736environment.
737
738Recipe-Space Metadata
739---------------------
740
741When stored in recipe-space, the kernel Metadata files reside in a
742directory hierarchy below
743:term:`FILESEXTRAPATHS`. For
744a linux-yocto recipe or for a Linux kernel recipe derived by copying and
745modifying
746``oe-core/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb`` to
747a recipe in your layer, ``FILESEXTRAPATHS`` is typically set to
748``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``.
749See the "`Modifying an Existing
750Recipe <#modifying-an-existing-recipe>`__" section for more information.
751
752Here is an example that shows a trivial tree of kernel Metadata stored
753in recipe-space within a BSP layer:
754::
755
756 meta-my_bsp_layer/
757 `-- recipes-kernel
758 `-- linux
759 `-- linux-yocto
760 |-- bsp-standard.scc
761 |-- bsp.cfg
762 `-- standard.cfg
763
764When the Metadata is stored in recipe-space, you must take steps to
765ensure BitBake has the necessary information to decide what files to
766fetch and when they need to be fetched again. It is only necessary to
767specify the ``.scc`` files on the
768:term:`SRC_URI`. BitBake parses them
769and fetches any files referenced in the ``.scc`` files by the
770``include``, ``patch``, or ``kconf`` commands. Because of this, it is
771necessary to bump the recipe :term:`PR`
772value when changing the content of files not explicitly listed in the
773``SRC_URI``.
774
775If the BSP description is in recipe space, you cannot simply list the
776``*.scc`` in the ``SRC_URI`` statement. You need to use the following
777form from your kernel append file:
778::
779
780 SRC_URI_append_myplatform = " \
781 file://myplatform;type=kmeta;destsuffix=myplatform \
782 "
783
784Metadata Outside the Recipe-Space
785---------------------------------
786
787When stored outside of the recipe-space, the kernel Metadata files
788reside in a separate repository. The OpenEmbedded build system adds the
789Metadata to the build as a "type=kmeta" repository through the
790:term:`SRC_URI` variable. As an
791example, consider the following ``SRC_URI`` statement from the
792``linux-yocto_4.12.bb`` kernel recipe:
793::
794
795 SRC_URI = "git://git.yoctoproject.org/linux-yocto-4.12.git;name=machine;branch=${KBRANCH}; \
796 git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
797
798
799``${KMETA}``, in this context, is simply used to name the directory into
800which the Git fetcher places the Metadata. This behavior is no different
801than any multi-repository ``SRC_URI`` statement used in a recipe (e.g.
802see the previous section).
803
804You can keep kernel Metadata in a "kernel-cache", which is a directory
805containing configuration fragments. As with any Metadata kept outside
806the recipe-space, you simply need to use the ``SRC_URI`` statement with
807the "type=kmeta" attribute. Doing so makes the kernel Metadata available
808during the configuration phase.
809
810If you modify the Metadata, you must not forget to update the ``SRCREV``
811statements in the kernel's recipe. In particular, you need to update the
812``SRCREV_meta`` variable to match the commit in the ``KMETA`` branch you
813wish to use. Changing the data in these branches and not updating the
814``SRCREV`` statements to match will cause the build to fetch an older
815commit.
816
817Organizing Your Source
818======================
819
820Many recipes based on the ``linux-yocto-custom.bb`` recipe use Linux
821kernel sources that have only a single branch - "master". This type of
822repository structure is fine for linear development supporting a single
823machine and architecture. However, if you work with multiple boards and
824architectures, a kernel source repository with multiple branches is more
825efficient. For example, suppose you need a series of patches for one
826board to boot. Sometimes, these patches are works-in-progress or
827fundamentally wrong, yet they are still necessary for specific boards.
828In these situations, you most likely do not want to include these
829patches in every kernel you build (i.e. have the patches as part of the
830lone "master" branch). It is situations like these that give rise to
831multiple branches used within a Linux kernel sources Git repository.
832
833Repository organization strategies exist that maximize source reuse,
834remove redundancy, and logically order your changes. This section
835presents strategies for the following cases:
836
837- Encapsulating patches in a feature description and only including the
838 patches in the BSP descriptions of the applicable boards.
839
840- Creating a machine branch in your kernel source repository and
841 applying the patches on that branch only.
842
843- Creating a feature branch in your kernel source repository and
844 merging that branch into your BSP when needed.
845
846The approach you take is entirely up to you and depends on what works
847best for your development model.
848
849Encapsulating Patches
850---------------------
851
852if you are reusing patches from an external tree and are not working on
853the patches, you might find the encapsulated feature to be appropriate.
854Given this scenario, you do not need to create any branches in the
855source repository. Rather, you just take the static patches you need and
856encapsulate them within a feature description. Once you have the feature
857description, you simply include that into the BSP description as
858described in the "`BSP Descriptions <#bsp-descriptions>`__" section.
859
860You can find information on how to create patches and BSP descriptions
861in the "`Patches <#patches>`__" and "`BSP
862Descriptions <#bsp-descriptions>`__" sections.
863
864Machine Branches
865----------------
866
867When you have multiple machines and architectures to support, or you are
868actively working on board support, it is more efficient to create
869branches in the repository based on individual machines. Having machine
870branches allows common source to remain in the "master" branch with any
871features specific to a machine stored in the appropriate machine branch.
872This organization method frees you from continually reintegrating your
873patches into a feature.
874
875Once you have a new branch, you can set up your kernel Metadata to use
876the branch a couple different ways. In the recipe, you can specify the
877new branch as the ``KBRANCH`` to use for the board as follows:
878::
879
880 KBRANCH = "mynewbranch"
881
882Another method is to use the ``branch`` command in the BSP
883description:
884
885 mybsp.scc:
886 define KMACHINE mybsp
887 define KTYPE standard
888 define KARCH i386
889 include standard.scc
890
891 branch mynewbranch
892
893 include mybsp-hw.scc
894
895If you find yourself with numerous branches, you might consider using a
896hierarchical branching system similar to what the Yocto Linux Kernel Git
897repositories use:
898::
899
900 common/kernel_type/machine
901
902If you had two kernel types, "standard" and "small" for instance, three
903machines, and common as ``mydir``, the branches in your Git repository
904might look like this:
905:
906
907 mydir/base
908 mydir/standard/base
909 mydir/standard/machine_a
910 mydir/standard/machine_b
911 mydir/standard/machine_c
912 mydir/small/base
913 mydir/small/machine_a
914
915This organization can help clarify the branch relationships. In this
916case, ``mydir/standard/machine_a`` includes everything in ``mydir/base``
917and ``mydir/standard/base``. The "standard" and "small" branches add
918sources specific to those kernel types that for whatever reason are not
919appropriate for the other branches.
920
921.. note::
922
923 The "base" branches are an artifact of the way Git manages its data
924 internally on the filesystem: Git will not allow you to use
925 mydir/standard
926 and
927 mydir/standard/machine_a
928 because it would have to create a file and a directory named
929 "standard".
930
931Feature Branches
932----------------
933
934When you are actively developing new features, it can be more efficient
935to work with that feature as a branch, rather than as a set of patches
936that have to be regularly updated. The Yocto Project Linux kernel tools
937provide for this with the ``git merge`` command.
938
939To merge a feature branch into a BSP, insert the ``git merge`` command
940after any ``branch`` commands:
941::
942
943 mybsp.scc:
944 define KMACHINE mybsp
945 define KTYPE standard
946 define KARCH i386
947 include standard.scc
948
949 branch mynewbranch
950 git merge myfeature
951
952 include mybsp-hw.scc
953
954.. _scc-reference:
955
956SCC Description File Reference
957==============================
958
959This section provides a brief reference for the commands you can use
960within an SCC description file (``.scc``):
961
962- ``branch [ref]``: Creates a new branch relative to the current branch
963 (typically ``${KTYPE}``) using the currently checked-out branch, or
964 "ref" if specified.
965
966- ``define``: Defines variables, such as
967 :term:`KMACHINE`,
968 :term:`KTYPE`,
969 :term:`KARCH`, and
970 :term:`KFEATURE_DESCRIPTION`.
971
972- ``include SCC_FILE``: Includes an SCC file in the current file. The
973 file is parsed as if you had inserted it inline.
974
975- ``kconf [hardware|non-hardware] CFG_FILE``: Queues a configuration
976 fragment for merging into the final Linux ``.config`` file.
977
978- ``git merge GIT_BRANCH``: Merges the feature branch into the current
979 branch.
980
981- ``patch PATCH_FILE``: Applies the patch to the current Git branch.
982
983
diff --git a/documentation/kernel-dev/kernel-dev-advanced.xml b/documentation/kernel-dev/kernel-dev-advanced.xml
index 5c76ed2391..37177966bf 100644
--- a/documentation/kernel-dev/kernel-dev-advanced.xml
+++ b/documentation/kernel-dev/kernel-dev-advanced.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='kernel-dev-advanced'> 6<chapter id='kernel-dev-advanced'>
6<title>Working with Advanced Metadata (<filename>yocto-kernel-cache</filename>)</title> 7<title>Working with Advanced Metadata (<filename>yocto-kernel-cache</filename>)</title>
diff --git a/documentation/kernel-dev/kernel-dev-common.rst b/documentation/kernel-dev/kernel-dev-common.rst
new file mode 100644
index 0000000000..d4b60a9dc9
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-common.rst
@@ -0,0 +1,2078 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Common Tasks
5************
6
7This chapter presents several common tasks you perform when you work
8with the Yocto Project Linux kernel. These tasks include preparing your
9host development system for kernel development, preparing a layer,
10modifying an existing recipe, patching the kernel, configuring the
11kernel, iterative development, working with your own sources, and
12incorporating out-of-tree modules.
13
14.. note::
15
16 The examples presented in this chapter work with the Yocto Project
17 2.4 Release and forward.
18
19Preparing the Build Host to Work on the Kernel
20==============================================
21
22Before you can do any kernel development, you need to be sure your build
23host is set up to use the Yocto Project. For information on how to get
24set up, see the ":doc:`../dev-manual/dev-manual-start`" section in
25the Yocto Project Development Tasks Manual. Part of preparing the system
26is creating a local Git repository of the
27:term:`Source Directory` (``poky``) on your system. Follow the steps in the
28":ref:`dev-manual/dev-manual-start:cloning the \`\`poky\`\` repository`"
29section in the Yocto Project Development Tasks Manual to set up your
30Source Directory.
31
32.. note::
33
34 Be sure you check out the appropriate development branch or you
35 create your local branch by checking out a specific tag to get the
36 desired version of Yocto Project. See the "
37 Checking Out by Branch in Poky
38 " and "
39 Checking Out by Tag in Poky
40 " sections in the Yocto Project Development Tasks Manual for more
41 information.
42
43Kernel development is best accomplished using
44:ref:`devtool <sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow>`
45and not through traditional kernel workflow methods. The remainder of
46this section provides information for both scenarios.
47
48Getting Ready to Develop Using ``devtool``
49------------------------------------------
50
51Follow these steps to prepare to update the kernel image using
52``devtool``. Completing this procedure leaves you with a clean kernel
53image and ready to make modifications as described in the "
54:ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
55section:
56
571. *Initialize the BitBake Environment:* Before building an extensible
58 SDK, you need to initialize the BitBake build environment by sourcing
59 the build environment script (i.e. :ref:`structure-core-script`):
60 ::
61
62 $ cd ~/poky
63 $ source oe-init-build-env
64
65 .. note::
66
67 The previous commands assume the
68 Source Repositories
69 (i.e.
70 poky
71 ) have been cloned using Git and the local repository is named
72 "poky".
73
742. *Prepare Your local.conf File:* By default, the
75 :term:`MACHINE` variable is set to
76 "qemux86-64", which is fine if you are building for the QEMU emulator
77 in 64-bit mode. However, if you are not, you need to set the
78 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
79 found in the
80 :term:`Build Directory` (i.e.
81 ``~/poky/build`` in this example).
82
83 Also, since you are preparing to work on the kernel image, you need
84 to set the
85 :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
86 variable to include kernel modules.
87
88 In this example we wish to build for qemux86 so we must set the
89 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
90 As described we do this by appending to ``conf/local.conf``:
91 ::
92
93 MACHINE = "qemux86"
94 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
95
963. *Create a Layer for Patches:* You need to create a layer to hold
97 patches created for the kernel image. You can use the
98 ``bitbake-layers create-layer`` command as follows:
99 ::
100
101 $ cd ~/poky/build
102 $ bitbake-layers create-layer ../../meta-mylayer
103 NOTE: Starting bitbake server...
104 Add your new layer with 'bitbake-layers add-layer ../../meta-mylayer'
105 $
106
107 .. note::
108
109 For background information on working with common and BSP layers,
110 see the "
111 Understanding and Creating Layers
112 " section in the Yocto Project Development Tasks Manual and the "
113 BSP Layers
114 " section in the Yocto Project Board Support (BSP) Developer's
115 Guide, respectively. For information on how to use the
116 bitbake-layers create-layer
117 command to quickly set up a layer, see the "
118 Creating a General Layer Using the
119 bitbake-layers
120 Script
121 " section in the Yocto Project Development Tasks Manual.
122
1234. *Inform the BitBake Build Environment About Your Layer:* As directed
124 when you created your layer, you need to add the layer to the
125 :term:`BBLAYERS` variable in the
126 ``bblayers.conf`` file as follows:
127 ::
128
129 $ cd ~/poky/build
130 $ bitbake-layers add-layer ../../meta-mylayer
131 NOTE: Starting bitbake server...
132 $
133
1345. *Build the Extensible SDK:* Use BitBake to build the extensible SDK
135 specifically for use with images to be run using QEMU:
136 ::
137
138 $ cd ~/poky/build
139 $ bitbake core-image-minimal -c populate_sdk_ext
140
141 Once
142 the build finishes, you can find the SDK installer file (i.e.
143 ``*.sh`` file) in the following directory:
144 ~/poky/build/tmp/deploy/sdk For this example, the installer file is
145 named
146 ``poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-DISTRO.sh``
147
1486. *Install the Extensible SDK:* Use the following command to install
149 the SDK. For this example, install the SDK in the default
150 ``~/poky_sdk`` directory:
151 ::
152
153 $ cd ~/poky/build/tmp/deploy/sdk
154 $ ./poky-glibc-x86_64-core-image-minimal-i586-toolchain-ext-3.1.2.sh
155 Poky (Yocto Project Reference Distro) Extensible SDK installer version 3.1.2
156 ============================================================================
157 Enter target directory for SDK (default: ~/poky_sdk):
158 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
159 Extracting SDK......................................done
160 Setting it up...
161 Extracting buildtools...
162 Preparing build system...
163 Parsing recipes: 100% |#################################################################| Time: 0:00:52
164 Initializing tasks: 100% |############## ###############################################| Time: 0:00:04
165 Checking sstate mirror object availability: 100% |######################################| Time: 0:00:00
166 Parsing recipes: 100% |#################################################################| Time: 0:00:33
167 Initializing tasks: 100% |##############################################################| Time: 0:00:00
168 done
169 SDK has been successfully set up and is ready to be used.
170 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
171 $ . /home/scottrif/poky_sdk/environment-setup-i586-poky-linux
172
173
1747. *Set Up a New Terminal to Work With the Extensible SDK:* You must set
175 up a new terminal to work with the SDK. You cannot use the same
176 BitBake shell used to build the installer.
177
178 After opening a new shell, run the SDK environment setup script as
179 directed by the output from installing the SDK:
180 ::
181
182 $ source ~/poky_sdk/environment-setup-i586-poky-linux
183 "SDK environment now set up; additionally you may now run devtool to perform development tasks.
184 Run devtool --help for further details.
185
186 .. note::
187
188 If you get a warning about attempting to use the extensible SDK in
189 an environment set up to run BitBake, you did not use a new shell.
190
1918. *Build the Clean Image:* The final step in preparing to work on the
192 kernel is to build an initial image using ``devtool`` in the new
193 terminal you just set up and initialized for SDK work:
194 ::
195
196 $ devtool build-image
197 Parsing recipes: 100% |##########################################| Time: 0:00:05
198 Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 47 skipped, 0 masked, 0 errors.
199 WARNING: No packages to add, building image core-image-minimal unmodified
200 Loading cache: 100% |############################################| Time: 0:00:00
201 Loaded 1299 entries from dependency cache.
202 NOTE: Resolving any missing task queue dependencies
203 Initializing tasks: 100% |#######################################| Time: 0:00:07
204 Checking sstate mirror object availability: 100% |###############| Time: 0:00:00
205 NOTE: Executing SetScene Tasks
206 NOTE: Executing RunQueue Tasks
207 NOTE: Tasks Summary: Attempted 2866 tasks of which 2604 didn't need to be rerun and all succeeded.
208 NOTE: Successfully built core-image-minimal. You can find output files in /home/scottrif/poky_sdk/tmp/deploy/images/qemux86
209
210 If you were
211 building for actual hardware and not for emulation, you could flash
212 the image to a USB stick on ``/dev/sdd`` and boot your device. For an
213 example that uses a Minnowboard, see the
214 `TipsAndTricks/KernelDevelopmentWithEsdk <https://wiki.yoctoproject.org/wiki/TipsAndTricks/KernelDevelopmentWithEsdk>`__
215 Wiki page.
216
217At this point you have set up to start making modifications to the
218kernel by using the extensible SDK. For a continued example, see the
219":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
220section.
221
222Getting Ready for Traditional Kernel Development
223------------------------------------------------
224
225Getting ready for traditional kernel development using the Yocto Project
226involves many of the same steps as described in the previous section.
227However, you need to establish a local copy of the kernel source since
228you will be editing these files.
229
230Follow these steps to prepare to update the kernel image using
231traditional kernel development flow with the Yocto Project. Completing
232this procedure leaves you ready to make modifications to the kernel
233source as described in the ":ref:`kernel-dev/kernel-dev-common:using traditional kernel development to patch the kernel`"
234section:
235
2361. *Initialize the BitBake Environment:* Before you can do anything
237 using BitBake, you need to initialize the BitBake build environment
238 by sourcing the build environment script (i.e.
239 :ref:`structure-core-script`).
240 Also, for this example, be sure that the local branch you have
241 checked out for ``poky`` is the Yocto Project &DISTRO_NAME; branch. If
242 you need to checkout out the &DISTRO_NAME; branch, see the
243 ":ref:`dev-manual/dev-manual-start:checking out by branch in poky`"
244 section in the Yocto Project Development Tasks Manual.
245 ::
246
247 $ cd ~/poky
248 $ git branch
249 master
250 * &DISTRO_NAME;
251 $ source oe-init-build-env
252
253 .. note::
254
255 The previous commands assume the
256 Source Repositories
257 (i.e.
258 poky
259 ) have been cloned using Git and the local repository is named
260 "poky".
261
2622. *Prepare Your local.conf File:* By default, the
263 :term:`MACHINE` variable is set to
264 "qemux86-64", which is fine if you are building for the QEMU emulator
265 in 64-bit mode. However, if you are not, you need to set the
266 ``MACHINE`` variable appropriately in your ``conf/local.conf`` file
267 found in the
268 :term:`Build Directory` (i.e.
269 ``~/poky/build`` in this example).
270
271 Also, since you are preparing to work on the kernel image, you need
272 to set the
273 :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
274 variable to include kernel modules.
275
276 In this example we wish to build for qemux86 so we must set the
277 ``MACHINE`` variable to "qemux86" and also add the "kernel-modules".
278 As described we do this by appending to ``conf/local.conf``:
279 ::
280
281 MACHINE = "qemux86"
282 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-modules"
283
2843. *Create a Layer for Patches:* You need to create a layer to hold
285 patches created for the kernel image. You can use the
286 ``bitbake-layers create-layer`` command as follows:
287 ::
288
289 $ cd ~/poky/build
290 $ bitbake-layers create-layer ../../meta-mylayer
291 NOTE: Starting bitbake server...
292 Add your new layer with 'bitbake-layers add-layer ../../meta-mylayer'
293
294 .. note::
295
296 For background information on working with common and BSP layers,
297 see the "
298 Understanding and Creating Layers
299 " section in the Yocto Project Development Tasks Manual and the "
300 BSP Layers
301 " section in the Yocto Project Board Support (BSP) Developer's
302 Guide, respectively. For information on how to use the
303 bitbake-layers create-layer
304 command to quickly set up a layer, see the "
305 Creating a General Layer Using the
306 bitbake-layers
307 Script
308 " section in the Yocto Project Development Tasks Manual.
309
3104. *Inform the BitBake Build Environment About Your Layer:* As directed
311 when you created your layer, you need to add the layer to the
312 :term:`BBLAYERS` variable in the
313 ``bblayers.conf`` file as follows:
314 ::
315
316 $ cd ~/poky/build
317 $ bitbake-layers add-layer ../../meta-mylayer
318 NOTE: Starting bitbake server ...
319 $
320
3215. *Create a Local Copy of the Kernel Git Repository:* You can find Git
322 repositories of supported Yocto Project kernels organized under
323 "Yocto Linux Kernel" in the Yocto Project Source Repositories at
324 :yocto_git:`/`.
325
326 For simplicity, it is recommended that you create your copy of the
327 kernel Git repository outside of the
328 :term:`Source Directory`, which is
329 usually named ``poky``. Also, be sure you are in the
330 ``standard/base`` branch.
331
332 The following commands show how to create a local copy of the
333 ``linux-yocto-4.12`` kernel and be in the ``standard/base`` branch.
334
335 .. note::
336
337 The
338 linux-yocto-4.12
339 kernel can be used with the Yocto Project 2.4 release and forward.
340 You cannot use the
341 linux-yocto-4.12
342 kernel with releases prior to Yocto Project 2.4:
343
344 ::
345
346 $ cd ~
347 $ git clone git://git.yoctoproject.org/linux-yocto-4.12 --branch standard/base
348 Cloning into 'linux-yocto-4.12'...
349 remote: Counting objects: 6097195, done.
350 remote: Compressing objects: 100% (901026/901026), done.
351 remote: Total 6097195 (delta 5152604), reused 6096847 (delta 5152256)
352 Receiving objects: 100% (6097195/6097195), 1.24 GiB | 7.81 MiB/s, done.
353 Resolving deltas: 100% (5152604/5152604), done. Checking connectivity... done.
354 Checking out files: 100% (59846/59846), done.
355
3566. *Create a Local Copy of the Kernel Cache Git Repository:* For
357 simplicity, it is recommended that you create your copy of the kernel
358 cache Git repository outside of the
359 :term:`Source Directory`, which is
360 usually named ``poky``. Also, for this example, be sure you are in
361 the ``yocto-4.12`` branch.
362
363 The following commands show how to create a local copy of the
364 ``yocto-kernel-cache`` and be in the ``yocto-4.12`` branch:
365 ::
366
367 $ cd ~
368 $ git clone git://git.yoctoproject.org/yocto-kernel-cache --branch yocto-4.12
369 Cloning into 'yocto-kernel-cache'...
370 remote: Counting objects: 22639, done.
371 remote: Compressing objects: 100% (9761/9761), done.
372 remote: Total 22639 (delta 12400), reused 22586 (delta 12347)
373 Receiving objects: 100% (22639/22639), 22.34 MiB | 6.27 MiB/s, done.
374 Resolving deltas: 100% (12400/12400), done.
375 Checking connectivity... done.
376
377At this point, you are ready to start making modifications to the kernel
378using traditional kernel development steps. For a continued example, see
379the "`Using Traditional Kernel Development to Patch the
380Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
381section.
382
383Creating and Preparing a Layer
384==============================
385
386If you are going to be modifying kernel recipes, it is recommended that
387you create and prepare your own layer in which to do your work. Your
388layer contains its own :term:`BitBake`
389append files (``.bbappend``) and provides a convenient mechanism to
390create your own recipe files (``.bb``) as well as store and use kernel
391patch files. For background information on working with layers, see the
392":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
393section in the Yocto Project Development Tasks Manual.
394
395.. note::
396
397 The Yocto Project comes with many tools that simplify tasks you need
398 to perform. One such tool is the
399 bitbake-layers create-layer
400 command, which simplifies creating a new layer. See the "
401 Creating a General Layer Using the
402 bitbake-layers
403 Script
404 " section in the Yocto Project Development Tasks Manual for
405 information on how to use this script to quick set up a new layer.
406
407To better understand the layer you create for kernel development, the
408following section describes how to create a layer without the aid of
409tools. These steps assume creation of a layer named ``mylayer`` in your
410home directory:
411
4121. *Create Structure*: Create the layer's structure:
413 ::
414
415 $ cd $HOME
416 $ mkdir meta-mylayer
417 $ mkdir meta-mylayer/conf
418 $ mkdir meta-mylayer/recipes-kernel
419 $ mkdir meta-mylayer/recipes-kernel/linux
420 $ mkdir meta-mylayer/recipes-kernel/linux/linux-yocto
421
422 The ``conf`` directory holds your configuration files, while the
423 ``recipes-kernel`` directory holds your append file and eventual
424 patch files.
425
4262. *Create the Layer Configuration File*: Move to the
427 ``meta-mylayer/conf`` directory and create the ``layer.conf`` file as
428 follows:
429 ::
430
431 # We have a conf and classes directory, add to BBPATH
432 BBPATH .= ":${LAYERDIR}"
433
434 # We have recipes-* directories, add to BBFILES
435 BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
436 ${LAYERDIR}/recipes-*/*/*.bbappend"
437
438 BBFILE_COLLECTIONS += "mylayer"
439 BBFILE_PATTERN_mylayer = "^${LAYERDIR}/"
440 BBFILE_PRIORITY_mylayer = "5"
441
442 Notice ``mylayer`` as part of the last three statements.
443
4443. *Create the Kernel Recipe Append File*: Move to the
445 ``meta-mylayer/recipes-kernel/linux`` directory and create the
446 kernel's append file. This example uses the ``linux-yocto-4.12``
447 kernel. Thus, the name of the append file is
448 ``linux-yocto_4.12.bbappend``:
449 ::
450
451 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
452
453 SRC_URI_append = " file://patch-file-one"
454 SRC_URI_append = " file://patch-file-two"
455 SRC_URI_append = " file://patch-file-three"
456
457 The :term:`FILESEXTRAPATHS` and :term:`SRC_URI` statements
458 enable the OpenEmbedded build system to find patch files. For more
459 information on using append files, see the
460 ":ref:`dev-manual/dev-manual-common-tasks:using .bbappend files in your layer`"
461 section in the Yocto Project Development Tasks Manual.
462
463Modifying an Existing Recipe
464============================
465
466In many cases, you can customize an existing linux-yocto recipe to meet
467the needs of your project. Each release of the Yocto Project provides a
468few Linux kernel recipes from which you can choose. These are located in
469the :term:`Source Directory` in
470``meta/recipes-kernel/linux``.
471
472Modifying an existing recipe can consist of the following:
473
474- Creating the append file
475
476- Applying patches
477
478- Changing the configuration
479
480Before modifying an existing recipe, be sure that you have created a
481minimal, custom layer from which you can work. See the "`Creating and
482Preparing a Layer <#creating-and-preparing-a-layer>`__" section for
483information.
484
485Creating the Append File
486------------------------
487
488You create this file in your custom layer. You also name it accordingly
489based on the linux-yocto recipe you are using. For example, if you are
490modifying the ``meta/recipes-kernel/linux/linux-yocto_4.12.bb`` recipe,
491the append file will typically be located as follows within your custom
492layer:
493::
494
495 your-layer/recipes-kernel/linux/linux-yocto_4.12.bbappend
496
497The append file should initially extend the
498:term:`FILESPATH` search path by
499prepending the directory that contains your files to the
500:term:`FILESEXTRAPATHS`
501variable as follows:
502::
503
504 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
505
506The path ``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``
507expands to "linux-yocto" in the current directory for this example. If
508you add any new files that modify the kernel recipe and you have
509extended ``FILESPATH`` as described above, you must place the files in
510your layer in the following area:
511::
512
513 your-layer/recipes-kernel/linux/linux-yocto/
514
515.. note::
516
517 If you are working on a new machine Board Support Package (BSP), be
518 sure to refer to the
519 Yocto Project Board Support Package (BSP) Developer's Guide
520 .
521
522As an example, consider the following append file used by the BSPs in
523``meta-yocto-bsp``:
524::
525
526 meta-yocto-bsp/recipes-kernel/linux/linux-yocto_4.12.bbappend
527
528The following listing shows the file. Be aware that the actual commit ID
529strings in this example listing might be different than the actual
530strings in the file from the ``meta-yocto-bsp`` layer upstream.
531::
532
533 KBRANCH_genericx86 = "standard/base"
534 KBRANCH_genericx86-64 = "standard/base"
535
536 KMACHINE_genericx86 ?= "common-pc"
537 KMACHINE_genericx86-64 ?= "common-pc-64"
538 KBRANCH_edgerouter = "standard/edgerouter"
539 KBRANCH_beaglebone = "standard/beaglebone"
540
541 SRCREV_machine_genericx86 ?= "d09f2ce584d60ecb7890550c22a80c48b83c2e19"
542 SRCREV_machine_genericx86-64 ?= "d09f2ce584d60ecb7890550c22a80c48b83c2e19"
543 SRCREV_machine_edgerouter ?= "b5c8cfda2dfe296410d51e131289fb09c69e1e7d"
544 SRCREV_machine_beaglebone ?= "b5c8cfda2dfe296410d51e131289fb09c69e1e7d"
545
546
547 COMPATIBLE_MACHINE_genericx86 = "genericx86"
548 COMPATIBLE_MACHINE_genericx86-64 = "genericx86-64"
549 COMPATIBLE_MACHINE_edgerouter = "edgerouter"
550 COMPATIBLE_MACHINE_beaglebone = "beaglebone"
551
552 LINUX_VERSION_genericx86 = "4.12.7"
553 LINUX_VERSION_genericx86-64 = "4.12.7"
554 LINUX_VERSION_edgerouter = "4.12.10"
555 LINUX_VERSION_beaglebone = "4.12.10"
556
557This append file
558contains statements used to support several BSPs that ship with the
559Yocto Project. The file defines machines using the
560:term:`COMPATIBLE_MACHINE`
561variable and uses the
562:term:`KMACHINE` variable to ensure
563the machine name used by the OpenEmbedded build system maps to the
564machine name used by the Linux Yocto kernel. The file also uses the
565optional :term:`KBRANCH` variable to
566ensure the build process uses the appropriate kernel branch.
567
568Although this particular example does not use it, the
569:term:`KERNEL_FEATURES`
570variable could be used to enable features specific to the kernel. The
571append file points to specific commits in the
572:term:`Source Directory` Git repository and
573the ``meta`` Git repository branches to identify the exact kernel needed
574to build the BSP.
575
576One thing missing in this particular BSP, which you will typically need
577when developing a BSP, is the kernel configuration file (``.config``)
578for your BSP. When developing a BSP, you probably have a kernel
579configuration file or a set of kernel configuration files that, when
580taken together, define the kernel configuration for your BSP. You can
581accomplish this definition by putting the configurations in a file or a
582set of files inside a directory located at the same level as your
583kernel's append file and having the same name as the kernel's main
584recipe file. With all these conditions met, simply reference those files
585in the :term:`SRC_URI` statement in
586the append file.
587
588For example, suppose you had some configuration options in a file called
589``network_configs.cfg``. You can place that file inside a directory
590named ``linux-yocto`` and then add a ``SRC_URI`` statement such as the
591following to the append file. When the OpenEmbedded build system builds
592the kernel, the configuration options are picked up and applied.
593::
594
595 SRC_URI += "file://network_configs.cfg"
596
597To group related configurations into multiple files, you perform a
598similar procedure. Here is an example that groups separate
599configurations specifically for Ethernet and graphics into their own
600files and adds the configurations by using a ``SRC_URI`` statement like
601the following in your append file:
602::
603
604 SRC_URI += "file://myconfig.cfg \
605 file://eth.cfg \
606 file://gfx.cfg"
607
608Another variable you can use in your kernel recipe append file is the
609:term:`FILESEXTRAPATHS`
610variable. When you use this statement, you are extending the locations
611used by the OpenEmbedded system to look for files and patches as the
612recipe is processed.
613
614.. note::
615
616 Other methods exist to accomplish grouping and defining configuration
617 options. For example, if you are working with a local clone of the
618 kernel repository, you could checkout the kernel's ``meta`` branch,
619 make your changes, and then push the changes to the local bare clone
620 of the kernel. The result is that you directly add configuration
621 options to the ``meta`` branch for your BSP. The configuration
622 options will likely end up in that location anyway if the BSP gets
623 added to the Yocto Project.
624
625 In general, however, the Yocto Project maintainers take care of
626 moving the ``SRC_URI``-specified configuration options to the
627 kernel's ``meta`` branch. Not only is it easier for BSP developers to
628 not have to worry about putting those configurations in the branch,
629 but having the maintainers do it allows them to apply 'global'
630 knowledge about the kinds of common configuration options multiple
631 BSPs in the tree are typically using. This allows for promotion of
632 common configurations into common features.
633
634Applying Patches
635----------------
636
637If you have a single patch or a small series of patches that you want to
638apply to the Linux kernel source, you can do so just as you would with
639any other recipe. You first copy the patches to the path added to
640:term:`FILESEXTRAPATHS` in
641your ``.bbappend`` file as described in the previous section, and then
642reference them in :term:`SRC_URI`
643statements.
644
645For example, you can apply a three-patch series by adding the following
646lines to your linux-yocto ``.bbappend`` file in your layer:
647::
648
649 SRC_URI += "file://0001-first-change.patch"
650 SRC_URI += "file://0002-second-change.patch"
651 SRC_URI += "file://0003-third-change.patch"
652
653The next time you run BitBake to build
654the Linux kernel, BitBake detects the change in the recipe and fetches
655and applies the patches before building the kernel.
656
657For a detailed example showing how to patch the kernel using
658``devtool``, see the
659":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
660and
661":ref:`kernel-dev/kernel-dev-common:using traditional kernel development to patch the kernel`"
662sections.
663
664Changing the Configuration
665--------------------------
666
667You can make wholesale or incremental changes to the final ``.config``
668file used for the eventual Linux kernel configuration by including a
669``defconfig`` file and by specifying configuration fragments in the
670:term:`SRC_URI` to be applied to that
671file.
672
673If you have a complete, working Linux kernel ``.config`` file you want
674to use for the configuration, as before, copy that file to the
675appropriate ``${PN}`` directory in your layer's ``recipes-kernel/linux``
676directory, and rename the copied file to "defconfig". Then, add the
677following lines to the linux-yocto ``.bbappend`` file in your layer:
678::
679
680 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
681 SRC_URI += "file://defconfig"
682
683The ``SRC_URI`` tells the build system how to search
684for the file, while the
685:term:`FILESEXTRAPATHS`
686extends the :term:`FILESPATH`
687variable (search directories) to include the ``${PN}`` directory you
688created to hold the configuration changes.
689
690.. note::
691
692 The build system applies the configurations from the
693 defconfig
694 file before applying any subsequent configuration fragments. The
695 final kernel configuration is a combination of the configurations in
696 the
697 defconfig
698 file and any configuration fragments you provide. You need to realize
699 that if you have any configuration fragments, the build system
700 applies these on top of and after applying the existing
701 defconfig
702 file configurations.
703
704Generally speaking, the preferred approach is to determine the
705incremental change you want to make and add that as a configuration
706fragment. For example, if you want to add support for a basic serial
707console, create a file named ``8250.cfg`` in the ``${PN}`` directory
708with the following content (without indentation):
709::
710
711 CONFIG_SERIAL_8250=y
712 CONFIG_SERIAL_8250_CONSOLE=y
713 CONFIG_SERIAL_8250_PCI=y
714 CONFIG_SERIAL_8250_NR_UARTS=4
715 CONFIG_SERIAL_8250_RUNTIME_UARTS=4
716 CONFIG_SERIAL_CORE=y
717 CONFIG_SERIAL_CORE_CONSOLE=y
718
719Next, include this
720configuration fragment and extend the ``FILESPATH`` variable in your
721``.bbappend`` file:
722::
723
724 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
725 SRC_URI += "file://8250.cfg"
726
727The next time you run BitBake to build the
728Linux kernel, BitBake detects the change in the recipe and fetches and
729applies the new configuration before building the kernel.
730
731For a detailed example showing how to configure the kernel, see the
732"`Configuring the Kernel <#configuring-the-kernel>`__" section.
733
734Using an "In-Tree"  ``defconfig`` File
735--------------------------------------
736
737It might be desirable to have kernel configuration fragment support
738through a ``defconfig`` file that is pulled from the kernel source tree
739for the configured machine. By default, the OpenEmbedded build system
740looks for ``defconfig`` files in the layer used for Metadata, which is
741"out-of-tree", and then configures them using the following:
742::
743
744 SRC_URI += "file://defconfig"
745
746If you do not want to maintain copies of
747``defconfig`` files in your layer but would rather allow users to use
748the default configuration from the kernel tree and still be able to add
749configuration fragments to the
750:term:`SRC_URI` through, for example,
751append files, you can direct the OpenEmbedded build system to use a
752``defconfig`` file that is "in-tree".
753
754To specify an "in-tree" ``defconfig`` file, use the following statement
755form:
756::
757
758 KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file
759
760Here is an example
761that assigns the ``KBUILD_DEFCONFIG`` variable based on "raspberrypi2"
762and provides the path to the "in-tree" ``defconfig`` file to be used for
763a Raspberry Pi 2, which is based on the Broadcom 2708/2709 chipset:
764::
765
766 KBUILD_DEFCONFIG_raspberrypi2 ?= "bcm2709_defconfig"
767
768Aside from modifying your kernel recipe and providing your own
769``defconfig`` file, you need to be sure no files or statements set
770``SRC_URI`` to use a ``defconfig`` other than your "in-tree" file (e.g.
771a kernel's ``linux-``\ machine\ ``.inc`` file). In other words, if the
772build system detects a statement that identifies an "out-of-tree"
773``defconfig`` file, that statement will override your
774``KBUILD_DEFCONFIG`` variable.
775
776See the
777:term:`KBUILD_DEFCONFIG`
778variable description for more information.
779
780Using ``devtool`` to Patch the Kernel
781=====================================
782
783The steps in this procedure show you how you can patch the kernel using
784the extensible SDK and ``devtool``.
785
786.. note::
787
788 Before attempting this procedure, be sure you have performed the
789 steps to get ready for updating the kernel as described in the "
790 Getting Ready to Develop Using
791 devtool
792 " section.
793
794Patching the kernel involves changing or adding configurations to an
795existing kernel, changing or adding recipes to the kernel that are
796needed to support specific hardware features, or even altering the
797source code itself.
798
799This example creates a simple patch by adding some QEMU emulator console
800output at boot time through ``printk`` statements in the kernel's
801``calibrate.c`` source code file. Applying the patch and booting the
802modified image causes the added messages to appear on the emulator's
803console. The example is a continuation of the setup procedure found in
804the ":ref:`kernel-dev/kernel-dev-common:getting ready to develop using \`\`devtool\`\``" Section.
805
8061. *Check Out the Kernel Source Files:* First you must use ``devtool``
807 to checkout the kernel source code in its workspace. Be sure you are
808 in the terminal set up to do work with the extensible SDK.
809
810 .. note::
811
812 See this
813 step
814 in the "
815 Getting Ready to Develop Using
816 devtool
817 " section for more information.
818
819 Use the following ``devtool`` command to check out the code:
820 ::
821
822 $ devtool modify linux-yocto
823
824 .. note::
825
826 During the checkout operation, a bug exists that could cause
827 errors such as the following to appear:
828 ::
829
830 ERROR: Taskhash mismatch 2c793438c2d9f8c3681fd5f7bc819efa versus
831 be3a89ce7c47178880ba7bf6293d7404 for
832 /path/to/esdk/layers/poky/meta/recipes-kernel/linux/linux-yocto_4.10.bb.do_unpack
833
834
835 You can safely ignore these messages. The source code is correctly
836 checked out.
837
8382. *Edit the Source Files* Follow these steps to make some simple
839 changes to the source files:
840
841 1. *Change the working directory*: In the previous step, the output
842 noted where you can find the source files (e.g.
843 ``~/poky_sdk/workspace/sources/linux-yocto``). Change to where the
844 kernel source code is before making your edits to the
845 ``calibrate.c`` file:
846 ::
847
848 $ cd ~/poky_sdk/workspace/sources/linux-yocto
849
850 2. *Edit the source file*: Edit the ``init/calibrate.c`` file to have
851 the following changes:
852 ::
853
854 void calibrate_delay(void)
855 {
856 unsigned long lpj;
857 static bool printed;
858 int this_cpu = smp_processor_id();
859
860 printk("*************************************\n");
861 printk("* *\n");
862 printk("* HELLO YOCTO KERNEL *\n");
863 printk("* *\n");
864 printk("*************************************\n");
865
866 if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
867 .
868 .
869 .
870
8713. *Build the Updated Kernel Source:* To build the updated kernel
872 source, use ``devtool``:
873 ::
874
875 $ devtool build linux-yocto
876
8774. *Create the Image With the New Kernel:* Use the
878 ``devtool build-image`` command to create a new image that has the
879 new kernel.
880
881 .. note::
882
883 If the image you originally created resulted in a Wic file, you
884 can use an alternate method to create the new image with the
885 updated kernel. For an example, see the steps in the
886 TipsAndTricks/KernelDevelopmentWithEsdk
887 Wiki Page.
888
889 ::
890
891 $ cd ~
892 $ devtool build-image core-image-minimal
893
8945. *Test the New Image:* For this example, you can run the new image
895 using QEMU to verify your changes:
896
897 1. *Boot the image*: Boot the modified image in the QEMU emulator
898 using this command:
899 ::
900
901 $ runqemu qemux86
902
903 2. *Verify the changes*: Log into the machine using ``root`` with no
904 password and then use the following shell command to scroll
905 through the console's boot output.
906 ::
907
908 # dmesg | less
909
910 You should see
911 the results of your ``printk`` statements as part of the output
912 when you scroll down the console window.
913
9146. *Stage and commit your changes*: Within your eSDK terminal, change
915 your working directory to where you modified the ``calibrate.c`` file
916 and use these Git commands to stage and commit your changes:
917 ::
918
919 $ cd ~/poky_sdk/workspace/sources/linux-yocto
920 $ git status
921 $ git add init/calibrate.c
922 $ git commit -m "calibrate: Add printk example"
923
9247. *Export the Patches and Create an Append File:* To export your
925 commits as patches and create a ``.bbappend`` file, use the following
926 command in the terminal used to work with the extensible SDK. This
927 example uses the previously established layer named ``meta-mylayer``.
928
929 .. note::
930
931 See Step 3 of the "
932 Getting Ready to Develop Using devtool
933 " section for information on setting up this layer.
934
935 $ devtool finish linux-yocto ~/meta-mylayer
936
937 Once the command
938 finishes, the patches and the ``.bbappend`` file are located in the
939 ``~/meta-mylayer/recipes-kernel/linux`` directory.
940
9418. *Build the Image With Your Modified Kernel:* You can now build an
942 image that includes your kernel patches. Execute the following
943 command from your
944 :term:`Build Directory` in the terminal
945 set up to run BitBake:
946 ::
947
948 $ cd ~/poky/build
949 $ bitbake core-image-minimal
950
951Using Traditional Kernel Development to Patch the Kernel
952========================================================
953
954The steps in this procedure show you how you can patch the kernel using
955traditional kernel development (i.e. not using ``devtool`` and the
956extensible SDK as described in the
957":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
958section).
959
960.. note::
961
962 Before attempting this procedure, be sure you have performed the
963 steps to get ready for updating the kernel as described in the "
964 Getting Ready for Traditional Kernel Development
965 " section.
966
967Patching the kernel involves changing or adding configurations to an
968existing kernel, changing or adding recipes to the kernel that are
969needed to support specific hardware features, or even altering the
970source code itself.
971
972The example in this section creates a simple patch by adding some QEMU
973emulator console output at boot time through ``printk`` statements in
974the kernel's ``calibrate.c`` source code file. Applying the patch and
975booting the modified image causes the added messages to appear on the
976emulator's console. The example is a continuation of the setup procedure
977found in the "`Getting Ready for Traditional Kernel
978Development <#getting-ready-for-traditional-kernel-development>`__"
979Section.
980
9811. *Edit the Source Files* Prior to this step, you should have used Git
982 to create a local copy of the repository for your kernel. Assuming
983 you created the repository as directed in the "`Getting Ready for
984 Traditional Kernel
985 Development <#getting-ready-for-traditional-kernel-development>`__"
986 section, use the following commands to edit the ``calibrate.c`` file:
987
988 1. *Change the working directory*: You need to locate the source
989 files in the local copy of the kernel Git repository: Change to
990 where the kernel source code is before making your edits to the
991 ``calibrate.c`` file:
992 ::
993
994 $ cd ~/linux-yocto-4.12/init
995
996 2. *Edit the source file*: Edit the ``calibrate.c`` file to have the
997 following changes:
998 ::
999
1000 void calibrate_delay(void)
1001 {
1002 unsigned long lpj;
1003 static bool printed;
1004 int this_cpu = smp_processor_id();
1005
1006 printk("*************************************\n");
1007 printk("* *\n");
1008 printk("* HELLO YOCTO KERNEL *\n");
1009 printk("* *\n");
1010 printk("*************************************\n");
1011
1012 if (per_cpu(cpu_loops_per_jiffy, this_cpu)) {
1013 .
1014 .
1015 .
1016
10172. *Stage and Commit Your Changes:* Use standard Git commands to stage
1018 and commit the changes you just made:
1019 ::
1020
1021 $ git add calibrate.c
1022 $ git commit -m "calibrate.c - Added some printk statements"
1023
1024 If you do not
1025 stage and commit your changes, the OpenEmbedded Build System will not
1026 pick up the changes.
1027
10283. *Update Your local.conf File to Point to Your Source Files:* In
1029 addition to your ``local.conf`` file specifying to use
1030 "kernel-modules" and the "qemux86" machine, it must also point to the
1031 updated kernel source files. Add
1032 :term:`SRC_URI` and
1033 :term:`SRCREV` statements similar
1034 to the following to your ``local.conf``:
1035 ::
1036
1037 $ cd ~/poky/build/conf
1038
1039 Add the following to the ``local.conf``:
1040 ::
1041
1042 SRC_URI_pn-linux-yocto = "git:///path-to/linux-yocto-4.12;protocol=file;name=machine;branch=standard/base; \
1043 git:///path-to/yocto-kernel-cache;protocol=file;type=kmeta;name=meta;branch=yocto-4.12;destsuffix=${KMETA}"
1044 SRCREV_meta_qemux86 = "${AUTOREV}"
1045 SRCREV_machine_qemux86 = "${AUTOREV}"
1046
1047 .. note::
1048
1049 Be sure to replace
1050 path-to
1051 with the pathname to your local Git repositories. Also, you must
1052 be sure to specify the correct branch and machine types. For this
1053 example, the branch is
1054 standard/base
1055 and the machine is "qemux86".
1056
10574. *Build the Image:* With the source modified, your changes staged and
1058 committed, and the ``local.conf`` file pointing to the kernel files,
1059 you can now use BitBake to build the image:
1060 ::
1061
1062 $ cd ~/poky/build
1063 $ bitbake core-image-minimal
1064
10655. *Boot the image*: Boot the modified image in the QEMU emulator using
1066 this command. When prompted to login to the QEMU console, use "root"
1067 with no password:
1068 ::
1069
1070 $ cd ~/poky/build
1071 $ runqemu qemux86
1072
10736. *Look for Your Changes:* As QEMU booted, you might have seen your
1074 changes rapidly scroll by. If not, use these commands to see your
1075 changes:
1076 ::
1077
1078 # dmesg | less
1079
1080 You should see the results of your
1081 ``printk`` statements as part of the output when you scroll down the
1082 console window.
1083
10847. *Generate the Patch File:* Once you are sure that your patch works
1085 correctly, you can generate a ``*.patch`` file in the kernel source
1086 repository:
1087 ::
1088
1089 $ cd ~/linux-yocto-4.12/init
1090 $ git format-patch -1
1091 0001-calibrate.c-Added-some-printk-statements.patch
1092
10938. *Move the Patch File to Your Layer:* In order for subsequent builds
1094 to pick up patches, you need to move the patch file you created in
1095 the previous step to your layer ``meta-mylayer``. For this example,
1096 the layer created earlier is located in your home directory as
1097 ``meta-mylayer``. When the layer was created using the
1098 ``yocto-create`` script, no additional hierarchy was created to
1099 support patches. Before moving the patch file, you need to add
1100 additional structure to your layer using the following commands:
1101 ::
1102
1103 $ cd ~/meta-mylayer
1104 $ mkdir recipes-kernel
1105 $ mkdir recipes-kernel/linux
1106 $ mkdir recipes-kernel/linux/linux-yocto
1107
1108 Once you have created this
1109 hierarchy in your layer, you can move the patch file using the
1110 following command:
1111 ::
1112
1113 $ mv ~/linux-yocto-4.12/init/0001-calibrate.c-Added-some-printk-statements.patch ~/meta-mylayer/recipes-kernel/linux/linux-yocto
1114
11159. *Create the Append File:* Finally, you need to create the
1116 ``linux-yocto_4.12.bbappend`` file and insert statements that allow
1117 the OpenEmbedded build system to find the patch. The append file
1118 needs to be in your layer's ``recipes-kernel/linux`` directory and it
1119 must be named ``linux-yocto_4.12.bbappend`` and have the following
1120 contents:
1121 ::
1122
1123 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
1124 SRC_URI_append = "file://0001-calibrate.c-Added-some-printk-statements.patch"
1125
1126 The :term:`FILESEXTRAPATHS` and :term:`SRC_URI` statements
1127 enable the OpenEmbedded build system to find the patch file.
1128
1129 For more information on append files and patches, see the "`Creating
1130 the Append File <#creating-the-append-file>`__" and "`Applying
1131 Patches <#applying-patches>`__" sections. You can also see the
1132 ":ref:`dev-manual/dev-manual-common-tasks:using .bbappend files in your layer`"
1133 section in the Yocto Project Development Tasks Manual.
1134
1135 .. note::
1136
1137 To build
1138 core-image-minimal
1139 again and see the effects of your patch, you can essentially
1140 eliminate the temporary source files saved in
1141 poky/build/tmp/work/...
1142 and residual effects of the build by entering the following
1143 sequence of commands:
1144 ::
1145
1146 $ cd ~/poky/build
1147 $ bitbake -c cleanall yocto-linux
1148 $ bitbake core-image-minimal -c cleanall
1149 $ bitbake core-image-minimal
1150 $ runqemu qemux86
1151
1152
1153Configuring the Kernel
1154======================
1155
1156Configuring the Yocto Project kernel consists of making sure the
1157``.config`` file has all the right information in it for the image you
1158are building. You can use the ``menuconfig`` tool and configuration
1159fragments to make sure your ``.config`` file is just how you need it.
1160You can also save known configurations in a ``defconfig`` file that the
1161build system can use for kernel configuration.
1162
1163This section describes how to use ``menuconfig``, create and use
1164configuration fragments, and how to interactively modify your
1165``.config`` file to create the leanest kernel configuration file
1166possible.
1167
1168For more information on kernel configuration, see the "`Changing the
1169Configuration <#changing-the-configuration>`__" section.
1170
1171Using  ``menuconfig``
1172---------------------
1173
1174The easiest way to define kernel configurations is to set them through
1175the ``menuconfig`` tool. This tool provides an interactive method with
1176which to set kernel configurations. For general information on
1177``menuconfig``, see http://en.wikipedia.org/wiki/Menuconfig.
1178
1179To use the ``menuconfig`` tool in the Yocto Project development
1180environment, you must do the following:
1181
1182- Because you launch ``menuconfig`` using BitBake, you must be sure to
1183 set up your environment by running the
1184 :ref:`structure-core-script` script found in
1185 the :term:`Build Directory`.
1186
1187- You must be sure of the state of your build's configuration in the
1188 :term:`Source Directory`.
1189
1190- Your build host must have the following two packages installed:
1191 ::
1192
1193 libncurses5-dev
1194 libtinfo-dev
1195
1196The following commands initialize the BitBake environment, run the
1197:ref:`ref-tasks-kernel_configme`
1198task, and launch ``menuconfig``. These commands assume the Source
1199Directory's top-level folder is ``~/poky``:
1200::
1201
1202 $ cd poky
1203 $ source oe-init-build-env
1204 $ bitbake linux-yocto -c kernel_configme -f
1205 $ bitbake linux-yocto -c menuconfig
1206
1207Once ``menuconfig`` comes up, its standard
1208interface allows you to interactively examine and configure all the
1209kernel configuration parameters. After making your changes, simply exit
1210the tool and save your changes to create an updated version of the
1211``.config`` configuration file.
1212
1213.. note::
1214
1215 You can use the entire
1216 .config
1217 file as the
1218 defconfig
1219 file. For information on
1220 defconfig
1221 files, see the "
1222 Changing the Configuration
1223 ", "
1224 Using an In-Tree
1225 defconfig
1226 File
1227 , and "
1228 Creating a
1229 defconfig
1230 File
1231 " sections.
1232
1233Consider an example that configures the "CONFIG_SMP" setting for the
1234``linux-yocto-4.12`` kernel.
1235
1236.. note::
1237
1238 The OpenEmbedded build system recognizes this kernel as
1239 linux-yocto
1240 through Metadata (e.g.
1241 PREFERRED_VERSION
1242 \_linux-yocto ?= "12.4%"
1243 ).
1244
1245Once ``menuconfig`` launches, use the interface to navigate through the
1246selections to find the configuration settings in which you are
1247interested. For this example, you deselect "CONFIG_SMP" by clearing the
1248"Symmetric Multi-Processing Support" option. Using the interface, you
1249can find the option under "Processor Type and Features". To deselect
1250"CONFIG_SMP", use the arrow keys to highlight "Symmetric
1251Multi-Processing Support" and enter "N" to clear the asterisk. When you
1252are finished, exit out and save the change.
1253
1254Saving the selections updates the ``.config`` configuration file. This
1255is the file that the OpenEmbedded build system uses to configure the
1256kernel during the build. You can find and examine this file in the Build
1257Directory in ``tmp/work/``. The actual ``.config`` is located in the
1258area where the specific kernel is built. For example, if you were
1259building a Linux Yocto kernel based on the ``linux-yocto-4.12`` kernel
1260and you were building a QEMU image targeted for ``x86`` architecture,
1261the ``.config`` file would be:
1262::
1263
1264 poky/build/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+gitAUTOINC+eda4d18...
1265 ...967-r0/linux-qemux86-standard-build/.config
1266
1267.. note::
1268
1269 The previous example directory is artificially split and many of the
1270 characters in the actual filename are omitted in order to make it
1271 more readable. Also, depending on the kernel you are using, the exact
1272 pathname might differ.
1273
1274Within the ``.config`` file, you can see the kernel settings. For
1275example, the following entry shows that symmetric multi-processor
1276support is not set:
1277::
1278
1279 # CONFIG_SMP is not set
1280
1281A good method to isolate changed configurations is to use a combination
1282of the ``menuconfig`` tool and simple shell commands. Before changing
1283configurations with ``menuconfig``, copy the existing ``.config`` and
1284rename it to something else, use ``menuconfig`` to make as many changes
1285as you want and save them, then compare the renamed configuration file
1286against the newly created file. You can use the resulting differences as
1287your base to create configuration fragments to permanently save in your
1288kernel layer.
1289
1290.. note::
1291
1292 Be sure to make a copy of the
1293 .config
1294 file and do not just rename it. The build system needs an existing
1295 .config
1296 file from which to work.
1297
1298Creating a  ``defconfig`` File
1299------------------------------
1300
1301A ``defconfig`` file in the context of the Yocto Project is often a
1302``.config`` file that is copied from a build or a ``defconfig`` taken
1303from the kernel tree and moved into recipe space. You can use a
1304``defconfig`` file to retain a known set of kernel configurations from
1305which the OpenEmbedded build system can draw to create the final
1306``.config`` file.
1307
1308.. note::
1309
1310 Out-of-the-box, the Yocto Project never ships a
1311 defconfig
1312 or
1313 .config
1314 file. The OpenEmbedded build system creates the final
1315 .config
1316 file used to configure the kernel.
1317
1318To create a ``defconfig``, start with a complete, working Linux kernel
1319``.config`` file. Copy that file to the appropriate
1320``${``\ :term:`PN`\ ``}`` directory in
1321your layer's ``recipes-kernel/linux`` directory, and rename the copied
1322file to "defconfig" (e.g.
1323``~/meta-mylayer/recipes-kernel/linux/linux-yocto/defconfig``). Then,
1324add the following lines to the linux-yocto ``.bbappend`` file in your
1325layer:
1326::
1327
1328 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
1329 SRC_URI += "file://defconfig"
1330
1331The :term:`SRC_URI` tells the build system how to search for the file, while the
1332:term:`FILESEXTRAPATHS` extends the :term:`FILESPATH`
1333variable (search directories) to include the ``${PN}`` directory you
1334created to hold the configuration changes.
1335
1336.. note::
1337
1338 The build system applies the configurations from the
1339 defconfig
1340 file before applying any subsequent configuration fragments. The
1341 final kernel configuration is a combination of the configurations in
1342 the
1343 defconfig
1344 file and any configuration fragments you provide. You need to realize
1345 that if you have any configuration fragments, the build system
1346 applies these on top of and after applying the existing defconfig
1347 file configurations.
1348
1349For more information on configuring the kernel, see the "`Changing the
1350Configuration <#changing-the-configuration>`__" section.
1351
1352.. _creating-config-fragments:
1353
1354Creating Configuration Fragments
1355--------------------------------
1356
1357Configuration fragments are simply kernel options that appear in a file
1358placed where the OpenEmbedded build system can find and apply them. The
1359build system applies configuration fragments after applying
1360configurations from a ``defconfig`` file. Thus, the final kernel
1361configuration is a combination of the configurations in the
1362``defconfig`` file and then any configuration fragments you provide. The
1363build system applies fragments on top of and after applying the existing
1364defconfig file configurations.
1365
1366Syntactically, the configuration statement is identical to what would
1367appear in the ``.config`` file, which is in the :term:`Build Directory`.
1368
1369.. note::
1370
1371 For more information about where the
1372 .config
1373 file is located, see the example in the
1374 ":ref:`kernel-dev/kernel-dev-common:using \`\`menuconfig\`\``"
1375 section.
1376
1377It is simple to create a configuration fragment. One method is to use
1378shell commands. For example, issuing the following from the shell
1379creates a configuration fragment file named ``my_smp.cfg`` that enables
1380multi-processor support within the kernel:
1381::
1382
1383 $ echo "CONFIG_SMP=y" >> my_smp.cfg
1384
1385.. note::
1386
1387 All configuration fragment files must use the
1388 .cfg
1389 extension in order for the OpenEmbedded build system to recognize
1390 them as a configuration fragment.
1391
1392Another method is to create a configuration fragment using the
1393differences between two configuration files: one previously created and
1394saved, and one freshly created using the ``menuconfig`` tool.
1395
1396To create a configuration fragment using this method, follow these
1397steps:
1398
13991. *Complete a Build Through Kernel Configuration:* Complete a build at
1400 least through the kernel configuration task as follows:
1401 ::
1402
1403 $ bitbake linux-yocto -c kernel_configme -f
1404
1405 This step ensures that you create a
1406 ``.config`` file from a known state. Because situations exist where
1407 your build state might become unknown, it is best to run this task
1408 prior to starting ``menuconfig``.
1409
14102. *Launch menuconfig:* Run the ``menuconfig`` command:
1411 ::
1412
1413 $ bitbake linux-yocto -c menuconfig
1414
14153. *Create the Configuration Fragment:* Run the ``diffconfig`` command
1416 to prepare a configuration fragment. The resulting file
1417 ``fragment.cfg`` is placed in the
1418 ``${``\ :term:`WORKDIR`\ ``}``
1419 directory:
1420 ::
1421
1422 $ bitbake linux-yocto -c diffconfig
1423
1424The ``diffconfig`` command creates a file that is a list of Linux kernel
1425``CONFIG_`` assignments. See the "`Changing the
1426Configuration <#changing-the-configuration>`__" section for additional
1427information on how to use the output as a configuration fragment.
1428
1429.. note::
1430
1431 You can also use this method to create configuration fragments for a
1432 BSP. See the "
1433 BSP Descriptions
1434 " section for more information.
1435
1436Where do you put your configuration fragment files? You can place these
1437files in an area pointed to by
1438:term:`SRC_URI` as directed by your
1439``bblayers.conf`` file, which is located in your layer. The OpenEmbedded
1440build system picks up the configuration and adds it to the kernel's
1441configuration. For example, suppose you had a set of configuration
1442options in a file called ``myconfig.cfg``. If you put that file inside a
1443directory named ``linux-yocto`` that resides in the same directory as
1444the kernel's append file within your layer and then add the following
1445statements to the kernel's append file, those configuration options will
1446be picked up and applied when the kernel is built:
1447::
1448
1449 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
1450 SRC_URI += "file://myconfig.cfg"
1451
1452As mentioned earlier, you can group related configurations into multiple
1453files and name them all in the ``SRC_URI`` statement as well. For
1454example, you could group separate configurations specifically for
1455Ethernet and graphics into their own files and add those by using a
1456``SRC_URI`` statement like the following in your append file:
1457::
1458
1459 SRC_URI += "file://myconfig.cfg \
1460 file://eth.cfg \
1461 file://gfx.cfg"
1462
1463Validating Configuration
1464------------------------
1465
1466You can use the
1467:ref:`ref-tasks-kernel_configcheck`
1468task to provide configuration validation:
1469::
1470
1471 $ bitbake linux-yocto -c kernel_configcheck -f
1472
1473Running this task produces warnings for when a
1474requested configuration does not appear in the final ``.config`` file or
1475when you override a policy configuration in a hardware configuration
1476fragment.
1477
1478In order to run this task, you must have an existing ``.config`` file.
1479See the ":ref:`kernel-dev/kernel-dev-common:using \`\`menuconfig\`\``" section for
1480information on how to create a configuration file.
1481
1482Following is sample output from the ``do_kernel_configcheck`` task:
1483::
1484
1485 Loading cache: 100% |########################################################| Time: 0:00:00
1486 Loaded 1275 entries from dependency cache.
1487 NOTE: Resolving any missing task queue dependencies
1488
1489 Build Configuration:
1490 .
1491 .
1492 .
1493
1494 NOTE: Executing SetScene Tasks
1495 NOTE: Executing RunQueue Tasks
1496 WARNING: linux-yocto-4.12.12+gitAUTOINC+eda4d18ce4_16de014967-r0 do_kernel_configcheck:
1497 [kernel config]: specified values did not make it into the kernel's final configuration:
1498
1499 ---------- CONFIG_X86_TSC -----------------
1500 Config: CONFIG_X86_TSC
1501 From: /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc-cpu.cfg
1502 Requested value: CONFIG_X86_TSC=y
1503 Actual value:
1504
1505
1506 ---------- CONFIG_X86_BIGSMP -----------------
1507 Config: CONFIG_X86_BIGSMP
1508 From: /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1509 /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1510 Requested value: # CONFIG_X86_BIGSMP is not set
1511 Actual value:
1512
1513
1514 ---------- CONFIG_NR_CPUS -----------------
1515 Config: CONFIG_NR_CPUS
1516 From: /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1517 /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/bsp/common-pc/common-pc.cfg
1518 /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1519 Requested value: CONFIG_NR_CPUS=8
1520 Actual value: CONFIG_NR_CPUS=1
1521
1522
1523 ---------- CONFIG_SCHED_SMT -----------------
1524 Config: CONFIG_SCHED_SMT
1525 From: /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/cfg/smp.cfg
1526 /home/scottrif/poky/build/tmp/work-shared/qemux86/kernel-source/.kernel-meta/configs/standard/defconfig
1527 Requested value: CONFIG_SCHED_SMT=y
1528 Actual value:
1529
1530
1531
1532 NOTE: Tasks Summary: Attempted 288 tasks of which 285 didn't need to be rerun and all succeeded.
1533
1534 Summary: There were 3 WARNING messages shown.
1535
1536.. note::
1537
1538 The previous output example has artificial line breaks to make it
1539 more readable.
1540
1541The output describes the various problems that you can encounter along
1542with where to find the offending configuration items. You can use the
1543information in the logs to adjust your configuration files and then
1544repeat the
1545:ref:`ref-tasks-kernel_configme`
1546and
1547:ref:`ref-tasks-kernel_configcheck`
1548tasks until they produce no warnings.
1549
1550For more information on how to use the ``menuconfig`` tool, see the
1551:ref:`kernel-dev/kernel-dev-common:using \`\`menuconfig\`\`` section.
1552
1553Fine-Tuning the Kernel Configuration File
1554-----------------------------------------
1555
1556You can make sure the ``.config`` file is as lean or efficient as
1557possible by reading the output of the kernel configuration fragment
1558audit, noting any issues, making changes to correct the issues, and then
1559repeating.
1560
1561As part of the kernel build process, the ``do_kernel_configcheck`` task
1562runs. This task validates the kernel configuration by checking the final
1563``.config`` file against the input files. During the check, the task
1564produces warning messages for the following issues:
1565
1566- Requested options that did not make the final ``.config`` file.
1567
1568- Configuration items that appear twice in the same configuration
1569 fragment.
1570
1571- Configuration items tagged as "required" that were overridden.
1572
1573- A board overrides a non-board specific option.
1574
1575- Listed options not valid for the kernel being processed. In other
1576 words, the option does not appear anywhere.
1577
1578.. note::
1579
1580 The
1581 do_kernel_configcheck
1582 task can also optionally report if an option is overridden during
1583 processing.
1584
1585For each output warning, a message points to the file that contains a
1586list of the options and a pointer to the configuration fragment that
1587defines them. Collectively, the files are the key to streamlining the
1588configuration.
1589
1590To streamline the configuration, do the following:
1591
15921. *Use a Working Configuration:* Start with a full configuration that
1593 you know works. Be sure the configuration builds and boots
1594 successfully. Use this configuration file as your baseline.
1595
15962. *Run Configure and Check Tasks:* Separately run the
1597 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks:
1598 ::
1599
1600 $ bitbake linux-yocto -c kernel_configme -f
1601 $ bitbake linux-yocto -c kernel_configcheck -f
1602
16033. *Process the Results:* Take the resulting list of files from the
1604 ``do_kernel_configcheck`` task warnings and do the following:
1605
1606 - Drop values that are redefined in the fragment but do not change
1607 the final ``.config`` file.
1608
1609 - Analyze and potentially drop values from the ``.config`` file that
1610 override required configurations.
1611
1612 - Analyze and potentially remove non-board specific options.
1613
1614 - Remove repeated and invalid options.
1615
16164. *Re-Run Configure and Check Tasks:* After you have worked through the
1617 output of the kernel configuration audit, you can re-run the
1618 ``do_kernel_configme`` and ``do_kernel_configcheck`` tasks to see the
1619 results of your changes. If you have more issues, you can deal with
1620 them as described in the previous step.
1621
1622Iteratively working through steps two through four eventually yields a
1623minimal, streamlined configuration file. Once you have the best
1624``.config``, you can build the Linux Yocto kernel.
1625
1626Expanding Variables
1627===================
1628
1629Sometimes it is helpful to determine what a variable expands to during a
1630build. You can do examine the values of variables by examining the
1631output of the ``bitbake -e`` command. The output is long and is more
1632easily managed in a text file, which allows for easy searches:
1633::
1634
1635 $ bitbake -e virtual/kernel > some_text_file
1636
1637Within the text file, you can see
1638exactly how each variable is expanded and used by the OpenEmbedded build
1639system.
1640
1641Working with a "Dirty" Kernel Version String
1642============================================
1643
1644If you build a kernel image and the version string has a "+" or a
1645"-dirty" at the end, uncommitted modifications exist in the kernel's
1646source directory. Follow these steps to clean up the version string:
1647
16481. *Discover the Uncommitted Changes:* Go to the kernel's locally cloned
1649 Git repository (source directory) and use the following Git command
1650 to list the files that have been changed, added, or removed:
1651 ::
1652
1653 $ git status
1654
16552. *Commit the Changes:* You should commit those changes to the kernel
1656 source tree regardless of whether or not you will save, export, or
1657 use the changes:
1658 ::
1659
1660 $ git add
1661 $ git commit -s -a -m "getting rid of -dirty"
1662
16633. *Rebuild the Kernel Image:* Once you commit the changes, rebuild the
1664 kernel.
1665
1666 Depending on your particular kernel development workflow, the
1667 commands you use to rebuild the kernel might differ. For information
1668 on building the kernel image when using ``devtool``, see the
1669 ":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
1670 section. For
1671 information on building the kernel image when using Bitbake, see the
1672 "`Using Traditional Kernel Development to Patch the
1673 Kernel <#using-traditional-kernel-development-to-patch-the-kernel>`__"
1674 section.
1675
1676Working With Your Own Sources
1677=============================
1678
1679If you cannot work with one of the Linux kernel versions supported by
1680existing linux-yocto recipes, you can still make use of the Yocto
1681Project Linux kernel tooling by working with your own sources. When you
1682use your own sources, you will not be able to leverage the existing
1683kernel :term:`Metadata` and stabilization
1684work of the linux-yocto sources. However, you will be able to manage
1685your own Metadata in the same format as the linux-yocto sources.
1686Maintaining format compatibility facilitates converging with linux-yocto
1687on a future, mutually-supported kernel version.
1688
1689To help you use your own sources, the Yocto Project provides a
1690linux-yocto custom recipe (``linux-yocto-custom.bb``) that uses
1691``kernel.org`` sources and the Yocto Project Linux kernel tools for
1692managing kernel Metadata. You can find this recipe in the ``poky`` Git
1693repository of the Yocto Project :yocto_git:`Source Repository <>`
1694at:
1695::
1696
1697 poky/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
1698
1699Here are some basic steps you can use to work with your own sources:
1700
17011. *Create a Copy of the Kernel Recipe:* Copy the
1702 ``linux-yocto-custom.bb`` recipe to your layer and give it a
1703 meaningful name. The name should include the version of the Yocto
1704 Linux kernel you are using (e.g. ``linux-yocto-myproject_4.12.bb``,
1705 where "4.12" is the base version of the Linux kernel with which you
1706 would be working).
1707
17082. *Create a Directory for Your Patches:* In the same directory inside
1709 your layer, create a matching directory to store your patches and
1710 configuration files (e.g. ``linux-yocto-myproject``).
1711
17123. *Ensure You Have Configurations:* Make sure you have either a
1713 ``defconfig`` file or configuration fragment files in your layer.
1714 When you use the ``linux-yocto-custom.bb`` recipe, you must specify a
1715 configuration. If you do not have a ``defconfig`` file, you can run
1716 the following:
1717 ::
1718
1719 $ make defconfig
1720
1721 After running the command, copy the
1722 resulting ``.config`` file to the ``files`` directory in your layer
1723 as "defconfig" and then add it to the
1724 :term:`SRC_URI` variable in the
1725 recipe.
1726
1727 Running the ``make defconfig`` command results in the default
1728 configuration for your architecture as defined by your kernel.
1729 However, no guarantee exists that this configuration is valid for
1730 your use case, or that your board will even boot. This is
1731 particularly true for non-x86 architectures.
1732
1733 To use non-x86 ``defconfig`` files, you need to be more specific and
1734 find one that matches your board (i.e. for arm, you look in
1735 ``arch/arm/configs`` and use the one that is the best starting point
1736 for your board).
1737
17384. *Edit the Recipe:* Edit the following variables in your recipe as
1739 appropriate for your project:
1740
1741 - :term:`SRC_URI`: The
1742 ``SRC_URI`` should specify a Git repository that uses one of the
1743 supported Git fetcher protocols (i.e. ``file``, ``git``, ``http``,
1744 and so forth). The ``SRC_URI`` variable should also specify either
1745 a ``defconfig`` file or some configuration fragment files. The
1746 skeleton recipe provides an example ``SRC_URI`` as a syntax
1747 reference.
1748
1749 - :term:`LINUX_VERSION`:
1750 The Linux kernel version you are using (e.g. "4.12").
1751
1752 - :term:`LINUX_VERSION_EXTENSION`:
1753 The Linux kernel ``CONFIG_LOCALVERSION`` that is compiled into the
1754 resulting kernel and visible through the ``uname`` command.
1755
1756 - :term:`SRCREV`: The commit ID
1757 from which you want to build.
1758
1759 - :term:`PR`: Treat this variable the
1760 same as you would in any other recipe. Increment the variable to
1761 indicate to the OpenEmbedded build system that the recipe has
1762 changed.
1763
1764 - :term:`PV`: The default ``PV``
1765 assignment is typically adequate. It combines the
1766 ``LINUX_VERSION`` with the Source Control Manager (SCM) revision
1767 as derived from the :term:`SRCPV`
1768 variable. The combined results are a string with the following
1769 form:
1770 3.19.11+git1+68a635bf8dfb64b02263c1ac80c948647cc76d5f_1+218bd8d2022b9852c60d32f0d770931e3cf343e2
1771 While lengthy, the extra verbosity in ``PV`` helps ensure you are
1772 using the exact sources from which you intend to build.
1773
1774 - :term:`COMPATIBLE_MACHINE`:
1775 A list of the machines supported by your new recipe. This variable
1776 in the example recipe is set by default to a regular expression
1777 that matches only the empty string, "(^$)". This default setting
1778 triggers an explicit build failure. You must change it to match a
1779 list of the machines that your new recipe supports. For example,
1780 to support the ``qemux86`` and ``qemux86-64`` machines, use the
1781 following form: COMPATIBLE_MACHINE = "qemux86|qemux86-64"
1782
17835. *Customize Your Recipe as Needed:* Provide further customizations to
1784 your recipe as needed just as you would customize an existing
1785 linux-yocto recipe. See the "`Modifying an Existing
1786 Recipe <#modifying-an-existing-recipe>`__" section for information.
1787
1788Working with Out-of-Tree Modules
1789================================
1790
1791This section describes steps to build out-of-tree modules on your target
1792and describes how to incorporate out-of-tree modules in the build.
1793
1794Building Out-of-Tree Modules on the Target
1795------------------------------------------
1796
1797While the traditional Yocto Project development model would be to
1798include kernel modules as part of the normal build process, you might
1799find it useful to build modules on the target. This could be the case if
1800your target system is capable and powerful enough to handle the
1801necessary compilation. Before deciding to build on your target, however,
1802you should consider the benefits of using a proper cross-development
1803environment from your build host.
1804
1805If you want to be able to build out-of-tree modules on the target, there
1806are some steps you need to take on the target that is running your SDK
1807image. Briefly, the ``kernel-dev`` package is installed by default on
1808all ``*.sdk`` images and the ``kernel-devsrc`` package is installed on
1809many of the ``*.sdk`` images. However, you need to create some scripts
1810prior to attempting to build the out-of-tree modules on the target that
1811is running that image.
1812
1813Prior to attempting to build the out-of-tree modules, you need to be on
1814the target as root and you need to change to the ``/usr/src/kernel``
1815directory. Next, ``make`` the scripts:
1816::
1817
1818 # cd /usr/src/kernel
1819 # make scripts
1820
1821Because all SDK image recipes include ``dev-pkgs``, the
1822``kernel-dev`` packages will be installed as part of the SDK image and
1823the ``kernel-devsrc`` packages will be installed as part of applicable
1824SDK images. The SDK uses the scripts when building out-of-tree modules.
1825Once you have switched to that directory and created the scripts, you
1826should be able to build your out-of-tree modules on the target.
1827
1828Incorporating Out-of-Tree Modules
1829---------------------------------
1830
1831While it is always preferable to work with sources integrated into the
1832Linux kernel sources, if you need an external kernel module, the
1833``hello-mod.bb`` recipe is available as a template from which you can
1834create your own out-of-tree Linux kernel module recipe.
1835
1836This template recipe is located in the ``poky`` Git repository of the
1837Yocto Project :yocto_git:`Source Repository <>` at:
1838::
1839
1840 poky/meta-skeleton/recipes-kernel/hello-mod/hello-mod_0.1.bb
1841
1842To get started, copy this recipe to your layer and give it a meaningful
1843name (e.g. ``mymodule_1.0.bb``). In the same directory, create a new
1844directory named ``files`` where you can store any source files, patches,
1845or other files necessary for building the module that do not come with
1846the sources. Finally, update the recipe as needed for the module.
1847Typically, you will need to set the following variables:
1848
1849- :term:`DESCRIPTION`
1850
1851- :term:`LICENSE* <LICENSE>`
1852
1853- :term:`SRC_URI`
1854
1855- :term:`PV`
1856
1857Depending on the build system used by the module sources, you might need
1858to make some adjustments. For example, a typical module ``Makefile``
1859looks much like the one provided with the ``hello-mod`` template:
1860::
1861
1862 obj-m := hello.o
1863
1864 SRC := $(shell pwd)
1865
1866 all:
1867 $(MAKE) -C $(KERNEL_SRC) M=$(SRC)
1868
1869 modules_install:
1870 $(MAKE) -C $(KERNEL_SRC) M=$(SRC) modules_install
1871 ...
1872
1873The important point to note here is the :term:`KERNEL_SRC` variable. The
1874:ref:`module <ref-classes-module>` class sets this variable and the
1875:term:`KERNEL_PATH` variable to
1876``${STAGING_KERNEL_DIR}`` with the necessary Linux kernel build
1877information to build modules. If your module ``Makefile`` uses a
1878different variable, you might want to override the
1879:ref:`ref-tasks-compile` step, or
1880create a patch to the ``Makefile`` to work with the more typical
1881``KERNEL_SRC`` or ``KERNEL_PATH`` variables.
1882
1883After you have prepared your recipe, you will likely want to include the
1884module in your images. To do this, see the documentation for the
1885following variables in the Yocto Project Reference Manual and set one of
1886them appropriately for your machine configuration file:
1887
1888- :term:`MACHINE_ESSENTIAL_EXTRA_RDEPENDS`
1889
1890- :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
1891
1892- :term:`MACHINE_EXTRA_RDEPENDS`
1893
1894- :term:`MACHINE_EXTRA_RRECOMMENDS`
1895
1896Modules are often not required for boot and can be excluded from certain
1897build configurations. The following allows for the most flexibility:
1898::
1899
1900 MACHINE_EXTRA_RRECOMMENDS += "kernel-module-mymodule"
1901
1902The value is
1903derived by appending the module filename without the ``.ko`` extension
1904to the string "kernel-module-".
1905
1906Because the variable is
1907:term:`RRECOMMENDS` and not a
1908:term:`RDEPENDS` variable, the build
1909will not fail if this module is not available to include in the image.
1910
1911Inspecting Changes and Commits
1912==============================
1913
1914A common question when working with a kernel is: "What changes have been
1915applied to this tree?" Rather than using "grep" across directories to
1916see what has changed, you can use Git to inspect or search the kernel
1917tree. Using Git is an efficient way to see what has changed in the tree.
1918
1919What Changed in a Kernel?
1920-------------------------
1921
1922Following are a few examples that show how to use Git commands to
1923examine changes. These examples are by no means the only way to see
1924changes.
1925
1926.. note::
1927
1928 In the following examples, unless you provide a commit range,
1929 kernel.org
1930 history is blended with Yocto Project kernel changes. You can form
1931 ranges by using branch names from the kernel tree as the upper and
1932 lower commit markers with the Git commands. You can see the branch
1933 names through the web interface to the Yocto Project source
1934 repositories at
1935 .
1936
1937To see a full range of the changes, use the ``git whatchanged`` command
1938and specify a commit range for the branch (commit\ ``..``\ commit).
1939
1940Here is an example that looks at what has changed in the ``emenlow``
1941branch of the ``linux-yocto-3.19`` kernel. The lower commit range is the
1942commit associated with the ``standard/base`` branch, while the upper
1943commit range is the commit associated with the ``standard/emenlow``
1944branch.
1945::
1946
1947 $ git whatchanged origin/standard/base..origin/standard/emenlow
1948
1949To see short, one line summaries of changes use the ``git log`` command:
1950::
1951
1952 $ git log --oneline origin/standard/base..origin/standard/emenlow
1953
1954Use this command to see code differences for the changes:
1955::
1956
1957 $ git diff origin/standard/base..origin/standard/emenlow
1958
1959Use this command to see the commit log messages and the text
1960differences:
1961::
1962
1963 $ git show origin/standard/base..origin/standard/emenlow
1964
1965Use this command to create individual patches for each change. Here is
1966an example that that creates patch files for each commit and places them
1967in your ``Documents`` directory:
1968::
1969
1970 $ git format-patch -o $HOME/Documents origin/standard/base..origin/standard/emenlow
1971
1972Showing a Particular Feature or Branch Change
1973---------------------------------------------
1974
1975Tags in the Yocto Project kernel tree divide changes for significant
1976features or branches. The ``git show`` tag command shows changes based
1977on a tag. Here is an example that shows ``systemtap`` changes:
1978::
1979
1980 $ git show systemtap
1981
1982You can use the ``git branch --contains`` tag command to
1983show the branches that contain a particular feature. This command shows
1984the branches that contain the ``systemtap`` feature:
1985::
1986
1987 $ git branch --contains systemtap
1988
1989Adding Recipe-Space Kernel Features
1990===================================
1991
1992You can add kernel features in the
1993`recipe-space <#recipe-space-metadata>`__ by using the
1994:term:`KERNEL_FEATURES`
1995variable and by specifying the feature's ``.scc`` file path in the
1996:term:`SRC_URI` statement. When you
1997add features using this method, the OpenEmbedded build system checks to
1998be sure the features are present. If the features are not present, the
1999build stops. Kernel features are the last elements processed for
2000configuring and patching the kernel. Therefore, adding features in this
2001manner is a way to enforce specific features are present and enabled
2002without needing to do a full audit of any other layer's additions to the
2003``SRC_URI`` statement.
2004
2005You add a kernel feature by providing the feature as part of the
2006``KERNEL_FEATURES`` variable and by providing the path to the feature's
2007``.scc`` file, which is relative to the root of the kernel Metadata. The
2008OpenEmbedded build system searches all forms of kernel Metadata on the
2009``SRC_URI`` statement regardless of whether the Metadata is in the
2010"kernel-cache", system kernel Metadata, or a recipe-space Metadata (i.e.
2011part of the kernel recipe). See the "`Kernel Metadata
2012Location <#kernel-metadata-location>`__" section for additional
2013information.
2014
2015When you specify the feature's ``.scc`` file on the ``SRC_URI``
2016statement, the OpenEmbedded build system adds the directory of that
2017``.scc`` file along with all its subdirectories to the kernel feature
2018search path. Because subdirectories are searched, you can reference a
2019single ``.scc`` file in the ``SRC_URI`` statement to reference multiple
2020kernel features.
2021
2022Consider the following example that adds the "test.scc" feature to the
2023build.
2024
20251. *Create the Feature File:* Create a ``.scc`` file and locate it just
2026 as you would any other patch file, ``.cfg`` file, or fetcher item you
2027 specify in the ``SRC_URI`` statement.
2028
2029 .. note::
2030
2031 - You must add the directory of the ``.scc`` file to the
2032 fetcher's search path in the same manner as you would add a
2033 ``.patch`` file.
2034
2035 - You can create additional ``.scc`` files beneath the directory
2036 that contains the file you are adding. All subdirectories are
2037 searched during the build as potential feature directories.
2038
2039 Continuing with the example, suppose the "test.scc" feature you are
2040 adding has a ``test.scc`` file in the following directory:
2041 ::
2042
2043 my_recipe
2044 |
2045 +-linux-yocto
2046 |
2047 +-test.cfg
2048 +-test.scc
2049
2050 In this example, the
2051 ``linux-yocto`` directory has both the feature ``test.scc`` file and
2052 a similarly named configuration fragment file ``test.cfg``.
2053
20542. *Add the Feature File to SRC_URI:* Add the ``.scc`` file to the
2055 recipe's ``SRC_URI`` statement:
2056 ::
2057
2058 SRC_URI_append = " file://test.scc"
2059
2060 The leading space before the path is important as the path is
2061 appended to the existing path.
2062
20633. *Specify the Feature as a Kernel Feature:* Use the
2064 ``KERNEL_FEATURES`` statement to specify the feature as a kernel
2065 feature:
2066 ::
2067
2068 KERNEL_FEATURES_append = " test.scc"
2069
2070 The OpenEmbedded build
2071 system processes the kernel feature when it builds the kernel.
2072
2073 .. note::
2074
2075 If other features are contained below "test.scc", then their
2076 directories are relative to the directory containing the
2077 test.scc
2078 file.
diff --git a/documentation/kernel-dev/kernel-dev-common.xml b/documentation/kernel-dev/kernel-dev-common.xml
index c1c2d6d703..8e8a6dbed4 100644
--- a/documentation/kernel-dev/kernel-dev-common.xml
+++ b/documentation/kernel-dev/kernel-dev-common.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='kernel-dev-common'> 6<chapter id='kernel-dev-common'>
6<title>Common Tasks</title> 7<title>Common Tasks</title>
diff --git a/documentation/kernel-dev/kernel-dev-concepts-appx.rst b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
new file mode 100644
index 0000000000..04cb1172b2
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-concepts-appx.rst
@@ -0,0 +1,426 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************************
4Advanced Kernel Concepts
5************************
6
7.. _kernel-big-picture:
8
9Yocto Project Kernel Development and Maintenance
10================================================
11
12Kernels available through the Yocto Project (Yocto Linux kernels), like
13other kernels, are based off the Linux kernel releases from
14http://www.kernel.org. At the beginning of a major Linux kernel
15development cycle, the Yocto Project team chooses a Linux kernel based
16on factors such as release timing, the anticipated release timing of
17final upstream ``kernel.org`` versions, and Yocto Project feature
18requirements. Typically, the Linux kernel chosen is in the final stages
19of development by the Linux community. In other words, the Linux kernel
20is in the release candidate or "rc" phase and has yet to reach final
21release. But, by being in the final stages of external development, the
22team knows that the ``kernel.org`` final release will clearly be within
23the early stages of the Yocto Project development window.
24
25This balance allows the Yocto Project team to deliver the most
26up-to-date Yocto Linux kernel possible, while still ensuring that the
27team has a stable official release for the baseline Linux kernel
28version.
29
30As implied earlier, the ultimate source for Yocto Linux kernels are
31released kernels from ``kernel.org``. In addition to a foundational
32kernel from ``kernel.org``, the available Yocto Linux kernels contain a
33mix of important new mainline developments, non-mainline developments
34(when no alternative exists), Board Support Package (BSP) developments,
35and custom features. These additions result in a commercially released
36Yocto Project Linux kernel that caters to specific embedded designer
37needs for targeted hardware.
38
39You can find a web interface to the Yocto Linux kernels in the
40:ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`
41at :yocto_git:`/`. If you look at the interface, you will see to
42the left a grouping of Git repositories titled "Yocto Linux Kernel".
43Within this group, you will find several Linux Yocto kernels developed
44and included with Yocto Project releases:
45
46- *linux-yocto-4.1:* The stable Yocto Project kernel to use with
47 the Yocto Project Release 2.0. This kernel is based on the Linux 4.1
48 released kernel.
49
50- *linux-yocto-4.4:* The stable Yocto Project kernel to use with
51 the Yocto Project Release 2.1. This kernel is based on the Linux 4.4
52 released kernel.
53
54- *linux-yocto-4.6:* A temporary kernel that is not tied to any
55 Yocto Project release.
56
57- *linux-yocto-4.8:* The stable yocto Project kernel to use with
58 the Yocto Project Release 2.2.
59
60- *linux-yocto-4.9:* The stable Yocto Project kernel to use with
61 the Yocto Project Release 2.3. This kernel is based on the Linux 4.9
62 released kernel.
63
64- *linux-yocto-4.10:* The default stable Yocto Project kernel to
65 use with the Yocto Project Release 2.3. This kernel is based on the
66 Linux 4.10 released kernel.
67
68- *linux-yocto-4.12:* The default stable Yocto Project kernel to
69 use with the Yocto Project Release 2.4. This kernel is based on the
70 Linux 4.12 released kernel.
71
72- *yocto-kernel-cache:* The ``linux-yocto-cache`` contains patches
73 and configurations for the linux-yocto kernel tree. This repository
74 is useful when working on the linux-yocto kernel. For more
75 information on this "Advanced Kernel Metadata", see the
76 ":doc:`kernel-dev-advanced`" Chapter.
77
78- *linux-yocto-dev:* A development kernel based on the latest
79 upstream release candidate available.
80
81.. note::
82
83 Long Term Support Initiative (LTSI) for Yocto Linux kernels is as
84 follows:
85
86 - For Yocto Project releases 1.7, 1.8, and 2.0, the LTSI kernel is
87 ``linux-yocto-3.14``.
88
89 - For Yocto Project releases 2.1, 2.2, and 2.3, the LTSI kernel is
90 ``linux-yocto-4.1``.
91
92 - For Yocto Project release 2.4, the LTSI kernel is
93 ``linux-yocto-4.9``
94
95 - ``linux-yocto-4.4`` is an LTS kernel.
96
97Once a Yocto Linux kernel is officially released, the Yocto Project team
98goes into their next development cycle, or upward revision (uprev)
99cycle, while still continuing maintenance on the released kernel. It is
100important to note that the most sustainable and stable way to include
101feature development upstream is through a kernel uprev process.
102Back-porting hundreds of individual fixes and minor features from
103various kernel versions is not sustainable and can easily compromise
104quality.
105
106During the uprev cycle, the Yocto Project team uses an ongoing analysis
107of Linux kernel development, BSP support, and release timing to select
108the best possible ``kernel.org`` Linux kernel version on which to base
109subsequent Yocto Linux kernel development. The team continually monitors
110Linux community kernel development to look for significant features of
111interest. The team does consider back-porting large features if they
112have a significant advantage. User or community demand can also trigger
113a back-port or creation of new functionality in the Yocto Project
114baseline kernel during the uprev cycle.
115
116Generally speaking, every new Linux kernel both adds features and
117introduces new bugs. These consequences are the basic properties of
118upstream Linux kernel development and are managed by the Yocto Project
119team's Yocto Linux kernel development strategy. It is the Yocto Project
120team's policy to not back-port minor features to the released Yocto
121Linux kernel. They only consider back-porting significant technological
122jumps DASH and, that is done after a complete gap analysis. The reason
123for this policy is that back-porting any small to medium sized change
124from an evolving Linux kernel can easily create mismatches,
125incompatibilities and very subtle errors.
126
127The policies described in this section result in both a stable and a
128cutting edge Yocto Linux kernel that mixes forward ports of existing
129Linux kernel features and significant and critical new functionality.
130Forward porting Linux kernel functionality into the Yocto Linux kernels
131available through the Yocto Project can be thought of as a "micro
132uprev." The many "micro uprevs" produce a Yocto Linux kernel version
133with a mix of important new mainline, non-mainline, BSP developments and
134feature integrations. This Yocto Linux kernel gives insight into new
135features and allows focused amounts of testing to be done on the kernel,
136which prevents surprises when selecting the next major uprev. The
137quality of these cutting edge Yocto Linux kernels is evolving and the
138kernels are used in leading edge feature and BSP development.
139
140Yocto Linux Kernel Architecture and Branching Strategies
141========================================================
142
143As mentioned earlier, a key goal of the Yocto Project is to present the
144developer with a kernel that has a clear and continuous history that is
145visible to the user. The architecture and mechanisms, in particular the
146branching strategies, used achieve that goal in a manner similar to
147upstream Linux kernel development in ``kernel.org``.
148
149You can think of a Yocto Linux kernel as consisting of a baseline Linux
150kernel with added features logically structured on top of the baseline.
151The features are tagged and organized by way of a branching strategy
152implemented by the Yocto Project team using the Source Code Manager
153(SCM) Git.
154
155.. note::
156
157 - Git is the obvious SCM for meeting the Yocto Linux kernel
158 organizational and structural goals described in this section. Not
159 only is Git the SCM for Linux kernel development in ``kernel.org``
160 but, Git continues to grow in popularity and supports many
161 different work flows, front-ends and management techniques.
162
163 - You can find documentation on Git at
164 http://git-scm.com/documentation. You can also get an
165 introduction to Git as it applies to the Yocto Project in the
166 ":ref:`overview-manual/overview-manual-development-environment:git`" section in the Yocto Project
167 Overview and Concepts Manual. The latter reference provides an
168 overview of Git and presents a minimal set of Git commands that
169 allows you to be functional using Git. You can use as much, or as
170 little, of what Git has to offer to accomplish what you need for
171 your project. You do not have to be a "Git Expert" in order to use
172 it with the Yocto Project.
173
174Using Git's tagging and branching features, the Yocto Project team
175creates kernel branches at points where functionality is no longer
176shared and thus, needs to be isolated. For example, board-specific
177incompatibilities would require different functionality and would
178require a branch to separate the features. Likewise, for specific kernel
179features, the same branching strategy is used.
180
181This "tree-like" architecture results in a structure that has features
182organized to be specific for particular functionality, single kernel
183types, or a subset of kernel types. Thus, the user has the ability to
184see the added features and the commits that make up those features. In
185addition to being able to see added features, the user can also view the
186history of what made up the baseline Linux kernel.
187
188Another consequence of this strategy results in not having to store the
189same feature twice internally in the tree. Rather, the kernel team
190stores the unique differences required to apply the feature onto the
191kernel type in question.
192
193.. note::
194
195 The Yocto Project team strives to place features in the tree such
196 that features can be shared by all boards and kernel types where
197 possible. However, during development cycles or when large features
198 are merged, the team cannot always follow this practice. In those
199 cases, the team uses isolated branches to merge features.
200
201BSP-specific code additions are handled in a similar manner to
202kernel-specific additions. Some BSPs only make sense given certain
203kernel types. So, for these types, the team creates branches off the end
204of that kernel type for all of the BSPs that are supported on that
205kernel type. From the perspective of the tools that create the BSP
206branch, the BSP is really no different than a feature. Consequently, the
207same branching strategy applies to BSPs as it does to kernel features.
208So again, rather than store the BSP twice, the team only stores the
209unique differences for the BSP across the supported multiple kernels.
210
211While this strategy can result in a tree with a significant number of
212branches, it is important to realize that from the developer's point of
213view, there is a linear path that travels from the baseline
214``kernel.org``, through a select group of features and ends with their
215BSP-specific commits. In other words, the divisions of the kernel are
216transparent and are not relevant to the developer on a day-to-day basis.
217From the developer's perspective, this path is the "master" branch in
218Git terms. The developer does not need to be aware of the existence of
219any other branches at all. Of course, value exists in the having these
220branches in the tree, should a person decide to explore them. For
221example, a comparison between two BSPs at either the commit level or at
222the line-by-line code ``diff`` level is now a trivial operation.
223
224The following illustration shows the conceptual Yocto Linux kernel.
225
226.. image:: figures/kernel-architecture-overview.png
227 :align: center
228
229In the illustration, the "Kernel.org Branch Point" marks the specific
230spot (or Linux kernel release) from which the Yocto Linux kernel is
231created. From this point forward in the tree, features and differences
232are organized and tagged.
233
234The "Yocto Project Baseline Kernel" contains functionality that is
235common to every kernel type and BSP that is organized further along in
236the tree. Placing these common features in the tree this way means
237features do not have to be duplicated along individual branches of the
238tree structure.
239
240From the "Yocto Project Baseline Kernel", branch points represent
241specific functionality for individual Board Support Packages (BSPs) as
242well as real-time kernels. The illustration represents this through
243three BSP-specific branches and a real-time kernel branch. Each branch
244represents some unique functionality for the BSP or for a real-time
245Yocto Linux kernel.
246
247In this example structure, the "Real-time (rt) Kernel" branch has common
248features for all real-time Yocto Linux kernels and contains more
249branches for individual BSP-specific real-time kernels. The illustration
250shows three branches as an example. Each branch points the way to
251specific, unique features for a respective real-time kernel as they
252apply to a given BSP.
253
254The resulting tree structure presents a clear path of markers (or
255branches) to the developer that, for all practical purposes, is the
256Yocto Linux kernel needed for any given set of requirements.
257
258.. note::
259
260 Keep in mind the figure does not take into account all the supported
261 Yocto Linux kernels, but rather shows a single generic kernel just
262 for conceptual purposes. Also keep in mind that this structure
263 represents the Yocto Project
264 Source Repositories
265 that are either pulled from during the build or established on the
266 host development system prior to the build by either cloning a
267 particular kernel's Git repository or by downloading and unpacking a
268 tarball.
269
270Working with the kernel as a structured tree follows recognized
271community best practices. In particular, the kernel as shipped with the
272product, should be considered an "upstream source" and viewed as a
273series of historical and documented modifications (commits). These
274modifications represent the development and stabilization done by the
275Yocto Project kernel development team.
276
277Because commits only change at significant release points in the product
278life cycle, developers can work on a branch created from the last
279relevant commit in the shipped Yocto Project Linux kernel. As mentioned
280previously, the structure is transparent to the developer because the
281kernel tree is left in this state after cloning and building the kernel.
282
283Kernel Build File Hierarchy
284===========================
285
286Upstream storage of all the available kernel source code is one thing,
287while representing and using the code on your host development system is
288another. Conceptually, you can think of the kernel source repositories
289as all the source files necessary for all the supported Yocto Linux
290kernels. As a developer, you are just interested in the source files for
291the kernel on which you are working. And, furthermore, you need them
292available on your host system.
293
294Kernel source code is available on your host system several different
295ways:
296
297- *Files Accessed While using devtool:* ``devtool``, which is
298 available with the Yocto Project, is the preferred method by which to
299 modify the kernel. See the ":ref:`kernel-dev/kernel-dev-intro:kernel modification workflow`" section.
300
301- *Cloned Repository:* If you are working in the kernel all the time,
302 you probably would want to set up your own local Git repository of
303 the Yocto Linux kernel tree. For information on how to clone a Yocto
304 Linux kernel Git repository, see the
305 ":ref:`kernel-dev/kernel-dev-common:preparing the build host to work on the kernel`"
306 section.
307
308- *Temporary Source Files from a Build:* If you just need to make some
309 patches to the kernel using a traditional BitBake workflow (i.e. not
310 using the ``devtool``), you can access temporary kernel source files
311 that were extracted and used during a kernel build.
312
313The temporary kernel source files resulting from a build using BitBake
314have a particular hierarchy. When you build the kernel on your
315development system, all files needed for the build are taken from the
316source repositories pointed to by the
317:term:`SRC_URI` variable and gathered
318in a temporary work area where they are subsequently used to create the
319unique kernel. Thus, in a sense, the process constructs a local source
320tree specific to your kernel from which to generate the new kernel
321image.
322
323The following figure shows the temporary file structure created on your
324host system when you build the kernel using Bitbake. This
325:term:`Build Directory` contains all the
326source files used during the build.
327
328.. image:: figures/kernel-overview-2-generic.png
329 :align: center
330
331Again, for additional information on the Yocto Project kernel's
332architecture and its branching strategy, see the
333":ref:`kernel-dev/kernel-dev-concepts-appx:yocto linux kernel architecture and branching strategies`"
334section. You can also reference the
335":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
336and
337":ref:`kernel-dev/kernel-dev-common:using traditional kernel development to patch the kernel`"
338sections for detailed example that modifies the kernel.
339
340Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase
341=======================================================================================
342
343This section describes part of the kernel configuration audit phase that
344most developers can ignore. For general information on kernel
345configuration including ``menuconfig``, ``defconfig`` files, and
346configuration fragments, see the
347":ref:`kernel-dev/kernel-dev-common:configuring the kernel`" section.
348
349During this part of the audit phase, the contents of the final
350``.config`` file are compared against the fragments specified by the
351system. These fragments can be system fragments, distro fragments, or
352user-specified configuration elements. Regardless of their origin, the
353OpenEmbedded build system warns the user if a specific option is not
354included in the final kernel configuration.
355
356By default, in order to not overwhelm the user with configuration
357warnings, the system only reports missing "hardware" options as they
358could result in a boot failure or indicate that important hardware is
359not available.
360
361To determine whether or not a given option is "hardware" or
362"non-hardware", the kernel Metadata in ``yocto-kernel-cache`` contains
363files that classify individual or groups of options as either hardware
364or non-hardware. To better show this, consider a situation where the
365``yocto-kernel-cache`` contains the following files:
366::
367
368 yocto-kernel-cache/features/drm-psb/hardware.cfg
369 yocto-kernel-cache/features/kgdb/hardware.cfg
370 yocto-kernel-cache/ktypes/base/hardware.cfg
371 yocto-kernel-cache/bsp/mti-malta32/hardware.cfg
372 yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg
373 yocto-kernel-cache/bsp/qemuarma9/hardware.cfg
374 yocto-kernel-cache/bsp/mti-malta64/hardware.cfg
375 yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
376 yocto-kernel-cache/bsp/common-pc/hardware.cfg
377 yocto-kernel-cache/bsp/common-pc-64/hardware.cfg
378 yocto-kernel-cache/features/rfkill/non-hardware.cfg
379 yocto-kernel-cache/ktypes/base/non-hardware.cfg
380 yocto-kernel-cache/features/aufs/non-hardware.kcf
381 yocto-kernel-cache/features/ocf/non-hardware.kcf
382 yocto-kernel-cache/ktypes/base/non-hardware.kcf
383 yocto-kernel-cache/ktypes/base/hardware.kcf
384 yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf
385
386The following list
387provides explanations for the various files:
388
389- ``hardware.kcf``: Specifies a list of kernel Kconfig files that
390 contain hardware options only.
391
392- ``non-hardware.kcf``: Specifies a list of kernel Kconfig files that
393 contain non-hardware options only.
394
395- ``hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options that
396 are hardware, regardless of whether or not they are within a Kconfig
397 file specified by a hardware or non-hardware Kconfig file (i.e.
398 ``hardware.kcf`` or ``non-hardware.kcf``).
399
400- ``non-hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options
401 that are not hardware, regardless of whether or not they are within a
402 Kconfig file specified by a hardware or non-hardware Kconfig file
403 (i.e. ``hardware.kcf`` or ``non-hardware.kcf``).
404
405Here is a specific example using the
406``kernel-cache/bsp/mti-malta32/hardware.cfg``:
407::
408
409 CONFIG_SERIAL_8250
410 CONFIG_SERIAL_8250_CONSOLE
411 CONFIG_SERIAL_8250_NR_UARTS
412 CONFIG_SERIAL_8250_PCI
413 CONFIG_SERIAL_CORE
414 CONFIG_SERIAL_CORE_CONSOLE
415 CONFIG_VGA_ARB
416
417The kernel configuration audit automatically detects
418these files (hence the names must be exactly the ones discussed here),
419and uses them as inputs when generating warnings about the final
420``.config`` file.
421
422A user-specified kernel Metadata repository, or recipe space feature,
423can use these same files to classify options that are found within its
424``.cfg`` files as hardware or non-hardware, to prevent the OpenEmbedded
425build system from producing an error or warning when an option is not in
426the final ``.config`` file.
diff --git a/documentation/kernel-dev/kernel-dev-concepts-appx.xml b/documentation/kernel-dev/kernel-dev-concepts-appx.xml
index 62c68527d2..bf0c525caf 100644
--- a/documentation/kernel-dev/kernel-dev-concepts-appx.xml
+++ b/documentation/kernel-dev/kernel-dev-concepts-appx.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='kernel-dev-concepts-appx'> 6<appendix id='kernel-dev-concepts-appx'>
6<title>Advanced Kernel Concepts</title> 7<title>Advanced Kernel Concepts</title>
@@ -191,7 +192,7 @@
191 Forward porting Linux kernel functionality into the Yocto Linux 192 Forward porting Linux kernel functionality into the Yocto Linux
192 kernels available through the Yocto Project can be thought of as 193 kernels available through the Yocto Project can be thought of as
193 a "micro uprev." 194 a "micro uprev."
194 The many micro uprevs produce a Yocto Linux kernel version with 195 The many "micro uprevs" produce a Yocto Linux kernel version with
195 a mix of important new mainline, non-mainline, BSP developments 196 a mix of important new mainline, non-mainline, BSP developments
196 and feature integrations. 197 and feature integrations.
197 This Yocto Linux kernel gives insight into new features and 198 This Yocto Linux kernel gives insight into new features and
diff --git a/documentation/kernel-dev/kernel-dev-customization.xsl b/documentation/kernel-dev/kernel-dev-customization.xsl
index 325b738e94..88bf7c678a 100644
--- a/documentation/kernel-dev/kernel-dev-customization.xsl
+++ b/documentation/kernel-dev/kernel-dev-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/kernel-dev/kernel-dev-faq.rst b/documentation/kernel-dev/kernel-dev-faq.rst
new file mode 100644
index 0000000000..b5e6a84eba
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-faq.rst
@@ -0,0 +1,81 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************
4Kernel Development FAQ
5**********************
6
7.. _kernel-dev-faq-section:
8
9Common Questions and Solutions
10==============================
11
12The following lists some solutions for common questions.
13
14How do I use my own Linux kernel ``.config`` file?
15--------------------------------------------------
16
17Refer to the
18":ref:`kernel-dev/kernel-dev-common:changing the configuration`"
19section for information.
20
21How do I create configuration fragments?
22----------------------------------------
23
24A: Refer to the
25":ref:`kernel-dev/kernel-dev-common:creating configuration fragments`"
26section for information.
27
28How do I use my own Linux kernel sources?
29-----------------------------------------
30
31Refer to the
32":ref:`kernel-dev/kernel-dev-common:working with your own sources`"
33section for information.
34
35How do I install/not-install the kernel image on the rootfs?
36------------------------------------------------------------
37
38The kernel image (e.g. ``vmlinuz``) is provided by the
39``kernel-image`` package. Image recipes depend on ``kernel-base``. To
40specify whether or not the kernel image is installed in the generated
41root filesystem, override ``RDEPENDS_kernel-base`` to include or not
42include "kernel-image". See the
43":ref:`dev-manual/dev-manual-common-tasks:using .bbappend files in your layer`"
44section in the
45Yocto Project Development Tasks Manual for information on how to use an
46append file to override metadata.
47
48How do I install a specific kernel module?
49------------------------------------------
50
51Linux kernel modules are packaged individually. To ensure a
52specific kernel module is included in an image, include it in the
53appropriate machine
54:term:`RRECOMMENDS` variable.
55These other variables are useful for installing specific modules:
56:term:`MACHINE_ESSENTIAL_EXTRA_RDEPENDS`
57:term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
58:term:`MACHINE_EXTRA_RDEPENDS`
59:term:`MACHINE_EXTRA_RRECOMMENDS`
60For example, set the following in the ``qemux86.conf`` file to include
61the ``ab123`` kernel modules with images built for the ``qemux86``
62machine:
63::
64
65 MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123"
66
67For more
68information, see the "`Incorporating Out-of-Tree
69Modules <#incorporating-out-of-tree-modules>`__" section.
70
71How do I change the Linux kernel command line?
72----------------------------------------------
73
74The Linux kernel command line is
75typically specified in the machine config using the ``APPEND`` variable.
76For example, you can add some helpful debug information doing the
77following:
78::
79
80 APPEND += "printk.time=y initcall_debug debug"
81
diff --git a/documentation/kernel-dev/kernel-dev-faq.xml b/documentation/kernel-dev/kernel-dev-faq.xml
index c3a20465a0..d76f0a4e32 100644
--- a/documentation/kernel-dev/kernel-dev-faq.xml
+++ b/documentation/kernel-dev/kernel-dev-faq.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='kernel-dev-faq'> 6<appendix id='kernel-dev-faq'>
6<title>Kernel Development FAQ</title> 7<title>Kernel Development FAQ</title>
diff --git a/documentation/kernel-dev/kernel-dev-intro.rst b/documentation/kernel-dev/kernel-dev-intro.rst
new file mode 100644
index 0000000000..21d43d5e80
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-intro.rst
@@ -0,0 +1,183 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Introduction
5************
6
7.. _kernel-dev-overview:
8
9Overview
10========
11
12Regardless of how you intend to make use of the Yocto Project, chances
13are you will work with the Linux kernel. This manual describes how to
14set up your build host to support kernel development, introduces the
15kernel development process, provides background information on the Yocto
16Linux kernel :term:`Metadata`, describes
17common tasks you can perform using the kernel tools, shows you how to
18use the kernel Metadata needed to work with the kernel inside the Yocto
19Project, and provides insight into how the Yocto Project team develops
20and maintains Yocto Linux kernel Git repositories and Metadata.
21
22Each Yocto Project release has a set of Yocto Linux kernel recipes,
23whose Git repositories you can view in the Yocto
24:yocto_git:`Source Repositories <>` under the "Yocto Linux Kernel"
25heading. New recipes for the release track the latest Linux kernel
26upstream developments from http://www.kernel.org> and introduce
27newly-supported platforms. Previous recipes in the release are refreshed
28and supported for at least one additional Yocto Project release. As they
29align, these previous releases are updated to include the latest from
30the Long Term Support Initiative (LTSI) project. You can learn more
31about Yocto Linux kernels and LTSI in the ":ref:`Yocto Project Kernel
32Development and Maintenance <kernel-big-picture>`" section.
33
34Also included is a Yocto Linux kernel development recipe
35(``linux-yocto-dev.bb``) should you want to work with the very latest in
36upstream Yocto Linux kernel development and kernel Metadata development.
37
38.. note::
39
40 For more on Yocto Linux kernels, see the "
41 Yocto Project Kernel Development and Maintenance
42 section.
43
44The Yocto Project also provides a powerful set of kernel tools for
45managing Yocto Linux kernel sources and configuration data. You can use
46these tools to make a single configuration change, apply multiple
47patches, or work with your own kernel sources.
48
49In particular, the kernel tools allow you to generate configuration
50fragments that specify only what you must, and nothing more.
51Configuration fragments only need to contain the highest level visible
52``CONFIG`` options as presented by the Yocto Linux kernel ``menuconfig``
53system. Contrast this against a complete Yocto Linux kernel ``.config``
54file, which includes all the automatically selected ``CONFIG`` options.
55This efficiency reduces your maintenance effort and allows you to
56further separate your configuration in ways that make sense for your
57project. A common split separates policy and hardware. For example, all
58your kernels might support the ``proc`` and ``sys`` filesystems, but
59only specific boards require sound, USB, or specific drivers. Specifying
60these configurations individually allows you to aggregate them together
61as needed, but maintains them in only one place. Similar logic applies
62to separating source changes.
63
64If you do not maintain your own kernel sources and need to make only
65minimal changes to the sources, the released recipes provide a vetted
66base upon which to layer your changes. Doing so allows you to benefit
67from the continual kernel integration and testing performed during
68development of the Yocto Project.
69
70If, instead, you have a very specific Linux kernel source tree and are
71unable to align with one of the official Yocto Linux kernel recipes, an
72alternative exists by which you can use the Yocto Project Linux kernel
73tools with your own kernel sources.
74
75The remainder of this manual provides instructions for completing
76specific Linux kernel development tasks. These instructions assume you
77are comfortable working with
78`BitBake <http://openembedded.org/wiki/Bitbake>`__ recipes and basic
79open-source development tools. Understanding these concepts will
80facilitate the process of working with the kernel recipes. If you find
81you need some additional background, please be sure to review and
82understand the following documentation:
83
84- :doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document.
85
86- :doc:`../overview-manual/overview-manual`.
87
88- :ref:`devtool
89 workflow <sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow>`
90 as described in the Yocto Project Application Development and the
91 Extensible Software Development Kit (eSDK) manual.
92
93- The ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
94 section in the Yocto Project Development Tasks Manual.
95
96- The "`Kernel Modification
97 Workflow <#kernel-modification-workflow>`__" section.
98
99Kernel Modification Workflow
100============================
101
102Kernel modification involves changing the Yocto Project kernel, which
103could involve changing configuration options as well as adding new
104kernel recipes. Configuration changes can be added in the form of
105configuration fragments, while recipe modification comes through the
106kernel's ``recipes-kernel`` area in a kernel layer you create.
107
108This section presents a high-level overview of the Yocto Project kernel
109modification workflow. The illustration and accompanying list provide
110general information and references for further information.
111
112.. image:: figures/kernel-dev-flow.png
113 :align: center
114
1151. *Set up Your Host Development System to Support Development Using the
116 Yocto Project*: See the ":doc:`../dev-manual/dev-manual-start`" section in
117 the Yocto Project Development Tasks Manual for options on how to get
118 a build host ready to use the Yocto Project.
119
1202. *Set Up Your Host Development System for Kernel Development:* It is
121 recommended that you use ``devtool`` and an extensible SDK for kernel
122 development. Alternatively, you can use traditional kernel
123 development methods with the Yocto Project. Either way, there are
124 steps you need to take to get the development environment ready.
125
126 Using ``devtool`` and the eSDK requires that you have a clean build
127 of the image and that you are set up with the appropriate eSDK. For
128 more information, see the
129 ":ref:`kernel-dev/kernel-dev-common:getting ready to develop using \`\`devtool\`\``"
130 section.
131
132 Using traditional kernel development requires that you have the
133 kernel source available in an isolated local Git repository. For more
134 information, see the
135 ":ref:`kernel-dev/kernel-dev-common:getting ready for traditional kernel development`"
136 section.
137
1383. *Make Changes to the Kernel Source Code if applicable:* Modifying the
139 kernel does not always mean directly changing source files. However,
140 if you have to do this, you make the changes to the files in the
141 eSDK's Build Directory if you are using ``devtool``. For more
142 information, see the
143 ":ref:`kernel-dev/kernel-dev-common:using \`\`devtool\`\` to patch the kernel`"
144 section.
145
146 If you are using traditional kernel development, you edit the source
147 files in the kernel's local Git repository. For more information, see the
148 ":ref:`kernel-dev/kernel-dev-common:using traditional kernel development to patch the kernel`"
149 section.
150
1514. *Make Kernel Configuration Changes if Applicable:* If your situation
152 calls for changing the kernel's configuration, you can use
153 :ref:`menuconfig <kernel-dev/kernel-dev-common:using \`\`menuconfig\`\`>`,
154 which allows you to
155 interactively develop and test the configuration changes you are
156 making to the kernel. Saving changes you make with ``menuconfig``
157 updates the kernel's ``.config`` file.
158
159 .. note::
160
161 Try to resist the temptation to directly edit an existing
162 .config
163 file, which is found in the Build Directory among the source code
164 used for the build. Doing so, can produce unexpected results when
165 the OpenEmbedded build system regenerates the configuration file.
166
167 Once you are satisfied with the configuration changes made using
168 ``menuconfig`` and you have saved them, you can directly compare the
169 resulting ``.config`` file against an existing original and gather
170 those changes into a `configuration fragment
171 file <#creating-config-fragments>`__ to be referenced from within the
172 kernel's ``.bbappend`` file.
173
174 Additionally, if you are working in a BSP layer and need to modify
175 the BSP's kernel's configuration, you can use ``menuconfig``.
176
1775. *Rebuild the Kernel Image With Your Changes:* Rebuilding the kernel
178 image applies your changes. Depending on your target hardware, you
179 can verify your changes on actual hardware or perhaps QEMU.
180
181The remainder of this developer's guide covers common tasks typically
182used during kernel development, advanced Metadata usage, and Yocto Linux
183kernel maintenance concepts.
diff --git a/documentation/kernel-dev/kernel-dev-intro.xml b/documentation/kernel-dev/kernel-dev-intro.xml
index 4e4fd282a5..7c1ea0e510 100644
--- a/documentation/kernel-dev/kernel-dev-intro.xml
+++ b/documentation/kernel-dev/kernel-dev-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='kernel-dev-intro'> 6<chapter id='kernel-dev-intro'>
6<title>Introduction</title> 7<title>Introduction</title>
diff --git a/documentation/kernel-dev/kernel-dev-maint-appx.rst b/documentation/kernel-dev/kernel-dev-maint-appx.rst
new file mode 100644
index 0000000000..5514dac876
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev-maint-appx.rst
@@ -0,0 +1,239 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************
4Kernel Maintenance
5******************
6
7Tree Construction
8=================
9
10This section describes construction of the Yocto Project kernel source
11repositories as accomplished by the Yocto Project team to create Yocto
12Linux kernel repositories. These kernel repositories are found under the
13heading "Yocto Linux Kernel" at :yocto_git:`/` and
14are shipped as part of a Yocto Project release. The team creates these
15repositories by compiling and executing the set of feature descriptions
16for every BSP and feature in the product. Those feature descriptions
17list all necessary patches, configurations, branches, tags, and feature
18divisions found in a Yocto Linux kernel. Thus, the Yocto Project Linux
19kernel repository (or tree) and accompanying Metadata in the
20``yocto-kernel-cache`` are built.
21
22The existence of these repositories allow you to access and clone a
23particular Yocto Project Linux kernel repository and use it to build
24images based on their configurations and features.
25
26You can find the files used to describe all the valid features and BSPs
27in the Yocto Project Linux kernel in any clone of the Yocto Project
28Linux kernel source repository and ``yocto-kernel-cache`` Git trees. For
29example, the following commands clone the Yocto Project baseline Linux
30kernel that branches off ``linux.org`` version 4.12 and the
31``yocto-kernel-cache``, which contains stores of kernel Metadata:
32::
33
34 $ git clone git://git.yoctoproject.org/linux-yocto-4.12
35 $ git clone git://git.yoctoproject.org/linux-kernel-cache
36
37For more information on
38how to set up a local Git repository of the Yocto Project Linux kernel
39files, see the
40":ref:`kernel-dev/kernel-dev-common:preparing the build host to work on the kernel`"
41section.
42
43Once you have cloned the kernel Git repository and the cache of Metadata
44on your local machine, you can discover the branches that are available
45in the repository using the following Git command: $ git branch -a
46Checking out a branch allows you to work with a particular Yocto Linux
47kernel. For example, the following commands check out the
48"standard/beagleboard" branch of the Yocto Linux kernel repository and
49the "yocto-4.12" branch of the ``yocto-kernel-cache`` repository:
50::
51
52 $ cd ~/linux-yocto-4.12
53 $ git checkout -b my-kernel-4.12 remotes/origin/standard/beagleboard
54 $ cd ~/linux-kernel-cache
55 $ git checkout -b my-4.12-metadata remotes/origin/yocto-4.12
56
57.. note::
58
59 Branches in the
60 yocto-kernel-cache
61 repository correspond to Yocto Linux kernel versions (e.g.
62 "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
63
64Once you have checked out and switched to appropriate branches, you can
65see a snapshot of all the kernel source files used to used to build that
66particular Yocto Linux kernel for a particular board.
67
68To see the features and configurations for a particular Yocto Linux
69kernel, you need to examine the ``yocto-kernel-cache`` Git repository.
70As mentioned, branches in the ``yocto-kernel-cache`` repository
71correspond to Yocto Linux kernel versions (e.g. ``yocto-4.12``).
72Branches contain descriptions in the form of ``.scc`` and ``.cfg``
73files.
74
75You should realize, however, that browsing your local
76``yocto-kernel-cache`` repository for feature descriptions and patches
77is not an effective way to determine what is in a particular kernel
78branch. Instead, you should use Git directly to discover the changes in
79a branch. Using Git is an efficient and flexible way to inspect changes
80to the kernel.
81
82.. note::
83
84 Ground up reconstruction of the complete kernel tree is an action
85 only taken by the Yocto Project team during an active development
86 cycle. When you create a clone of the kernel Git repository, you are
87 simply making it efficiently available for building and development.
88
89The following steps describe what happens when the Yocto Project Team
90constructs the Yocto Project kernel source Git repository (or tree)
91found at :yocto_git:`/` given the introduction of a new
92top-level kernel feature or BSP. The following actions effectively
93provide the Metadata and create the tree that includes the new feature,
94patch, or BSP:
95
961. *Pass Feature to the OpenEmbedded Build System:* A top-level kernel
97 feature is passed to the kernel build subsystem. Normally, this
98 feature is a BSP for a particular kernel type.
99
1002. *Locate Feature:* The file that describes the top-level feature is
101 located by searching these system directories:
102
103 - The in-tree kernel-cache directories, which are located in the
104 :yocto_git:`yocto-kernel-cache </cgit/cgit.cgi/yocto-kernel-cache/tree/bsp>`
105 repository organized under the "Yocto Linux Kernel" heading in the
106 :yocto_git:`Yocto Project Source Repositories <>`.
107
108 - Areas pointed to by ``SRC_URI`` statements found in kernel recipes
109
110 For a typical build, the target of the search is a feature
111 description in an ``.scc`` file whose name follows this format (e.g.
112 ``beaglebone-standard.scc`` and ``beaglebone-preempt-rt.scc``):
113 ::
114
115 bsp_root_name-kernel_type.scc
116
1173. *Expand Feature:* Once located, the feature description is either
118 expanded into a simple script of actions, or into an existing
119 equivalent script that is already part of the shipped kernel.
120
1214. *Append Extra Features:* Extra features are appended to the top-level
122 feature description. These features can come from the
123 :term:`KERNEL_FEATURES`
124 variable in recipes.
125
1265. *Locate, Expand, and Append Each Feature:* Each extra feature is
127 located, expanded and appended to the script as described in step
128 three.
129
1306. *Execute the Script:* The script is executed to produce files
131 ``.scc`` and ``.cfg`` files in appropriate directories of the
132 ``yocto-kernel-cache`` repository. These files are descriptions of
133 all the branches, tags, patches and configurations that need to be
134 applied to the base Git repository to completely create the source
135 (build) branch for the new BSP or feature.
136
1377. *Clone Base Repository:* The base repository is cloned, and the
138 actions listed in the ``yocto-kernel-cache`` directories are applied
139 to the tree.
140
1418. *Perform Cleanup:* The Git repositories are left with the desired
142 branches checked out and any required branching, patching and tagging
143 has been performed.
144
145The kernel tree and cache are ready for developer consumption to be
146locally cloned, configured, and built into a Yocto Project kernel
147specific to some target hardware.
148
149.. note::
150
151 - The generated ``yocto-kernel-cache`` repository adds to the kernel
152 as shipped with the Yocto Project release. Any add-ons and
153 configuration data are applied to the end of an existing branch.
154 The full repository generation that is found in the official Yocto
155 Project kernel repositories at :yocto_git:`/` is the
156 combination of all supported boards and configurations.
157
158 - The technique the Yocto Project team uses is flexible and allows
159 for seamless blending of an immutable history with additional
160 patches specific to a deployment. Any additions to the kernel
161 become an integrated part of the branches.
162
163 - The full kernel tree that you see on :yocto_git:`/` is
164 generated through repeating the above steps for all valid BSPs.
165 The end result is a branched, clean history tree that makes up the
166 kernel for a given release. You can see the script (``kgit-scc``)
167 responsible for this in the
168 :yocto_git:`yocto-kernel-tools </cgit.cgi/yocto-kernel-tools/tree/tools>`
169 repository.
170
171 - The steps used to construct the full kernel tree are the same
172 steps that BitBake uses when it builds a kernel image.
173
174Build Strategy
175==============
176
177Once you have cloned a Yocto Linux kernel repository and the cache
178repository (``yocto-kernel-cache``) onto your development system, you
179can consider the compilation phase of kernel development, which is
180building a kernel image. Some prerequisites exist that are validated by
181the build process before compilation starts:
182
183- The :term:`SRC_URI` points to the
184 kernel Git repository.
185
186- A BSP build branch with Metadata exists in the ``yocto-kernel-cache``
187 repository. The branch is based on the Yocto Linux kernel version and
188 has configurations and features grouped under the
189 ``yocto-kernel-cache/bsp`` directory. For example, features and
190 configurations for the BeagleBone Board assuming a
191 ``linux-yocto_4.12`` kernel reside in the following area of the
192 ``yocto-kernel-cache`` repository: yocto-kernel-cache/bsp/beaglebone
193
194 .. note::
195
196 In the previous example, the "yocto-4.12" branch is checked out in
197 the
198 yocto-kernel-cache
199 repository.
200
201The OpenEmbedded build system makes sure these conditions exist before
202attempting compilation. Other means, however, do exist, such as as
203bootstrapping a BSP.
204
205Before building a kernel, the build process verifies the tree and
206configures the kernel by processing all of the configuration "fragments"
207specified by feature descriptions in the ``.scc`` files. As the features
208are compiled, associated kernel configuration fragments are noted and
209recorded in the series of directories in their compilation order. The
210fragments are migrated, pre-processed and passed to the Linux Kernel
211Configuration subsystem (``lkc``) as raw input in the form of a
212``.config`` file. The ``lkc`` uses its own internal dependency
213constraints to do the final processing of that information and generates
214the final ``.config`` file that is used during compilation.
215
216Using the board's architecture and other relevant values from the
217board's template, kernel compilation is started and a kernel image is
218produced.
219
220The other thing that you notice once you configure a kernel is that the
221build process generates a build tree that is separate from your kernel's
222local Git source repository tree. This build tree has a name that uses
223the following form, where ``${MACHINE}`` is the metadata name of the
224machine (BSP) and "kernel_type" is one of the Yocto Project supported
225kernel types (e.g. "standard"):
226::
227
228 linux-${MACHINE}-kernel_type-build
229
230The existing support in the ``kernel.org`` tree achieves this default
231functionality.
232
233This behavior means that all the generated files for a particular
234machine or BSP are now in the build tree directory. The files include
235the final ``.config`` file, all the ``.o`` files, the ``.a`` files, and
236so forth. Since each machine or BSP has its own separate
237:term:`Build Directory` in its own separate
238branch of the Git repository, you can easily switch between different
239builds.
diff --git a/documentation/kernel-dev/kernel-dev-maint-appx.xml b/documentation/kernel-dev/kernel-dev-maint-appx.xml
index b825ae7ea5..3d9c7c66fd 100644
--- a/documentation/kernel-dev/kernel-dev-maint-appx.xml
+++ b/documentation/kernel-dev/kernel-dev-maint-appx.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='kernel-dev-maint-appx'> 6<appendix id='kernel-dev-maint-appx'>
6<title>Kernel Maintenance</title> 7<title>Kernel Maintenance</title>
diff --git a/documentation/kernel-dev/kernel-dev-style.css b/documentation/kernel-dev/kernel-dev-style.css
index 9c01aa7983..fc6fbb8de1 100644
--- a/documentation/kernel-dev/kernel-dev-style.css
+++ b/documentation/kernel-dev/kernel-dev-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/kernel-dev/kernel-dev.rst b/documentation/kernel-dev/kernel-dev.rst
new file mode 100644
index 0000000000..332e089b03
--- /dev/null
+++ b/documentation/kernel-dev/kernel-dev.rst
@@ -0,0 +1,21 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=============================================
4Yocto Project Linux Kernel Development Manual
5=============================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 kernel-dev-intro
14 kernel-dev-common
15 kernel-dev-advanced
16 kernel-dev-concepts-appx
17 kernel-dev-maint-appx
18 kernel-dev-faq
19 history
20
21.. include:: /boilerplate.rst
diff --git a/documentation/kernel-dev/kernel-dev.xml b/documentation/kernel-dev/kernel-dev.xml
index 76256c9c3e..887ff836f1 100755
--- a/documentation/kernel-dev/kernel-dev.xml
+++ b/documentation/kernel-dev/kernel-dev.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='kernel-dev' lang='en' 6<book id='kernel-dev' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -102,28 +103,8 @@
102 </revision> 103 </revision>
103 <revision> 104 <revision>
104 <revnumber>3.1</revnumber> 105 <revnumber>3.1</revnumber>
105 <date>April 2020</date>
106 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
107 </revision>
108 <revision>
109 <revnumber>3.1.1</revnumber>
110 <date>June 2020</date>
111 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
112 </revision>
113 <revision>
114 <revnumber>3.1.2</revnumber>
115 <date>August 2020</date>
116 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
117 </revision>
118 <revision>
119 <revnumber>3.1.3</revnumber>
120 <date>October 2020</date>
121 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
122 </revision>
123 <revision>
124 <revnumber>3.1.4</revnumber>
125 <date>&REL_MONTH_YEAR;</date> 106 <date>&REL_MONTH_YEAR;</date>
126 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 107 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
127 </revision> 108 </revision>
128 </revhistory> 109 </revhistory>
129 110
diff --git a/documentation/mega-manual/mega-manual-customization.xsl b/documentation/mega-manual/mega-manual-customization.xsl
index b52b5b2aa3..33a6e16325 100644
--- a/documentation/mega-manual/mega-manual-customization.xsl
+++ b/documentation/mega-manual/mega-manual-customization.xsl
@@ -1,4 +1,5 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 3<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 4
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 5 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/mega-manual/mega-manual.xml b/documentation/mega-manual/mega-manual.xml
index cea1464890..d9912fa442 100755
--- a/documentation/mega-manual/mega-manual.xml
+++ b/documentation/mega-manual/mega-manual.xml
@@ -1,7 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4 4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5 5
6<book id='mega-manual' lang='en' 6<book id='mega-manual' lang='en'
7 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -93,28 +93,8 @@
93 </revision> 93 </revision>
94 <revision> 94 <revision>
95 <revnumber>3.1</revnumber> 95 <revnumber>3.1</revnumber>
96 <date>April 2020</date>
97 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
98 </revision>
99 <revision>
100 <revnumber>3.1.1</revnumber>
101 <date>June 2020</date>
102 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
103 </revision>
104 <revision>
105 <revnumber>3.1.2</revnumber>
106 <date>August 2020</date>
107 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
108 </revision>
109 <revision>
110 <revnumber>3.1.3</revnumber>
111 <date>October 2020</date>
112 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
113 </revision>
114 <revision>
115 <revnumber>3.1.4</revnumber>
116 <date>&REL_MONTH_YEAR;</date> 96 <date>&REL_MONTH_YEAR;</date>
117 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 97 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
118 </revision> 98 </revision>
119 </revhistory> 99 </revhistory>
120 100
diff --git a/documentation/mega-manual/mega-style.css b/documentation/mega-manual/mega-style.css
index cd71eb6425..7748320fb2 100644
--- a/documentation/mega-manual/mega-style.css
+++ b/documentation/mega-manual/mega-style.css
@@ -1,4 +1,6 @@
1/* 1/*
2 SPDX-License-Identifier: CC-BY-2.0-UK
3
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 4 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 5
4 Browser wrangling and typographic design by 6 Browser wrangling and typographic design by
diff --git a/documentation/overview-manual/history.rst b/documentation/overview-manual/history.rst
new file mode 100644
index 0000000000..0273d28b90
--- /dev/null
+++ b/documentation/overview-manual/history.rst
@@ -0,0 +1,28 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 2.5
15 - May 2018
16 - The initial document released with the Yocto Project 2.5 Release
17 * - 2.6
18 - November 2018
19 - Released with the Yocto Project 2.6 Release.
20 * - 2.7
21 - May 2019
22 - Released with the Yocto Project 2.7 Release.
23 * - 3.0
24 - October 2019
25 - Released with the Yocto Project 3.0 Release.
26 * - 3.1
27 - April 2020
28 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/overview-manual/overview-manual-concepts.rst b/documentation/overview-manual/overview-manual-concepts.rst
new file mode 100644
index 0000000000..6ce5f80af3
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-concepts.rst
@@ -0,0 +1,2185 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************
4Yocto Project Concepts
5**********************
6
7This chapter provides explanations for Yocto Project concepts that go
8beyond the surface of "how-to" information and reference (or look-up)
9material. Concepts such as components, the :term:`OpenEmbedded Build System`
10workflow,
11cross-development toolchains, shared state cache, and so forth are
12explained.
13
14Yocto Project Components
15========================
16
17The :term:`BitBake` task executor
18together with various types of configuration files form the
19:term:`OpenEmbedded-Core (OE-Core)`. This section
20overviews these components by describing their use and how they
21interact.
22
23BitBake handles the parsing and execution of the data files. The data
24itself is of various types:
25
26- *Recipes:* Provides details about particular pieces of software.
27
28- *Class Data:* Abstracts common build information (e.g. how to build a
29 Linux kernel).
30
31- *Configuration Data:* Defines machine-specific settings, policy
32 decisions, and so forth. Configuration data acts as the glue to bind
33 everything together.
34
35BitBake knows how to combine multiple data sources together and refers
36to each data source as a layer. For information on layers, see the
37":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
38section of the Yocto Project Development Tasks Manual.
39
40Following are some brief details on these core components. For
41additional information on how these components interact during a build,
42see the
43":ref:`overview-manual/overview-manual-concepts:openembedded build system concepts`"
44section.
45
46.. _usingpoky-components-bitbake:
47
48BitBake
49-------
50
51BitBake is the tool at the heart of the :term:`OpenEmbedded Build System`
52and is responsible
53for parsing the :term:`Metadata`, generating
54a list of tasks from it, and then executing those tasks.
55
56This section briefly introduces BitBake. If you want more information on
57BitBake, see the :doc:`BitBake User Manual <bitbake:index>`.
58
59To see a list of the options BitBake supports, use either of the
60following commands:
61::
62
63 $ bitbake -h
64 $ bitbake --help
65
66The most common usage for BitBake is ``bitbake recipename``, where
67``recipename`` is the name of the recipe you want to build (referred
68to as the "target"). The target often equates to the first part of a
69recipe's filename (e.g. "foo" for a recipe named ``foo_1.3.0-r0.bb``).
70So, to process the ``matchbox-desktop_1.2.3.bb`` recipe file, you might
71type the following:
72::
73
74 $ bitbake matchbox-desktop
75
76Several different
77versions of ``matchbox-desktop`` might exist. BitBake chooses the one
78selected by the distribution configuration. You can get more details
79about how BitBake chooses between different target versions and
80providers in the
81":ref:`Preferences <bitbake:bb-bitbake-preferences>`" section
82of the BitBake User Manual.
83
84BitBake also tries to execute any dependent tasks first. So for example,
85before building ``matchbox-desktop``, BitBake would build a cross
86compiler and ``glibc`` if they had not already been built.
87
88A useful BitBake option to consider is the ``-k`` or ``--continue``
89option. This option instructs BitBake to try and continue processing the
90job as long as possible even after encountering an error. When an error
91occurs, the target that failed and those that depend on it cannot be
92remade. However, when you use this option other dependencies can still
93be processed.
94
95.. _overview-components-recipes:
96
97Recipes
98-------
99
100Files that have the ``.bb`` suffix are "recipes" files. In general, a
101recipe contains information about a single piece of software. This
102information includes the location from which to download the unaltered
103source, any source patches to be applied to that source (if needed),
104which special configuration options to apply, how to compile the source
105files, and how to package the compiled output.
106
107The term "package" is sometimes used to refer to recipes. However, since
108the word "package" is used for the packaged output from the OpenEmbedded
109build system (i.e. ``.ipk`` or ``.deb`` files), this document avoids
110using the term "package" when referring to recipes.
111
112.. _overview-components-classes:
113
114Classes
115-------
116
117Class files (``.bbclass``) contain information that is useful to share
118between recipes files. An example is the
119:ref:`autotools <ref-classes-autotools>` class,
120which contains common settings for any application that Autotools uses.
121The ":ref:`ref-manual/ref-classes:Classes`" chapter in the
122Yocto Project Reference Manual provides details about classes and how to
123use them.
124
125.. _overview-components-configurations:
126
127Configurations
128--------------
129
130The configuration files (``.conf``) define various configuration
131variables that govern the OpenEmbedded build process. These files fall
132into several areas that define machine configuration options,
133distribution configuration options, compiler tuning options, general
134common configuration options, and user configuration options in
135``conf/local.conf``, which is found in the :term:`Build Directory`.
136
137
138.. _overview-layers:
139
140Layers
141======
142
143Layers are repositories that contain related metadata (i.e. sets of
144instructions) that tell the OpenEmbedded build system how to build a
145target. Yocto Project's `layer model <#the-yocto-project-layer-model>`__
146facilitates collaboration, sharing, customization, and reuse within the
147Yocto Project development environment. Layers logically separate
148information for your project. For example, you can use a layer to hold
149all the configurations for a particular piece of hardware. Isolating
150hardware-specific configurations allows you to share other metadata by
151using a different layer where that metadata might be common across
152several pieces of hardware.
153
154Many layers exist that work in the Yocto Project development
155environment. The `Yocto Project Curated Layer
156Index <https://www.yoctoproject.org/software-overview/layers/>`__
157and `OpenEmbedded Layer
158Index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__
159both contain layers from which you can use or leverage.
160
161By convention, layers in the Yocto Project follow a specific form.
162Conforming to a known structure allows BitBake to make assumptions
163during builds on where to find types of metadata. You can find
164procedures and learn about tools (i.e. ``bitbake-layers``) for creating
165layers suitable for the Yocto Project in the
166":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
167section of the Yocto Project Development Tasks Manual.
168
169.. _openembedded-build-system-build-concepts:
170
171OpenEmbedded Build System Concepts
172==================================
173
174This section takes a more detailed look inside the build process used by
175the :term:`OpenEmbedded Build System`,
176which is the build
177system specific to the Yocto Project. At the heart of the build system
178is BitBake, the task executor.
179
180The following diagram represents the high-level workflow of a build. The
181remainder of this section expands on the fundamental input, output,
182process, and metadata logical blocks that make up the workflow.
183
184.. image:: figures/YP-flow-diagram.png
185 :align: center
186
187In general, the build's workflow consists of several functional areas:
188
189- *User Configuration:* metadata you can use to control the build
190 process.
191
192- *Metadata Layers:* Various layers that provide software, machine, and
193 distro metadata.
194
195- *Source Files:* Upstream releases, local projects, and SCMs.
196
197- *Build System:* Processes under the control of
198 :term:`BitBake`. This block expands
199 on how BitBake fetches source, applies patches, completes
200 compilation, analyzes output for package generation, creates and
201 tests packages, generates images, and generates cross-development
202 tools.
203
204- *Package Feeds:* Directories containing output packages (RPM, DEB or
205 IPK), which are subsequently used in the construction of an image or
206 Software Development Kit (SDK), produced by the build system. These
207 feeds can also be copied and shared using a web server or other means
208 to facilitate extending or updating existing images on devices at
209 runtime if runtime package management is enabled.
210
211- *Images:* Images produced by the workflow.
212
213- *Application Development SDK:* Cross-development tools that are
214 produced along with an image or separately with BitBake.
215
216User Configuration
217------------------
218
219User configuration helps define the build. Through user configuration,
220you can tell BitBake the target architecture for which you are building
221the image, where to store downloaded source, and other build properties.
222
223The following figure shows an expanded representation of the "User
224Configuration" box of the `general workflow
225figure <#general-workflow-figure>`__:
226
227.. image:: figures/user-configuration.png
228 :align: center
229
230BitBake needs some basic configuration files in order to complete a
231build. These files are ``*.conf`` files. The minimally necessary ones
232reside as example files in the ``build/conf`` directory of the
233:term:`Source Directory`. For simplicity,
234this section refers to the Source Directory as the "Poky Directory."
235
236When you clone the :term:`Poky` Git repository
237or you download and unpack a Yocto Project release, you can set up the
238Source Directory to be named anything you want. For this discussion, the
239cloned repository uses the default name ``poky``.
240
241.. note::
242
243 The Poky repository is primarily an aggregation of existing
244 repositories. It is not a canonical upstream source.
245
246The ``meta-poky`` layer inside Poky contains a ``conf`` directory that
247has example configuration files. These example files are used as a basis
248for creating actual configuration files when you source
249:ref:`structure-core-script`, which is the
250build environment script.
251
252Sourcing the build environment script creates a
253:term:`Build Directory` if one does not
254already exist. BitBake uses the Build Directory for all its work during
255builds. The Build Directory has a ``conf`` directory that contains
256default versions of your ``local.conf`` and ``bblayers.conf``
257configuration files. These default configuration files are created only
258if versions do not already exist in the Build Directory at the time you
259source the build environment setup script.
260
261Because the Poky repository is fundamentally an aggregation of existing
262repositories, some users might be familiar with running the
263:ref:`structure-core-script` script in the context of separate
264:term:`OpenEmbedded-Core (OE-Core)` and BitBake
265repositories rather than a single Poky repository. This discussion
266assumes the script is executed from within a cloned or unpacked version
267of Poky.
268
269Depending on where the script is sourced, different sub-scripts are
270called to set up the Build Directory (Yocto or OpenEmbedded).
271Specifically, the script ``scripts/oe-setup-builddir`` inside the poky
272directory sets up the Build Directory and seeds the directory (if
273necessary) with configuration files appropriate for the Yocto Project
274development environment.
275
276.. note::
277
278 The
279 scripts/oe-setup-builddir
280 script uses the
281 ``$TEMPLATECONF``
282 variable to determine which sample configuration files to locate.
283
284The ``local.conf`` file provides many basic variables that define a
285build environment. Here is a list of a few. To see the default
286configurations in a ``local.conf`` file created by the build environment
287script, see the
288:yocto_git:`local.conf.sample </cgit/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample>`
289in the ``meta-poky`` layer:
290
291- *Target Machine Selection:* Controlled by the
292 :term:`MACHINE` variable.
293
294- *Download Directory:* Controlled by the
295 :term:`DL_DIR` variable.
296
297- *Shared State Directory:* Controlled by the
298 :term:`SSTATE_DIR` variable.
299
300- *Build Output:* Controlled by the
301 :term:`TMPDIR` variable.
302
303- *Distribution Policy:* Controlled by the
304 :term:`DISTRO` variable.
305
306- *Packaging Format:* Controlled by the
307 :term:`PACKAGE_CLASSES`
308 variable.
309
310- *SDK Target Architecture:* Controlled by the
311 :term:`SDKMACHINE` variable.
312
313- *Extra Image Packages:* Controlled by the
314 :term:`EXTRA_IMAGE_FEATURES`
315 variable.
316
317.. note::
318
319 Configurations set in the
320 conf/local.conf
321 file can also be set in the
322 conf/site.conf
323 and
324 conf/auto.conf
325 configuration files.
326
327The ``bblayers.conf`` file tells BitBake what layers you want considered
328during the build. By default, the layers listed in this file include
329layers minimally needed by the build system. However, you must manually
330add any custom layers you have created. You can find more information on
331working with the ``bblayers.conf`` file in the
332":ref:`dev-manual/dev-manual-common-tasks:enabling your layer`"
333section in the Yocto Project Development Tasks Manual.
334
335The files ``site.conf`` and ``auto.conf`` are not created by the
336environment initialization script. If you want the ``site.conf`` file,
337you need to create that yourself. The ``auto.conf`` file is typically
338created by an autobuilder:
339
340- *site.conf:* You can use the ``conf/site.conf`` configuration
341 file to configure multiple build directories. For example, suppose
342 you had several build environments and they shared some common
343 features. You can set these default build properties here. A good
344 example is perhaps the packaging format to use through the
345 :term:`PACKAGE_CLASSES`
346 variable.
347
348 One useful scenario for using the ``conf/site.conf`` file is to
349 extend your :term:`BBPATH` variable
350 to include the path to a ``conf/site.conf``. Then, when BitBake looks
351 for Metadata using ``BBPATH``, it finds the ``conf/site.conf`` file
352 and applies your common configurations found in the file. To override
353 configurations in a particular build directory, alter the similar
354 configurations within that build directory's ``conf/local.conf``
355 file.
356
357- *auto.conf:* The file is usually created and written to by an
358 autobuilder. The settings put into the file are typically the same as
359 you would find in the ``conf/local.conf`` or the ``conf/site.conf``
360 files.
361
362You can edit all configuration files to further define any particular
363build environment. This process is represented by the "User
364Configuration Edits" box in the figure.
365
366When you launch your build with the ``bitbake target`` command, BitBake
367sorts out the configurations to ultimately define your build
368environment. It is important to understand that the
369:term:`OpenEmbedded Build System` reads the
370configuration files in a specific order: ``site.conf``, ``auto.conf``,
371and ``local.conf``. And, the build system applies the normal assignment
372statement rules as described in the
373":doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata`" chapter
374of the BitBake User Manual. Because the files are parsed in a specific
375order, variable assignments for the same variable could be affected. For
376example, if the ``auto.conf`` file and the ``local.conf`` set variable1
377to different values, because the build system parses ``local.conf``
378after ``auto.conf``, variable1 is assigned the value from the
379``local.conf`` file.
380
381Metadata, Machine Configuration, and Policy Configuration
382---------------------------------------------------------
383
384The previous section described the user configurations that define
385BitBake's global behavior. This section takes a closer look at the
386layers the build system uses to further control the build. These layers
387provide Metadata for the software, machine, and policies.
388
389In general, three types of layer input exists. You can see them below
390the "User Configuration" box in the `general workflow
391figure <#general-workflow-figure>`__:
392
393- *Metadata (.bb + Patches):* Software layers containing
394 user-supplied recipe files, patches, and append files. A good example
395 of a software layer might be the
396 `meta-qt5 layer <https://github.com/meta-qt5/meta-qt5>`__ from
397 the `OpenEmbedded Layer
398 Index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
399 This layer is for version 5.0 of the popular
400 `Qt <https://wiki.qt.io/About_Qt>`__ cross-platform application
401 development framework for desktop, embedded and mobile.
402
403- *Machine BSP Configuration:* Board Support Package (BSP) layers (i.e.
404 "BSP Layer" in the following figure) providing machine-specific
405 configurations. This type of information is specific to a particular
406 target architecture. A good example of a BSP layer from the `Poky
407 Reference Distribution <#gs-reference-distribution-poky>`__ is the
408 :yocto_git:`meta-yocto-bsp </cgit/cgit.cgi/poky/tree/meta-yocto-bsp>`
409 layer.
410
411- *Policy Configuration:* Distribution Layers (i.e. "Distro Layer" in
412 the following figure) providing top-level or general policies for the
413 images or SDKs being built for a particular distribution. For
414 example, in the Poky Reference Distribution the distro layer is the
415 :yocto_git:`meta-poky </cgit/cgit.cgi/poky/tree/meta-poky>`
416 layer. Within the distro layer is a ``conf/distro`` directory that
417 contains distro configuration files (e.g.
418 :yocto_git:`poky.conf </cgit/cgit.cgi/poky/tree/meta-poky/conf/distro/poky.conf>`
419 that contain many policy configurations for the Poky distribution.
420
421The following figure shows an expanded representation of these three
422layers from the `general workflow figure <#general-workflow-figure>`__:
423
424.. image:: figures/layer-input.png
425 :align: center
426
427In general, all layers have a similar structure. They all contain a
428licensing file (e.g. ``COPYING.MIT``) if the layer is to be distributed,
429a ``README`` file as good practice and especially if the layer is to be
430distributed, a configuration directory, and recipe directories. You can
431learn about the general structure for layers used with the Yocto Project
432in the
433":ref:`dev-manual/dev-manual-common-tasks:creating your own layer`"
434section in the
435Yocto Project Development Tasks Manual. For a general discussion on
436layers and the many layers from which you can draw, see the
437"`Layers <#overview-layers>`__" and "`The Yocto Project Layer
438Model <#the-yocto-project-layer-model>`__" sections both earlier in this
439manual.
440
441If you explored the previous links, you discovered some areas where many
442layers that work with the Yocto Project exist. The `Source
443Repositories <http://git.yoctoproject.org/>`__ also shows layers
444categorized under "Yocto Metadata Layers."
445
446.. note::
447
448 Layers exist in the Yocto Project Source Repositories that cannot be
449 found in the OpenEmbedded Layer Index. These layers are either
450 deprecated or experimental in nature.
451
452BitBake uses the ``conf/bblayers.conf`` file, which is part of the user
453configuration, to find what layers it should be using as part of the
454build.
455
456Distro Layer
457~~~~~~~~~~~~
458
459The distribution layer provides policy configurations for your
460distribution. Best practices dictate that you isolate these types of
461configurations into their own layer. Settings you provide in
462``conf/distro/distro.conf`` override similar settings that BitBake finds
463in your ``conf/local.conf`` file in the Build Directory.
464
465The following list provides some explanation and references for what you
466typically find in the distribution layer:
467
468- *classes:* Class files (``.bbclass``) hold common functionality that
469 can be shared among recipes in the distribution. When your recipes
470 inherit a class, they take on the settings and functions for that
471 class. You can read more about class files in the
472 ":ref:`ref-manual/ref-classes:Classes`" chapter of the Yocto
473 Reference Manual.
474
475- *conf:* This area holds configuration files for the layer
476 (``conf/layer.conf``), the distribution
477 (``conf/distro/distro.conf``), and any distribution-wide include
478 files.
479
480- *recipes-*:* Recipes and append files that affect common
481 functionality across the distribution. This area could include
482 recipes and append files to add distribution-specific configuration,
483 initialization scripts, custom image recipes, and so forth. Examples
484 of ``recipes-*`` directories are ``recipes-core`` and
485 ``recipes-extra``. Hierarchy and contents within a ``recipes-*``
486 directory can vary. Generally, these directories contain recipe files
487 (``*.bb``), recipe append files (``*.bbappend``), directories that
488 are distro-specific for configuration files, and so forth.
489
490BSP Layer
491~~~~~~~~~
492
493The BSP Layer provides machine configurations that target specific
494hardware. Everything in this layer is specific to the machine for which
495you are building the image or the SDK. A common structure or form is
496defined for BSP layers. You can learn more about this structure in the
497:doc:`../bsp-guide/bsp-guide`.
498
499.. note::
500
501 In order for a BSP layer to be considered compliant with the Yocto
502 Project, it must meet some structural requirements.
503
504The BSP Layer's configuration directory contains configuration files for
505the machine (``conf/machine/machine.conf``) and, of course, the layer
506(``conf/layer.conf``).
507
508The remainder of the layer is dedicated to specific recipes by function:
509``recipes-bsp``, ``recipes-core``, ``recipes-graphics``,
510``recipes-kernel``, and so forth. Metadata can exist for multiple
511formfactors, graphics support systems, and so forth.
512
513.. note::
514
515 While the figure shows several
516 recipes-\*
517 directories, not all these directories appear in all BSP layers.
518
519Software Layer
520~~~~~~~~~~~~~~
521
522The software layer provides the Metadata for additional software
523packages used during the build. This layer does not include Metadata
524that is specific to the distribution or the machine, which are found in
525their respective layers.
526
527This layer contains any recipes, append files, and patches, that your
528project needs.
529
530.. _sources-dev-environment:
531
532Sources
533-------
534
535In order for the OpenEmbedded build system to create an image or any
536target, it must be able to access source files. The `general workflow
537figure <#general-workflow-figure>`__ represents source files using the
538"Upstream Project Releases", "Local Projects", and "SCMs (optional)"
539boxes. The figure represents mirrors, which also play a role in locating
540source files, with the "Source Materials" box.
541
542The method by which source files are ultimately organized is a function
543of the project. For example, for released software, projects tend to use
544tarballs or other archived files that can capture the state of a release
545guaranteeing that it is statically represented. On the other hand, for a
546project that is more dynamic or experimental in nature, a project might
547keep source files in a repository controlled by a Source Control Manager
548(SCM) such as Git. Pulling source from a repository allows you to
549control the point in the repository (the revision) from which you want
550to build software. Finally, a combination of the two might exist, which
551would give the consumer a choice when deciding where to get source
552files.
553
554BitBake uses the :term:`SRC_URI`
555variable to point to source files regardless of their location. Each
556recipe must have a ``SRC_URI`` variable that points to the source.
557
558Another area that plays a significant role in where source files come
559from is pointed to by the
560:term:`DL_DIR` variable. This area is
561a cache that can hold previously downloaded source. You can also
562instruct the OpenEmbedded build system to create tarballs from Git
563repositories, which is not the default behavior, and store them in the
564``DL_DIR`` by using the
565:term:`BB_GENERATE_MIRROR_TARBALLS`
566variable.
567
568Judicious use of a ``DL_DIR`` directory can save the build system a trip
569across the Internet when looking for files. A good method for using a
570download directory is to have ``DL_DIR`` point to an area outside of
571your Build Directory. Doing so allows you to safely delete the Build
572Directory if needed without fear of removing any downloaded source file.
573
574The remainder of this section provides a deeper look into the source
575files and the mirrors. Here is a more detailed look at the source file
576area of the `general workflow figure <#general-workflow-figure>`__:
577
578.. image:: figures/source-input.png
579 :align: center
580
581Upstream Project Releases
582~~~~~~~~~~~~~~~~~~~~~~~~~
583
584Upstream project releases exist anywhere in the form of an archived file
585(e.g. tarball or zip file). These files correspond to individual
586recipes. For example, the figure uses specific releases each for
587BusyBox, Qt, and Dbus. An archive file can be for any released product
588that can be built using a recipe.
589
590Local Projects
591~~~~~~~~~~~~~~
592
593Local projects are custom bits of software the user provides. These bits
594reside somewhere local to a project - perhaps a directory into which the
595user checks in items (e.g. a local directory containing a development
596source tree used by the group).
597
598The canonical method through which to include a local project is to use
599the :ref:`externalsrc <ref-classes-externalsrc>`
600class to include that local project. You use either the ``local.conf``
601or a recipe's append file to override or set the recipe to point to the
602local directory on your disk to pull in the whole source tree.
603
604.. _scms:
605
606Source Control Managers (Optional)
607~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
608
609Another place from which the build system can get source files is with
610:ref:`fetchers <bitbake:bb-fetchers>` employing various Source
611Control Managers (SCMs) such as Git or Subversion. In such cases, a
612repository is cloned or checked out. The
613:ref:`ref-tasks-fetch` task inside
614BitBake uses the :term:`SRC_URI`
615variable and the argument's prefix to determine the correct fetcher
616module.
617
618.. note::
619
620 For information on how to have the OpenEmbedded build system generate
621 tarballs for Git repositories and place them in the
622 DL_DIR
623 directory, see the :term:`BB_GENERATE_MIRROR_TARBALLS`
624 variable in the Yocto Project Reference Manual.
625
626When fetching a repository, BitBake uses the
627:term:`SRCREV` variable to determine
628the specific revision from which to build.
629
630Source Mirror(s)
631~~~~~~~~~~~~~~~~
632
633Two kinds of mirrors exist: pre-mirrors and regular mirrors. The
634:term:`PREMIRRORS` and
635:term:`MIRRORS` variables point to
636these, respectively. BitBake checks pre-mirrors before looking upstream
637for any source files. Pre-mirrors are appropriate when you have a shared
638directory that is not a directory defined by the
639:term:`DL_DIR` variable. A Pre-mirror
640typically points to a shared directory that is local to your
641organization.
642
643Regular mirrors can be any site across the Internet that is used as an
644alternative location for source code should the primary site not be
645functioning for some reason or another.
646
647.. _package-feeds-dev-environment:
648
649Package Feeds
650-------------
651
652When the OpenEmbedded build system generates an image or an SDK, it gets
653the packages from a package feed area located in the
654:term:`Build Directory`. The `general
655workflow figure <#general-workflow-figure>`__ shows this package feeds
656area in the upper-right corner.
657
658This section looks a little closer into the package feeds area used by
659the build system. Here is a more detailed look at the area:
660
661.. image:: figures/package-feeds.png
662 :align: center
663
664Package feeds are an intermediary step in the build process. The
665OpenEmbedded build system provides classes to generate different package
666types, and you specify which classes to enable through the
667:term:`PACKAGE_CLASSES`
668variable. Before placing the packages into package feeds, the build
669process validates them with generated output quality assurance checks
670through the :ref:`insane <ref-classes-insane>`
671class.
672
673The package feed area resides in the Build Directory. The directory the
674build system uses to temporarily store packages is determined by a
675combination of variables and the particular package manager in use. See
676the "Package Feeds" box in the illustration and note the information to
677the right of that area. In particular, the following defines where
678package files are kept:
679
680- :term:`DEPLOY_DIR`: Defined as
681 ``tmp/deploy`` in the Build Directory.
682
683- ``DEPLOY_DIR_*``: Depending on the package manager used, the package
684 type sub-folder. Given RPM, IPK, or DEB packaging and tarball
685 creation, the
686 :term:`DEPLOY_DIR_RPM`,
687 :term:`DEPLOY_DIR_IPK`,
688 :term:`DEPLOY_DIR_DEB`, or
689 :term:`DEPLOY_DIR_TAR`,
690 variables are used, respectively.
691
692- :term:`PACKAGE_ARCH`: Defines
693 architecture-specific sub-folders. For example, packages could exist
694 for the i586 or qemux86 architectures.
695
696BitBake uses the
697:ref:`do_package_write_* <ref-tasks-package_write_deb>`
698tasks to generate packages and place them into the package holding area
699(e.g. ``do_package_write_ipk`` for IPK packages). See the
700":ref:`ref-tasks-package_write_deb`",
701":ref:`ref-tasks-package_write_ipk`",
702":ref:`ref-tasks-package_write_rpm`",
703and
704":ref:`ref-tasks-package_write_tar`"
705sections in the Yocto Project Reference Manual for additional
706information. As an example, consider a scenario where an IPK packaging
707manager is being used and package architecture support for both i586 and
708qemux86 exist. Packages for the i586 architecture are placed in
709``build/tmp/deploy/ipk/i586``, while packages for the qemux86
710architecture are placed in ``build/tmp/deploy/ipk/qemux86``.
711
712.. _bitbake-dev-environment:
713
714BitBake Tool
715------------
716
717The OpenEmbedded build system uses
718:term:`BitBake` to produce images and
719Software Development Kits (SDKs). You can see from the `general workflow
720figure <#general-workflow-figure>`__, the BitBake area consists of
721several functional areas. This section takes a closer look at each of
722those areas.
723
724.. note::
725
726 Separate documentation exists for the BitBake tool. See the
727 BitBake User Manual
728 for reference material on BitBake.
729
730.. _source-fetching-dev-environment:
731
732Source Fetching
733~~~~~~~~~~~~~~~
734
735The first stages of building a recipe are to fetch and unpack the source
736code:
737
738.. image:: figures/source-fetching.png
739 :align: center
740
741The :ref:`ref-tasks-fetch` and
742:ref:`ref-tasks-unpack` tasks fetch
743the source files and unpack them into the
744:term:`Build Directory`.
745
746.. note::
747
748 For every local file (e.g.
749 file://
750 ) that is part of a recipe's
751 SRC_URI
752 statement, the OpenEmbedded build system takes a checksum of the file
753 for the recipe and inserts the checksum into the signature for the
754 do_fetch
755 task. If any local file has been modified, the
756 do_fetch
757 task and all tasks that depend on it are re-executed.
758
759By default, everything is accomplished in the Build Directory, which has
760a defined structure. For additional general information on the Build
761Directory, see the ":ref:`structure-core-build`" section in
762the Yocto Project Reference Manual.
763
764Each recipe has an area in the Build Directory where the unpacked source
765code resides. The :term:`S` variable points
766to this area for a recipe's unpacked source code. The name of that
767directory for any given recipe is defined from several different
768variables. The preceding figure and the following list describe the
769Build Directory's hierarchy:
770
771- :term:`TMPDIR`: The base directory
772 where the OpenEmbedded build system performs all its work during the
773 build. The default base directory is the ``tmp`` directory.
774
775- :term:`PACKAGE_ARCH`: The
776 architecture of the built package or packages. Depending on the
777 eventual destination of the package or packages (i.e. machine
778 architecture, :term:`Build Host`, SDK, or
779 specific machine), ``PACKAGE_ARCH`` varies. See the variable's
780 description for details.
781
782- :term:`TARGET_OS`: The operating
783 system of the target device. A typical value would be "linux" (e.g.
784 "qemux86-poky-linux").
785
786- :term:`PN`: The name of the recipe used
787 to build the package. This variable can have multiple meanings.
788 However, when used in the context of input files, ``PN`` represents
789 the name of the recipe.
790
791- :term:`WORKDIR`: The location
792 where the OpenEmbedded build system builds a recipe (i.e. does the
793 work to create the package).
794
795 - :term:`PV`: The version of the
796 recipe used to build the package.
797
798 - :term:`PR`: The revision of the
799 recipe used to build the package.
800
801- :term:`S`: Contains the unpacked source
802 files for a given recipe.
803
804 - :term:`BPN`: The name of the recipe
805 used to build the package. The ``BPN`` variable is a version of
806 the ``PN`` variable but with common prefixes and suffixes removed.
807
808 - :term:`PV`: The version of the
809 recipe used to build the package.
810
811.. note::
812
813 In the previous figure, notice that two sample hierarchies exist: one
814 based on package architecture (i.e.
815 PACKAGE_ARCH
816 ) and one based on a machine (i.e.
817 MACHINE
818 ). The underlying structures are identical. The differentiator being
819 what the OpenEmbedded build system is using as a build target (e.g.
820 general architecture, a build host, an SDK, or a specific machine).
821
822.. _patching-dev-environment:
823
824Patching
825~~~~~~~~
826
827Once source code is fetched and unpacked, BitBake locates patch files
828and applies them to the source files:
829
830.. image:: figures/patching.png
831 :align: center
832
833The :ref:`ref-tasks-patch` task uses a
834recipe's :term:`SRC_URI` statements
835and the :term:`FILESPATH` variable
836to locate applicable patch files.
837
838Default processing for patch files assumes the files have either
839``*.patch`` or ``*.diff`` file types. You can use ``SRC_URI`` parameters
840to change the way the build system recognizes patch files. See the
841:ref:`ref-tasks-patch` task for more
842information.
843
844BitBake finds and applies multiple patches for a single recipe in the
845order in which it locates the patches. The ``FILESPATH`` variable
846defines the default set of directories that the build system uses to
847search for patch files. Once found, patches are applied to the recipe's
848source files, which are located in the
849:term:`S` directory.
850
851For more information on how the source directories are created, see the
852"`Source Fetching <#source-fetching-dev-environment>`__" section. For
853more information on how to create patches and how the build system
854processes patches, see the
855":ref:`dev-manual/dev-manual-common-tasks:patching code`"
856section in the
857Yocto Project Development Tasks Manual. You can also see the
858":ref:`sdk-manual/sdk-extensible:use \`\`devtool modify\`\` to modify the source of an existing component`"
859section in the Yocto Project Application Development and the Extensible
860Software Development Kit (SDK) manual and the
861":ref:`kernel-dev/kernel-dev-common:using traditional kernel development to patch the kernel`"
862section in the Yocto Project Linux Kernel Development Manual.
863
864.. _configuration-compilation-and-staging-dev-environment:
865
866Configuration, Compilation, and Staging
867~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
868
869After source code is patched, BitBake executes tasks that configure and
870compile the source code. Once compilation occurs, the files are copied
871to a holding area (staged) in preparation for packaging:
872
873.. image:: figures/configuration-compile-autoreconf.png
874 :align: center
875
876This step in the build process consists of the following tasks:
877
878- :ref:`ref-tasks-prepare_recipe_sysroot`:
879 This task sets up the two sysroots in
880 ``${``\ :term:`WORKDIR`\ ``}``
881 (i.e. ``recipe-sysroot`` and ``recipe-sysroot-native``) so that
882 during the packaging phase the sysroots can contain the contents of
883 the
884 :ref:`ref-tasks-populate_sysroot`
885 tasks of the recipes on which the recipe containing the tasks
886 depends. A sysroot exists for both the target and for the native
887 binaries, which run on the host system.
888
889- *do_configure*: This task configures the source by enabling and
890 disabling any build-time and configuration options for the software
891 being built. Configurations can come from the recipe itself as well
892 as from an inherited class. Additionally, the software itself might
893 configure itself depending on the target for which it is being built.
894
895 The configurations handled by the
896 :ref:`ref-tasks-configure` task
897 are specific to configurations for the source code being built by the
898 recipe.
899
900 If you are using the
901 :ref:`autotools <ref-classes-autotools>` class,
902 you can add additional configuration options by using the
903 :term:`EXTRA_OECONF` or
904 :term:`PACKAGECONFIG_CONFARGS`
905 variables. For information on how this variable works within that
906 class, see the
907 :ref:`autotools <ref-classes-autotools>` class
908 :yocto_git:`here </cgit/cgit.cgi/poky/tree/meta/classes/autotools.bbclass>`.
909
910- *do_compile*: Once a configuration task has been satisfied,
911 BitBake compiles the source using the
912 :ref:`ref-tasks-compile` task.
913 Compilation occurs in the directory pointed to by the
914 :term:`B` variable. Realize that the
915 ``B`` directory is, by default, the same as the
916 :term:`S` directory.
917
918- *do_install*: After compilation completes, BitBake executes the
919 :ref:`ref-tasks-install` task.
920 This task copies files from the ``B`` directory and places them in a
921 holding area pointed to by the :term:`D`
922 variable. Packaging occurs later using files from this holding
923 directory.
924
925.. _package-splitting-dev-environment:
926
927Package Splitting
928~~~~~~~~~~~~~~~~~
929
930After source code is configured, compiled, and staged, the build system
931analyzes the results and splits the output into packages:
932
933.. image:: figures/analysis-for-package-splitting.png
934 :align: center
935
936The :ref:`ref-tasks-package` and
937:ref:`ref-tasks-packagedata`
938tasks combine to analyze the files found in the
939:term:`D` directory and split them into
940subsets based on available packages and files. Analysis involves the
941following as well as other items: splitting out debugging symbols,
942looking at shared library dependencies between packages, and looking at
943package relationships.
944
945The ``do_packagedata`` task creates package metadata based on the
946analysis such that the build system can generate the final packages. The
947:ref:`ref-tasks-populate_sysroot`
948task stages (copies) a subset of the files installed by the
949:ref:`ref-tasks-install` task into
950the appropriate sysroot. Working, staged, and intermediate results of
951the analysis and package splitting process use several areas:
952
953- :term:`PKGD`: The destination
954 directory (i.e. ``package``) for packages before they are split into
955 individual packages.
956
957- :term:`PKGDESTWORK`: A
958 temporary work area (i.e. ``pkgdata``) used by the ``do_package``
959 task to save package metadata.
960
961- :term:`PKGDEST`: The parent
962 directory (i.e. ``packages-split``) for packages after they have been
963 split.
964
965- :term:`PKGDATA_DIR`: A shared,
966 global-state directory that holds packaging metadata generated during
967 the packaging process. The packaging process copies metadata from
968 ``PKGDESTWORK`` to the ``PKGDATA_DIR`` area where it becomes globally
969 available.
970
971- :term:`STAGING_DIR_HOST`:
972 The path for the sysroot for the system on which a component is built
973 to run (i.e. ``recipe-sysroot``).
974
975- :term:`STAGING_DIR_NATIVE`:
976 The path for the sysroot used when building components for the build
977 host (i.e. ``recipe-sysroot-native``).
978
979- :term:`STAGING_DIR_TARGET`:
980 The path for the sysroot used when a component that is built to
981 execute on a system and it generates code for yet another machine
982 (e.g. cross-canadian recipes).
983
984The :term:`FILES` variable defines the
985files that go into each package in
986:term:`PACKAGES`. If you want
987details on how this is accomplished, you can look at
988:yocto_git:`package.bbclass </cgit/cgit.cgi/poky/tree/meta/classes/package.bbclass>`.
989
990Depending on the type of packages being created (RPM, DEB, or IPK), the
991:ref:`do_package_write_* <ref-tasks-package_write_deb>`
992task creates the actual packages and places them in the Package Feed
993area, which is ``${TMPDIR}/deploy``. You can see the "`Package
994Feeds <#package-feeds-dev-environment>`__" section for more detail on
995that part of the build process.
996
997.. note::
998
999 Support for creating feeds directly from the
1000 deploy/\*
1001 directories does not exist. Creating such feeds usually requires some
1002 kind of feed maintenance mechanism that would upload the new packages
1003 into an official package feed (e.g. the Ångström distribution). This
1004 functionality is highly distribution-specific and thus is not
1005 provided out of the box.
1006
1007.. _image-generation-dev-environment:
1008
1009Image Generation
1010~~~~~~~~~~~~~~~~
1011
1012Once packages are split and stored in the Package Feeds area, the build
1013system uses BitBake to generate the root filesystem image:
1014
1015.. image:: figures/image-generation.png
1016 :align: center
1017
1018The image generation process consists of several stages and depends on
1019several tasks and variables. The
1020:ref:`ref-tasks-rootfs` task creates
1021the root filesystem (file and directory structure) for an image. This
1022task uses several key variables to help create the list of packages to
1023actually install:
1024
1025- :term:`IMAGE_INSTALL`: Lists
1026 out the base set of packages from which to install from the Package
1027 Feeds area.
1028
1029- :term:`PACKAGE_EXCLUDE`:
1030 Specifies packages that should not be installed into the image.
1031
1032- :term:`IMAGE_FEATURES`:
1033 Specifies features to include in the image. Most of these features
1034 map to additional packages for installation.
1035
1036- :term:`PACKAGE_CLASSES`:
1037 Specifies the package backend (e.g. RPM, DEB, or IPK) to use and
1038 consequently helps determine where to locate packages within the
1039 Package Feeds area.
1040
1041- :term:`IMAGE_LINGUAS`:
1042 Determines the language(s) for which additional language support
1043 packages are installed.
1044
1045- :term:`PACKAGE_INSTALL`:
1046 The final list of packages passed to the package manager for
1047 installation into the image.
1048
1049With :term:`IMAGE_ROOTFS`
1050pointing to the location of the filesystem under construction and the
1051``PACKAGE_INSTALL`` variable providing the final list of packages to
1052install, the root file system is created.
1053
1054Package installation is under control of the package manager (e.g.
1055dnf/rpm, opkg, or apt/dpkg) regardless of whether or not package
1056management is enabled for the target. At the end of the process, if
1057package management is not enabled for the target, the package manager's
1058data files are deleted from the root filesystem. As part of the final
1059stage of package installation, post installation scripts that are part
1060of the packages are run. Any scripts that fail to run on the build host
1061are run on the target when the target system is first booted. If you are
1062using a
1063:ref:`read-only root filesystem <dev-manual/dev-manual-common-tasks:creating a read-only root filesystem>`,
1064all the post installation scripts must succeed on the build host during
1065the package installation phase since the root filesystem on the target
1066is read-only.
1067
1068The final stages of the ``do_rootfs`` task handle post processing. Post
1069processing includes creation of a manifest file and optimizations.
1070
1071The manifest file (``.manifest``) resides in the same directory as the
1072root filesystem image. This file lists out, line-by-line, the installed
1073packages. The manifest file is useful for the
1074:ref:`testimage <ref-classes-testimage*>` class,
1075for example, to determine whether or not to run specific tests. See the
1076:term:`IMAGE_MANIFEST`
1077variable for additional information.
1078
1079Optimizing processes that are run across the image include ``mklibs``,
1080``prelink``, and any other post-processing commands as defined by the
1081:term:`ROOTFS_POSTPROCESS_COMMAND`
1082variable. The ``mklibs`` process optimizes the size of the libraries,
1083while the ``prelink`` process optimizes the dynamic linking of shared
1084libraries to reduce start up time of executables.
1085
1086After the root filesystem is built, processing begins on the image
1087through the :ref:`ref-tasks-image`
1088task. The build system runs any pre-processing commands as defined by
1089the
1090:term:`IMAGE_PREPROCESS_COMMAND`
1091variable. This variable specifies a list of functions to call before the
1092build system creates the final image output files.
1093
1094The build system dynamically creates ``do_image_*`` tasks as needed,
1095based on the image types specified in the
1096:term:`IMAGE_FSTYPES` variable.
1097The process turns everything into an image file or a set of image files
1098and can compress the root filesystem image to reduce the overall size of
1099the image. The formats used for the root filesystem depend on the
1100``IMAGE_FSTYPES`` variable. Compression depends on whether the formats
1101support compression.
1102
1103As an example, a dynamically created task when creating a particular
1104image type would take the following form:
1105::
1106
1107 do_image_type
1108
1109So, if the type
1110as specified by the ``IMAGE_FSTYPES`` were ``ext4``, the dynamically
1111generated task would be as follows:
1112::
1113
1114 do_image_ext4
1115
1116The final task involved in image creation is the
1117:ref:`do_image_complete <ref-tasks-image-complete>`
1118task. This task completes the image by applying any image post
1119processing as defined through the
1120:term:`IMAGE_POSTPROCESS_COMMAND`
1121variable. The variable specifies a list of functions to call once the
1122build system has created the final image output files.
1123
1124.. note::
1125
1126 The entire image generation process is run under
1127 Pseudo. Running under Pseudo ensures that the files in the root filesystem
1128 have correct ownership.
1129
1130.. _sdk-generation-dev-environment:
1131
1132SDK Generation
1133~~~~~~~~~~~~~~
1134
1135The OpenEmbedded build system uses BitBake to generate the Software
1136Development Kit (SDK) installer scripts for both the standard SDK and
1137the extensible SDK (eSDK):
1138
1139.. image:: figures/sdk-generation.png
1140 :align: center
1141
1142.. note::
1143
1144 For more information on the cross-development toolchain generation,
1145 see the ":ref:`overview-manual/overview-manual-concepts:cross-development toolchain generation`"
1146 section. For information on advantages gained when building a
1147 cross-development toolchain using the do_populate_sdk task, see the
1148 ":ref:`sdk-manual/sdk-appendix-obtain:building an sdk installer`" section in
1149 the Yocto Project Application Development and the Extensible Software
1150 Development Kit (eSDK) manual.
1151
1152Like image generation, the SDK script process consists of several stages
1153and depends on many variables. The
1154:ref:`ref-tasks-populate_sdk`
1155and
1156:ref:`ref-tasks-populate_sdk_ext`
1157tasks use these key variables to help create the list of packages to
1158actually install. For information on the variables listed in the figure,
1159see the "`Application Development SDK <#sdk-dev-environment>`__"
1160section.
1161
1162The ``do_populate_sdk`` task helps create the standard SDK and handles
1163two parts: a target part and a host part. The target part is the part
1164built for the target hardware and includes libraries and headers. The
1165host part is the part of the SDK that runs on the
1166:term:`SDKMACHINE`.
1167
1168The ``do_populate_sdk_ext`` task helps create the extensible SDK and
1169handles host and target parts differently than its counter part does for
1170the standard SDK. For the extensible SDK, the task encapsulates the
1171build system, which includes everything needed (host and target) for the
1172SDK.
1173
1174Regardless of the type of SDK being constructed, the tasks perform some
1175cleanup after which a cross-development environment setup script and any
1176needed configuration files are created. The final output is the
1177Cross-development toolchain installation script (``.sh`` file), which
1178includes the environment setup script.
1179
1180Stamp Files and the Rerunning of Tasks
1181~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1182
1183For each task that completes successfully, BitBake writes a stamp file
1184into the :term:`STAMPS_DIR`
1185directory. The beginning of the stamp file's filename is determined by
1186the :term:`STAMP` variable, and the end
1187of the name consists of the task's name and current `input
1188checksum <#overview-checksums>`__.
1189
1190.. note::
1191
1192 This naming scheme assumes that
1193 BB_SIGNATURE_HANDLER
1194 is "OEBasicHash", which is almost always the case in current
1195 OpenEmbedded.
1196
1197To determine if a task needs to be rerun, BitBake checks if a stamp file
1198with a matching input checksum exists for the task. If such a stamp file
1199exists, the task's output is assumed to exist and still be valid. If the
1200file does not exist, the task is rerun.
1201
1202.. note::
1203
1204 The stamp mechanism is more general than the shared state (sstate)
1205 cache mechanism described in the "`Setscene Tasks and Shared
1206 State <#setscene-tasks-and-shared-state>`__" section. BitBake avoids
1207 rerunning any task that has a valid stamp file, not just tasks that
1208 can be accelerated through the sstate cache.
1209
1210 However, you should realize that stamp files only serve as a marker
1211 that some work has been done and that these files do not record task
1212 output. The actual task output would usually be somewhere in
1213 :term:`TMPDIR` (e.g. in some
1214 recipe's :term:`WORKDIR`.) What
1215 the sstate cache mechanism adds is a way to cache task output that
1216 can then be shared between build machines.
1217
1218Since ``STAMPS_DIR`` is usually a subdirectory of ``TMPDIR``, removing
1219``TMPDIR`` will also remove ``STAMPS_DIR``, which means tasks will
1220properly be rerun to repopulate ``TMPDIR``.
1221
1222If you want some task to always be considered "out of date", you can
1223mark it with the :ref:`nostamp <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`
1224varflag. If some other task depends on such a task, then that task will
1225also always be considered out of date, which might not be what you want.
1226
1227For details on how to view information about a task's signature, see the
1228":ref:`dev-manual/dev-manual-common-tasks:viewing task variable dependencies`"
1229section in the Yocto Project Development Tasks Manual.
1230
1231Setscene Tasks and Shared State
1232~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1233
1234The description of tasks so far assumes that BitBake needs to build
1235everything and no available prebuilt objects exist. BitBake does support
1236skipping tasks if prebuilt objects are available. These objects are
1237usually made available in the form of a shared state (sstate) cache.
1238
1239.. note::
1240
1241 For information on variables affecting sstate, see the
1242 :term:`SSTATE_DIR`
1243 and
1244 :term:`SSTATE_MIRRORS`
1245 variables.
1246
1247The idea of a setscene task (i.e ``do_``\ taskname\ ``_setscene``) is a
1248version of the task where instead of building something, BitBake can
1249skip to the end result and simply place a set of files into specific
1250locations as needed. In some cases, it makes sense to have a setscene
1251task variant (e.g. generating package files in the
1252:ref:`do_package_write_* <ref-tasks-package_write_deb>`
1253task). In other cases, it does not make sense (e.g. a
1254:ref:`ref-tasks-patch` task or a
1255:ref:`ref-tasks-unpack` task) since
1256the work involved would be equal to or greater than the underlying task.
1257
1258In the build system, the common tasks that have setscene variants are
1259:ref:`ref-tasks-package`,
1260``do_package_write_*``,
1261:ref:`ref-tasks-deploy`,
1262:ref:`ref-tasks-packagedata`, and
1263:ref:`ref-tasks-populate_sysroot`.
1264Notice that these tasks represent most of the tasks whose output is an
1265end result.
1266
1267The build system has knowledge of the relationship between these tasks
1268and other preceding tasks. For example, if BitBake runs
1269``do_populate_sysroot_setscene`` for something, it does not make sense
1270to run any of the ``do_fetch``, ``do_unpack``, ``do_patch``,
1271``do_configure``, ``do_compile``, and ``do_install`` tasks. However, if
1272``do_package`` needs to be run, BitBake needs to run those other tasks.
1273
1274It becomes more complicated if everything can come from an sstate cache
1275because some objects are simply not required at all. For example, you do
1276not need a compiler or native tools, such as quilt, if nothing exists to
1277compile or patch. If the ``do_package_write_*`` packages are available
1278from sstate, BitBake does not need the ``do_package`` task data.
1279
1280To handle all these complexities, BitBake runs in two phases. The first
1281is the "setscene" stage. During this stage, BitBake first checks the
1282sstate cache for any targets it is planning to build. BitBake does a
1283fast check to see if the object exists rather than a complete download.
1284If nothing exists, the second phase, which is the setscene stage,
1285completes and the main build proceeds.
1286
1287If objects are found in the sstate cache, the build system works
1288backwards from the end targets specified by the user. For example, if an
1289image is being built, the build system first looks for the packages
1290needed for that image and the tools needed to construct an image. If
1291those are available, the compiler is not needed. Thus, the compiler is
1292not even downloaded. If something was found to be unavailable, or the
1293download or setscene task fails, the build system then tries to install
1294dependencies, such as the compiler, from the cache.
1295
1296The availability of objects in the sstate cache is handled by the
1297function specified by the
1298:term:`bitbake:BB_HASHCHECK_FUNCTION`
1299variable and returns a list of available objects. The function specified
1300by the
1301:term:`bitbake:BB_SETSCENE_DEPVALID`
1302variable is the function that determines whether a given dependency
1303needs to be followed, and whether for any given relationship the
1304function needs to be passed. The function returns a True or False value.
1305
1306.. _images-dev-environment:
1307
1308Images
1309------
1310
1311The images produced by the build system are compressed forms of the root
1312filesystem and are ready to boot on a target device. You can see from
1313the `general workflow figure <#general-workflow-figure>`__ that BitBake
1314output, in part, consists of images. This section takes a closer look at
1315this output:
1316
1317.. image:: figures/images.png
1318 :align: center
1319
1320.. note::
1321
1322 For a list of example images that the Yocto Project provides, see the
1323 ":doc:`../ref-manual/ref-images`" chapter in the Yocto Project Reference
1324 Manual.
1325
1326The build process writes images out to the :term:`Build Directory`
1327inside the
1328``tmp/deploy/images/machine/`` folder as shown in the figure. This
1329folder contains any files expected to be loaded on the target device.
1330The :term:`DEPLOY_DIR` variable
1331points to the ``deploy`` directory, while the
1332:term:`DEPLOY_DIR_IMAGE`
1333variable points to the appropriate directory containing images for the
1334current configuration.
1335
1336- kernel-image: A kernel binary file. The
1337 :term:`KERNEL_IMAGETYPE`
1338 variable determines the naming scheme for the kernel image file.
1339 Depending on this variable, the file could begin with a variety of
1340 naming strings. The ``deploy/images/``\ machine directory can contain
1341 multiple image files for the machine.
1342
1343- root-filesystem-image: Root filesystems for the target device (e.g.
1344 ``*.ext3`` or ``*.bz2`` files). The
1345 :term:`IMAGE_FSTYPES`
1346 variable determines the root filesystem image type. The
1347 ``deploy/images/``\ machine directory can contain multiple root
1348 filesystems for the machine.
1349
1350- kernel-modules: Tarballs that contain all the modules built for the
1351 kernel. Kernel module tarballs exist for legacy purposes and can be
1352 suppressed by setting the
1353 :term:`MODULE_TARBALL_DEPLOY`
1354 variable to "0". The ``deploy/images/``\ machine directory can
1355 contain multiple kernel module tarballs for the machine.
1356
1357- bootloaders: If applicable to the target machine, bootloaders
1358 supporting the image. The ``deploy/images/``\ machine directory can
1359 contain multiple bootloaders for the machine.
1360
1361- symlinks: The ``deploy/images/``\ machine folder contains a symbolic
1362 link that points to the most recently built file for each machine.
1363 These links might be useful for external scripts that need to obtain
1364 the latest version of each file.
1365
1366.. _sdk-dev-environment:
1367
1368Application Development SDK
1369---------------------------
1370
1371In the `general workflow figure <#general-workflow-figure>`__, the
1372output labeled "Application Development SDK" represents an SDK. The SDK
1373generation process differs depending on whether you build an extensible
1374SDK (e.g. ``bitbake -c populate_sdk_ext`` imagename) or a standard SDK
1375(e.g. ``bitbake -c populate_sdk`` imagename). This section takes a
1376closer look at this output:
1377
1378.. image:: figures/sdk.png
1379 :align: center
1380
1381The specific form of this output is a set of files that includes a
1382self-extracting SDK installer (``*.sh``), host and target manifest
1383files, and files used for SDK testing. When the SDK installer file is
1384run, it installs the SDK. The SDK consists of a cross-development
1385toolchain, a set of libraries and headers, and an SDK environment setup
1386script. Running this installer essentially sets up your
1387cross-development environment. You can think of the cross-toolchain as
1388the "host" part because it runs on the SDK machine. You can think of the
1389libraries and headers as the "target" part because they are built for
1390the target hardware. The environment setup script is added so that you
1391can initialize the environment before using the tools.
1392
1393.. note::
1394
1395 - The Yocto Project supports several methods by which you can set up
1396 this cross-development environment. These methods include
1397 downloading pre-built SDK installers or building and installing
1398 your own SDK installer.
1399
1400 - For background information on cross-development toolchains in the
1401 Yocto Project development environment, see the "`Cross-Development
1402 Toolchain Generation <#cross-development-toolchain-generation>`__"
1403 section.
1404
1405 - For information on setting up a cross-development environment, see
1406 the :doc:`../sdk-manual/sdk-manual` manual.
1407
1408All the output files for an SDK are written to the ``deploy/sdk`` folder
1409inside the :term:`Build Directory` as
1410shown in the previous figure. Depending on the type of SDK, several
1411variables exist that help configure these files. The following list
1412shows the variables associated with an extensible SDK:
1413
1414- :term:`DEPLOY_DIR`: Points to
1415 the ``deploy`` directory.
1416
1417- :term:`SDK_EXT_TYPE`:
1418 Controls whether or not shared state artifacts are copied into the
1419 extensible SDK. By default, all required shared state artifacts are
1420 copied into the SDK.
1421
1422- :term:`SDK_INCLUDE_PKGDATA`:
1423 Specifies whether or not packagedata is included in the extensible
1424 SDK for all recipes in the "world" target.
1425
1426- :term:`SDK_INCLUDE_TOOLCHAIN`:
1427 Specifies whether or not the toolchain is included when building the
1428 extensible SDK.
1429
1430- :term:`SDK_LOCAL_CONF_WHITELIST`:
1431 A list of variables allowed through from the build system
1432 configuration into the extensible SDK configuration.
1433
1434- :term:`SDK_LOCAL_CONF_BLACKLIST`:
1435 A list of variables not allowed through from the build system
1436 configuration into the extensible SDK configuration.
1437
1438- :term:`SDK_INHERIT_BLACKLIST`:
1439 A list of classes to remove from the
1440 :term:`INHERIT` value globally
1441 within the extensible SDK configuration.
1442
1443This next list, shows the variables associated with a standard SDK:
1444
1445- :term:`DEPLOY_DIR`: Points to
1446 the ``deploy`` directory.
1447
1448- :term:`SDKMACHINE`: Specifies
1449 the architecture of the machine on which the cross-development tools
1450 are run to create packages for the target hardware.
1451
1452- :term:`SDKIMAGE_FEATURES`:
1453 Lists the features to include in the "target" part of the SDK.
1454
1455- :term:`TOOLCHAIN_HOST_TASK`:
1456 Lists packages that make up the host part of the SDK (i.e. the part
1457 that runs on the ``SDKMACHINE``). When you use
1458 ``bitbake -c populate_sdk imagename`` to create the SDK, a set of
1459 default packages apply. This variable allows you to add more
1460 packages.
1461
1462- :term:`TOOLCHAIN_TARGET_TASK`:
1463 Lists packages that make up the target part of the SDK (i.e. the part
1464 built for the target hardware).
1465
1466- :term:`SDKPATH`: Defines the
1467 default SDK installation path offered by the installation script.
1468
1469- :term:`SDK_HOST_MANIFEST`:
1470 Lists all the installed packages that make up the host part of the
1471 SDK. This variable also plays a minor role for extensible SDK
1472 development as well. However, it is mainly used for the standard SDK.
1473
1474- :term:`SDK_TARGET_MANIFEST`:
1475 Lists all the installed packages that make up the target part of the
1476 SDK. This variable also plays a minor role for extensible SDK
1477 development as well. However, it is mainly used for the standard SDK.
1478
1479Cross-Development Toolchain Generation
1480======================================
1481
1482The Yocto Project does most of the work for you when it comes to
1483creating :ref:`sdk-manual/sdk-intro:the cross-development toolchain`. This
1484section provides some technical background on how cross-development
1485toolchains are created and used. For more information on toolchains, you
1486can also see the :doc:`../sdk-manual/sdk-manual` manual.
1487
1488In the Yocto Project development environment, cross-development
1489toolchains are used to build images and applications that run on the
1490target hardware. With just a few commands, the OpenEmbedded build system
1491creates these necessary toolchains for you.
1492
1493The following figure shows a high-level build environment regarding
1494toolchain construction and use.
1495
1496.. image:: figures/cross-development-toolchains.png
1497 :align: center
1498
1499Most of the work occurs on the Build Host. This is the machine used to
1500build images and generally work within the the Yocto Project
1501environment. When you run
1502:term:`BitBake` to create an image, the
1503OpenEmbedded build system uses the host ``gcc`` compiler to bootstrap a
1504cross-compiler named ``gcc-cross``. The ``gcc-cross`` compiler is what
1505BitBake uses to compile source files when creating the target image. You
1506can think of ``gcc-cross`` simply as an automatically generated
1507cross-compiler that is used internally within BitBake only.
1508
1509.. note::
1510
1511 The extensible SDK does not use
1512 gcc-cross-canadian
1513 since this SDK ships a copy of the OpenEmbedded build system and the
1514 sysroot within it contains
1515 gcc-cross
1516 .
1517
1518The chain of events that occurs when ``gcc-cross`` is bootstrapped is as
1519follows:
1520::
1521
1522 gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime
1523
1524- ``gcc``: The build host's GNU Compiler Collection (GCC).
1525
1526- ``binutils-cross``: The bare minimum binary utilities needed in order
1527 to run the ``gcc-cross-initial`` phase of the bootstrap operation.
1528
1529- ``gcc-cross-initial``: An early stage of the bootstrap process for
1530 creating the cross-compiler. This stage builds enough of the
1531 ``gcc-cross``, the C library, and other pieces needed to finish
1532 building the final cross-compiler in later stages. This tool is a
1533 "native" package (i.e. it is designed to run on the build host).
1534
1535- ``linux-libc-headers``: Headers needed for the cross-compiler.
1536
1537- ``glibc-initial``: An initial version of the Embedded GNU C Library
1538 (GLIBC) needed to bootstrap ``glibc``.
1539
1540- ``glibc``: The GNU C Library.
1541
1542- ``gcc-cross``: The final stage of the bootstrap process for the
1543 cross-compiler. This stage results in the actual cross-compiler that
1544 BitBake uses when it builds an image for a targeted device.
1545
1546 .. note::
1547
1548 If you are replacing this cross compiler toolchain with a custom
1549 version, you must replace
1550 gcc-cross
1551 .
1552
1553 This tool is also a "native" package (i.e. it is designed to run on
1554 the build host).
1555
1556- ``gcc-runtime``: Runtime libraries resulting from the toolchain
1557 bootstrapping process. This tool produces a binary that consists of
1558 the runtime libraries need for the targeted device.
1559
1560You can use the OpenEmbedded build system to build an installer for the
1561relocatable SDK used to develop applications. When you run the
1562installer, it installs the toolchain, which contains the development
1563tools (e.g., ``gcc-cross-canadian``, ``binutils-cross-canadian``, and
1564other ``nativesdk-*`` tools), which are tools native to the SDK (i.e.
1565native to :term:`SDK_ARCH`), you
1566need to cross-compile and test your software. The figure shows the
1567commands you use to easily build out this toolchain. This
1568cross-development toolchain is built to execute on the
1569:term:`SDKMACHINE`, which might or
1570might not be the same machine as the Build Host.
1571
1572.. note::
1573
1574 If your target architecture is supported by the Yocto Project, you
1575 can take advantage of pre-built images that ship with the Yocto
1576 Project and already contain cross-development toolchain installers.
1577
1578Here is the bootstrap process for the relocatable toolchain:
1579::
1580
1581 gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers -> glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian
1582
1583- ``gcc``: The build host's GNU Compiler Collection (GCC).
1584
1585- ``binutils-crosssdk``: The bare minimum binary utilities needed in
1586 order to run the ``gcc-crosssdk-initial`` phase of the bootstrap
1587 operation.
1588
1589- ``gcc-crosssdk-initial``: An early stage of the bootstrap process for
1590 creating the cross-compiler. This stage builds enough of the
1591 ``gcc-crosssdk`` and supporting pieces so that the final stage of the
1592 bootstrap process can produce the finished cross-compiler. This tool
1593 is a "native" binary that runs on the build host.
1594
1595- ``linux-libc-headers``: Headers needed for the cross-compiler.
1596
1597- ``glibc-initial``: An initial version of the Embedded GLIBC needed to
1598 bootstrap ``nativesdk-glibc``.
1599
1600- ``nativesdk-glibc``: The Embedded GLIBC needed to bootstrap the
1601 ``gcc-crosssdk``.
1602
1603- ``gcc-crosssdk``: The final stage of the bootstrap process for the
1604 relocatable cross-compiler. The ``gcc-crosssdk`` is a transitory
1605 compiler and never leaves the build host. Its purpose is to help in
1606 the bootstrap process to create the eventual ``gcc-cross-canadian``
1607 compiler, which is relocatable. This tool is also a "native" package
1608 (i.e. it is designed to run on the build host).
1609
1610- ``gcc-cross-canadian``: The final relocatable cross-compiler. When
1611 run on the :term:`SDKMACHINE`,
1612 this tool produces executable code that runs on the target device.
1613 Only one cross-canadian compiler is produced per architecture since
1614 they can be targeted at different processor optimizations using
1615 configurations passed to the compiler through the compile commands.
1616 This circumvents the need for multiple compilers and thus reduces the
1617 size of the toolchains.
1618
1619.. note::
1620
1621 For information on advantages gained when building a
1622 cross-development toolchain installer, see the
1623 ":ref:`sdk-manual/sdk-appendix-obtain:building an sdk installer`" appendix
1624 in the Yocto Project Application Development and the
1625 Extensible Software Development Kit (eSDK) manual.
1626
1627Shared State Cache
1628==================
1629
1630By design, the OpenEmbedded build system builds everything from scratch
1631unless :term:`BitBake` can determine
1632that parts do not need to be rebuilt. Fundamentally, building from
1633scratch is attractive as it means all parts are built fresh and no
1634possibility of stale data exists that can cause problems. When
1635developers hit problems, they typically default back to building from
1636scratch so they have a know state from the start.
1637
1638Building an image from scratch is both an advantage and a disadvantage
1639to the process. As mentioned in the previous paragraph, building from
1640scratch ensures that everything is current and starts from a known
1641state. However, building from scratch also takes much longer as it
1642generally means rebuilding things that do not necessarily need to be
1643rebuilt.
1644
1645The Yocto Project implements shared state code that supports incremental
1646builds. The implementation of the shared state code answers the
1647following questions that were fundamental roadblocks within the
1648OpenEmbedded incremental build support system:
1649
1650- What pieces of the system have changed and what pieces have not
1651 changed?
1652
1653- How are changed pieces of software removed and replaced?
1654
1655- How are pre-built components that do not need to be rebuilt from
1656 scratch used when they are available?
1657
1658For the first question, the build system detects changes in the "inputs"
1659to a given task by creating a checksum (or signature) of the task's
1660inputs. If the checksum changes, the system assumes the inputs have
1661changed and the task needs to be rerun. For the second question, the
1662shared state (sstate) code tracks which tasks add which output to the
1663build process. This means the output from a given task can be removed,
1664upgraded or otherwise manipulated. The third question is partly
1665addressed by the solution for the second question assuming the build
1666system can fetch the sstate objects from remote locations and install
1667them if they are deemed to be valid.
1668
1669.. note::
1670
1671 - The build system does not maintain
1672 :term:`PR` information as part of
1673 the shared state packages. Consequently, considerations exist that
1674 affect maintaining shared state feeds. For information on how the
1675 build system works with packages and can track incrementing ``PR``
1676 information, see the ":ref:`dev-manual/dev-manual-common-tasks:automatically incrementing a package version number`"
1677 section in the Yocto Project Development Tasks Manual.
1678
1679 - The code in the build system that supports incremental builds is
1680 not simple code. For techniques that help you work around issues
1681 related to shared state code, see the
1682 ":ref:`dev-manual/dev-manual-common-tasks:viewing metadata used to create the input signature of a shared state task`"
1683 and
1684 ":ref:`dev-manual/dev-manual-common-tasks:invalidating shared state to force a task to run`"
1685 sections both in the Yocto Project Development Tasks Manual.
1686
1687The rest of this section goes into detail about the overall incremental
1688build architecture, the checksums (signatures), and shared state.
1689
1690.. _concepts-overall-architecture:
1691
1692Overall Architecture
1693--------------------
1694
1695When determining what parts of the system need to be built, BitBake
1696works on a per-task basis rather than a per-recipe basis. You might
1697wonder why using a per-task basis is preferred over a per-recipe basis.
1698To help explain, consider having the IPK packaging backend enabled and
1699then switching to DEB. In this case, the
1700:ref:`ref-tasks-install` and
1701:ref:`ref-tasks-package` task outputs
1702are still valid. However, with a per-recipe approach, the build would
1703not include the ``.deb`` files. Consequently, you would have to
1704invalidate the whole build and rerun it. Rerunning everything is not the
1705best solution. Also, in this case, the core must be "taught" much about
1706specific tasks. This methodology does not scale well and does not allow
1707users to easily add new tasks in layers or as external recipes without
1708touching the packaged-staging core.
1709
1710.. _overview-checksums:
1711
1712Checksums (Signatures)
1713----------------------
1714
1715The shared state code uses a checksum, which is a unique signature of a
1716task's inputs, to determine if a task needs to be run again. Because it
1717is a change in a task's inputs that triggers a rerun, the process needs
1718to detect all the inputs to a given task. For shell tasks, this turns
1719out to be fairly easy because the build process generates a "run" shell
1720script for each task and it is possible to create a checksum that gives
1721you a good idea of when the task's data changes.
1722
1723To complicate the problem, there are things that should not be included
1724in the checksum. First, there is the actual specific build path of a
1725given task - the :term:`WORKDIR`. It
1726does not matter if the work directory changes because it should not
1727affect the output for target packages. Also, the build process has the
1728objective of making native or cross packages relocatable.
1729
1730.. note::
1731
1732 Both native and cross packages run on the
1733 build host. However, cross packages generate output for the target
1734 architecture.
1735
1736The checksum therefore needs to exclude ``WORKDIR``. The simplistic
1737approach for excluding the work directory is to set ``WORKDIR`` to some
1738fixed value and create the checksum for the "run" script.
1739
1740Another problem results from the "run" scripts containing functions that
1741might or might not get called. The incremental build solution contains
1742code that figures out dependencies between shell functions. This code is
1743used to prune the "run" scripts down to the minimum set, thereby
1744alleviating this problem and making the "run" scripts much more readable
1745as a bonus.
1746
1747So far, solutions for shell scripts exist. What about Python tasks? The
1748same approach applies even though these tasks are more difficult. The
1749process needs to figure out what variables a Python function accesses
1750and what functions it calls. Again, the incremental build solution
1751contains code that first figures out the variable and function
1752dependencies, and then creates a checksum for the data used as the input
1753to the task.
1754
1755Like the ``WORKDIR`` case, situations exist where dependencies should be
1756ignored. For these situations, you can instruct the build process to
1757ignore a dependency by using a line like the following:
1758::
1759
1760 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
1761
1762This example ensures that the :term:`PACKAGE_ARCHS` variable
1763does not depend on the value of :term:`MACHINE`, even if it does
1764reference it.
1765
1766Equally, there are cases where you need to add dependencies BitBake is
1767not able to find. You can accomplish this by using a line like the
1768following:
1769::
1770
1771 PACKAGE_ARCHS[vardeps] = "MACHINE"
1772
1773This example explicitly
1774adds the ``MACHINE`` variable as a dependency for ``PACKAGE_ARCHS``.
1775
1776As an example, consider a case with in-line Python where BitBake is not
1777able to figure out dependencies. When running in debug mode (i.e. using
1778``-DDD``), BitBake produces output when it discovers something for which
1779it cannot figure out dependencies. The Yocto Project team has currently
1780not managed to cover those dependencies in detail and is aware of the
1781need to fix this situation.
1782
1783Thus far, this section has limited discussion to the direct inputs into
1784a task. Information based on direct inputs is referred to as the
1785"basehash" in the code. However, the question of a task's indirect
1786inputs still exits - items already built and present in the
1787:term:`Build Directory`. The checksum (or
1788signature) for a particular task needs to add the hashes of all the
1789tasks on which the particular task depends. Choosing which dependencies
1790to add is a policy decision. However, the effect is to generate a master
1791checksum that combines the basehash and the hashes of the task's
1792dependencies.
1793
1794At the code level, a variety of ways exist by which both the basehash
1795and the dependent task hashes can be influenced. Within the BitBake
1796configuration file, you can give BitBake some extra information to help
1797it construct the basehash. The following statement effectively results
1798in a list of global variable dependency excludes (i.e. variables never
1799included in any checksum):
1800::
1801
1802 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \\
1803 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \\
1804 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \\
1805 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \\
1806 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
1807
1808The
1809previous example excludes
1810:term:`WORKDIR` since that variable
1811is actually constructed as a path within
1812:term:`TMPDIR`, which is on the
1813whitelist.
1814
1815The rules for deciding which hashes of dependent tasks to include
1816through dependency chains are more complex and are generally
1817accomplished with a Python function. The code in
1818``meta/lib/oe/sstatesig.py`` shows two examples of this and also
1819illustrates how you can insert your own policy into the system if so
1820desired. This file defines the two basic signature generators
1821:term:`OpenEmbedded-Core (OE-Core)` uses: "OEBasic" and
1822"OEBasicHash". By default, a dummy "noop" signature handler is enabled
1823in BitBake. This means that behavior is unchanged from previous
1824versions. OE-Core uses the "OEBasicHash" signature handler by default
1825through this setting in the ``bitbake.conf`` file:
1826::
1827
1828 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
1829
1830The "OEBasicHash" ``BB_SIGNATURE_HANDLER`` is the same
1831as the "OEBasic" version but adds the task hash to the `stamp
1832files <#stamp-files-and-the-rerunning-of-tasks>`__. This results in any
1833metadata change that changes the task hash, automatically causing the
1834task to be run again. This removes the need to bump
1835:term:`PR` values, and changes to metadata
1836automatically ripple across the build.
1837
1838It is also worth noting that the end result of these signature
1839generators is to make some dependency and hash information available to
1840the build. This information includes:
1841
1842- ``BB_BASEHASH_task-``\ taskname: The base hashes for each task in the
1843 recipe.
1844
1845- ``BB_BASEHASH_``\ filename\ ``:``\ taskname: The base hashes for each
1846 dependent task.
1847
1848- ``BBHASHDEPS_``\ filename\ ``:``\ taskname: The task dependencies for
1849 each task.
1850
1851- ``BB_TASKHASH``: The hash of the currently running task.
1852
1853Shared State
1854------------
1855
1856Checksums and dependencies, as discussed in the previous section, solve
1857half the problem of supporting a shared state. The other half of the
1858problem is being able to use checksum information during the build and
1859being able to reuse or rebuild specific components.
1860
1861The :ref:`sstate <ref-classes-sstate>` class is a
1862relatively generic implementation of how to "capture" a snapshot of a
1863given task. The idea is that the build process does not care about the
1864source of a task's output. Output could be freshly built or it could be
1865downloaded and unpacked from somewhere. In other words, the build
1866process does not need to worry about its origin.
1867
1868Two types of output exist. One type is just about creating a directory
1869in :term:`WORKDIR`. A good example is
1870the output of either
1871:ref:`ref-tasks-install` or
1872:ref:`ref-tasks-package`. The other
1873type of output occurs when a set of data is merged into a shared
1874directory tree such as the sysroot.
1875
1876The Yocto Project team has tried to keep the details of the
1877implementation hidden in ``sstate`` class. From a user's perspective,
1878adding shared state wrapping to a task is as simple as this
1879:ref:`ref-tasks-deploy` example taken
1880from the :ref:`deploy <ref-classes-deploy>` class:
1881::
1882
1883 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
1884 SSTATETASKS += "do_deploy"
1885 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
1886 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
1887
1888 python do_deploy_setscene () {
1889 sstate_setscene(d)
1890 }
1891 addtask do_deploy_setscene
1892 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
1893 do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"
1894
1895The following list explains the previous example:
1896
1897- Adding "do_deploy" to ``SSTATETASKS`` adds some required
1898 sstate-related processing, which is implemented in the
1899 :ref:`sstate <ref-classes-sstate>` class, to
1900 before and after the
1901 :ref:`ref-tasks-deploy` task.
1902
1903- The ``do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"`` declares that
1904 ``do_deploy`` places its output in ``${DEPLOYDIR}`` when run normally
1905 (i.e. when not using the sstate cache). This output becomes the input
1906 to the shared state cache.
1907
1908- The ``do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"`` line
1909 causes the contents of the shared state cache to be copied to
1910 ``${DEPLOY_DIR_IMAGE}``.
1911
1912 .. note::
1913
1914 If ``do_deploy`` is not already in the shared state cache or if its input
1915 checksum (signature) has changed from when the output was cached, the task
1916 runs to populate the shared state cache, after which the contents of the
1917 shared state cache is copied to ${:term:`DEPLOY_DIR_IMAGE`}. If
1918 ``do_deploy`` is in the shared state cache and its signature indicates
1919 that the cached output is still valid (i.e. if no relevant task inputs
1920 have changed), then the contents of the shared state cache copies
1921 directly to ${``DEPLOY_DIR_IMAGE``} by the ``do_deploy_setscene`` task
1922 instead, skipping the ``do_deploy`` task.
1923
1924- The following task definition is glue logic needed to make the
1925 previous settings effective:
1926 ::
1927
1928 python do_deploy_setscene () {
1929 sstate_setscene(d)
1930 }
1931 addtask do_deploy_setscene
1932
1933 ``sstate_setscene()`` takes the flags above as input and accelerates the ``do_deploy`` task
1934 through the shared state cache if possible. If the task was
1935 accelerated, ``sstate_setscene()`` returns True. Otherwise, it
1936 returns False, and the normal ``do_deploy`` task runs. For more
1937 information, see the ":ref:`setscene <bitbake:bitbake-user-manual/bitbake-user-manual-execution:setscene>`"
1938 section in the BitBake User Manual.
1939
1940- The ``do_deploy[dirs] = "${DEPLOYDIR} ${B}"`` line creates
1941 ``${DEPLOYDIR}`` and ``${B}`` before the ``do_deploy`` task runs, and
1942 also sets the current working directory of ``do_deploy`` to ``${B}``.
1943 For more information, see the ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags`"
1944 section in the BitBake
1945 User Manual.
1946
1947 .. note::
1948
1949 In cases where ``sstate-inputdirs`` and ``sstate-outputdirs`` would be
1950 the same, you can use ``sstate-plaindirs``. For example, to preserve the
1951 ${:term:`PKGD`} and ${:term:`PKGDEST`} output from the ``do_package``
1952 task, use the following:
1953 ::
1954
1955 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
1956
1957
1958- The ``do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"`` line appends
1959 extra metadata to the `stamp
1960 file <#stamp-files-and-the-rerunning-of-tasks>`__. In this case, the
1961 metadata makes the task specific to a machine's architecture. See
1962 ":ref:`bitbake:ref-bitbake-tasklist`"
1963 section in the BitBake User Manual for more information on the
1964 ``stamp-extra-info`` flag.
1965
1966- ``sstate-inputdirs`` and ``sstate-outputdirs`` can also be used with
1967 multiple directories. For example, the following declares
1968 ``PKGDESTWORK`` and ``SHLIBWORK`` as shared state input directories,
1969 which populates the shared state cache, and ``PKGDATA_DIR`` and
1970 ``SHLIBSDIR`` as the corresponding shared state output directories:
1971 ::
1972
1973 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
1974 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
1975
1976- These methods also include the ability to take a lockfile when
1977 manipulating shared state directory structures, for cases where file
1978 additions or removals are sensitive:
1979 ::
1980
1981 do_package[sstate-lockfile] = "${PACKAGELOCK}"
1982
1983Behind the scenes, the shared state code works by looking in
1984:term:`SSTATE_DIR` and
1985:term:`SSTATE_MIRRORS` for
1986shared state files. Here is an example:
1987::
1988
1989 SSTATE_MIRRORS ?= "\
1990 file://.\* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \n \
1991 file://.\* file:///some/local/dir/sstate/PATH"
1992
1993.. note::
1994
1995 The shared state directory (``SSTATE_DIR``) is organized into two-character
1996 subdirectories, where the subdirectory names are based on the first two
1997 characters of the hash.
1998 If the shared state directory structure for a mirror has the same structure
1999 as ``SSTATE_DIR``, you must specify "PATH" as part of the URI to enable the build
2000 system to map to the appropriate subdirectory.
2001
2002The shared state package validity can be detected just by looking at the
2003filename since the filename contains the task checksum (or signature) as
2004described earlier in this section. If a valid shared state package is
2005found, the build process downloads it and uses it to accelerate the
2006task.
2007
2008The build processes use the ``*_setscene`` tasks for the task
2009acceleration phase. BitBake goes through this phase before the main
2010execution code and tries to accelerate any tasks for which it can find
2011shared state packages. If a shared state package for a task is
2012available, the shared state package is used. This means the task and any
2013tasks on which it is dependent are not executed.
2014
2015As a real world example, the aim is when building an IPK-based image,
2016only the
2017:ref:`ref-tasks-package_write_ipk`
2018tasks would have their shared state packages fetched and extracted.
2019Since the sysroot is not used, it would never get extracted. This is
2020another reason why a task-based approach is preferred over a
2021recipe-based approach, which would have to install the output from every
2022task.
2023
2024Automatically Added Runtime Dependencies
2025========================================
2026
2027The OpenEmbedded build system automatically adds common types of runtime
2028dependencies between packages, which means that you do not need to
2029explicitly declare the packages using
2030:term:`RDEPENDS`. Three automatic
2031mechanisms exist (``shlibdeps``, ``pcdeps``, and ``depchains``) that
2032handle shared libraries, package configuration (pkg-config) modules, and
2033``-dev`` and ``-dbg`` packages, respectively. For other types of runtime
2034dependencies, you must manually declare the dependencies.
2035
2036- ``shlibdeps``: During the
2037 :ref:`ref-tasks-package` task of
2038 each recipe, all shared libraries installed by the recipe are
2039 located. For each shared library, the package that contains the
2040 shared library is registered as providing the shared library. More
2041 specifically, the package is registered as providing the
2042 `soname <https://en.wikipedia.org/wiki/Soname>`__ of the library. The
2043 resulting shared-library-to-package mapping is saved globally in
2044 :term:`PKGDATA_DIR` by the
2045 :ref:`ref-tasks-packagedata`
2046 task.
2047
2048 Simultaneously, all executables and shared libraries installed by the
2049 recipe are inspected to see what shared libraries they link against.
2050 For each shared library dependency that is found, ``PKGDATA_DIR`` is
2051 queried to see if some package (likely from a different recipe)
2052 contains the shared library. If such a package is found, a runtime
2053 dependency is added from the package that depends on the shared
2054 library to the package that contains the library.
2055
2056 The automatically added runtime dependency also includes a version
2057 restriction. This version restriction specifies that at least the
2058 current version of the package that provides the shared library must
2059 be used, as if "package (>= version)" had been added to ``RDEPENDS``.
2060 This forces an upgrade of the package containing the shared library
2061 when installing the package that depends on the library, if needed.
2062
2063 If you want to avoid a package being registered as providing a
2064 particular shared library (e.g. because the library is for internal
2065 use only), then add the library to
2066 :term:`PRIVATE_LIBS` inside
2067 the package's recipe.
2068
2069- ``pcdeps``: During the ``do_package`` task of each recipe, all
2070 pkg-config modules (``*.pc`` files) installed by the recipe are
2071 located. For each module, the package that contains the module is
2072 registered as providing the module. The resulting module-to-package
2073 mapping is saved globally in ``PKGDATA_DIR`` by the
2074 ``do_packagedata`` task.
2075
2076 Simultaneously, all pkg-config modules installed by the recipe are
2077 inspected to see what other pkg-config modules they depend on. A
2078 module is seen as depending on another module if it contains a
2079 "Requires:" line that specifies the other module. For each module
2080 dependency, ``PKGDATA_DIR`` is queried to see if some package
2081 contains the module. If such a package is found, a runtime dependency
2082 is added from the package that depends on the module to the package
2083 that contains the module.
2084
2085 .. note::
2086
2087 The
2088 pcdeps
2089 mechanism most often infers dependencies between
2090 -dev
2091 packages.
2092
2093- ``depchains``: If a package ``foo`` depends on a package ``bar``,
2094 then ``foo-dev`` and ``foo-dbg`` are also made to depend on
2095 ``bar-dev`` and ``bar-dbg``, respectively. Taking the ``-dev``
2096 packages as an example, the ``bar-dev`` package might provide headers
2097 and shared library symlinks needed by ``foo-dev``, which shows the
2098 need for a dependency between the packages.
2099
2100 The dependencies added by ``depchains`` are in the form of
2101 :term:`RRECOMMENDS`.
2102
2103 .. note::
2104
2105 By default, ``foo-dev`` also has an ``RDEPENDS``-style dependency on
2106 ``foo``, because the default value of ``RDEPENDS_${PN}-dev`` (set in
2107 bitbake.conf) includes "${PN}".
2108
2109 To ensure that the dependency chain is never broken, ``-dev`` and
2110 ``-dbg`` packages are always generated by default, even if the
2111 packages turn out to be empty. See the
2112 :term:`ALLOW_EMPTY` variable
2113 for more information.
2114
2115The ``do_package`` task depends on the ``do_packagedata`` task of each
2116recipe in :term:`DEPENDS` through use
2117of a ``[``\ :ref:`deptask <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]``
2118declaration, which guarantees that the required
2119shared-library/module-to-package mapping information will be available
2120when needed as long as ``DEPENDS`` has been correctly set.
2121
2122Fakeroot and Pseudo
2123===================
2124
2125Some tasks are easier to implement when allowed to perform certain
2126operations that are normally reserved for the root user (e.g.
2127:ref:`ref-tasks-install`,
2128:ref:`do_package_write* <ref-tasks-package_write_deb>`,
2129:ref:`ref-tasks-rootfs`, and
2130:ref:`do_image* <ref-tasks-image>`). For example,
2131the ``do_install`` task benefits from being able to set the UID and GID
2132of installed files to arbitrary values.
2133
2134One approach to allowing tasks to perform root-only operations would be
2135to require :term:`BitBake` to run as
2136root. However, this method is cumbersome and has security issues. The
2137approach that is actually used is to run tasks that benefit from root
2138privileges in a "fake" root environment. Within this environment, the
2139task and its child processes believe that they are running as the root
2140user, and see an internally consistent view of the filesystem. As long
2141as generating the final output (e.g. a package or an image) does not
2142require root privileges, the fact that some earlier steps ran in a fake
2143root environment does not cause problems.
2144
2145The capability to run tasks in a fake root environment is known as
2146"`fakeroot <http://man.he.net/man1/fakeroot>`__", which is derived from
2147the BitBake keyword/variable flag that requests a fake root environment
2148for a task.
2149
2150In the :term:`OpenEmbedded Build System`,
2151the program that
2152implements fakeroot is known as
2153`Pseudo <https://www.yoctoproject.org/software-item/pseudo/>`__. Pseudo
2154overrides system calls by using the environment variable ``LD_PRELOAD``,
2155which results in the illusion of running as root. To keep track of
2156"fake" file ownership and permissions resulting from operations that
2157require root permissions, Pseudo uses an SQLite 3 database. This
2158database is stored in
2159``${``\ :term:`WORKDIR`\ ``}/pseudo/files.db``
2160for individual recipes. Storing the database in a file as opposed to in
2161memory gives persistence between tasks and builds, which is not
2162accomplished using fakeroot.
2163
2164.. note::
2165
2166 If you add your own task that manipulates the same files or
2167 directories as a fakeroot task, then that task also needs to run
2168 under fakeroot. Otherwise, the task cannot run root-only operations,
2169 and cannot see the fake file ownership and permissions set by the
2170 other task. You need to also add a dependency on
2171 virtual/fakeroot-native:do_populate_sysroot
2172 , giving the following:
2173 ::
2174
2175 fakeroot do_mytask () {
2176 ...
2177 }
2178 do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot"
2179
2180
2181For more information, see the
2182:term:`FAKEROOT* <bitbake:FAKEROOT>` variables in the
2183BitBake User Manual. You can also reference the "`Why Not
2184Fakeroot? <https://github.com/wrpseudo/pseudo/wiki/WhyNotFakeroot>`__"
2185article for background information on Fakeroot and Pseudo.
diff --git a/documentation/overview-manual/overview-manual-concepts.xml b/documentation/overview-manual/overview-manual-concepts.xml
index f085dd710d..58b64bd269 100644
--- a/documentation/overview-manual/overview-manual-concepts.xml
+++ b/documentation/overview-manual/overview-manual-concepts.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id=' overview-manual-concepts'> 6<chapter id=' overview-manual-concepts'>
6<title>Yocto Project Concepts</title> 7<title>Yocto Project Concepts</title>
diff --git a/documentation/overview-manual/overview-manual-customization.xsl b/documentation/overview-manual/overview-manual-customization.xsl
index 22360e7bab..1dd91bde80 100644
--- a/documentation/overview-manual/overview-manual-customization.xsl
+++ b/documentation/overview-manual/overview-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/overview-manual/overview-manual-development-environment.rst b/documentation/overview-manual/overview-manual-development-environment.rst
new file mode 100644
index 0000000000..bb2c8e72e7
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-development-environment.rst
@@ -0,0 +1,672 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************************
4The Yocto Project Development Environment
5*****************************************
6
7This chapter takes a look at the Yocto Project development environment.
8The chapter provides Yocto Project Development environment concepts that
9help you understand how work is accomplished in an open source
10environment, which is very different as compared to work accomplished in
11a closed, proprietary environment.
12
13Specifically, this chapter addresses open source philosophy, source
14repositories, workflows, Git, and licensing.
15
16Open Source Philosophy
17======================
18
19Open source philosophy is characterized by software development directed
20by peer production and collaboration through an active community of
21developers. Contrast this to the more standard centralized development
22models used by commercial software companies where a finite set of
23developers produces a product for sale using a defined set of procedures
24that ultimately result in an end product whose architecture and source
25material are closed to the public.
26
27Open source projects conceptually have differing concurrent agendas,
28approaches, and production. These facets of the development process can
29come from anyone in the public (community) who has a stake in the
30software project. The open source environment contains new copyright,
31licensing, domain, and consumer issues that differ from the more
32traditional development environment. In an open source environment, the
33end product, source material, and documentation are all available to the
34public at no cost.
35
36A benchmark example of an open source project is the Linux kernel, which
37was initially conceived and created by Finnish computer science student
38Linus Torvalds in 1991. Conversely, a good example of a non-open source
39project is the Windows family of operating systems developed by
40Microsoft Corporation.
41
42Wikipedia has a good historical description of the Open Source
43Philosophy `here <http://en.wikipedia.org/wiki/Open_source>`__. You can
44also find helpful information on how to participate in the Linux
45Community
46`here <https://www.kernel.org/doc/html/latest/process/index.html>`__.
47
48.. _gs-the-development-host:
49
50The Development Host
51====================
52
53A development host or :term:`Build Host` is key to
54using the Yocto Project. Because the goal of the Yocto Project is to
55develop images or applications that run on embedded hardware,
56development of those images and applications generally takes place on a
57system not intended to run the software - the development host.
58
59You need to set up a development host in order to use it with the Yocto
60Project. Most find that it is best to have a native Linux machine
61function as the development host. However, it is possible to use a
62system that does not run Linux as its operating system as your
63development host. When you have a Mac or Windows-based system, you can
64set it up as the development host by using
65`CROPS <https://github.com/crops/poky-container>`__, which leverages
66`Docker Containers <https://www.docker.com/>`__. Once you take the steps
67to set up a CROPS machine, you effectively have access to a shell
68environment that is similar to what you see when using a Linux-based
69development host. For the steps needed to set up a system using CROPS,
70see the
71":ref:`dev-manual/dev-manual-start:setting up to use cross platforms (crops)`"
72section in
73the Yocto Project Development Tasks Manual.
74
75If your development host is going to be a system that runs a Linux
76distribution, steps still exist that you must take to prepare the system
77for use with the Yocto Project. You need to be sure that the Linux
78distribution on the system is one that supports the Yocto Project. You
79also need to be sure that the correct set of host packages are installed
80that allow development using the Yocto Project. For the steps needed to
81set up a development host that runs Linux, see the
82":ref:`dev-manual/dev-manual-start:setting up a native linux host`"
83section in the Yocto Project Development Tasks Manual.
84
85Once your development host is set up to use the Yocto Project, several
86methods exist for you to do work in the Yocto Project environment:
87
88- *Command Lines, BitBake, and Shells:* Traditional development in the
89 Yocto Project involves using the :term:`OpenEmbedded Build System`,
90 which uses
91 BitBake, in a command-line environment from a shell on your
92 development host. You can accomplish this from a host that is a
93 native Linux machine or from a host that has been set up with CROPS.
94 Either way, you create, modify, and build images and applications all
95 within a shell-based environment using components and tools available
96 through your Linux distribution and the Yocto Project.
97
98 For a general flow of the build procedures, see the
99 ":ref:`dev-manual/dev-manual-common-tasks:building a simple image`"
100 section in the Yocto Project Development Tasks Manual.
101
102- *Board Support Package (BSP) Development:* Development of BSPs
103 involves using the Yocto Project to create and test layers that allow
104 easy development of images and applications targeted for specific
105 hardware. To development BSPs, you need to take some additional steps
106 beyond what was described in setting up a development host.
107
108 The :doc:`../bsp-guide/bsp-guide` provides BSP-related development
109 information. For specifics on development host preparation, see the
110 ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
111 section in the Yocto Project Board Support Package (BSP) Developer's
112 Guide.
113
114- *Kernel Development:* If you are going to be developing kernels using
115 the Yocto Project you likely will be using ``devtool``. A workflow
116 using ``devtool`` makes kernel development quicker by reducing
117 iteration cycle times.
118
119 The :doc:`../kernel-dev/kernel-dev` provides kernel-related
120 development information. For specifics on development host
121 preparation, see the
122 ":ref:`kernel-dev/kernel-dev-common:preparing the build host to work on the kernel`"
123 section in the Yocto Project Linux Kernel Development Manual.
124
125- *Using Toaster:* The other Yocto Project development method that
126 involves an interface that effectively puts the Yocto Project into
127 the background is Toaster. Toaster provides an interface to the
128 OpenEmbedded build system. The interface enables you to configure and
129 run your builds. Information about builds is collected and stored in
130 a database. You can use Toaster to configure and start builds on
131 multiple remote build servers.
132
133 For steps that show you how to set up your development host to use
134 Toaster and on how to use Toaster in general, see the
135 :doc:`../toaster-manual/toaster-manual`.
136
137.. _yocto-project-repositories:
138
139Yocto Project Source Repositories
140=================================
141
142The Yocto Project team maintains complete source repositories for all
143Yocto Project files at :yocto_git:`/`. This web-based source
144code browser is organized into categories by function such as IDE
145Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth. From the
146interface, you can click on any particular item in the "Name" column and
147see the URL at the bottom of the page that you need to clone a Git
148repository for that particular item. Having a local Git repository of
149the :term:`Source Directory`, which
150is usually named "poky", allows you to make changes, contribute to the
151history, and ultimately enhance the Yocto Project's tools, Board Support
152Packages, and so forth.
153
154For any supported release of Yocto Project, you can also go to the
155:yocto_home:`Yocto Project Website <>` and select the "DOWNLOADS"
156item from the "SOFTWARE" menu and get a released tarball of the ``poky``
157repository, any supported BSP tarball, or Yocto Project tools. Unpacking
158these tarballs gives you a snapshot of the released files.
159
160.. note::
161
162 - The recommended method for setting up the Yocto Project
163 :term:`Source Directory` and the files
164 for supported BSPs (e.g., ``meta-intel``) is to use `Git <#git>`__
165 to create a local copy of the upstream repositories.
166
167 - Be sure to always work in matching branches for both the selected
168 BSP repository and the Source Directory (i.e. ``poky``)
169 repository. For example, if you have checked out the "master"
170 branch of ``poky`` and you are going to use ``meta-intel``, be
171 sure to checkout the "master" branch of ``meta-intel``.
172
173In summary, here is where you can get the project files needed for
174development:
175
176- :yocto_git:`Source Repositories: <>` This area contains IDE
177 Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and
178 Yocto Metadata Layers. You can create local copies of Git
179 repositories for each of these areas.
180
181 .. image:: figures/source-repos.png
182 :align: center
183
184 For steps on how to view and access these upstream Git repositories,
185 see the ":ref:`dev-manual/dev-manual-start:accessing source repositories`"
186 Section in the Yocto Project Development Tasks Manual.
187
188- :yocto_dl:`Index of /releases: </releases>` This is an index
189 of releases such as Poky, Pseudo, installers for cross-development
190 toolchains, miscellaneous support and all released versions of Yocto
191 Project in the form of images or tarballs. Downloading and extracting
192 these files does not produce a local copy of the Git repository but
193 rather a snapshot of a particular release or image.
194
195 .. image:: figures/index-downloads.png
196 :align: center
197
198 For steps on how to view and access these files, see the
199 ":ref:`dev-manual/dev-manual-start:accessing index of releases`"
200 section in the Yocto Project Development Tasks Manual.
201
202- *"DOWNLOADS" page for the* :yocto_home:`Yocto Project Website <>` *:*
203
204 The Yocto Project website includes a "DOWNLOADS" page accessible
205 through the "SOFTWARE" menu that allows you to download any Yocto
206 Project release, tool, and Board Support Package (BSP) in tarball
207 form. The tarballs are similar to those found in the
208 :yocto_dl:`Index of /releases: </releases>` area.
209
210 .. image:: figures/yp-download.png
211 :align: center
212
213 For steps on how to use the "DOWNLOADS" page, see the
214 ":ref:`dev-manual/dev-manual-start:using the downloads page`"
215 section in the Yocto Project Development Tasks Manual.
216
217.. _gs-git-workflows-and-the-yocto-project:
218
219Git Workflows and the Yocto Project
220===================================
221
222Developing using the Yocto Project likely requires the use of
223`Git <#git>`__. Git is a free, open source distributed version control
224system used as part of many collaborative design environments. This
225section provides workflow concepts using the Yocto Project and Git. In
226particular, the information covers basic practices that describe roles
227and actions in a collaborative development environment.
228
229.. note::
230
231 If you are familiar with this type of development environment, you
232 might not want to read this section.
233
234The Yocto Project files are maintained using Git in "branches" whose Git
235histories track every change and whose structures provide branches for
236all diverging functionality. Although there is no need to use Git, many
237open source projects do so.
238
239For the Yocto Project, a key individual called the "maintainer" is
240responsible for the integrity of the "master" branch of a given Git
241repository. The "master" branch is the "upstream" repository from which
242final or most recent builds of a project occur. The maintainer is
243responsible for accepting changes from other developers and for
244organizing the underlying branch structure to reflect release strategies
245and so forth.
246
247.. note::
248
249 For information on finding out who is responsible for (maintains) a
250 particular area of code in the Yocto Project, see the
251 ":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
252 section of the Yocto Project Development Tasks Manual.
253
254The Yocto Project ``poky`` Git repository also has an upstream
255contribution Git repository named ``poky-contrib``. You can see all the
256branches in this repository using the web interface of the
257:yocto_git:`Source Repositories <>` organized within the "Poky Support"
258area. These branches hold changes (commits) to the project that have
259been submitted or committed by the Yocto Project development team and by
260community members who contribute to the project. The maintainer
261determines if the changes are qualified to be moved from the "contrib"
262branches into the "master" branch of the Git repository.
263
264Developers (including contributing community members) create and
265maintain cloned repositories of upstream branches. The cloned
266repositories are local to their development platforms and are used to
267develop changes. When a developer is satisfied with a particular feature
268or change, they "push" the change to the appropriate "contrib"
269repository.
270
271Developers are responsible for keeping their local repository up-to-date
272with whatever upstream branch they are working against. They are also
273responsible for straightening out any conflicts that might arise within
274files that are being worked on simultaneously by more than one person.
275All this work is done locally on the development host before anything is
276pushed to a "contrib" area and examined at the maintainer's level.
277
278A somewhat formal method exists by which developers commit changes and
279push them into the "contrib" area and subsequently request that the
280maintainer include them into an upstream branch. This process is called
281"submitting a patch" or "submitting a change." For information on
282submitting patches and changes, see the
283":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
284section in the Yocto Project Development Tasks Manual.
285
286In summary, a single point of entry exists for changes into a "master"
287or development branch of the Git repository, which is controlled by the
288project's maintainer. And, a set of developers exist who independently
289develop, test, and submit changes to "contrib" areas for the maintainer
290to examine. The maintainer then chooses which changes are going to
291become a permanent part of the project.
292
293.. image:: figures/git-workflow.png
294 :align: center
295
296While each development environment is unique, there are some best
297practices or methods that help development run smoothly. The following
298list describes some of these practices. For more information about Git
299workflows, see the workflow topics in the `Git Community
300Book <http://book.git-scm.com>`__.
301
302- *Make Small Changes:* It is best to keep the changes you commit small
303 as compared to bundling many disparate changes into a single commit.
304 This practice not only keeps things manageable but also allows the
305 maintainer to more easily include or refuse changes.
306
307- *Make Complete Changes:* It is also good practice to leave the
308 repository in a state that allows you to still successfully build
309 your project. In other words, do not commit half of a feature, then
310 add the other half as a separate, later commit. Each commit should
311 take you from one buildable project state to another buildable state.
312
313- *Use Branches Liberally:* It is very easy to create, use, and delete
314 local branches in your working Git repository on the development
315 host. You can name these branches anything you like. It is helpful to
316 give them names associated with the particular feature or change on
317 which you are working. Once you are done with a feature or change and
318 have merged it into your local master branch, simply discard the
319 temporary branch.
320
321- *Merge Changes:* The ``git merge`` command allows you to take the
322 changes from one branch and fold them into another branch. This
323 process is especially helpful when more than a single developer might
324 be working on different parts of the same feature. Merging changes
325 also automatically identifies any collisions or "conflicts" that
326 might happen as a result of the same lines of code being altered by
327 two different developers.
328
329- *Manage Branches:* Because branches are easy to use, you should use a
330 system where branches indicate varying levels of code readiness. For
331 example, you can have a "work" branch to develop in, a "test" branch
332 where the code or change is tested, a "stage" branch where changes
333 are ready to be committed, and so forth. As your project develops,
334 you can merge code across the branches to reflect ever-increasing
335 stable states of the development.
336
337- *Use Push and Pull:* The push-pull workflow is based on the concept
338 of developers "pushing" local commits to a remote repository, which
339 is usually a contribution repository. This workflow is also based on
340 developers "pulling" known states of the project down into their
341 local development repositories. The workflow easily allows you to
342 pull changes submitted by other developers from the upstream
343 repository into your work area ensuring that you have the most recent
344 software on which to develop. The Yocto Project has two scripts named
345 ``create-pull-request`` and ``send-pull-request`` that ship with the
346 release to facilitate this workflow. You can find these scripts in
347 the ``scripts`` folder of the
348 :term:`Source Directory`. For information
349 on how to use these scripts, see the
350 ":ref:`dev-manual/dev-manual-common-tasks:using scripts to push a change upstream and request a pull`"
351 section in the Yocto Project Development Tasks Manual.
352
353- *Patch Workflow:* This workflow allows you to notify the maintainer
354 through an email that you have a change (or patch) you would like
355 considered for the "master" branch of the Git repository. To send
356 this type of change, you format the patch and then send the email
357 using the Git commands ``git format-patch`` and ``git send-email``.
358 For information on how to use these scripts, see the
359 ":ref:`dev-manual/dev-manual-common-tasks:submitting a change to the yocto project`"
360 section in the Yocto Project Development Tasks Manual.
361
362Git
363===
364
365The Yocto Project makes extensive use of Git, which is a free, open
366source distributed version control system. Git supports distributed
367development, non-linear development, and can handle large projects. It
368is best that you have some fundamental understanding of how Git tracks
369projects and how to work with Git if you are going to use the Yocto
370Project for development. This section provides a quick overview of how
371Git works and provides you with a summary of some essential Git
372commands.
373
374.. note::
375
376 - For more information on Git, see
377 http://git-scm.com/documentation.
378
379 - If you need to download Git, it is recommended that you add Git to
380 your system through your distribution's "software store" (e.g. for
381 Ubuntu, use the Ubuntu Software feature). For the Git download
382 page, see http://git-scm.com/download.
383
384 - For information beyond the introductory nature in this section,
385 see the ":ref:`dev-manual/dev-manual-start:locating yocto project source files`"
386 section in the Yocto Project Development Tasks Manual.
387
388Repositories, Tags, and Branches
389--------------------------------
390
391As mentioned briefly in the previous section and also in the "`Git
392Workflows and the Yocto
393Project <#gs-git-workflows-and-the-yocto-project>`__" section, the Yocto
394Project maintains source repositories at :yocto_git:`/`. If you
395look at this web-interface of the repositories, each item is a separate
396Git repository.
397
398Git repositories use branching techniques that track content change (not
399files) within a project (e.g. a new feature or updated documentation).
400Creating a tree-like structure based on project divergence allows for
401excellent historical information over the life of a project. This
402methodology also allows for an environment from which you can do lots of
403local experimentation on projects as you develop changes or new
404features.
405
406A Git repository represents all development efforts for a given project.
407For example, the Git repository ``poky`` contains all changes and
408developments for that repository over the course of its entire life.
409That means that all changes that make up all releases are captured. The
410repository maintains a complete history of changes.
411
412You can create a local copy of any repository by "cloning" it with the
413``git clone`` command. When you clone a Git repository, you end up with
414an identical copy of the repository on your development system. Once you
415have a local copy of a repository, you can take steps to develop
416locally. For examples on how to clone Git repositories, see the
417":ref:`dev-manual/dev-manual-start:locating yocto project source files`"
418section in the Yocto Project Development Tasks Manual.
419
420It is important to understand that Git tracks content change and not
421files. Git uses "branches" to organize different development efforts.
422For example, the ``poky`` repository has several branches that include
423the current "&DISTRO_NAME_NO_CAP;" branch, the "master" branch, and many
424branches for past Yocto Project releases. You can see all the branches
425by going to https://git.yoctoproject.org/cgit.cgi/poky/ and clicking on the
426``[...]`` link beneath the "Branch" heading.
427
428Each of these branches represents a specific area of development. The
429"master" branch represents the current or most recent development. All
430other branches represent offshoots of the "master" branch.
431
432When you create a local copy of a Git repository, the copy has the same
433set of branches as the original. This means you can use Git to create a
434local working area (also called a branch) that tracks a specific
435development branch from the upstream source Git repository. in other
436words, you can define your local Git environment to work on any
437development branch in the repository. To help illustrate, consider the
438following example Git commands:
439::
440
441 $ cd ~
442 $ git clone git://git.yoctoproject.org/poky
443 $ cd poky
444 $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
445
446In the previous example
447after moving to the home directory, the ``git clone`` command creates a
448local copy of the upstream ``poky`` Git repository. By default, Git
449checks out the "master" branch for your work. After changing the working
450directory to the new local repository (i.e. ``poky``), the
451``git checkout`` command creates and checks out a local branch named
452"&DISTRO_NAME_NO_CAP;", which tracks the upstream
453"origin/&DISTRO_NAME_NO_CAP;" branch. Changes you make while in this
454branch would ultimately affect the upstream "&DISTRO_NAME_NO_CAP;" branch
455of the ``poky`` repository.
456
457It is important to understand that when you create and checkout a local
458working branch based on a branch name, your local environment matches
459the "tip" of that particular development branch at the time you created
460your local branch, which could be different from the files in the
461"master" branch of the upstream repository. In other words, creating and
462checking out a local branch based on the "&DISTRO_NAME_NO_CAP;" branch
463name is not the same as checking out the "master" branch in the
464repository. Keep reading to see how you create a local snapshot of a
465Yocto Project Release.
466
467Git uses "tags" to mark specific changes in a repository branch
468structure. Typically, a tag is used to mark a special point such as the
469final change (or commit) before a project is released. You can see the
470tags used with the ``poky`` Git repository by going to
471https://git.yoctoproject.org/cgit.cgi/poky/ and clicking on the ``[...]`` link
472beneath the "Tag" heading.
473
474Some key tags for the ``poky`` repository are ``jethro-14.0.3``,
475``morty-16.0.1``, ``pyro-17.0.0``, and
476``&DISTRO_NAME_NO_CAP;-&POKYVERSION;``. These tags represent Yocto Project
477releases.
478
479When you create a local copy of the Git repository, you also have access
480to all the tags in the upstream repository. Similar to branches, you can
481create and checkout a local working Git branch based on a tag name. When
482you do this, you get a snapshot of the Git repository that reflects the
483state of the files when the change was made associated with that tag.
484The most common use is to checkout a working branch that matches a
485specific Yocto Project release. Here is an example:
486::
487
488 $ cd ~
489 $ git clone git://git.yoctoproject.org/poky
490 $ cd poky
491 $ git fetch --tags
492 $ git checkout tags/rocko-18.0.0 -b my_rocko-18.0.0
493
494In this example, the name
495of the top-level directory of your local Yocto Project repository is
496``poky``. After moving to the ``poky`` directory, the ``git fetch``
497command makes all the upstream tags available locally in your
498repository. Finally, the ``git checkout`` command creates and checks out
499a branch named "my-rocko-18.0.0" that is based on the upstream branch
500whose "HEAD" matches the commit in the repository associated with the
501"rocko-18.0.0" tag. The files in your repository now exactly match that
502particular Yocto Project release as it is tagged in the upstream Git
503repository. It is important to understand that when you create and
504checkout a local working branch based on a tag, your environment matches
505a specific point in time and not the entire development branch (i.e.
506from the "tip" of the branch backwards).
507
508Basic Commands
509--------------
510
511Git has an extensive set of commands that lets you manage changes and
512perform collaboration over the life of a project. Conveniently though,
513you can manage with a small set of basic operations and workflows once
514you understand the basic philosophy behind Git. You do not have to be an
515expert in Git to be functional. A good place to look for instruction on
516a minimal set of Git commands is
517`here <http://git-scm.com/documentation>`__.
518
519The following list of Git commands briefly describes some basic Git
520operations as a way to get started. As with any set of commands, this
521list (in most cases) simply shows the base command and omits the many
522arguments it supports. See the Git documentation for complete
523descriptions and strategies on how to use these commands:
524
525- *git init:* Initializes an empty Git repository. You cannot use
526 Git commands unless you have a ``.git`` repository.
527
528- *git clone:* Creates a local clone of a Git repository that is on
529 equal footing with a fellow developer's Git repository or an upstream
530 repository.
531
532- *git add:* Locally stages updated file contents to the index that
533 Git uses to track changes. You must stage all files that have changed
534 before you can commit them.
535
536- *git commit:* Creates a local "commit" that documents the changes
537 you made. Only changes that have been staged can be committed.
538 Commits are used for historical purposes, for determining if a
539 maintainer of a project will allow the change, and for ultimately
540 pushing the change from your local Git repository into the project's
541 upstream repository.
542
543- *git status:* Reports any modified files that possibly need to be
544 staged and gives you a status of where you stand regarding local
545 commits as compared to the upstream repository.
546
547- *git checkout branch-name:* Changes your local working branch and
548 in this form assumes the local branch already exists. This command is
549 analogous to "cd".
550
551- *git checkout –b working-branch upstream-branch:* Creates and
552 checks out a working branch on your local machine. The local branch
553 tracks the upstream branch. You can use your local branch to isolate
554 your work. It is a good idea to use local branches when adding
555 specific features or changes. Using isolated branches facilitates
556 easy removal of changes if they do not work out.
557
558- *git branch:* Displays the existing local branches associated
559 with your local repository. The branch that you have currently
560 checked out is noted with an asterisk character.
561
562- *git branch -D branch-name:* Deletes an existing local branch.
563 You need to be in a local branch other than the one you are deleting
564 in order to delete branch-name.
565
566- *git pull --rebase:* Retrieves information from an upstream Git
567 repository and places it in your local Git repository. You use this
568 command to make sure you are synchronized with the repository from
569 which you are basing changes (.e.g. the "master" branch). The
570 "--rebase" option ensures that any local commits you have in your
571 branch are preserved at the top of your local branch.
572
573- *git push repo-name local-branch:upstream-branch:* Sends
574 all your committed local changes to the upstream Git repository that
575 your local repository is tracking (e.g. a contribution repository).
576 The maintainer of the project draws from these repositories to merge
577 changes (commits) into the appropriate branch of project's upstream
578 repository.
579
580- *git merge:* Combines or adds changes from one local branch of
581 your repository with another branch. When you create a local Git
582 repository, the default branch is named "master". A typical workflow
583 is to create a temporary branch that is based off "master" that you
584 would use for isolated work. You would make your changes in that
585 isolated branch, stage and commit them locally, switch to the
586 "master" branch, and then use the ``git merge`` command to apply the
587 changes from your isolated branch into the currently checked out
588 branch (e.g. "master"). After the merge is complete and if you are
589 done with working in that isolated branch, you can safely delete the
590 isolated branch.
591
592- *git cherry-pick commits:* Choose and apply specific commits from
593 one branch into another branch. There are times when you might not be
594 able to merge all the changes in one branch with another but need to
595 pick out certain ones.
596
597- *gitk:* Provides a GUI view of the branches and changes in your
598 local Git repository. This command is a good way to graphically see
599 where things have diverged in your local repository.
600
601 .. note::
602
603 You need to install the
604 gitk
605 package on your development system to use this command.
606
607- *git log:* Reports a history of your commits to the repository.
608 This report lists all commits regardless of whether you have pushed
609 them upstream or not.
610
611- *git diff:* Displays line-by-line differences between a local
612 working file and the same file as understood by Git. This command is
613 useful to see what you have changed in any given file.
614
615Licensing
616=========
617
618Because open source projects are open to the public, they have different
619licensing structures in place. License evolution for both Open Source
620and Free Software has an interesting history. If you are interested in
621this history, you can find basic information here:
622
623- `Open source license
624 history <http://en.wikipedia.org/wiki/Open-source_license>`__
625
626- `Free software license
627 history <http://en.wikipedia.org/wiki/Free_software_license>`__
628
629In general, the Yocto Project is broadly licensed under the
630Massachusetts Institute of Technology (MIT) License. MIT licensing
631permits the reuse of software within proprietary software as long as the
632license is distributed with that software. MIT is also compatible with
633the GNU General Public License (GPL). Patches to the Yocto Project
634follow the upstream licensing scheme. You can find information on the
635MIT license
636`here <http://www.opensource.org/licenses/mit-license.php>`__. You can
637find information on the GNU GPL
638`here <http://www.opensource.org/licenses/LGPL-3.0>`__.
639
640When you build an image using the Yocto Project, the build process uses
641a known list of licenses to ensure compliance. You can find this list in
642the :term:`Source Directory` at
643``meta/files/common-licenses``. Once the build completes, the list of
644all licenses found and used during that build are kept in the
645:term:`Build Directory` at
646``tmp/deploy/licenses``.
647
648If a module requires a license that is not in the base list, the build
649process generates a warning during the build. These tools make it easier
650for a developer to be certain of the licenses with which their shipped
651products must comply. However, even with these tools it is still up to
652the developer to resolve potential licensing issues.
653
654The base list of licenses used by the build process is a combination of
655the Software Package Data Exchange (SPDX) list and the Open Source
656Initiative (OSI) projects. `SPDX Group <http://spdx.org>`__ is a working
657group of the Linux Foundation that maintains a specification for a
658standard format for communicating the components, licenses, and
659copyrights associated with a software package.
660`OSI <http://opensource.org>`__ is a corporation dedicated to the Open
661Source Definition and the effort for reviewing and approving licenses
662that conform to the Open Source Definition (OSD).
663
664You can find a list of the combined SPDX and OSI licenses that the Yocto
665Project uses in the ``meta/files/common-licenses`` directory in your
666:term:`Source Directory`.
667
668For information that can help you maintain compliance with various open
669source licensing during the lifecycle of a product created using the
670Yocto Project, see the
671":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
672section in the Yocto Project Development Tasks Manual.
diff --git a/documentation/overview-manual/overview-manual-development-environment.xml b/documentation/overview-manual/overview-manual-development-environment.xml
index 36ebf8a321..08ad071316 100644
--- a/documentation/overview-manual/overview-manual-development-environment.xml
+++ b/documentation/overview-manual/overview-manual-development-environment.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='overview-development-environment'> 6<chapter id='overview-development-environment'>
6<title>The Yocto Project Development Environment</title> 7<title>The Yocto Project Development Environment</title>
@@ -326,7 +327,7 @@
326 For the Yocto Project, a key individual called the "maintainer" is 327 For the Yocto Project, a key individual called the "maintainer" is
327 responsible for the integrity of the "master" branch of a given Git 328 responsible for the integrity of the "master" branch of a given Git
328 repository. 329 repository.
329 The "master" branch is the upstream repository from which final or 330 The "master" branch is the "upstream" repository from which final or
330 most recent builds of a project occur. 331 most recent builds of a project occur.
331 The maintainer is responsible for accepting changes from other 332 The maintainer is responsible for accepting changes from other
332 developers and for organizing the underlying branch structure to 333 developers and for organizing the underlying branch structure to
@@ -371,7 +372,7 @@
371 might arise within files that are being worked on simultaneously by 372 might arise within files that are being worked on simultaneously by
372 more than one person. 373 more than one person.
373 All this work is done locally on the development host before 374 All this work is done locally on the development host before
374 anything is pushed to a "contrib" area and examined at the maintainers 375 anything is pushed to a "contrib" area and examined at the maintainer's
375 level. 376 level.
376 </para> 377 </para>
377 378
@@ -379,7 +380,7 @@
379 A somewhat formal method exists by which developers commit changes 380 A somewhat formal method exists by which developers commit changes
380 and push them into the "contrib" area and subsequently request that 381 and push them into the "contrib" area and subsequently request that
381 the maintainer include them into an upstream branch. 382 the maintainer include them into an upstream branch.
382 This process is called submitting a patch or "submitting a change." 383 This process is called "submitting a patch" or "submitting a change."
383 For information on submitting patches and changes, see the 384 For information on submitting patches and changes, see the
384 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" 385 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
385 section in the Yocto Project Development Tasks Manual. 386 section in the Yocto Project Development Tasks Manual.
@@ -388,7 +389,7 @@
388 <para> 389 <para>
389 In summary, a single point of entry 390 In summary, a single point of entry
390 exists for changes into a "master" or development branch of the 391 exists for changes into a "master" or development branch of the
391 Git repository, which is controlled by the projects maintainer. 392 Git repository, which is controlled by the project's maintainer.
392 And, a set of developers exist who independently develop, test, and 393 And, a set of developers exist who independently develop, test, and
393 submit changes to "contrib" areas for the maintainer to examine. 394 submit changes to "contrib" areas for the maintainer to examine.
394 The maintainer then chooses which changes are going to become a 395 The maintainer then chooses which changes are going to become a
@@ -733,7 +734,7 @@
733 <listitem><para id='git-commands-clone'> 734 <listitem><para id='git-commands-clone'>
734 <emphasis><filename>git clone</filename>:</emphasis> 735 <emphasis><filename>git clone</filename>:</emphasis>
735 Creates a local clone of a Git repository that is on 736 Creates a local clone of a Git repository that is on
736 equal footing with a fellow developers Git repository 737 equal footing with a fellow developer's Git repository
737 or an upstream repository. 738 or an upstream repository.
738 </para></listitem> 739 </para></listitem>
739 <listitem><para> 740 <listitem><para>
@@ -751,7 +752,7 @@
751 Commits are used for historical purposes, for determining 752 Commits are used for historical purposes, for determining
752 if a maintainer of a project will allow the change, 753 if a maintainer of a project will allow the change,
753 and for ultimately pushing the change from your local 754 and for ultimately pushing the change from your local
754 Git repository into the projects upstream repository. 755 Git repository into the project's upstream repository.
755 </para></listitem> 756 </para></listitem>
756 <listitem><para> 757 <listitem><para>
757 <emphasis><filename>git status</filename>:</emphasis> 758 <emphasis><filename>git status</filename>:</emphasis>
diff --git a/documentation/overview-manual/overview-manual-intro.rst b/documentation/overview-manual/overview-manual-intro.rst
new file mode 100644
index 0000000000..3f206fd54b
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-intro.rst
@@ -0,0 +1,74 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************************************
4The Yocto Project Overview and Concepts Manual
5**********************************************
6
7.. _overview-manual-welcome:
8
9Welcome
10=======
11
12Welcome to the Yocto Project Overview and Concepts Manual! This manual
13introduces the Yocto Project by providing concepts, software overviews,
14best-known-methods (BKMs), and any other high-level introductory
15information suitable for a new Yocto Project user.
16
17The following list describes what you can get from this manual:
18
19- `Introducing the Yocto Project <#overview-yp>`__\ *:* This chapter
20 provides an introduction to the Yocto Project. You will learn about
21 features and challenges of the Yocto Project, the layer model,
22 components and tools, development methods, the
23 :term:`Poky` reference distribution, the
24 OpenEmbedded build system workflow, and some basic Yocto terms.
25
26- `The Yocto Project Development
27 Environment <#overview-development-environment>`__\ *:* This chapter
28 helps you get started understanding the Yocto Project development
29 environment. You will learn about open source, development hosts,
30 Yocto Project source repositories, workflows using Git and the Yocto
31 Project, a Git primer, and information about licensing.
32
33- :doc:`overview-manual-concepts` *:* This
34 chapter presents various concepts regarding the Yocto Project. You
35 can find conceptual information about components, development,
36 cross-toolchains, and so forth.
37
38This manual does not give you the following:
39
40- *Step-by-step Instructions for Development Tasks:* Instructional
41 procedures reside in other manuals within the Yocto Project
42 documentation set. For example, the :doc:`../dev-manual/dev-manual`
43 provides examples on how to perform
44 various development tasks. As another example, the
45 :doc:`../sdk-manual/sdk-manual` manual contains detailed
46 instructions on how to install an SDK, which is used to develop
47 applications for target hardware.
48
49- *Reference Material:* This type of material resides in an appropriate
50 reference manual. For example, system variables are documented in the
51 :doc:`../ref-manual/ref-manual`. As another
52 example, the :doc:`../bsp-guide/bsp-guide` contains reference information on
53 BSPs.
54
55- *Detailed Public Information Not Specific to the Yocto Project:* For
56 example, exhaustive information on how to use the Source Control
57 Manager Git is better covered with Internet searches and official Git
58 Documentation than through the Yocto Project documentation.
59
60.. _overview-manual-other-information:
61
62Other Information
63=================
64
65Because this manual presents information for many different topics,
66supplemental information is recommended for full comprehension. For
67additional introductory information on the Yocto Project, see the
68:yocto_home:`Yocto Project Website <>`. If you want to build an image
69with no knowledge of Yocto Project as a way of quickly testing it out,
70see the :doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document.
71For a comprehensive list of links and other documentation, see the
72":ref:`Links and Related
73Documentation <resources-links-and-related-documentation>`"
74section in the Yocto Project Reference Manual.
diff --git a/documentation/overview-manual/overview-manual-intro.xml b/documentation/overview-manual/overview-manual-intro.xml
index 39433aa41b..0e0bfed6e5 100644
--- a/documentation/overview-manual/overview-manual-intro.xml
+++ b/documentation/overview-manual/overview-manual-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='overview-manual-intro'> 6<chapter id='overview-manual-intro'>
6 7
diff --git a/documentation/overview-manual/overview-manual-style.css b/documentation/overview-manual/overview-manual-style.css
index 97a364b125..eec934161a 100644
--- a/documentation/overview-manual/overview-manual-style.css
+++ b/documentation/overview-manual/overview-manual-style.css
@@ -1,4 +1,6 @@
1/* 1/*
2 SPDX-License-Identifier: CC-BY-2.0-UK
3
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 4 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 5
4 Browser wrangling and typographic design by 6 Browser wrangling and typographic design by
diff --git a/documentation/overview-manual/overview-manual-yp-intro.rst b/documentation/overview-manual/overview-manual-yp-intro.rst
new file mode 100644
index 0000000000..5cdab7ca4a
--- /dev/null
+++ b/documentation/overview-manual/overview-manual-yp-intro.rst
@@ -0,0 +1,941 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************
4Introducing the Yocto Project
5*****************************
6
7What is the Yocto Project?
8==========================
9
10The Yocto Project is an open source collaboration project that helps
11developers create custom Linux-based systems that are designed for
12embedded products regardless of the product's hardware architecture.
13Yocto Project provides a flexible toolset and a development environment
14that allows embedded device developers across the world to collaborate
15through shared technologies, software stacks, configurations, and best
16practices used to create these tailored Linux images.
17
18Thousands of developers worldwide have discovered that Yocto Project
19provides advantages in both systems and applications development,
20archival and management benefits, and customizations used for speed,
21footprint, and memory utilization. The project is a standard when it
22comes to delivering embedded software stacks. The project allows
23software customizations and build interchange for multiple hardware
24platforms as well as software stacks that can be maintained and scaled.
25
26.. image:: figures/key-dev-elements.png
27 :align: center
28
29For further introductory information on the Yocto Project, you might be
30interested in this
31`article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__
32by Drew Moseley and in this short introductory
33`video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__.
34
35The remainder of this section overviews advantages and challenges tied
36to the Yocto Project.
37
38.. _gs-features:
39
40Features
41--------
42
43The following list describes features and advantages of the Yocto
44Project:
45
46- *Widely Adopted Across the Industry:* Semiconductor, operating
47 system, software, and service vendors exist whose products and
48 services adopt and support the Yocto Project. For a look at the Yocto
49 Project community and the companies involved with the Yocto Project,
50 see the "COMMUNITY" and "ECOSYSTEM" tabs on the
51 :yocto_home:`Yocto Project <>` home page.
52
53- *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS,
54 AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors
55 create and supply BSPs that support their hardware. If you have
56 custom silicon, you can create a BSP that supports that architecture.
57
58 Aside from lots of architecture support, the Yocto Project fully
59 supports a wide range of device emulation through the Quick EMUlator
60 (QEMU).
61
62- *Images and Code Transfer Easily:* Yocto Project output can easily
63 move between architectures without moving to new development
64 environments. Additionally, if you have used the Yocto Project to
65 create an image or application and you find yourself not able to
66 support it, commercial Linux vendors such as Wind River, Mentor
67 Graphics, Timesys, and ENEA could take it and provide ongoing
68 support. These vendors have offerings that are built using the Yocto
69 Project.
70
71- *Flexibility:* Corporations use the Yocto Project many different
72 ways. One example is to create an internal Linux distribution as a
73 code base the corporation can use across multiple product groups.
74 Through customization and layering, a project group can leverage the
75 base Linux distribution to create a distribution that works for their
76 product needs.
77
78- *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux
79 distribution, you can use the Yocto Project to create exactly what
80 you need for embedded devices. You only add the feature support or
81 packages that you absolutely need for the device. For devices that
82 have display hardware, you can use available system components such
83 as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich
84 user experience. For devices that do not have a display or where you
85 want to use alternative UI frameworks, you can choose to not install
86 these components.
87
88- *Comprehensive Toolchain Capabilities:* Toolchains for supported
89 architectures satisfy most use cases. However, if your hardware
90 supports features that are not part of a standard toolchain, you can
91 easily customize that toolchain through specification of
92 platform-specific tuning parameters. And, should you need to use a
93 third-party toolchain, mechanisms built into the Yocto Project allow
94 for that.
95
96- *Mechanism Rules Over Policy:* Focusing on mechanism rather than
97 policy ensures that you are free to set policies based on the needs
98 of your design instead of adopting decisions enforced by some system
99 software provider.
100
101- *Uses a Layer Model:* The Yocto Project `layer
102 infrastructure <#the-yocto-project-layer-model>`__ groups related
103 functionality into separate bundles. You can incrementally add these
104 grouped functionalities to your project as needed. Using layers to
105 isolate and group functionality reduces project complexity and
106 redundancy, allows you to easily extend the system, make
107 customizations, and keep functionality organized.
108
109- *Supports Partial Builds:* You can build and rebuild individual
110 packages as needed. Yocto Project accomplishes this through its
111 `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being
112 able to build and debug components individually eases project
113 development.
114
115- *Releases According to a Strict Schedule:* Major releases occur on a
116 :doc:`six-month cycle <../ref-manual/ref-release-process>`
117 predictably in October and April. The most recent two releases
118 support point releases to address common vulnerabilities and
119 exposures. This predictability is crucial for projects based on the
120 Yocto Project and allows development teams to plan activities.
121
122- *Rich Ecosystem of Individuals and Organizations:* For open source
123 projects, the value of community is very important. Support forums,
124 expertise, and active developers who continue to push the Yocto
125 Project forward are readily available.
126
127- *Binary Reproducibility:* The Yocto Project allows you to be very
128 specific about dependencies and achieves very high percentages of
129 binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When
130 distributions are not specific about which packages are pulled in and
131 in what order to support dependencies, other build systems can
132 arbitrarily include packages.
133
134- *License Manifest:* The Yocto Project provides a :ref:`license
135 manifest <dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle>`
136 for review by people who need to track the use of open source
137 licenses (e.g. legal teams).
138
139.. _gs-challenges:
140
141Challenges
142----------
143
144The following list presents challenges you might encounter when
145developing using the Yocto Project:
146
147- *Steep Learning Curve:* The Yocto Project has a steep learning curve
148 and has many different ways to accomplish similar tasks. It can be
149 difficult to choose how to proceed when varying methods exist by
150 which to accomplish a given task.
151
152- *Understanding What Changes You Need to Make For Your Design Requires
153 Some Research:* Beyond the simple tutorial stage, understanding what
154 changes need to be made for your particular design can require a
155 significant amount of research and investigation. For information
156 that helps you transition from trying out the Yocto Project to using
157 it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and
158 ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`"
159 documents on the Yocto Project website.
160
161- *Project Workflow Could Be Confusing:* The `Yocto Project
162 workflow <#overview-development-environment>`__ could be confusing if
163 you are used to traditional desktop and server software development.
164 In a desktop development environment, mechanisms exist to easily pull
165 and install new packages, which are typically pre-compiled binaries
166 from servers accessible over the Internet. Using the Yocto Project,
167 you must modify your configuration and rebuild to add additional
168 packages.
169
170- *Working in a Cross-Build Environment Can Feel Unfamiliar:* When
171 developing code to run on a target, compilation, execution, and
172 testing done on the actual target can be faster than running a
173 BitBake build on a development host and then deploying binaries to
174 the target for test. While the Yocto Project does support development
175 tools on the target, the additional step of integrating your changes
176 back into the Yocto Project build environment would be required.
177 Yocto Project supports an intermediate approach that involves making
178 changes on the development system within the BitBake environment and
179 then deploying only the updated packages to the target.
180
181 The Yocto Project :term:`OpenEmbedded Build System`
182 produces packages
183 in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy
184 these packages into the running system on the target by using
185 utilities on the target such as ``rpm`` or ``ipk``.
186
187- *Initial Build Times Can be Significant:* Long initial build times
188 are unfortunately unavoidable due to the large number of packages
189 initially built from scratch for a fully functioning Linux system.
190 Once that initial build is completed, however, the shared-state
191 (sstate) cache mechanism Yocto Project uses keeps the system from
192 rebuilding packages that have not been "touched" since the last
193 build. The sstate mechanism significantly reduces times for
194 successive builds.
195
196The Yocto Project Layer Model
197=============================
198
199The Yocto Project's "Layer Model" is a development model for embedded
200and IoT Linux creation that distinguishes the Yocto Project from other
201simple build systems. The Layer Model simultaneously supports
202collaboration and customization. Layers are repositories that contain
203related sets of instructions that tell the :term:`OpenEmbedded Build System`
204what to do. You can
205collaborate, share, and reuse layers.
206
207Layers can contain changes to previous instructions or settings at any
208time. This powerful override capability is what allows you to customize
209previously supplied collaborative or community layers to suit your
210product requirements.
211
212You use different layers to logically separate information in your
213build. As an example, you could have BSP, GUI, distro configuration,
214middleware, or application layers. Putting your entire build into one
215layer limits and complicates future customization and reuse. Isolating
216information into layers, on the other hand, helps simplify future
217customizations and reuse. You might find it tempting to keep everything
218in one layer when working on a single project. However, the more modular
219your Metadata, the easier it is to cope with future changes.
220
221.. note::
222
223 - Use Board Support Package (BSP) layers from silicon vendors when
224 possible.
225
226 - Familiarize yourself with the `Yocto Project curated layer
227 index <https://www.yoctoproject.org/software-overview/layers/>`__
228 or the `OpenEmbedded layer
229 index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__.
230 The latter contains more layers but they are less universally
231 validated.
232
233 - Layers support the inclusion of technologies, hardware components,
234 and software components. The :ref:`Yocto Project
235 Compatible <dev-manual/dev-manual-common-tasks:making sure your layer is compatible with yocto project>`
236 designation provides a minimum level of standardization that
237 contributes to a strong ecosystem. "YP Compatible" is applied to
238 appropriate products and software components such as BSPs, other
239 OE-compatible layers, and related open-source projects, allowing
240 the producer to use Yocto Project badges and branding assets.
241
242To illustrate how layers are used to keep things modular, consider
243machine customizations. These types of customizations typically reside
244in a special layer, rather than a general layer, called a BSP Layer.
245Furthermore, the machine customizations should be isolated from recipes
246and Metadata that support a new GUI environment, for example. This
247situation gives you a couple of layers: one for the machine
248configurations, and one for the GUI environment. It is important to
249understand, however, that the BSP layer can still make machine-specific
250additions to recipes within the GUI environment layer without polluting
251the GUI layer itself with those machine-specific changes. You can
252accomplish this through a recipe that is a BitBake append
253(``.bbappend``) file, which is described later in this section.
254
255.. note::
256
257 For general information on BSP layer structure, see the
258 :doc:`../bsp-guide/bsp-guide`
259 .
260
261The :term:`Source Directory`
262contains both general layers and BSP layers right out of the box. You
263can easily identify layers that ship with a Yocto Project release in the
264Source Directory by their names. Layers typically have names that begin
265with the string ``meta-``.
266
267.. note::
268
269 It is not a requirement that a layer name begin with the prefix
270 meta-
271 , but it is a commonly accepted standard in the Yocto Project
272 community.
273
274For example, if you were to examine the `tree
275view <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/>`__ of the
276``poky`` repository, you will see several layers: ``meta``,
277``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and
278``meta-yocto-bsp``. Each of these repositories represents a distinct
279layer.
280
281For procedures on how to create layers, see the
282":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
283section in the Yocto Project Development Tasks Manual.
284
285Components and Tools
286====================
287
288The Yocto Project employs a collection of components and tools used by
289the project itself, by project developers, and by those using the Yocto
290Project. These components and tools are open source projects and
291metadata that are separate from the reference distribution
292(:term:`Poky`) and the
293:term:`OpenEmbedded Build System`. Most of the
294components and tools are downloaded separately.
295
296This section provides brief overviews of the components and tools
297associated with the Yocto Project.
298
299.. _gs-development-tools:
300
301Development Tools
302-----------------
303
304The following list consists of tools that help you develop images and
305applications using the Yocto Project:
306
307- *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an
308 open source, cross-platform development framework that leverages
309 `Docker Containers <https://www.docker.com/>`__. CROPS provides an
310 easily managed, extensible environment that allows you to build
311 binaries for a variety of architectures on Windows, Linux and Mac OS
312 X hosts.
313
314- *devtool:* This command-line tool is available as part of the
315 extensible SDK (eSDK) and is its cornerstone. You can use ``devtool``
316 to help build, test, and package software within the eSDK. You can
317 use the tool to optionally integrate what you build into an image
318 built by the OpenEmbedded build system.
319
320 The ``devtool`` command employs a number of sub-commands that allow
321 you to add, modify, and upgrade recipes. As with the OpenEmbedded
322 build system, "recipes" represent software packages within
323 ``devtool``. When you use ``devtool add``, a recipe is automatically
324 created. When you use ``devtool modify``, the specified existing
325 recipe is used in order to determine where to get the source code and
326 how to patch it. In both cases, an environment is set up so that when
327 you build the recipe a source tree that is under your control is used
328 in order to allow you to make changes to the source as desired. By
329 default, both new recipes and the source go into a "workspace"
330 directory under the eSDK. The ``devtool upgrade`` command updates an
331 existing recipe so that you can build it for an updated set of source
332 files.
333
334 You can read about the ``devtool`` workflow in the Yocto Project
335 Application Development and Extensible Software Development Kit
336 (eSDK) Manual in the
337 ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`"
338 section.
339
340- *Extensible Software Development Kit (eSDK):* The eSDK provides a
341 cross-development toolchain and libraries tailored to the contents of
342 a specific image. The eSDK makes it easy to add new applications and
343 libraries to an image, modify the source for an existing component,
344 test changes on the target hardware, and integrate into the rest of
345 the OpenEmbedded build system. The eSDK gives you a toolchain
346 experience supplemented with the powerful set of ``devtool`` commands
347 tailored for the Yocto Project environment.
348
349 For information on the eSDK, see the :doc:`../sdk-manual/sdk-manual` Manual.
350
351- *Toaster:* Toaster is a web interface to the Yocto Project
352 OpenEmbedded build system. Toaster allows you to configure, run, and
353 view information about builds. For information on Toaster, see the
354 :doc:`../toaster-manual/toaster-manual`.
355
356.. _gs-production-tools:
357
358Production Tools
359----------------
360
361The following list consists of tools that help production related
362activities using the Yocto Project:
363
364- *Auto Upgrade Helper:* This utility when used in conjunction with the
365 :term:`OpenEmbedded Build System`
366 (BitBake and
367 OE-Core) automatically generates upgrades for recipes that are based
368 on new versions of the recipes published upstream. See
369 :ref:`dev-manual/dev-manual-common-tasks:using the auto upgrade helper (auh)`
370 for how to set it up.
371
372- *Recipe Reporting System:* The Recipe Reporting System tracks recipe
373 versions available for Yocto Project. The main purpose of the system
374 is to help you manage the recipes you maintain and to offer a dynamic
375 overview of the project. The Recipe Reporting System is built on top
376 of the `OpenEmbedded Layer
377 Index <http://layers.openembedded.org/layerindex/layers/>`__, which
378 is a website that indexes OpenEmbedded-Core layers.
379
380- *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__
381 is a fork of a project originally started by
382 `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking
383 system designed to streamline the process of bringing contributions
384 into a project. The Yocto Project uses Patchwork as an organizational
385 tool to handle patches, which number in the thousands for every
386 release.
387
388- *AutoBuilder:* AutoBuilder is a project that automates build tests
389 and quality assurance (QA). By using the public AutoBuilder, anyone
390 can determine the status of the current "master" branch of Poky.
391
392 .. note::
393
394 AutoBuilder is based on buildbot.
395
396 A goal of the Yocto Project is to lead the open source industry with
397 a project that automates testing and QA procedures. In doing so, the
398 project encourages a development community that publishes QA and test
399 plans, publicly demonstrates QA and test plans, and encourages
400 development of tools that automate and test and QA procedures for the
401 benefit of the development community.
402
403 You can learn more about the AutoBuilder used by the Yocto Project
404 Autobuilder :doc:`here <../test-manual/test-manual-understand-autobuilder>`.
405
406- *Cross-Prelink:* Prelinking is the process of pre-computing the load
407 addresses and link tables generated by the dynamic linker as compared
408 to doing this at runtime. Doing this ahead of time results in
409 performance improvements when the application is launched and reduced
410 memory usage for libraries shared by many applications.
411
412 Historically, cross-prelink is a variant of prelink, which was
413 conceived by `Jakub
414 Jelínek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of
415 years ago. Both prelink and cross-prelink are maintained in the same
416 repository albeit on separate branches. By providing an emulated
417 runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation),
418 the cross-prelink project extends the prelink software's ability to
419 prelink a sysroot environment. Additionally, the cross-prelink
420 software enables the ability to work in sysroot style environments.
421
422 The dynamic linker determines standard load address calculations
423 based on a variety of factors such as mapping addresses, library
424 usage, and library function conflicts. The prelink tool uses this
425 information, from the dynamic linker, to determine unique load
426 addresses for executable and linkable format (ELF) binaries that are
427 shared libraries and dynamically linked. The prelink tool modifies
428 these ELF binaries with the pre-computed information. The result is
429 faster loading and often lower memory consumption because more of the
430 library code can be re-used from shared Copy-On-Write (COW) pages.
431
432 The original upstream prelink project only supports running prelink
433 on the end target device due to the reliance on the target device's
434 dynamic linker. This restriction causes issues when developing a
435 cross-compiled system. The cross-prelink adds a synthesized dynamic
436 loader that runs on the host, thus permitting cross-prelinking
437 without ever having to run on a read-write target filesystem.
438
439- *Pseudo:* Pseudo is the Yocto Project implementation of
440 `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run
441 commands in an environment that seemingly has root privileges.
442
443 During a build, it can be necessary to perform operations that
444 require system administrator privileges. For example, file ownership
445 or permissions might need definition. Pseudo is a tool that you can
446 either use directly or through the environment variable
447 ``LD_PRELOAD``. Either method allows these operations to succeed as
448 if system administrator privileges exist even when they do not.
449
450 You can read more about Pseudo in the "`Fakeroot and
451 Pseudo <#fakeroot-and-pseudo>`__" section.
452
453.. _gs-openembedded-build-system:
454
455Open-Embedded Build System Components
456-------------------------------------
457
458The following list consists of components associated with the
459:term:`OpenEmbedded Build System`:
460
461- *BitBake:* BitBake is a core component of the Yocto Project and is
462 used by the OpenEmbedded build system to build images. While BitBake
463 is key to the build system, BitBake is maintained separately from the
464 Yocto Project.
465
466 BitBake is a generic task execution engine that allows shell and
467 Python tasks to be run efficiently and in parallel while working
468 within complex inter-task dependency constraints. In short, BitBake
469 is a build engine that works through recipes written in a specific
470 format in order to perform sets of tasks.
471
472 You can learn more about BitBake in the :doc:`BitBake User
473 Manual <bitbake:index>`.
474
475- *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of
476 metadata (i.e. recipes, classes, and associated files) used by
477 OpenEmbedded-derived systems, which includes the Yocto Project. The
478 Yocto Project and the OpenEmbedded Project both maintain the
479 OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto
480 Project :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/meta>`.
481
482 Historically, the Yocto Project integrated the OE-Core metadata
483 throughout the Yocto Project source repository reference system
484 (Poky). After Yocto Project Version 1.0, the Yocto Project and
485 OpenEmbedded agreed to work together and share a common core set of
486 metadata (OE-Core), which contained much of the functionality
487 previously found in Poky. This collaboration achieved a long-standing
488 OpenEmbedded objective for having a more tightly controlled and
489 quality-assured core. The results also fit well with the Yocto
490 Project objective of achieving a smaller number of fully featured
491 tools as compared to many different ones.
492
493 Sharing a core set of metadata results in Poky as an integration
494 layer on top of OE-Core. You can see that in this
495 `figure <#yp-key-dev-elements>`__. The Yocto Project combines various
496 components such as BitBake, OE-Core, script "glue", and documentation
497 for its build system.
498
499.. _gs-reference-distribution-poky:
500
501Reference Distribution (Poky)
502-----------------------------
503
504Poky is the Yocto Project reference distribution. It contains the
505:term:`OpenEmbedded Build System`
506(BitBake and OE-Core) as well as a set of metadata to get you started
507building your own distribution. See the
508`figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?"
509section for an illustration that shows Poky and its relationship with
510other parts of the Yocto Project.
511
512To use the Yocto Project tools and components, you can download
513(``clone``) Poky and use it to bootstrap your own distribution.
514
515.. note::
516
517 Poky does not contain binary files. It is a working example of how to
518 build your own custom Linux distribution from source.
519
520You can read more about Poky in the "`Reference Embedded Distribution
521(Poky) <#reference-embedded-distribution>`__" section.
522
523.. _gs-packages-for-finished-targets:
524
525Packages for Finished Targets
526-----------------------------
527
528The following lists components associated with packages for finished
529targets:
530
531- *Matchbox:* Matchbox is an Open Source, base environment for the X
532 Window System running on non-desktop, embedded platforms such as
533 handhelds, set-top boxes, kiosks, and anything else for which screen
534 space, input mechanisms, or system resources are limited.
535
536 Matchbox consists of a number of interchangeable and optional
537 applications that you can tailor to a specific, non-desktop platform
538 to enhance usability in constrained environments.
539
540 You can find the Matchbox source in the Yocto Project
541 :yocto_git:`Source Repositories <>`.
542
543- *Opkg:* Open PacKaGe management (opkg) is a lightweight package
544 management system based on the itsy package (ipkg) management system.
545 Opkg is written in C and resembles Advanced Package Tool (APT) and
546 Debian Package (dpkg) in operation.
547
548 Opkg is intended for use on embedded Linux devices and is used in
549 this capacity in the
550 `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and
551 `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto
552 Project.
553
554 .. note::
555
556 As best it can, opkg maintains backwards compatibility with ipkg
557 and conforms to a subset of Debian's policy manual regarding
558 control files.
559
560 You can find the opkg source in the Yocto Project
561 :yocto_git:`Source Repositories <>`.
562
563.. _gs-archived-components:
564
565Archived Components
566-------------------
567
568The Build Appliance is a virtual machine image that enables you to build
569and boot a custom embedded Linux image with the Yocto Project using a
570non-Linux development system.
571
572Historically, the Build Appliance was the second of three methods by
573which you could use the Yocto Project on a system that was not native to
574Linux.
575
5761. *Hob:* Hob, which is now deprecated and is no longer available since
577 the 2.1 release of the Yocto Project provided a rudimentary,
578 GUI-based interface to the Yocto Project. Toaster has fully replaced
579 Hob.
580
5812. *Build Appliance:* Post Hob, the Build Appliance became available. It
582 was never recommended that you use the Build Appliance as a
583 day-to-day production development environment with the Yocto Project.
584 Build Appliance was useful as a way to try out development in the
585 Yocto Project environment.
586
5873. *CROPS:* The final and best solution available now for developing
588 using the Yocto Project on a system not native to Linux is with
589 `CROPS <#gs-crops-overview>`__.
590
591.. _gs-development-methods:
592
593Development Methods
594===================
595
596The Yocto Project development environment usually involves a
597:term:`Build Host` and target
598hardware. You use the Build Host to build images and develop
599applications, while you use the target hardware to test deployed
600software.
601
602This section provides an introduction to the choices or development
603methods you have when setting up your Build Host. Depending on the your
604particular workflow preference and the type of operating system your
605Build Host runs, several choices exist that allow you to use the Yocto
606Project.
607
608.. note::
609
610 For additional detail about the Yocto Project development
611 environment, see the ":doc:`overview-manual-development-environment`"
612 chapter.
613
614- *Native Linux Host:* By far the best option for a Build Host. A
615 system running Linux as its native operating system allows you to
616 develop software by directly using the
617 :term:`BitBake` tool. You can
618 accomplish all aspects of development from a familiar shell of a
619 supported Linux distribution.
620
621 For information on how to set up a Build Host on a system running
622 Linux as its native operating system, see the
623 ":ref:`dev-manual/dev-manual-start:setting up a native linux host`"
624 section in the Yocto Project Development Tasks Manual.
625
626- *CROss PlatformS (CROPS):* Typically, you use
627 `CROPS <https://github.com/crops/poky-container/>`__, which leverages
628 `Docker Containers <https://www.docker.com/>`__, to set up a Build
629 Host that is not running Linux (e.g. Microsoft Windows or macOS).
630
631 .. note::
632
633 You can, however, use CROPS on a Linux-based system.
634
635 CROPS is an open source, cross-platform development framework that
636 provides an easily managed, extensible environment for building
637 binaries targeted for a variety of architectures on Windows, macOS,
638 or Linux hosts. Once the Build Host is set up using CROPS, you can
639 prepare a shell environment to mimic that of a shell being used on a
640 system natively running Linux.
641
642 For information on how to set up a Build Host with CROPS, see the
643 ":ref:`dev-manual/dev-manual-start:setting up to use cross platforms (crops)`"
644 section in the Yocto Project Development Tasks Manual.
645
646- *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem
647 For Linux v2 to set up a build host using Windows 10.
648
649 .. note::
650
651 The Yocto Project is not compatible with WSLv1, it is compatible
652 but not officially supported nor validated with WSLv2, if you
653 still decide to use WSL please upgrade to WSLv2.
654
655 The Windows Subsystem For Linux allows Windows 10 to run a real Linux
656 kernel inside of a lightweight utility virtual machine (VM) using
657 virtualization technology.
658
659 For information on how to set up a Build Host with WSLv2, see the
660 ":ref:`dev-manual/dev-manual-start:setting up to use windows subsystem for linux (wslv2)`"
661 section in the Yocto Project Development Tasks Manual.
662
663- *Toaster:* Regardless of what your Build Host is running, you can use
664 Toaster to develop software using the Yocto Project. Toaster is a web
665 interface to the Yocto Project's :term:`OpenEmbedded Build System`.
666 The interface
667 enables you to configure and run your builds. Information about
668 builds is collected and stored in a database. You can use Toaster to
669 configure and start builds on multiple remote build servers.
670
671 For information about and how to use Toaster, see the
672 :doc:`../toaster-manual/toaster-manual`.
673
674.. _reference-embedded-distribution:
675
676Reference Embedded Distribution (Poky)
677======================================
678
679"Poky", which is pronounced *Pock*-ee, is the name of the Yocto
680Project's reference distribution or Reference OS Kit. Poky contains the
681:term:`OpenEmbedded Build System`
682(:term:`BitBake` and
683:term:`OpenEmbedded-Core (OE-Core)`) as well as a set
684of :term:`Metadata` to get you started
685building your own distro. In other words, Poky is a base specification
686of the functionality needed for a typical embedded system as well as the
687components from the Yocto Project that allow you to build a distribution
688into a usable binary image.
689
690Poky is a combined repository of BitBake, OpenEmbedded-Core (which is
691found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation
692provided all together and known to work well together. You can view
693these items that make up the Poky repository in the
694:yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/>`.
695
696.. note::
697
698 If you are interested in all the contents of the
699 poky
700 Git repository, see the ":ref:`ref-manual/ref-structure:top-level core components`"
701 section in the Yocto Project Reference Manual.
702
703The following figure illustrates what generally comprises Poky:
704
705.. image:: figures/poky-reference-distribution.png
706 :align: center
707
708- BitBake is a task executor and scheduler that is the heart of the
709 OpenEmbedded build system.
710
711- ``meta-poky``, which is Poky-specific metadata.
712
713- ``meta-yocto-bsp``, which are Yocto Project-specific Board Support
714 Packages (BSPs).
715
716- OpenEmbedded-Core (OE-Core) metadata, which includes shared
717 configurations, global variable definitions, shared classes,
718 packaging, and recipes. Classes define the encapsulation and
719 inheritance of build logic. Recipes are the logical units of software
720 and images to be built.
721
722- Documentation, which contains the Yocto Project source files used to
723 make the set of user manuals.
724
725.. note::
726
727 While Poky is a "complete" distribution specification and is tested
728 and put through QA, you cannot use it as a product "out of the box"
729 in its current form.
730
731To use the Yocto Project tools, you can use Git to clone (download) the
732Poky repository then use your local copy of the reference distribution
733to bootstrap your own distribution.
734
735.. note::
736
737 Poky does not contain binary files. It is a working example of how to
738 build your own custom Linux distribution from source.
739
740Poky has a regular, well established, six-month release cycle under its
741own version. Major releases occur at the same time major releases (point
742releases) occur for the Yocto Project, which are typically in the Spring
743and Fall. For more information on the Yocto Project release schedule and
744cadence, see the ":doc:`../ref-manual/ref-release-process`" chapter in the
745Yocto Project Reference Manual.
746
747Much has been said about Poky being a "default configuration". A default
748configuration provides a starting image footprint. You can use Poky out
749of the box to create an image ranging from a shell-accessible minimal
750image all the way up to a Linux Standard Base-compliant image that uses
751a GNOME Mobile and Embedded (GMAE) based reference user interface called
752Sato.
753
754One of the most powerful properties of Poky is that every aspect of a
755build is controlled by the metadata. You can use metadata to augment
756these base image types by adding metadata
757`layers <#the-yocto-project-layer-model>`__ that extend functionality.
758These layers can provide, for example, an additional software stack for
759an image type, add a board support package (BSP) for additional
760hardware, or even create a new image type.
761
762Metadata is loosely grouped into configuration files or package recipes.
763A recipe is a collection of non-executable metadata used by BitBake to
764set variables or define additional build-time tasks. A recipe contains
765fields such as the recipe description, the recipe version, the license
766of the package and the upstream source repository. A recipe might also
767indicate that the build process uses autotools, make, distutils or any
768other build process, in which case the basic functionality can be
769defined by the classes it inherits from the OE-Core layer's class
770definitions in ``./meta/classes``. Within a recipe you can also define
771additional tasks as well as task prerequisites. Recipe syntax through
772BitBake also supports both ``_prepend`` and ``_append`` operators as a
773method of extending task functionality. These operators inject code into
774the beginning or end of a task. For information on these BitBake
775operators, see the
776":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`"
777section in the BitBake User's Manual.
778
779.. _openembedded-build-system-workflow:
780
781The OpenEmbedded Build System Workflow
782======================================
783
784The :term:`OpenEmbedded Build System` uses a "workflow" to
785accomplish image and SDK generation. The following figure overviews that
786workflow:
787
788.. image:: figures/YP-flow-diagram.png
789 :align: center
790
791Following is a brief summary of the "workflow":
792
7931. Developers specify architecture, policies, patches and configuration
794 details.
795
7962. The build system fetches and downloads the source code from the
797 specified location. The build system supports standard methods such
798 as tarballs or source code repositories systems such as Git.
799
8003. Once source code is downloaded, the build system extracts the sources
801 into a local work area where patches are applied and common steps for
802 configuring and compiling the software are run.
803
8044. The build system then installs the software into a temporary staging
805 area where the binary package format you select (DEB, RPM, or IPK) is
806 used to roll up the software.
807
8085. Different QA and sanity checks run throughout entire build process.
809
8106. After the binaries are created, the build system generates a binary
811 package feed that is used to create the final root file image.
812
8137. The build system generates the file system image and a customized
814 Extensible SDK (eSDK) for application development in parallel.
815
816For a very detailed look at this workflow, see the "`OpenEmbedded Build
817System Concepts <#openembedded-build-system-build-concepts>`__" section.
818
819Some Basic Terms
820================
821
822It helps to understand some basic fundamental terms when learning the
823Yocto Project. Although a list of terms exists in the ":doc:`Yocto Project
824Terms <../ref-manual/ref-terms>`" section of the Yocto Project
825Reference Manual, this section provides the definitions of some terms
826helpful for getting started:
827
828- *Configuration Files:* Files that hold global definitions of
829 variables, user-defined variables, and hardware configuration
830 information. These files tell the :term:`OpenEmbedded Build System`
831 what to build and
832 what to put into the image to support a particular platform.
833
834- *Extensible Software Development Kit (eSDK):* A custom SDK for
835 application developers. This eSDK allows developers to incorporate
836 their library and programming changes back into the image to make
837 their code available to other application developers. For information
838 on the eSDK, see the :doc:`../sdk-manual/sdk-manual` manual.
839
840- *Layer:* A collection of related recipes. Layers allow you to
841 consolidate related metadata to customize your build. Layers also
842 isolate information used when building for multiple architectures.
843 Layers are hierarchical in their ability to override previous
844 specifications. You can include any number of available layers from
845 the Yocto Project and customize the build by adding your layers after
846 them. You can search the Layer Index for layers used within Yocto
847 Project.
848
849 For more detailed information on layers, see the
850 ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
851 section in the Yocto Project Development Tasks Manual. For a
852 discussion specifically on BSP Layers, see the
853 ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto
854 Project Board Support Packages (BSP) Developer's Guide.
855
856- *Metadata:* A key element of the Yocto Project is the Metadata that
857 is used to construct a Linux distribution and is contained in the
858 files that the OpenEmbedded build system parses when building an
859 image. In general, Metadata includes recipes, configuration files,
860 and other information that refers to the build instructions
861 themselves, as well as the data used to control what things get built
862 and the effects of the build. Metadata also includes commands and
863 data used to indicate what versions of software are used, from where
864 they are obtained, and changes or additions to the software itself
865 (patches or auxiliary files) that are used to fix bugs or customize
866 the software for use in a particular situation. OpenEmbedded-Core is
867 an important set of validated metadata.
868
869- *OpenEmbedded Build System:* The terms "BitBake" and "build system"
870 are sometimes used for the OpenEmbedded Build System.
871
872 BitBake is a task scheduler and execution engine that parses
873 instructions (i.e. recipes) and configuration data. After a parsing
874 phase, BitBake creates a dependency tree to order the compilation,
875 schedules the compilation of the included code, and finally executes
876 the building of the specified custom Linux image (distribution).
877 BitBake is similar to the ``make`` tool.
878
879 During a build process, the build system tracks dependencies and
880 performs a native or cross-compilation of the package. As a first
881 step in a cross-build setup, the framework attempts to create a
882 cross-compiler toolchain (i.e. Extensible SDK) suited for the target
883 platform.
884
885- *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of
886 foundation recipes, classes, and associated files that are meant to
887 be common among many different OpenEmbedded-derived systems,
888 including the Yocto Project. OE-Core is a curated subset of an
889 original repository developed by the OpenEmbedded community that has
890 been pared down into a smaller, core set of continuously validated
891 recipes. The result is a tightly controlled and quality-assured core
892 set of recipes.
893
894 You can see the Metadata in the ``meta`` directory of the Yocto
895 Project `Source
896 Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__.
897
898- *Packages:* In the context of the Yocto Project, this term refers to
899 a recipe's packaged output produced by BitBake (i.e. a "baked
900 recipe"). A package is generally the compiled binaries produced from
901 the recipe's sources. You "bake" something by running it through
902 BitBake.
903
904 It is worth noting that the term "package" can, in general, have
905 subtle meanings. For example, the packages referred to in the
906 ":ref:`ref-manual/ref-system-requirements:required packages for the build host`"
907 section in the Yocto Project Reference Manual are compiled binaries
908 that, when installed, add functionality to your Linux distribution.
909
910 Another point worth noting is that historically within the Yocto
911 Project, recipes were referred to as packages - thus, the existence
912 of several BitBake variables that are seemingly mis-named, (e.g.
913 :term:`PR`,
914 :term:`PV`, and
915 :term:`PE`).
916
917- *Poky:* Poky is a reference embedded distribution and a reference
918 test configuration. Poky provides the following:
919
920 - A base-level functional distro used to illustrate how to customize
921 a distribution.
922
923 - A means by which to test the Yocto Project components (i.e. Poky
924 is used to validate the Yocto Project).
925
926 - A vehicle through which you can download the Yocto Project.
927
928 Poky is not a product level distro. Rather, it is a good starting
929 point for customization.
930
931 .. note::
932
933 Poky is an integration layer on top of OE-Core.
934
935- *Recipe:* The most common form of metadata. A recipe contains a list
936 of settings and tasks (i.e. instructions) for building packages that
937 are then used to build the binary image. A recipe describes where you
938 get source code and which patches to apply. Recipes describe
939 dependencies for libraries or for other recipes as well as
940 configuration and compilation options. Related recipes are
941 consolidated into a layer.
diff --git a/documentation/overview-manual/overview-manual-yp-intro.xml b/documentation/overview-manual/overview-manual-yp-intro.xml
index 1b60a30302..a2a1f494bb 100644
--- a/documentation/overview-manual/overview-manual-yp-intro.xml
+++ b/documentation/overview-manual/overview-manual-yp-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='overview-yp'> 6<chapter id='overview-yp'>
6 <title>Introducing the Yocto Project</title> 7 <title>Introducing the Yocto Project</title>
@@ -458,7 +459,7 @@
458 <para>The <filename>devtool</filename> command employs 459 <para>The <filename>devtool</filename> command employs
459 a number of sub-commands that allow you to add, modify, 460 a number of sub-commands that allow you to add, modify,
460 and upgrade recipes. 461 and upgrade recipes.
461 As with the OpenEmbedded build system, recipes 462 As with the OpenEmbedded build system, "recipes"
462 represent software packages within 463 represent software packages within
463 <filename>devtool</filename>. 464 <filename>devtool</filename>.
464 When you use <filename>devtool add</filename>, a recipe 465 When you use <filename>devtool add</filename>, a recipe
@@ -471,7 +472,7 @@
471 control is used in order to allow you to make changes 472 control is used in order to allow you to make changes
472 to the source as desired. 473 to the source as desired.
473 By default, both new recipes and the source go into 474 By default, both new recipes and the source go into
474 a workspace directory under the eSDK. 475 a "workspace" directory under the eSDK.
475 The <filename>devtool upgrade</filename> command 476 The <filename>devtool upgrade</filename> command
476 updates an existing recipe so that you can build it 477 updates an existing recipe so that you can build it
477 for an updated set of source files.</para> 478 for an updated set of source files.</para>
@@ -597,7 +598,7 @@
597 By providing an emulated runtime dynamic linker 598 By providing an emulated runtime dynamic linker
598 (i.e. <filename>glibc</filename>-derived 599 (i.e. <filename>glibc</filename>-derived
599 <filename>ld.so</filename> emulation), the 600 <filename>ld.so</filename> emulation), the
600 cross-prelink project extends the prelink softwares 601 cross-prelink project extends the prelink software's
601 ability to prelink a sysroot environment. 602 ability to prelink a sysroot environment.
602 Additionally, the cross-prelink software enables the 603 Additionally, the cross-prelink software enables the
603 ability to work in sysroot style environments.</para> 604 ability to work in sysroot style environments.</para>
@@ -619,7 +620,7 @@
619 620
620 <para>The original upstream prelink project only 621 <para>The original upstream prelink project only
621 supports running prelink on the end target device 622 supports running prelink on the end target device
622 due to the reliance on the target devices dynamic 623 due to the reliance on the target device's dynamic
623 linker. 624 linker.
624 This restriction causes issues when developing a 625 This restriction causes issues when developing a
625 cross-compiled system. 626 cross-compiled system.
@@ -712,7 +713,7 @@
712 You can see that in this 713 You can see that in this
713 <link linkend='yp-key-dev-elements'>figure</link>. 714 <link linkend='yp-key-dev-elements'>figure</link>.
714 The Yocto Project combines various components such as 715 The Yocto Project combines various components such as
715 BitBake, OE-Core, script glue, and documentation 716 BitBake, OE-Core, script "glue", and documentation
716 for its build system. 717 for its build system.
717 </para></listitem> 718 </para></listitem>
718 </itemizedlist> 719 </itemizedlist>
@@ -790,7 +791,7 @@
790 <note> 791 <note>
791 As best it can, opkg maintains backwards 792 As best it can, opkg maintains backwards
792 compatibility with ipkg and conforms to a subset 793 compatibility with ipkg and conforms to a subset
793 of Debians policy manual regarding control files. 794 of Debian's policy manual regarding control files.
794 </note> 795 </note>
795 </para></listitem> 796 </para></listitem>
796 </itemizedlist> 797 </itemizedlist>
diff --git a/documentation/overview-manual/overview-manual.rst b/documentation/overview-manual/overview-manual.rst
new file mode 100644
index 0000000000..80ce9aae76
--- /dev/null
+++ b/documentation/overview-manual/overview-manual.rst
@@ -0,0 +1,19 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3==========================================
4Yocto Project Overview and Concepts Manual
5==========================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 overview-manual-intro
14 overview-manual-yp-intro
15 overview-manual-development-environment
16 overview-manual-concepts
17 history
18
19.. include:: /boilerplate.rst
diff --git a/documentation/overview-manual/overview-manual.xml b/documentation/overview-manual/overview-manual.xml
index 7c75e5086c..8021a2e95e 100755
--- a/documentation/overview-manual/overview-manual.xml
+++ b/documentation/overview-manual/overview-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='overview-manual' lang='en' 6<book id='overview-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -52,28 +53,8 @@
52 </revision> 53 </revision>
53 <revision> 54 <revision>
54 <revnumber>3.1</revnumber> 55 <revnumber>3.1</revnumber>
55 <date>April 2020</date>
56 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
57 </revision>
58 <revision>
59 <revnumber>3.1.1</revnumber>
60 <date>June 2020</date>
61 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
62 </revision>
63 <revision>
64 <revnumber>3.1.2</revnumber>
65 <date>August 2020</date>
66 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
67 </revision>
68 <revision>
69 <revnumber>3.1.3</revnumber>
70 <date>October 2020</date>
71 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
72 </revision>
73 <revision>
74 <revnumber>3.1.4</revnumber>
75 <date>&REL_MONTH_YEAR;</date> 56 <date>&REL_MONTH_YEAR;</date>
76 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 57 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
77 </revision> 58 </revision>
78 </revhistory> 59 </revhistory>
79 60
diff --git a/documentation/poky.ent b/documentation/poky.ent
index 7c00d3a41b..aec8d455e8 100755
--- a/documentation/poky.ent
+++ b/documentation/poky.ent
@@ -1,17 +1,17 @@
1<!ENTITY DISTRO "3.1.4"> 1<!ENTITY DISTRO "3.1">
2<!ENTITY DISTRO_COMPRESSED "314"> 2<!ENTITY DISTRO_COMPRESSED "31">
3<!ENTITY DISTRO_NAME_NO_CAP "dunfell"> 3<!ENTITY DISTRO_NAME_NO_CAP "dunfell">
4<!ENTITY DISTRO_NAME "Dunfell"> 4<!ENTITY DISTRO_NAME "Dunfell">
5<!ENTITY DISTRO_NAME_NO_CAP_MINUS_ONE "zeus"> 5<!ENTITY DISTRO_NAME_NO_CAP_MINUS_ONE "zeus">
6<!ENTITY DISTRO_NAME_MINUS_ONE "Zeus"> 6<!ENTITY DISTRO_NAME_MINUS_ONE "Zeus">
7<!ENTITY YOCTO_DOC_VERSION "3.1.4"> 7<!ENTITY YOCTO_DOC_VERSION "3.1">
8<!ENTITY YOCTO_DOC_VERSION_MINUS_ONE "3.0.2"> 8<!ENTITY YOCTO_DOC_VERSION_MINUS_ONE "3.0.2">
9<!ENTITY DISTRO_REL_TAG "yocto-3.1.4"> 9<!ENTITY DISTRO_REL_TAG "yocto-3.1">
10<!ENTITY METAINTELVERSION "12.0"> 10<!ENTITY METAINTELVERSION "12.0">
11<!ENTITY REL_MONTH_YEAR "November 2020"> 11<!ENTITY REL_MONTH_YEAR "April 2020">
12<!ENTITY META_INTEL_REL_TAG "&METAINTELVERSION;-&DISTRO_NAME_NO_CAP;-&YOCTO_DOC_VERSION;"> 12<!ENTITY META_INTEL_REL_TAG "&METAINTELVERSION;-&DISTRO_NAME_NO_CAP;-&YOCTO_DOC_VERSION;">
13<!ENTITY POKYVERSION "23.0.4"> 13<!ENTITY POKYVERSION "23.0.0">
14<!ENTITY POKYVERSION_COMPRESSED "2304"> 14<!ENTITY POKYVERSION_COMPRESSED "2300">
15<!ENTITY YOCTO_POKY "poky-&DISTRO_NAME_NO_CAP;-&POKYVERSION;"> 15<!ENTITY YOCTO_POKY "poky-&DISTRO_NAME_NO_CAP;-&POKYVERSION;">
16<!ENTITY COPYRIGHT_YEAR "2010-2020"> 16<!ENTITY COPYRIGHT_YEAR "2010-2020">
17<!ENTITY ORGNAME "The Yocto Project"> 17<!ENTITY ORGNAME "The Yocto Project">
@@ -62,22 +62,22 @@
62<!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo gcc-multilib \ 62<!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo gcc-multilib \
63 build-essential chrpath socat cpio python3 python3-pip python3-pexpect \ 63 build-essential chrpath socat cpio python3 python3-pip python3-pexpect \
64 xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev \ 64 xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev \
65 pylint3 xterm"> 65 pylint3 xterm python3-subunit mesa-common-dev">
66<!ENTITY FEDORA_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python3 unzip perl patch \ 66<!ENTITY FEDORA_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python3 unzip perl patch \
67 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \ 67 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
68 ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue perl-bignum socat \ 68 ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue perl-bignum socat \
69 python3-pexpect findutils which file cpio python python3-pip xz python3-GitPython \ 69 python3-pexpect findutils which file cpio python python3-pip xz python3-GitPython \
70 python3-jinja2 SDL-devel xterm rpcgen"> 70 python3-jinja2 SDL-devel xterm rpcgen mesa-libGL-devel">
71<!ENTITY OPENSUSE_HOST_PACKAGES_ESSENTIAL "python gcc gcc-c++ git chrpath make wget python-xml \ 71<!ENTITY OPENSUSE_HOST_PACKAGES_ESSENTIAL "python gcc gcc-c++ git chrpath make wget python-xml \
72 diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip \ 72 diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip \
73 python3-pexpect xz which python3-Jinja2 Mesa-libEGL1 libSDL-devel xterm rpcgen 73 python3-pexpect xz which python3-Jinja2 Mesa-libEGL1 libSDL-devel xterm rpcgen Mesa-dri-devel
74 $ sudo pip3 install GitPython"> 74 $ sudo pip3 install GitPython">
75<!ENTITY CENTOS7_HOST_PACKAGES_ESSENTIAL "-y epel-release 75<!ENTITY CENTOS7_HOST_PACKAGES_ESSENTIAL "-y epel-release
76 $ sudo yum makecache 76 $ sudo yum makecache
77 $ sudo yum install gawk make wget tar bzip2 gzip python3 unzip perl patch \ 77 $ sudo yum install gawk make wget tar bzip2 gzip python3 unzip perl patch \
78 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \ 78 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
79 perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python36-pip xz \ 79 perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python36-pip xz \
80 which SDL-devel xterm 80 which SDL-devel xterm mesa-libGL-devel
81 $ sudo pip3 install GitPython jinja2"> 81 $ sudo pip3 install GitPython jinja2">
82<!ENTITY CENTOS8_HOST_PACKAGES_ESSENTIAL "-y epel-release 82<!ENTITY CENTOS8_HOST_PACKAGES_ESSENTIAL "-y epel-release
83 $ sudo dnf config-manager --set-enabled PowerTools 83 $ sudo dnf config-manager --set-enabled PowerTools
@@ -86,4 +86,4 @@
86 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath ccache \ 86 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath ccache \
87 socat perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python3-pip \ 87 socat perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python3-pip \
88 python3-GitPython python3-jinja2 python3-pexpect xz which SDL-devel xterm \ 88 python3-GitPython python3-jinja2 python3-pexpect xz which SDL-devel xterm \
89 rpcgen"> 89 rpcgen mesa-libGL-devel">
diff --git a/documentation/poky.yaml b/documentation/poky.yaml
new file mode 100644
index 0000000000..7d544b41aa
--- /dev/null
+++ b/documentation/poky.yaml
@@ -0,0 +1,89 @@
1DISTRO : "3.1"
2DISTRO_COMPRESSED : "31"
3DISTRO_NAME_NO_CAP : "dunfell"
4DISTRO_NAME : "Dunfell"
5DISTRO_NAME_NO_CAP_MINUS_ONE : "zeus"
6DISTRO_NAME_MINUS_ONE : "Zeus"
7YOCTO_DOC_VERSION : "3.1"
8YOCTO_DOC_VERSION_MINUS_ONE : "3.0.2"
9DISTRO_REL_TAG : "yocto-3.1"
10METAINTELVERSION : "12.0"
11REL_MONTH_YEAR : "April 2020"
12META_INTEL_REL_TAG : "&METAINTELVERSION;-&DISTRO_NAME_NO_CAP;-&YOCTO_DOC_VERSION;"
13POKYVERSION : "23.0.0"
14POKYVERSION_COMPRESSED : "2300"
15YOCTO_POKY : "poky-&DISTRO_NAME_NO_CAP;-&POKYVERSION;"
16COPYRIGHT_YEAR : "2010-2020"
17ORGNAME : "The Yocto Project"
18ORGEMAIL : "docs@lists.yoctoproject.org"
19YOCTO_DL_URL : "http://downloads.yoctoproject.org"
20YOCTO_HOME_URL : "http://www.yoctoproject.org"
21YOCTO_LISTS_URL : "http://lists.yoctoproject.org"
22YOCTO_BUGZILLA_URL : "http://bugzilla.yoctoproject.org"
23YOCTO_WIKI_URL : "https://wiki.yoctoproject.org"
24YOCTO_AB_URL : "http://autobuilder.yoctoproject.org"
25YOCTO_GIT_URL : "http://git.yoctoproject.org"
26YOCTO_ADTREPO_URL : "http://adtrepo.yoctoproject.org"
27OE_HOME_URL : "http://www.openembedded.org"
28OE_LISTS_URL : "http://lists.openembedded.org/mailman"
29OE_DOCS_URL : "http://docs.openembedded.org"
30OH_HOME_URL : "http://o-hand.com"
31BITBAKE_HOME_URL : "http://developer.berlios.de/projects/bitbake/"
32YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
33YOCTO_SOURCES_URL : "&YOCTO_HOME_URL;/sources/"
34YOCTO_AB_PORT_URL : "https://autobuilder.yocto.io/"
35YOCTO_AB_NIGHTLY_URL : "&YOCTO_AB_PORT_URL;/pub/nightly/"
36YOCTO_POKY_URL : "&YOCTO_DL_URL;/releases/poky/"
37YOCTO_RELEASE_DL_URL : "&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;"
38YOCTO_TOOLCHAIN_DL_URL : "&YOCTO_RELEASE_DL_URL;/toolchain/"
39YOCTO_ADTINSTALLER_DL_URL : "&YOCTO_RELEASE_DL_URL;/adt-installer"
40YOCTO_POKY_DL_URL : "&YOCTO_RELEASE_DL_URL;/&YOCTO_POKY;.tar.bz2"
41YOCTO_MACHINES_DL_URL : "&YOCTO_RELEASE_DL_URL;/machines"
42YOCTO_QEMU_DL_URL : "&YOCTO_MACHINES_DL_URL;/qemu"
43YOCTO_PYTHON-i686_DL_URL : "&YOCTO_DL_URL;/releases/miscsupport/python-nativesdk-standalone-i686.tar.bz2"
44YOCTO_PYTHON-x86_64_DL_URL : "&YOCTO_DL_URL;/releases/miscsupport/python-nativesdk-standalone-x86_64.tar.bz2"
45YOCTO_DOCS_QS_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/yocto-project-qs/yocto-project-qs.html"
46YOCTO_DOCS_ADT_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/adt-manual/adt-manual.html"
47YOCTO_DOCS_REF_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/ref-manual/ref-manual.html"
48YOCTO_DOCS_BSP_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/bsp-guide/bsp-guide.html"
49YOCTO_DOCS_DEV_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/dev-manual/dev-manual.html"
50YOCTO_DOCS_KERNEL_DEV_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/kernel-dev/kernel-dev.html"
51YOCTO_DOCS_PROF_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/profile-manual/profile-manual.html"
52YOCTO_DOCS_MM_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/mega-manual/mega-manual.html"
53YOCTO_DOCS_BB_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/bitbake-user-manual/bitbake-user-manual.html"
54YOCTO_DOCS_TOAST_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/toaster-manual/toaster-manual.html"
55YOCTO_DOCS_SDK_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/sdk-manual/sdk-manual.html"
56YOCTO_DOCS_OM_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/overview-manual/overview-manual.html"
57YOCTO_DOCS_BRIEF_URL : "&YOCTO_DOCS_URL;/&YOCTO_DOC_VERSION;/brief-yoctoprojectqs/brief-yoctoprojectqs.html"
58YOCTO_ADTPATH_DIR : "/opt/poky/&DISTRO;"
59YOCTO_POKY_TARBALL : "&YOCTO_POKY;.tar.bz2"
60OE_INIT_PATH : "&YOCTO_POKY;/oe-init-build-env"
61OE_INIT_FILE : "oe-init-build-env"
62UBUNTU_HOST_PACKAGES_ESSENTIAL : "gawk wget git-core diffstat unzip texinfo gcc-multilib \
63 build-essential chrpath socat cpio python3 python3-pip python3-pexpect \
64 xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev \
65 pylint3 xterm python3-subunit mesa-common-dev"
66FEDORA_HOST_PACKAGES_ESSENTIAL : "gawk make wget tar bzip2 gzip python3 unzip perl patch \
67 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
68 ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue perl-bignum socat \
69 python3-pexpect findutils which file cpio python python3-pip xz python3-GitPython \
70 python3-jinja2 SDL-devel xterm rpcgen mesa-libGL-devel"
71OPENSUSE_HOST_PACKAGES_ESSENTIAL : "python gcc gcc-c++ git chrpath make wget python-xml \
72 diffstat makeinfo python-curses patch socat python3 python3-curses tar python3-pip \
73 python3-pexpect xz which python3-Jinja2 Mesa-libEGL1 libSDL-devel xterm rpcgen Mesa-dri-devel
74 $ sudo pip3 install GitPython"
75CENTOS7_HOST_PACKAGES_ESSENTIAL : "-y epel-release
76 $ sudo yum makecache
77 $ sudo yum install gawk make wget tar bzip2 gzip python3 unzip perl patch \
78 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat \
79 perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python36-pip xz \
80 which SDL-devel xterm mesa-libGL-devel
81 $ sudo pip3 install GitPython jinja2"
82CENTOS8_HOST_PACKAGES_ESSENTIAL : "-y epel-release
83 $ sudo dnf config-manager --set-enabled PowerTools
84 $ sudo dnf makecache
85 $ sudo dnf install gawk make wget tar bzip2 gzip python3 unzip perl patch \
86 diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath ccache \
87 socat perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python3-pip \
88 python3-GitPython python3-jinja2 python3-pexpect xz which SDL-devel xterm \
89 rpcgen mesa-libGL-devel"
diff --git a/documentation/profile-manual/history.rst b/documentation/profile-manual/history.rst
new file mode 100644
index 0000000000..3ffb7eacb8
--- /dev/null
+++ b/documentation/profile-manual/history.rst
@@ -0,0 +1,58 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 1.4
15 - April 2013
16 - The initial document released with the Yocto Project 1.4 Release
17 * - 1.5
18 - October 2013
19 - Released with the Yocto Project 1.5 Release.
20 * - 1.6
21 - April 2014
22 - Released with the Yocto Project 1.6 Release.
23 * - 1.7
24 - October 2014
25 - Released with the Yocto Project 1.7 Release.
26 * - 1.8
27 - April 2015
28 - Released with the Yocto Project 1.8 Release.
29 * - 2.0
30 - October 2015
31 - Released with the Yocto Project 2.0 Release.
32 * - 2.1
33 - April 2016
34 - Released with the Yocto Project 2.1 Release.
35 * - 2.2
36 - October 2016
37 - Released with the Yocto Project 2.2 Release.
38 * - 2.3
39 - May 2017
40 - Released with the Yocto Project 2.3 Release.
41 * - 2.4
42 - October 2017
43 - Released with the Yocto Project 2.4 Release.
44 * - 2.5
45 - May 2018
46 - Released with the Yocto Project 2.5 Release.
47 * - 2.6
48 - November 2018
49 - Released with the Yocto Project 2.6 Release.
50 * - 2.7
51 - May 2019
52 - Released with the Yocto Project 2.7 Release.
53 * - 3.0
54 - October 2019
55 - Released with the Yocto Project 3.0 Release.
56 * - 3.1
57 - April 2020
58 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/profile-manual/profile-manual-arch.rst b/documentation/profile-manual/profile-manual-arch.rst
new file mode 100644
index 0000000000..9e1e400e42
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-arch.rst
@@ -0,0 +1,29 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*************************************************************
4Overall Architecture of the Linux Tracing and Profiling Tools
5*************************************************************
6
7Architecture of the Tracing and Profiling Tools
8===============================================
9
10It may seem surprising to see a section covering an 'overall
11architecture' for what seems to be a random collection of tracing tools
12that together make up the Linux tracing and profiling space. The fact
13is, however, that in recent years this seemingly disparate set of tools
14has started to converge on a 'core' set of underlying mechanisms:
15
16- static tracepoints
17- dynamic tracepoints
18
19 - kprobes
20 - uprobes
21
22- the perf_events subsystem
23- debugfs
24
25.. admonition:: Tying it Together
26
27 Rather than enumerating here how each tool makes use of these common
28 mechanisms, textboxes like this will make note of the specific usages
29 in each tool as they come up in the course of the text.
diff --git a/documentation/profile-manual/profile-manual-arch.xml b/documentation/profile-manual/profile-manual-arch.xml
index 19d1155229..8eb7bbfabd 100644
--- a/documentation/profile-manual/profile-manual-arch.xml
+++ b/documentation/profile-manual/profile-manual-arch.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='profile-manual-arch'> 6<chapter id='profile-manual-arch'>
6 7
diff --git a/documentation/profile-manual/profile-manual-customization.xsl b/documentation/profile-manual/profile-manual-customization.xsl
index caa57ef342..d995e0b3cb 100644
--- a/documentation/profile-manual/profile-manual-customization.xsl
+++ b/documentation/profile-manual/profile-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/profile-manual/profile-manual-examples.rst b/documentation/profile-manual/profile-manual-examples.rst
new file mode 100644
index 0000000000..32ccd37b88
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-examples.rst
@@ -0,0 +1,24 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************
4Real-World Examples
5*******************
6
7|
8
9This chapter contains real-world examples.
10
11Slow Write Speed on Live Images
12===============================
13
14In one of our previous releases (denzil), users noticed that booting off
15of a live image and writing to disk was noticeably slower. This included
16the boot itself, especially the first one, since first boots tend to do
17a significant amount of writing due to certain post-install scripts.
18
19The problem (and solution) was discovered by using the Yocto tracing
20tools, in this case 'perf stat', 'perf script', 'perf record' and 'perf
21report'.
22
23See all the unvarnished details of how this bug was diagnosed and solved
24here: Yocto Bug #3049
diff --git a/documentation/profile-manual/profile-manual-examples.xml b/documentation/profile-manual/profile-manual-examples.xml
index 9630c6c307..91e06fcd1c 100644
--- a/documentation/profile-manual/profile-manual-examples.xml
+++ b/documentation/profile-manual/profile-manual-examples.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='profile-manual-examples'> 6<chapter id='profile-manual-examples'>
6 7
diff --git a/documentation/profile-manual/profile-manual-intro.rst b/documentation/profile-manual/profile-manual-intro.rst
new file mode 100644
index 0000000000..994b1c5086
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-intro.rst
@@ -0,0 +1,79 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************************************
4Yocto Project Profiling and Tracing Manual
5******************************************
6
7.. _profile-intro:
8
9Introduction
10============
11
12Yocto bundles a number of tracing and profiling tools - this 'HOWTO'
13describes their basic usage and shows by example how to make use of them
14to examine application and system behavior.
15
16The tools presented are for the most part completely open-ended and have
17quite good and/or extensive documentation of their own which can be used
18to solve just about any problem you might come across in Linux. Each
19section that describes a particular tool has links to that tool's
20documentation and website.
21
22The purpose of this 'HOWTO' is to present a set of common and generally
23useful tracing and profiling idioms along with their application (as
24appropriate) to each tool, in the context of a general-purpose
25'drill-down' methodology that can be applied to solving a large number
26(90%?) of problems. For help with more advanced usages and problems,
27please see the documentation and/or websites listed for each tool.
28
29The final section of this 'HOWTO' is a collection of real-world examples
30which we'll be continually adding to as we solve more problems using the
31tools - feel free to add your own examples to the list!
32
33.. _profile-manual-general-setup:
34
35General Setup
36=============
37
38Most of the tools are available only in 'sdk' images or in images built
39after adding 'tools-profile' to your local.conf. So, in order to be able
40to access all of the tools described here, please first build and boot
41an 'sdk' image e.g. ::
42
43 $ bitbake core-image-sato-sdk
44
45or alternatively by adding 'tools-profile' to the EXTRA_IMAGE_FEATURES line in
46your local.conf: ::
47
48 EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile"
49
50If you use the 'tools-profile' method, you don't need to build an sdk image -
51the tracing and profiling tools will be included in non-sdk images as well e.g.: ::
52
53 $ bitbake core-image-sato
54
55.. note::
56
57 By default, the Yocto build system strips symbols from the binaries
58 it packages, which makes it difficult to use some of the tools.
59
60 You can prevent that by setting the
61 :term:`INHIBIT_PACKAGE_STRIP`
62 variable to "1" in your ``local.conf`` when you build the image: ::
63
64 INHIBIT_PACKAGE_STRIP = "1"
65
66 The above setting will noticeably increase the size of your image.
67
68If you've already built a stripped image, you can generate debug
69packages (xxx-dbg) which you can manually install as needed.
70
71To generate debug info for packages, you can add dbg-pkgs to
72EXTRA_IMAGE_FEATURES in local.conf. For example: ::
73
74 EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs"
75
76Additionally, in order to generate the right type of debuginfo, we also need to
77set :term:`PACKAGE_DEBUG_SPLIT_STYLE` in the ``local.conf`` file: ::
78
79 PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory'
diff --git a/documentation/profile-manual/profile-manual-intro.xml b/documentation/profile-manual/profile-manual-intro.xml
index f16db3f0f2..a2d2f80ec0 100644
--- a/documentation/profile-manual/profile-manual-intro.xml
+++ b/documentation/profile-manual/profile-manual-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='profile-manual-intro'> 6<chapter id='profile-manual-intro'>
6 7
diff --git a/documentation/profile-manual/profile-manual-style.css b/documentation/profile-manual/profile-manual-style.css
index f3cca8536d..8502c11090 100644
--- a/documentation/profile-manual/profile-manual-style.css
+++ b/documentation/profile-manual/profile-manual-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/profile-manual/profile-manual-usage.rst b/documentation/profile-manual/profile-manual-usage.rst
new file mode 100644
index 0000000000..32b04f6ff7
--- /dev/null
+++ b/documentation/profile-manual/profile-manual-usage.rst
@@ -0,0 +1,2624 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2.. highlight:: shell
3
4***************************************************************
5Basic Usage (with examples) for each of the Yocto Tracing Tools
6***************************************************************
7
8|
9
10This chapter presents basic usage examples for each of the tracing
11tools.
12
13.. _profile-manual-perf:
14
15perf
16====
17
18The 'perf' tool is the profiling and tracing tool that comes bundled
19with the Linux kernel.
20
21Don't let the fact that it's part of the kernel fool you into thinking
22that it's only for tracing and profiling the kernel - you can indeed use
23it to trace and profile just the kernel, but you can also use it to
24profile specific applications separately (with or without kernel
25context), and you can also use it to trace and profile the kernel and
26all applications on the system simultaneously to gain a system-wide view
27of what's going on.
28
29In many ways, perf aims to be a superset of all the tracing and
30profiling tools available in Linux today, including all the other tools
31covered in this HOWTO. The past couple of years have seen perf subsume a
32lot of the functionality of those other tools and, at the same time,
33those other tools have removed large portions of their previous
34functionality and replaced it with calls to the equivalent functionality
35now implemented by the perf subsystem. Extrapolation suggests that at
36some point those other tools will simply become completely redundant and
37go away; until then, we'll cover those other tools in these pages and in
38many cases show how the same things can be accomplished in perf and the
39other tools when it seems useful to do so.
40
41The coverage below details some of the most common ways you'll likely
42want to apply the tool; full documentation can be found either within
43the tool itself or in the man pages at
44`perf(1) <http://linux.die.net/man/1/perf>`__.
45
46.. _perf-setup:
47
48Perf Setup
49----------
50
51For this section, we'll assume you've already performed the basic setup
52outlined in the ":ref:`profile-manual/profile-manual-intro:General Setup`" section.
53
54In particular, you'll get the most mileage out of perf if you profile an
55image built with the following in your ``local.conf`` file: ::
56
57 INHIBIT_PACKAGE_STRIP = "1"
58
59perf runs on the target system for the most part. You can archive
60profile data and copy it to the host for analysis, but for the rest of
61this document we assume you've ssh'ed to the host and will be running
62the perf commands on the target.
63
64.. _perf-basic-usage:
65
66Basic Perf Usage
67----------------
68
69The perf tool is pretty much self-documenting. To remind yourself of the
70available commands, simply type 'perf', which will show you basic usage
71along with the available perf subcommands: ::
72
73 root@crownbay:~# perf
74
75 usage: perf [--version] [--help] COMMAND [ARGS]
76
77 The most commonly used perf commands are:
78 annotate Read perf.data (created by perf record) and display annotated code
79 archive Create archive with object files with build-ids found in perf.data file
80 bench General framework for benchmark suites
81 buildid-cache Manage build-id cache.
82 buildid-list List the buildids in a perf.data file
83 diff Read two perf.data files and display the differential profile
84 evlist List the event names in a perf.data file
85 inject Filter to augment the events stream with additional information
86 kmem Tool to trace/measure kernel memory(slab) properties
87 kvm Tool to trace/measure kvm guest os
88 list List all symbolic event types
89 lock Analyze lock events
90 probe Define new dynamic tracepoints
91 record Run a command and record its profile into perf.data
92 report Read perf.data (created by perf record) and display the profile
93 sched Tool to trace/measure scheduler properties (latencies)
94 script Read perf.data (created by perf record) and display trace output
95 stat Run a command and gather performance counter statistics
96 test Runs sanity tests.
97 timechart Tool to visualize total system behavior during a workload
98 top System profiling tool.
99
100 See 'perf help COMMAND' for more information on a specific command.
101
102
103Using perf to do Basic Profiling
104~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
105
106As a simple test case, we'll profile the 'wget' of a fairly large file,
107which is a minimally interesting case because it has both file and
108network I/O aspects, and at least in the case of standard Yocto images,
109it's implemented as part of busybox, so the methods we use to analyze it
110can be used in a very similar way to the whole host of supported busybox
111applets in Yocto. ::
112
113 root@crownbay:~# rm linux-2.6.19.2.tar.bz2; \
114 wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
115
116The quickest and easiest way to get some basic overall data about what's
117going on for a particular workload is to profile it using 'perf stat'.
118'perf stat' basically profiles using a few default counters and displays
119the summed counts at the end of the run: ::
120
121 root@crownbay:~# perf stat wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
122 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
123 linux-2.6.19.2.tar.b 100% |***************************************************| 41727k 0:00:00 ETA
124
125 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
126
127 4597.223902 task-clock # 0.077 CPUs utilized
128 23568 context-switches # 0.005 M/sec
129 68 CPU-migrations # 0.015 K/sec
130 241 page-faults # 0.052 K/sec
131 3045817293 cycles # 0.663 GHz
132 <not supported> stalled-cycles-frontend
133 <not supported> stalled-cycles-backend
134 858909167 instructions # 0.28 insns per cycle
135 165441165 branches # 35.987 M/sec
136 19550329 branch-misses # 11.82% of all branches
137
138 59.836627620 seconds time elapsed
139
140Many times such a simple-minded test doesn't yield much of
141interest, but sometimes it does (see Real-world Yocto bug (slow
142loop-mounted write speed)).
143
144Also, note that 'perf stat' isn't restricted to a fixed set of counters
145- basically any event listed in the output of 'perf list' can be tallied
146by 'perf stat'. For example, suppose we wanted to see a summary of all
147the events related to kernel memory allocation/freeing along with cache
148hits and misses: ::
149
150 root@crownbay:~# perf stat -e kmem:* -e cache-references -e cache-misses wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
151 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
152 linux-2.6.19.2.tar.b 100% |***************************************************| 41727k 0:00:00 ETA
153
154 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
155
156 5566 kmem:kmalloc
157 125517 kmem:kmem_cache_alloc
158 0 kmem:kmalloc_node
159 0 kmem:kmem_cache_alloc_node
160 34401 kmem:kfree
161 69920 kmem:kmem_cache_free
162 133 kmem:mm_page_free
163 41 kmem:mm_page_free_batched
164 11502 kmem:mm_page_alloc
165 11375 kmem:mm_page_alloc_zone_locked
166 0 kmem:mm_page_pcpu_drain
167 0 kmem:mm_page_alloc_extfrag
168 66848602 cache-references
169 2917740 cache-misses # 4.365 % of all cache refs
170
171 44.831023415 seconds time elapsed
172
173So 'perf stat' gives us a nice easy
174way to get a quick overview of what might be happening for a set of
175events, but normally we'd need a little more detail in order to
176understand what's going on in a way that we can act on in a useful way.
177
178To dive down into a next level of detail, we can use 'perf record'/'perf
179report' which will collect profiling data and present it to use using an
180interactive text-based UI (or simply as text if we specify --stdio to
181'perf report').
182
183As our first attempt at profiling this workload, we'll simply run 'perf
184record', handing it the workload we want to profile (everything after
185'perf record' and any perf options we hand it - here none - will be
186executed in a new shell). perf collects samples until the process exits
187and records them in a file named 'perf.data' in the current working
188directory. ::
189
190 root@crownbay:~# perf record wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
191
192 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
193 linux-2.6.19.2.tar.b 100% |************************************************| 41727k 0:00:00 ETA
194 [ perf record: Woken up 1 times to write data ]
195 [ perf record: Captured and wrote 0.176 MB perf.data (~7700 samples) ]
196
197To see the results in a
198'text-based UI' (tui), simply run 'perf report', which will read the
199perf.data file in the current working directory and display the results
200in an interactive UI: ::
201
202 root@crownbay:~# perf report
203
204.. image:: figures/perf-wget-flat-stripped.png
205 :align: center
206
207The above screenshot displays a 'flat' profile, one entry for each
208'bucket' corresponding to the functions that were profiled during the
209profiling run, ordered from the most popular to the least (perf has
210options to sort in various orders and keys as well as display entries
211only above a certain threshold and so on - see the perf documentation
212for details). Note that this includes both userspace functions (entries
213containing a [.]) and kernel functions accounted to the process (entries
214containing a [k]). (perf has command-line modifiers that can be used to
215restrict the profiling to kernel or userspace, among others).
216
217Notice also that the above report shows an entry for 'busybox', which is
218the executable that implements 'wget' in Yocto, but that instead of a
219useful function name in that entry, it displays a not-so-friendly hex
220value instead. The steps below will show how to fix that problem.
221
222Before we do that, however, let's try running a different profile, one
223which shows something a little more interesting. The only difference
224between the new profile and the previous one is that we'll add the -g
225option, which will record not just the address of a sampled function,
226but the entire callchain to the sampled function as well: ::
227
228 root@crownbay:~# perf record -g wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
229 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
230 linux-2.6.19.2.tar.b 100% |************************************************| 41727k 0:00:00 ETA
231 [ perf record: Woken up 3 times to write data ]
232 [ perf record: Captured and wrote 0.652 MB perf.data (~28476 samples) ]
233
234
235 root@crownbay:~# perf report
236
237.. image:: figures/perf-wget-g-copy-to-user-expanded-stripped.png
238 :align: center
239
240Using the callgraph view, we can actually see not only which functions
241took the most time, but we can also see a summary of how those functions
242were called and learn something about how the program interacts with the
243kernel in the process.
244
245Notice that each entry in the above screenshot now contains a '+' on the
246left-hand side. This means that we can expand the entry and drill down
247into the callchains that feed into that entry. Pressing 'enter' on any
248one of them will expand the callchain (you can also press 'E' to expand
249them all at the same time or 'C' to collapse them all).
250
251In the screenshot above, we've toggled the ``__copy_to_user_ll()`` entry
252and several subnodes all the way down. This lets us see which callchains
253contributed to the profiled ``__copy_to_user_ll()`` function which
254contributed 1.77% to the total profile.
255
256As a bit of background explanation for these callchains, think about
257what happens at a high level when you run wget to get a file out on the
258network. Basically what happens is that the data comes into the kernel
259via the network connection (socket) and is passed to the userspace
260program 'wget' (which is actually a part of busybox, but that's not
261important for now), which takes the buffers the kernel passes to it and
262writes it to a disk file to save it.
263
264The part of this process that we're looking at in the above call stacks
265is the part where the kernel passes the data it's read from the socket
266down to wget i.e. a copy-to-user.
267
268Notice also that here there's also a case where the hex value is
269displayed in the callstack, here in the expanded ``sys_clock_gettime()``
270function. Later we'll see it resolve to a userspace function call in
271busybox.
272
273.. image:: figures/perf-wget-g-copy-from-user-expanded-stripped.png
274 :align: center
275
276The above screenshot shows the other half of the journey for the data -
277from the wget program's userspace buffers to disk. To get the buffers to
278disk, the wget program issues a ``write(2)``, which does a ``copy-from-user`` to
279the kernel, which then takes care via some circuitous path (probably
280also present somewhere in the profile data), to get it safely to disk.
281
282Now that we've seen the basic layout of the profile data and the basics
283of how to extract useful information out of it, let's get back to the
284task at hand and see if we can get some basic idea about where the time
285is spent in the program we're profiling, wget. Remember that wget is
286actually implemented as an applet in busybox, so while the process name
287is 'wget', the executable we're actually interested in is busybox. So
288let's expand the first entry containing busybox:
289
290.. image:: figures/perf-wget-busybox-expanded-stripped.png
291 :align: center
292
293Again, before we expanded we saw that the function was labeled with a
294hex value instead of a symbol as with most of the kernel entries.
295Expanding the busybox entry doesn't make it any better.
296
297The problem is that perf can't find the symbol information for the
298busybox binary, which is actually stripped out by the Yocto build
299system.
300
301One way around that is to put the following in your ``local.conf`` file
302when you build the image: ::
303
304 INHIBIT_PACKAGE_STRIP = "1"
305
306However, we already have an image with the binaries stripped, so
307what can we do to get perf to resolve the symbols? Basically we need to
308install the debuginfo for the busybox package.
309
310To generate the debug info for the packages in the image, we can add
311``dbg-pkgs`` to :term:`EXTRA_IMAGE_FEATURES` in ``local.conf``. For example: ::
312
313 EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs"
314
315Additionally, in order to generate the type of debuginfo that perf
316understands, we also need to set
317:term:`PACKAGE_DEBUG_SPLIT_STYLE`
318in the ``local.conf`` file: ::
319
320 PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory'
321
322Once we've done that, we can install the
323debuginfo for busybox. The debug packages once built can be found in
324``build/tmp/deploy/rpm/*`` on the host system. Find the busybox-dbg-...rpm
325file and copy it to the target. For example: ::
326
327 [trz@empanada core2]$ scp /home/trz/yocto/crownbay-tracing-dbg/build/tmp/deploy/rpm/core2_32/busybox-dbg-1.20.2-r2.core2_32.rpm root@192.168.1.31:
328 busybox-dbg-1.20.2-r2.core2_32.rpm 100% 1826KB 1.8MB/s 00:01
329
330Now install the debug rpm on the target: ::
331
332 root@crownbay:~# rpm -i busybox-dbg-1.20.2-r2.core2_32.rpm
333
334Now that the debuginfo is installed, we see that the busybox entries now display
335their functions symbolically:
336
337.. image:: figures/perf-wget-busybox-debuginfo.png
338 :align: center
339
340If we expand one of the entries and press 'enter' on a leaf node, we're
341presented with a menu of actions we can take to get more information
342related to that entry:
343
344.. image:: figures/perf-wget-busybox-dso-zoom-menu.png
345 :align: center
346
347One of these actions allows us to show a view that displays a
348busybox-centric view of the profiled functions (in this case we've also
349expanded all the nodes using the 'E' key):
350
351.. image:: figures/perf-wget-busybox-dso-zoom.png
352 :align: center
353
354Finally, we can see that now that the busybox debuginfo is installed,
355the previously unresolved symbol in the ``sys_clock_gettime()`` entry
356mentioned previously is now resolved, and shows that the
357sys_clock_gettime system call that was the source of 6.75% of the
358copy-to-user overhead was initiated by the ``handle_input()`` busybox
359function:
360
361.. image:: figures/perf-wget-g-copy-to-user-expanded-debuginfo.png
362 :align: center
363
364At the lowest level of detail, we can dive down to the assembly level
365and see which instructions caused the most overhead in a function.
366Pressing 'enter' on the 'udhcpc_main' function, we're again presented
367with a menu:
368
369.. image:: figures/perf-wget-busybox-annotate-menu.png
370 :align: center
371
372Selecting 'Annotate udhcpc_main', we get a detailed listing of
373percentages by instruction for the udhcpc_main function. From the
374display, we can see that over 50% of the time spent in this function is
375taken up by a couple tests and the move of a constant (1) to a register:
376
377.. image:: figures/perf-wget-busybox-annotate-udhcpc.png
378 :align: center
379
380As a segue into tracing, let's try another profile using a different
381counter, something other than the default 'cycles'.
382
383The tracing and profiling infrastructure in Linux has become unified in
384a way that allows us to use the same tool with a completely different
385set of counters, not just the standard hardware counters that
386traditional tools have had to restrict themselves to (of course the
387traditional tools can also make use of the expanded possibilities now
388available to them, and in some cases have, as mentioned previously).
389
390We can get a list of the available events that can be used to profile a
391workload via 'perf list': ::
392
393 root@crownbay:~# perf list
394
395 List of pre-defined events (to be used in -e):
396 cpu-cycles OR cycles [Hardware event]
397 stalled-cycles-frontend OR idle-cycles-frontend [Hardware event]
398 stalled-cycles-backend OR idle-cycles-backend [Hardware event]
399 instructions [Hardware event]
400 cache-references [Hardware event]
401 cache-misses [Hardware event]
402 branch-instructions OR branches [Hardware event]
403 branch-misses [Hardware event]
404 bus-cycles [Hardware event]
405 ref-cycles [Hardware event]
406
407 cpu-clock [Software event]
408 task-clock [Software event]
409 page-faults OR faults [Software event]
410 minor-faults [Software event]
411 major-faults [Software event]
412 context-switches OR cs [Software event]
413 cpu-migrations OR migrations [Software event]
414 alignment-faults [Software event]
415 emulation-faults [Software event]
416
417 L1-dcache-loads [Hardware cache event]
418 L1-dcache-load-misses [Hardware cache event]
419 L1-dcache-prefetch-misses [Hardware cache event]
420 L1-icache-loads [Hardware cache event]
421 L1-icache-load-misses [Hardware cache event]
422 .
423 .
424 .
425 rNNN [Raw hardware event descriptor]
426 cpu/t1=v1[,t2=v2,t3 ...]/modifier [Raw hardware event descriptor]
427 (see 'perf list --help' on how to encode it)
428
429 mem:<addr>[:access] [Hardware breakpoint]
430
431 sunrpc:rpc_call_status [Tracepoint event]
432 sunrpc:rpc_bind_status [Tracepoint event]
433 sunrpc:rpc_connect_status [Tracepoint event]
434 sunrpc:rpc_task_begin [Tracepoint event]
435 skb:kfree_skb [Tracepoint event]
436 skb:consume_skb [Tracepoint event]
437 skb:skb_copy_datagram_iovec [Tracepoint event]
438 net:net_dev_xmit [Tracepoint event]
439 net:net_dev_queue [Tracepoint event]
440 net:netif_receive_skb [Tracepoint event]
441 net:netif_rx [Tracepoint event]
442 napi:napi_poll [Tracepoint event]
443 sock:sock_rcvqueue_full [Tracepoint event]
444 sock:sock_exceed_buf_limit [Tracepoint event]
445 udp:udp_fail_queue_rcv_skb [Tracepoint event]
446 hda:hda_send_cmd [Tracepoint event]
447 hda:hda_get_response [Tracepoint event]
448 hda:hda_bus_reset [Tracepoint event]
449 scsi:scsi_dispatch_cmd_start [Tracepoint event]
450 scsi:scsi_dispatch_cmd_error [Tracepoint event]
451 scsi:scsi_eh_wakeup [Tracepoint event]
452 drm:drm_vblank_event [Tracepoint event]
453 drm:drm_vblank_event_queued [Tracepoint event]
454 drm:drm_vblank_event_delivered [Tracepoint event]
455 random:mix_pool_bytes [Tracepoint event]
456 random:mix_pool_bytes_nolock [Tracepoint event]
457 random:credit_entropy_bits [Tracepoint event]
458 gpio:gpio_direction [Tracepoint event]
459 gpio:gpio_value [Tracepoint event]
460 block:block_rq_abort [Tracepoint event]
461 block:block_rq_requeue [Tracepoint event]
462 block:block_rq_issue [Tracepoint event]
463 block:block_bio_bounce [Tracepoint event]
464 block:block_bio_complete [Tracepoint event]
465 block:block_bio_backmerge [Tracepoint event]
466 .
467 .
468 writeback:writeback_wake_thread [Tracepoint event]
469 writeback:writeback_wake_forker_thread [Tracepoint event]
470 writeback:writeback_bdi_register [Tracepoint event]
471 .
472 .
473 writeback:writeback_single_inode_requeue [Tracepoint event]
474 writeback:writeback_single_inode [Tracepoint event]
475 kmem:kmalloc [Tracepoint event]
476 kmem:kmem_cache_alloc [Tracepoint event]
477 kmem:mm_page_alloc [Tracepoint event]
478 kmem:mm_page_alloc_zone_locked [Tracepoint event]
479 kmem:mm_page_pcpu_drain [Tracepoint event]
480 kmem:mm_page_alloc_extfrag [Tracepoint event]
481 vmscan:mm_vmscan_kswapd_sleep [Tracepoint event]
482 vmscan:mm_vmscan_kswapd_wake [Tracepoint event]
483 vmscan:mm_vmscan_wakeup_kswapd [Tracepoint event]
484 vmscan:mm_vmscan_direct_reclaim_begin [Tracepoint event]
485 .
486 .
487 module:module_get [Tracepoint event]
488 module:module_put [Tracepoint event]
489 module:module_request [Tracepoint event]
490 sched:sched_kthread_stop [Tracepoint event]
491 sched:sched_wakeup [Tracepoint event]
492 sched:sched_wakeup_new [Tracepoint event]
493 sched:sched_process_fork [Tracepoint event]
494 sched:sched_process_exec [Tracepoint event]
495 sched:sched_stat_runtime [Tracepoint event]
496 rcu:rcu_utilization [Tracepoint event]
497 workqueue:workqueue_queue_work [Tracepoint event]
498 workqueue:workqueue_execute_end [Tracepoint event]
499 signal:signal_generate [Tracepoint event]
500 signal:signal_deliver [Tracepoint event]
501 timer:timer_init [Tracepoint event]
502 timer:timer_start [Tracepoint event]
503 timer:hrtimer_cancel [Tracepoint event]
504 timer:itimer_state [Tracepoint event]
505 timer:itimer_expire [Tracepoint event]
506 irq:irq_handler_entry [Tracepoint event]
507 irq:irq_handler_exit [Tracepoint event]
508 irq:softirq_entry [Tracepoint event]
509 irq:softirq_exit [Tracepoint event]
510 irq:softirq_raise [Tracepoint event]
511 printk:console [Tracepoint event]
512 task:task_newtask [Tracepoint event]
513 task:task_rename [Tracepoint event]
514 syscalls:sys_enter_socketcall [Tracepoint event]
515 syscalls:sys_exit_socketcall [Tracepoint event]
516 .
517 .
518 .
519 syscalls:sys_enter_unshare [Tracepoint event]
520 syscalls:sys_exit_unshare [Tracepoint event]
521 raw_syscalls:sys_enter [Tracepoint event]
522 raw_syscalls:sys_exit [Tracepoint event]
523
524.. admonition:: Tying it Together
525
526 These are exactly the same set of events defined by the trace event
527 subsystem and exposed by ftrace/tracecmd/kernelshark as files in
528 /sys/kernel/debug/tracing/events, by SystemTap as
529 kernel.trace("tracepoint_name") and (partially) accessed by LTTng.
530
531Only a subset of these would be of interest to us when looking at this
532workload, so let's choose the most likely subsystems (identified by the
533string before the colon in the Tracepoint events) and do a 'perf stat'
534run using only those wildcarded subsystems: ::
535
536 root@crownbay:~# perf stat -e skb:* -e net:* -e napi:* -e sched:* -e workqueue:* -e irq:* -e syscalls:* wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
537 Performance counter stats for 'wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2':
538
539 23323 skb:kfree_skb
540 0 skb:consume_skb
541 49897 skb:skb_copy_datagram_iovec
542 6217 net:net_dev_xmit
543 6217 net:net_dev_queue
544 7962 net:netif_receive_skb
545 2 net:netif_rx
546 8340 napi:napi_poll
547 0 sched:sched_kthread_stop
548 0 sched:sched_kthread_stop_ret
549 3749 sched:sched_wakeup
550 0 sched:sched_wakeup_new
551 0 sched:sched_switch
552 29 sched:sched_migrate_task
553 0 sched:sched_process_free
554 1 sched:sched_process_exit
555 0 sched:sched_wait_task
556 0 sched:sched_process_wait
557 0 sched:sched_process_fork
558 1 sched:sched_process_exec
559 0 sched:sched_stat_wait
560 2106519415641 sched:sched_stat_sleep
561 0 sched:sched_stat_iowait
562 147453613 sched:sched_stat_blocked
563 12903026955 sched:sched_stat_runtime
564 0 sched:sched_pi_setprio
565 3574 workqueue:workqueue_queue_work
566 3574 workqueue:workqueue_activate_work
567 0 workqueue:workqueue_execute_start
568 0 workqueue:workqueue_execute_end
569 16631 irq:irq_handler_entry
570 16631 irq:irq_handler_exit
571 28521 irq:softirq_entry
572 28521 irq:softirq_exit
573 28728 irq:softirq_raise
574 1 syscalls:sys_enter_sendmmsg
575 1 syscalls:sys_exit_sendmmsg
576 0 syscalls:sys_enter_recvmmsg
577 0 syscalls:sys_exit_recvmmsg
578 14 syscalls:sys_enter_socketcall
579 14 syscalls:sys_exit_socketcall
580 .
581 .
582 .
583 16965 syscalls:sys_enter_read
584 16965 syscalls:sys_exit_read
585 12854 syscalls:sys_enter_write
586 12854 syscalls:sys_exit_write
587 .
588 .
589 .
590
591 58.029710972 seconds time elapsed
592
593
594
595Let's pick one of these tracepoints
596and tell perf to do a profile using it as the sampling event: ::
597
598 root@crownbay:~# perf record -g -e sched:sched_wakeup wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
599
600.. image:: figures/sched-wakeup-profile.png
601 :align: center
602
603The screenshot above shows the results of running a profile using
604sched:sched_switch tracepoint, which shows the relative costs of various
605paths to sched_wakeup (note that sched_wakeup is the name of the
606tracepoint - it's actually defined just inside ttwu_do_wakeup(), which
607accounts for the function name actually displayed in the profile:
608
609.. code-block:: c
610
611 /*
612 * Mark the task runnable and perform wakeup-preemption.
613 */
614 static void
615 ttwu_do_wakeup(struct rq *rq, struct task_struct *p, int wake_flags)
616 {
617 trace_sched_wakeup(p, true);
618 .
619 .
620 .
621 }
622
623A couple of the more interesting
624callchains are expanded and displayed above, basically some network
625receive paths that presumably end up waking up wget (busybox) when
626network data is ready.
627
628Note that because tracepoints are normally used for tracing, the default
629sampling period for tracepoints is 1 i.e. for tracepoints perf will
630sample on every event occurrence (this can be changed using the -c
631option). This is in contrast to hardware counters such as for example
632the default 'cycles' hardware counter used for normal profiling, where
633sampling periods are much higher (in the thousands) because profiling
634should have as low an overhead as possible and sampling on every cycle
635would be prohibitively expensive.
636
637Using perf to do Basic Tracing
638~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
639
640Profiling is a great tool for solving many problems or for getting a
641high-level view of what's going on with a workload or across the system.
642It is however by definition an approximation, as suggested by the most
643prominent word associated with it, 'sampling'. On the one hand, it
644allows a representative picture of what's going on in the system to be
645cheaply taken, but on the other hand, that cheapness limits its utility
646when that data suggests a need to 'dive down' more deeply to discover
647what's really going on. In such cases, the only way to see what's really
648going on is to be able to look at (or summarize more intelligently) the
649individual steps that go into the higher-level behavior exposed by the
650coarse-grained profiling data.
651
652As a concrete example, we can trace all the events we think might be
653applicable to our workload: ::
654
655 root@crownbay:~# perf record -g -e skb:* -e net:* -e napi:* -e sched:sched_switch -e sched:sched_wakeup -e irq:*
656 -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write
657 wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
658
659We can look at the raw trace output using 'perf script' with no
660arguments: ::
661
662 root@crownbay:~# perf script
663
664 perf 1262 [000] 11624.857082: sys_exit_read: 0x0
665 perf 1262 [000] 11624.857193: sched_wakeup: comm=migration/0 pid=6 prio=0 success=1 target_cpu=000
666 wget 1262 [001] 11624.858021: softirq_raise: vec=1 [action=TIMER]
667 wget 1262 [001] 11624.858074: softirq_entry: vec=1 [action=TIMER]
668 wget 1262 [001] 11624.858081: softirq_exit: vec=1 [action=TIMER]
669 wget 1262 [001] 11624.858166: sys_enter_read: fd: 0x0003, buf: 0xbf82c940, count: 0x0200
670 wget 1262 [001] 11624.858177: sys_exit_read: 0x200
671 wget 1262 [001] 11624.858878: kfree_skb: skbaddr=0xeb248d80 protocol=0 location=0xc15a5308
672 wget 1262 [001] 11624.858945: kfree_skb: skbaddr=0xeb248000 protocol=0 location=0xc15a5308
673 wget 1262 [001] 11624.859020: softirq_raise: vec=1 [action=TIMER]
674 wget 1262 [001] 11624.859076: softirq_entry: vec=1 [action=TIMER]
675 wget 1262 [001] 11624.859083: softirq_exit: vec=1 [action=TIMER]
676 wget 1262 [001] 11624.859167: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
677 wget 1262 [001] 11624.859192: sys_exit_read: 0x1d7
678 wget 1262 [001] 11624.859228: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
679 wget 1262 [001] 11624.859233: sys_exit_read: 0x0
680 wget 1262 [001] 11624.859573: sys_enter_read: fd: 0x0003, buf: 0xbf82c580, count: 0x0200
681 wget 1262 [001] 11624.859584: sys_exit_read: 0x200
682 wget 1262 [001] 11624.859864: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
683 wget 1262 [001] 11624.859888: sys_exit_read: 0x400
684 wget 1262 [001] 11624.859935: sys_enter_read: fd: 0x0003, buf: 0xb7720000, count: 0x0400
685 wget 1262 [001] 11624.859944: sys_exit_read: 0x400
686
687This gives us a detailed timestamped sequence of events that occurred within the
688workload with respect to those events.
689
690In many ways, profiling can be viewed as a subset of tracing -
691theoretically, if you have a set of trace events that's sufficient to
692capture all the important aspects of a workload, you can derive any of
693the results or views that a profiling run can.
694
695Another aspect of traditional profiling is that while powerful in many
696ways, it's limited by the granularity of the underlying data. Profiling
697tools offer various ways of sorting and presenting the sample data,
698which make it much more useful and amenable to user experimentation, but
699in the end it can't be used in an open-ended way to extract data that
700just isn't present as a consequence of the fact that conceptually, most
701of it has been thrown away.
702
703Full-blown detailed tracing data does however offer the opportunity to
704manipulate and present the information collected during a tracing run in
705an infinite variety of ways.
706
707Another way to look at it is that there are only so many ways that the
708'primitive' counters can be used on their own to generate interesting
709output; to get anything more complicated than simple counts requires
710some amount of additional logic, which is typically very specific to the
711problem at hand. For example, if we wanted to make use of a 'counter'
712that maps to the value of the time difference between when a process was
713scheduled to run on a processor and the time it actually ran, we
714wouldn't expect such a counter to exist on its own, but we could derive
715one called say 'wakeup_latency' and use it to extract a useful view of
716that metric from trace data. Likewise, we really can't figure out from
717standard profiling tools how much data every process on the system reads
718and writes, along with how many of those reads and writes fail
719completely. If we have sufficient trace data, however, we could with the
720right tools easily extract and present that information, but we'd need
721something other than pre-canned profiling tools to do that.
722
723Luckily, there is a general-purpose way to handle such needs, called
724'programming languages'. Making programming languages easily available
725to apply to such problems given the specific format of data is called a
726'programming language binding' for that data and language. Perf supports
727two programming language bindings, one for Python and one for Perl.
728
729.. admonition:: Tying it Together
730
731 Language bindings for manipulating and aggregating trace data are of
732 course not a new idea. One of the first projects to do this was IBM's
733 DProbes dpcc compiler, an ANSI C compiler which targeted a low-level
734 assembly language running on an in-kernel interpreter on the target
735 system. This is exactly analogous to what Sun's DTrace did, except
736 that DTrace invented its own language for the purpose. Systemtap,
737 heavily inspired by DTrace, also created its own one-off language,
738 but rather than running the product on an in-kernel interpreter,
739 created an elaborate compiler-based machinery to translate its
740 language into kernel modules written in C.
741
742Now that we have the trace data in perf.data, we can use 'perf script
743-g' to generate a skeleton script with handlers for the read/write
744entry/exit events we recorded: ::
745
746 root@crownbay:~# perf script -g python
747 generated Python script: perf-script.py
748
749The skeleton script simply creates a python function for each event type in the
750perf.data file. The body of each function simply prints the event name along
751with its parameters. For example:
752
753.. code-block:: python
754
755 def net__netif_rx(event_name, context, common_cpu,
756 common_secs, common_nsecs, common_pid, common_comm,
757 skbaddr, len, name):
758 print_header(event_name, common_cpu, common_secs, common_nsecs,
759 common_pid, common_comm)
760
761 print "skbaddr=%u, len=%u, name=%s\n" % (skbaddr, len, name),
762
763We can run that script directly to print all of the events contained in the
764perf.data file: ::
765
766 root@crownbay:~# perf script -s perf-script.py
767
768 in trace_begin
769 syscalls__sys_exit_read 0 11624.857082795 1262 perf nr=3, ret=0
770 sched__sched_wakeup 0 11624.857193498 1262 perf comm=migration/0, pid=6, prio=0, success=1, target_cpu=0
771 irq__softirq_raise 1 11624.858021635 1262 wget vec=TIMER
772 irq__softirq_entry 1 11624.858074075 1262 wget vec=TIMER
773 irq__softirq_exit 1 11624.858081389 1262 wget vec=TIMER
774 syscalls__sys_enter_read 1 11624.858166434 1262 wget nr=3, fd=3, buf=3213019456, count=512
775 syscalls__sys_exit_read 1 11624.858177924 1262 wget nr=3, ret=512
776 skb__kfree_skb 1 11624.858878188 1262 wget skbaddr=3945041280, location=3243922184, protocol=0
777 skb__kfree_skb 1 11624.858945608 1262 wget skbaddr=3945037824, location=3243922184, protocol=0
778 irq__softirq_raise 1 11624.859020942 1262 wget vec=TIMER
779 irq__softirq_entry 1 11624.859076935 1262 wget vec=TIMER
780 irq__softirq_exit 1 11624.859083469 1262 wget vec=TIMER
781 syscalls__sys_enter_read 1 11624.859167565 1262 wget nr=3, fd=3, buf=3077701632, count=1024
782 syscalls__sys_exit_read 1 11624.859192533 1262 wget nr=3, ret=471
783 syscalls__sys_enter_read 1 11624.859228072 1262 wget nr=3, fd=3, buf=3077701632, count=1024
784 syscalls__sys_exit_read 1 11624.859233707 1262 wget nr=3, ret=0
785 syscalls__sys_enter_read 1 11624.859573008 1262 wget nr=3, fd=3, buf=3213018496, count=512
786 syscalls__sys_exit_read 1 11624.859584818 1262 wget nr=3, ret=512
787 syscalls__sys_enter_read 1 11624.859864562 1262 wget nr=3, fd=3, buf=3077701632, count=1024
788 syscalls__sys_exit_read 1 11624.859888770 1262 wget nr=3, ret=1024
789 syscalls__sys_enter_read 1 11624.859935140 1262 wget nr=3, fd=3, buf=3077701632, count=1024
790 syscalls__sys_exit_read 1 11624.859944032 1262 wget nr=3, ret=1024
791
792That in itself isn't very useful; after all, we can accomplish pretty much the
793same thing by simply running 'perf script' without arguments in the same
794directory as the perf.data file.
795
796We can however replace the print statements in the generated function
797bodies with whatever we want, and thereby make it infinitely more
798useful.
799
800As a simple example, let's just replace the print statements in the
801function bodies with a simple function that does nothing but increment a
802per-event count. When the program is run against a perf.data file, each
803time a particular event is encountered, a tally is incremented for that
804event. For example:
805
806.. code-block:: python
807
808 def net__netif_rx(event_name, context, common_cpu,
809 common_secs, common_nsecs, common_pid, common_comm,
810 skbaddr, len, name):
811 inc_counts(event_name)
812
813Each event handler function in the generated code
814is modified to do this. For convenience, we define a common function
815called inc_counts() that each handler calls; inc_counts() simply tallies
816a count for each event using the 'counts' hash, which is a specialized
817hash function that does Perl-like autovivification, a capability that's
818extremely useful for kinds of multi-level aggregation commonly used in
819processing traces (see perf's documentation on the Python language
820binding for details):
821
822.. code-block:: python
823
824 counts = autodict()
825
826 def inc_counts(event_name):
827 try:
828 counts[event_name] += 1
829 except TypeError:
830 counts[event_name] = 1
831
832Finally, at the end of the trace processing run, we want to print the
833result of all the per-event tallies. For that, we use the special
834'trace_end()' function:
835
836.. code-block:: python
837
838 def trace_end():
839 for event_name, count in counts.iteritems():
840 print "%-40s %10s\n" % (event_name, count)
841
842The end result is a summary of all the events recorded in the trace: ::
843
844 skb__skb_copy_datagram_iovec 13148
845 irq__softirq_entry 4796
846 irq__irq_handler_exit 3805
847 irq__softirq_exit 4795
848 syscalls__sys_enter_write 8990
849 net__net_dev_xmit 652
850 skb__kfree_skb 4047
851 sched__sched_wakeup 1155
852 irq__irq_handler_entry 3804
853 irq__softirq_raise 4799
854 net__net_dev_queue 652
855 syscalls__sys_enter_read 17599
856 net__netif_receive_skb 1743
857 syscalls__sys_exit_read 17598
858 net__netif_rx 2
859 napi__napi_poll 1877
860 syscalls__sys_exit_write 8990
861
862Note that this is
863pretty much exactly the same information we get from 'perf stat', which
864goes a little way to support the idea mentioned previously that given
865the right kind of trace data, higher-level profiling-type summaries can
866be derived from it.
867
868Documentation on using the `'perf script' python
869binding <http://linux.die.net/man/1/perf-script-python>`__.
870
871System-Wide Tracing and Profiling
872~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
873
874The examples so far have focused on tracing a particular program or
875workload - in other words, every profiling run has specified the program
876to profile in the command-line e.g. 'perf record wget ...'.
877
878It's also possible, and more interesting in many cases, to run a
879system-wide profile or trace while running the workload in a separate
880shell.
881
882To do system-wide profiling or tracing, you typically use the -a flag to
883'perf record'.
884
885To demonstrate this, open up one window and start the profile using the
886-a flag (press Ctrl-C to stop tracing): ::
887
888 root@crownbay:~# perf record -g -a
889 ^C[ perf record: Woken up 6 times to write data ]
890 [ perf record: Captured and wrote 1.400 MB perf.data (~61172 samples) ]
891
892In another window, run the wget test: ::
893
894 root@crownbay:~# wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2
895 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
896 linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA
897
898Here we see entries not only for our wget load, but for
899other processes running on the system as well:
900
901.. image:: figures/perf-systemwide.png
902 :align: center
903
904In the snapshot above, we can see callchains that originate in libc, and
905a callchain from Xorg that demonstrates that we're using a proprietary X
906driver in userspace (notice the presence of 'PVR' and some other
907unresolvable symbols in the expanded Xorg callchain).
908
909Note also that we have both kernel and userspace entries in the above
910snapshot. We can also tell perf to focus on userspace but providing a
911modifier, in this case 'u', to the 'cycles' hardware counter when we
912record a profile: ::
913
914 root@crownbay:~# perf record -g -a -e cycles:u
915 ^C[ perf record: Woken up 2 times to write data ]
916 [ perf record: Captured and wrote 0.376 MB perf.data (~16443 samples) ]
917
918.. image:: figures/perf-report-cycles-u.png
919 :align: center
920
921Notice in the screenshot above, we see only userspace entries ([.])
922
923Finally, we can press 'enter' on a leaf node and select the 'Zoom into
924DSO' menu item to show only entries associated with a specific DSO. In
925the screenshot below, we've zoomed into the 'libc' DSO which shows all
926the entries associated with the libc-xxx.so DSO.
927
928.. image:: figures/perf-systemwide-libc.png
929 :align: center
930
931We can also use the system-wide -a switch to do system-wide tracing.
932Here we'll trace a couple of scheduler events: ::
933
934 root@crownbay:~# perf record -a -e sched:sched_switch -e sched:sched_wakeup
935 ^C[ perf record: Woken up 38 times to write data ]
936 [ perf record: Captured and wrote 9.780 MB perf.data (~427299 samples) ]
937
938We can look at the raw output using 'perf script' with no arguments: ::
939
940 root@crownbay:~# perf script
941
942 perf 1383 [001] 6171.460045: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
943 perf 1383 [001] 6171.460066: sched_switch: prev_comm=perf prev_pid=1383 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1 next_pid=21 next_prio=120
944 kworker/1:1 21 [001] 6171.460093: sched_switch: prev_comm=kworker/1:1 prev_pid=21 prev_prio=120 prev_state=S ==> next_comm=perf next_pid=1383 next_prio=120
945 swapper 0 [000] 6171.468063: sched_wakeup: comm=kworker/0:3 pid=1209 prio=120 success=1 target_cpu=000
946 swapper 0 [000] 6171.468107: sched_switch: prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=kworker/0:3 next_pid=1209 next_prio=120
947 kworker/0:3 1209 [000] 6171.468143: sched_switch: prev_comm=kworker/0:3 prev_pid=1209 prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0 next_prio=120
948 perf 1383 [001] 6171.470039: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
949 perf 1383 [001] 6171.470058: sched_switch: prev_comm=perf prev_pid=1383 prev_prio=120 prev_state=R+ ==> next_comm=kworker/1:1 next_pid=21 next_prio=120
950 kworker/1:1 21 [001] 6171.470082: sched_switch: prev_comm=kworker/1:1 prev_pid=21 prev_prio=120 prev_state=S ==> next_comm=perf next_pid=1383 next_prio=120
951 perf 1383 [001] 6171.480035: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
952
953.. _perf-filtering:
954
955Filtering
956^^^^^^^^^
957
958Notice that there are a lot of events that don't really have anything to
959do with what we're interested in, namely events that schedule 'perf'
960itself in and out or that wake perf up. We can get rid of those by using
961the '--filter' option - for each event we specify using -e, we can add a
962--filter after that to filter out trace events that contain fields with
963specific values: ::
964
965 root@crownbay:~# perf record -a -e sched:sched_switch --filter 'next_comm != perf && prev_comm != perf' -e sched:sched_wakeup --filter 'comm != perf'
966 ^C[ perf record: Woken up 38 times to write data ]
967 [ perf record: Captured and wrote 9.688 MB perf.data (~423279 samples) ]
968
969
970 root@crownbay:~# perf script
971
972 swapper 0 [000] 7932.162180: sched_switch: prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=kworker/0:3 next_pid=1209 next_prio=120
973 kworker/0:3 1209 [000] 7932.162236: sched_switch: prev_comm=kworker/0:3 prev_pid=1209 prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0 next_prio=120
974 perf 1407 [001] 7932.170048: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
975 perf 1407 [001] 7932.180044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
976 perf 1407 [001] 7932.190038: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
977 perf 1407 [001] 7932.200044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
978 perf 1407 [001] 7932.210044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
979 perf 1407 [001] 7932.220044: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
980 swapper 0 [001] 7932.230111: sched_wakeup: comm=kworker/1:1 pid=21 prio=120 success=1 target_cpu=001
981 swapper 0 [001] 7932.230146: sched_switch: prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=kworker/1:1 next_pid=21 next_prio=120
982 kworker/1:1 21 [001] 7932.230205: sched_switch: prev_comm=kworker/1:1 prev_pid=21 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120
983 swapper 0 [000] 7932.326109: sched_wakeup: comm=kworker/0:3 pid=1209 prio=120 success=1 target_cpu=000
984 swapper 0 [000] 7932.326171: sched_switch: prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=kworker/0:3 next_pid=1209 next_prio=120
985 kworker/0:3 1209 [000] 7932.326214: sched_switch: prev_comm=kworker/0:3 prev_pid=1209 prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0 next_prio=120
986
987In this case, we've filtered out all events that have
988'perf' in their 'comm' or 'comm_prev' or 'comm_next' fields. Notice that
989there are still events recorded for perf, but notice that those events
990don't have values of 'perf' for the filtered fields. To completely
991filter out anything from perf will require a bit more work, but for the
992purpose of demonstrating how to use filters, it's close enough.
993
994.. admonition:: Tying it Together
995
996 These are exactly the same set of event filters defined by the trace
997 event subsystem. See the ftrace/tracecmd/kernelshark section for more
998 discussion about these event filters.
999
1000.. admonition:: Tying it Together
1001
1002 These event filters are implemented by a special-purpose
1003 pseudo-interpreter in the kernel and are an integral and
1004 indispensable part of the perf design as it relates to tracing.
1005 kernel-based event filters provide a mechanism to precisely throttle
1006 the event stream that appears in user space, where it makes sense to
1007 provide bindings to real programming languages for postprocessing the
1008 event stream. This architecture allows for the intelligent and
1009 flexible partitioning of processing between the kernel and user
1010 space. Contrast this with other tools such as SystemTap, which does
1011 all of its processing in the kernel and as such requires a special
1012 project-defined language in order to accommodate that design, or
1013 LTTng, where everything is sent to userspace and as such requires a
1014 super-efficient kernel-to-userspace transport mechanism in order to
1015 function properly. While perf certainly can benefit from for instance
1016 advances in the design of the transport, it doesn't fundamentally
1017 depend on them. Basically, if you find that your perf tracing
1018 application is causing buffer I/O overruns, it probably means that
1019 you aren't taking enough advantage of the kernel filtering engine.
1020
1021Using Dynamic Tracepoints
1022~~~~~~~~~~~~~~~~~~~~~~~~~
1023
1024perf isn't restricted to the fixed set of static tracepoints listed by
1025'perf list'. Users can also add their own 'dynamic' tracepoints anywhere
1026in the kernel. For instance, suppose we want to define our own
1027tracepoint on do_fork(). We can do that using the 'perf probe' perf
1028subcommand: ::
1029
1030 root@crownbay:~# perf probe do_fork
1031 Added new event:
1032 probe:do_fork (on do_fork)
1033
1034 You can now use it in all perf tools, such as:
1035
1036 perf record -e probe:do_fork -aR sleep 1
1037
1038Adding a new tracepoint via
1039'perf probe' results in an event with all the expected files and format
1040in /sys/kernel/debug/tracing/events, just the same as for static
1041tracepoints (as discussed in more detail in the trace events subsystem
1042section: ::
1043
1044 root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork# ls -al
1045 drwxr-xr-x 2 root root 0 Oct 28 11:42 .
1046 drwxr-xr-x 3 root root 0 Oct 28 11:42 ..
1047 -rw-r--r-- 1 root root 0 Oct 28 11:42 enable
1048 -rw-r--r-- 1 root root 0 Oct 28 11:42 filter
1049 -r--r--r-- 1 root root 0 Oct 28 11:42 format
1050 -r--r--r-- 1 root root 0 Oct 28 11:42 id
1051
1052 root@crownbay:/sys/kernel/debug/tracing/events/probe/do_fork# cat format
1053 name: do_fork
1054 ID: 944
1055 format:
1056 field:unsigned short common_type; offset:0; size:2; signed:0;
1057 field:unsigned char common_flags; offset:2; size:1; signed:0;
1058 field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
1059 field:int common_pid; offset:4; size:4; signed:1;
1060 field:int common_padding; offset:8; size:4; signed:1;
1061
1062 field:unsigned long __probe_ip; offset:12; size:4; signed:0;
1063
1064 print fmt: "(%lx)", REC->__probe_ip
1065
1066We can list all dynamic tracepoints currently in
1067existence: ::
1068
1069 root@crownbay:~# perf probe -l
1070 probe:do_fork (on do_fork)
1071 probe:schedule (on schedule)
1072
1073Let's record system-wide ('sleep 30' is a
1074trick for recording system-wide but basically do nothing and then wake
1075up after 30 seconds): ::
1076
1077 root@crownbay:~# perf record -g -a -e probe:do_fork sleep 30
1078 [ perf record: Woken up 1 times to write data ]
1079 [ perf record: Captured and wrote 0.087 MB perf.data (~3812 samples) ]
1080
1081Using 'perf script' we can see each do_fork event that fired: ::
1082
1083 root@crownbay:~# perf script
1084
1085 # ========
1086 # captured on: Sun Oct 28 11:55:18 2012
1087 # hostname : crownbay
1088 # os release : 3.4.11-yocto-standard
1089 # perf version : 3.4.11
1090 # arch : i686
1091 # nrcpus online : 2
1092 # nrcpus avail : 2
1093 # cpudesc : Intel(R) Atom(TM) CPU E660 @ 1.30GHz
1094 # cpuid : GenuineIntel,6,38,1
1095 # total memory : 1017184 kB
1096 # cmdline : /usr/bin/perf record -g -a -e probe:do_fork sleep 30
1097 # event : name = probe:do_fork, type = 2, config = 0x3b0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern
1098 = 0, id = { 5, 6 }
1099 # HEADER_CPU_TOPOLOGY info available, use -I to display
1100 # ========
1101 #
1102 matchbox-deskto 1197 [001] 34211.378318: do_fork: (c1028460)
1103 matchbox-deskto 1295 [001] 34211.380388: do_fork: (c1028460)
1104 pcmanfm 1296 [000] 34211.632350: do_fork: (c1028460)
1105 pcmanfm 1296 [000] 34211.639917: do_fork: (c1028460)
1106 matchbox-deskto 1197 [001] 34217.541603: do_fork: (c1028460)
1107 matchbox-deskto 1299 [001] 34217.543584: do_fork: (c1028460)
1108 gthumb 1300 [001] 34217.697451: do_fork: (c1028460)
1109 gthumb 1300 [001] 34219.085734: do_fork: (c1028460)
1110 gthumb 1300 [000] 34219.121351: do_fork: (c1028460)
1111 gthumb 1300 [001] 34219.264551: do_fork: (c1028460)
1112 pcmanfm 1296 [000] 34219.590380: do_fork: (c1028460)
1113 matchbox-deskto 1197 [001] 34224.955965: do_fork: (c1028460)
1114 matchbox-deskto 1306 [001] 34224.957972: do_fork: (c1028460)
1115 matchbox-termin 1307 [000] 34225.038214: do_fork: (c1028460)
1116 matchbox-termin 1307 [001] 34225.044218: do_fork: (c1028460)
1117 matchbox-termin 1307 [000] 34225.046442: do_fork: (c1028460)
1118 matchbox-deskto 1197 [001] 34237.112138: do_fork: (c1028460)
1119 matchbox-deskto 1311 [001] 34237.114106: do_fork: (c1028460)
1120 gaku 1312 [000] 34237.202388: do_fork: (c1028460)
1121
1122And using 'perf report' on the same file, we can see the
1123callgraphs from starting a few programs during those 30 seconds:
1124
1125.. image:: figures/perf-probe-do_fork-profile.png
1126 :align: center
1127
1128.. admonition:: Tying it Together
1129
1130 The trace events subsystem accommodate static and dynamic tracepoints
1131 in exactly the same way - there's no difference as far as the
1132 infrastructure is concerned. See the ftrace section for more details
1133 on the trace event subsystem.
1134
1135.. admonition:: Tying it Together
1136
1137 Dynamic tracepoints are implemented under the covers by kprobes and
1138 uprobes. kprobes and uprobes are also used by and in fact are the
1139 main focus of SystemTap.
1140
1141.. _perf-documentation:
1142
1143Perf Documentation
1144------------------
1145
1146Online versions of the man pages for the commands discussed in this
1147section can be found here:
1148
1149- The `'perf stat' manpage <http://linux.die.net/man/1/perf-stat>`__.
1150
1151- The `'perf record'
1152 manpage <http://linux.die.net/man/1/perf-record>`__.
1153
1154- The `'perf report'
1155 manpage <http://linux.die.net/man/1/perf-report>`__.
1156
1157- The `'perf probe' manpage <http://linux.die.net/man/1/perf-probe>`__.
1158
1159- The `'perf script'
1160 manpage <http://linux.die.net/man/1/perf-script>`__.
1161
1162- Documentation on using the `'perf script' python
1163 binding <http://linux.die.net/man/1/perf-script-python>`__.
1164
1165- The top-level `perf(1) manpage <http://linux.die.net/man/1/perf>`__.
1166
1167Normally, you should be able to invoke the man pages via perf itself
1168e.g. 'perf help' or 'perf help record'.
1169
1170However, by default Yocto doesn't install man pages, but perf invokes
1171the man pages for most help functionality. This is a bug and is being
1172addressed by a Yocto bug: `Bug 3388 - perf: enable man pages for basic
1173'help'
1174functionality <https://bugzilla.yoctoproject.org/show_bug.cgi?id=3388>`__.
1175
1176The man pages in text form, along with some other files, such as a set
1177of examples, can be found in the 'perf' directory of the kernel tree: ::
1178
1179 tools/perf/Documentation
1180
1181There's also a nice perf tutorial on the perf
1182wiki that goes into more detail than we do here in certain areas: `Perf
1183Tutorial <https://perf.wiki.kernel.org/index.php/Tutorial>`__
1184
1185.. _profile-manual-ftrace:
1186
1187ftrace
1188======
1189
1190'ftrace' literally refers to the 'ftrace function tracer' but in reality
1191this encompasses a number of related tracers along with the
1192infrastructure that they all make use of.
1193
1194.. _ftrace-setup:
1195
1196ftrace Setup
1197------------
1198
1199For this section, we'll assume you've already performed the basic setup
1200outlined in the ":ref:`profile-manual/profile-manual-intro:General Setup`" section.
1201
1202ftrace, trace-cmd, and kernelshark run on the target system, and are
1203ready to go out-of-the-box - no additional setup is necessary. For the
1204rest of this section we assume you've ssh'ed to the host and will be
1205running ftrace on the target. kernelshark is a GUI application and if
1206you use the '-X' option to ssh you can have the kernelshark GUI run on
1207the target but display remotely on the host if you want.
1208
1209Basic ftrace usage
1210------------------
1211
1212'ftrace' essentially refers to everything included in the /tracing
1213directory of the mounted debugfs filesystem (Yocto follows the standard
1214convention and mounts it at /sys/kernel/debug). Here's a listing of all
1215the files found in /sys/kernel/debug/tracing on a Yocto system: ::
1216
1217 root@sugarbay:/sys/kernel/debug/tracing# ls
1218 README kprobe_events trace
1219 available_events kprobe_profile trace_clock
1220 available_filter_functions options trace_marker
1221 available_tracers per_cpu trace_options
1222 buffer_size_kb printk_formats trace_pipe
1223 buffer_total_size_kb saved_cmdlines tracing_cpumask
1224 current_tracer set_event tracing_enabled
1225 dyn_ftrace_total_info set_ftrace_filter tracing_on
1226 enabled_functions set_ftrace_notrace tracing_thresh
1227 events set_ftrace_pid
1228 free_buffer set_graph_function
1229
1230The files listed above are used for various purposes
1231- some relate directly to the tracers themselves, others are used to set
1232tracing options, and yet others actually contain the tracing output when
1233a tracer is in effect. Some of the functions can be guessed from their
1234names, others need explanation; in any case, we'll cover some of the
1235files we see here below but for an explanation of the others, please see
1236the ftrace documentation.
1237
1238We'll start by looking at some of the available built-in tracers.
1239
1240cat'ing the 'available_tracers' file lists the set of available tracers: ::
1241
1242 root@sugarbay:/sys/kernel/debug/tracing# cat available_tracers
1243 blk function_graph function nop
1244
1245The 'current_tracer' file contains the tracer currently in effect: ::
1246
1247 root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer
1248 nop
1249
1250The above listing of current_tracer shows that the
1251'nop' tracer is in effect, which is just another way of saying that
1252there's actually no tracer currently in effect.
1253
1254echo'ing one of the available_tracers into current_tracer makes the
1255specified tracer the current tracer: ::
1256
1257 root@sugarbay:/sys/kernel/debug/tracing# echo function > current_tracer
1258 root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer
1259 function
1260
1261The above sets the current tracer to be the 'function tracer'. This tracer
1262traces every function call in the kernel and makes it available as the
1263contents of the 'trace' file. Reading the 'trace' file lists the
1264currently buffered function calls that have been traced by the function
1265tracer: ::
1266
1267 root@sugarbay:/sys/kernel/debug/tracing# cat trace | less
1268
1269 # tracer: function
1270 #
1271 # entries-in-buffer/entries-written: 310629/766471 #P:8
1272 #
1273 # _-----=> irqs-off
1274 # / _----=> need-resched
1275 # | / _---=> hardirq/softirq
1276 # || / _--=> preempt-depth
1277 # ||| / delay
1278 # TASK-PID CPU# |||| TIMESTAMP FUNCTION
1279 # | | | |||| | |
1280 <idle>-0 [004] d..1 470.867169: ktime_get_real <-intel_idle
1281 <idle>-0 [004] d..1 470.867170: getnstimeofday <-ktime_get_real
1282 <idle>-0 [004] d..1 470.867171: ns_to_timeval <-intel_idle
1283 <idle>-0 [004] d..1 470.867171: ns_to_timespec <-ns_to_timeval
1284 <idle>-0 [004] d..1 470.867172: smp_apic_timer_interrupt <-apic_timer_interrupt
1285 <idle>-0 [004] d..1 470.867172: native_apic_mem_write <-smp_apic_timer_interrupt
1286 <idle>-0 [004] d..1 470.867172: irq_enter <-smp_apic_timer_interrupt
1287 <idle>-0 [004] d..1 470.867172: rcu_irq_enter <-irq_enter
1288 <idle>-0 [004] d..1 470.867173: rcu_idle_exit_common.isra.33 <-rcu_irq_enter
1289 <idle>-0 [004] d..1 470.867173: local_bh_disable <-irq_enter
1290 <idle>-0 [004] d..1 470.867173: add_preempt_count <-local_bh_disable
1291 <idle>-0 [004] d.s1 470.867174: tick_check_idle <-irq_enter
1292 <idle>-0 [004] d.s1 470.867174: tick_check_oneshot_broadcast <-tick_check_idle
1293 <idle>-0 [004] d.s1 470.867174: ktime_get <-tick_check_idle
1294 <idle>-0 [004] d.s1 470.867174: tick_nohz_stop_idle <-tick_check_idle
1295 <idle>-0 [004] d.s1 470.867175: update_ts_time_stats <-tick_nohz_stop_idle
1296 <idle>-0 [004] d.s1 470.867175: nr_iowait_cpu <-update_ts_time_stats
1297 <idle>-0 [004] d.s1 470.867175: tick_do_update_jiffies64 <-tick_check_idle
1298 <idle>-0 [004] d.s1 470.867175: _raw_spin_lock <-tick_do_update_jiffies64
1299 <idle>-0 [004] d.s1 470.867176: add_preempt_count <-_raw_spin_lock
1300 <idle>-0 [004] d.s2 470.867176: do_timer <-tick_do_update_jiffies64
1301 <idle>-0 [004] d.s2 470.867176: _raw_spin_lock <-do_timer
1302 <idle>-0 [004] d.s2 470.867176: add_preempt_count <-_raw_spin_lock
1303 <idle>-0 [004] d.s3 470.867177: ntp_tick_length <-do_timer
1304 <idle>-0 [004] d.s3 470.867177: _raw_spin_lock_irqsave <-ntp_tick_length
1305 .
1306 .
1307 .
1308
1309Each line in the trace above shows what was happening in the kernel on a given
1310cpu, to the level of detail of function calls. Each entry shows the function
1311called, followed by its caller (after the arrow).
1312
1313The function tracer gives you an extremely detailed idea of what the
1314kernel was doing at the point in time the trace was taken, and is a
1315great way to learn about how the kernel code works in a dynamic sense.
1316
1317.. admonition:: Tying it Together
1318
1319 The ftrace function tracer is also available from within perf, as the
1320 ftrace:function tracepoint.
1321
1322It is a little more difficult to follow the call chains than it needs to
1323be - luckily there's a variant of the function tracer that displays the
1324callchains explicitly, called the 'function_graph' tracer: ::
1325
1326 root@sugarbay:/sys/kernel/debug/tracing# echo function_graph > current_tracer
1327 root@sugarbay:/sys/kernel/debug/tracing# cat trace | less
1328
1329 tracer: function_graph
1330
1331 CPU DURATION FUNCTION CALLS
1332 | | | | | | |
1333 7) 0.046 us | pick_next_task_fair();
1334 7) 0.043 us | pick_next_task_stop();
1335 7) 0.042 us | pick_next_task_rt();
1336 7) 0.032 us | pick_next_task_fair();
1337 7) 0.030 us | pick_next_task_idle();
1338 7) | _raw_spin_unlock_irq() {
1339 7) 0.033 us | sub_preempt_count();
1340 7) 0.258 us | }
1341 7) 0.032 us | sub_preempt_count();
1342 7) + 13.341 us | } /* __schedule */
1343 7) 0.095 us | } /* sub_preempt_count */
1344 7) | schedule() {
1345 7) | __schedule() {
1346 7) 0.060 us | add_preempt_count();
1347 7) 0.044 us | rcu_note_context_switch();
1348 7) | _raw_spin_lock_irq() {
1349 7) 0.033 us | add_preempt_count();
1350 7) 0.247 us | }
1351 7) | idle_balance() {
1352 7) | _raw_spin_unlock() {
1353 7) 0.031 us | sub_preempt_count();
1354 7) 0.246 us | }
1355 7) | update_shares() {
1356 7) 0.030 us | __rcu_read_lock();
1357 7) 0.029 us | __rcu_read_unlock();
1358 7) 0.484 us | }
1359 7) 0.030 us | __rcu_read_lock();
1360 7) | load_balance() {
1361 7) | find_busiest_group() {
1362 7) 0.031 us | idle_cpu();
1363 7) 0.029 us | idle_cpu();
1364 7) 0.035 us | idle_cpu();
1365 7) 0.906 us | }
1366 7) 1.141 us | }
1367 7) 0.022 us | msecs_to_jiffies();
1368 7) | load_balance() {
1369 7) | find_busiest_group() {
1370 7) 0.031 us | idle_cpu();
1371 .
1372 .
1373 .
1374 4) 0.062 us | msecs_to_jiffies();
1375 4) 0.062 us | __rcu_read_unlock();
1376 4) | _raw_spin_lock() {
1377 4) 0.073 us | add_preempt_count();
1378 4) 0.562 us | }
1379 4) + 17.452 us | }
1380 4) 0.108 us | put_prev_task_fair();
1381 4) 0.102 us | pick_next_task_fair();
1382 4) 0.084 us | pick_next_task_stop();
1383 4) 0.075 us | pick_next_task_rt();
1384 4) 0.062 us | pick_next_task_fair();
1385 4) 0.066 us | pick_next_task_idle();
1386 ------------------------------------------
1387 4) kworker-74 => <idle>-0
1388 ------------------------------------------
1389
1390 4) | finish_task_switch() {
1391 4) | _raw_spin_unlock_irq() {
1392 4) 0.100 us | sub_preempt_count();
1393 4) 0.582 us | }
1394 4) 1.105 us | }
1395 4) 0.088 us | sub_preempt_count();
1396 4) ! 100.066 us | }
1397 .
1398 .
1399 .
1400 3) | sys_ioctl() {
1401 3) 0.083 us | fget_light();
1402 3) | security_file_ioctl() {
1403 3) 0.066 us | cap_file_ioctl();
1404 3) 0.562 us | }
1405 3) | do_vfs_ioctl() {
1406 3) | drm_ioctl() {
1407 3) 0.075 us | drm_ut_debug_printk();
1408 3) | i915_gem_pwrite_ioctl() {
1409 3) | i915_mutex_lock_interruptible() {
1410 3) 0.070 us | mutex_lock_interruptible();
1411 3) 0.570 us | }
1412 3) | drm_gem_object_lookup() {
1413 3) | _raw_spin_lock() {
1414 3) 0.080 us | add_preempt_count();
1415 3) 0.620 us | }
1416 3) | _raw_spin_unlock() {
1417 3) 0.085 us | sub_preempt_count();
1418 3) 0.562 us | }
1419 3) 2.149 us | }
1420 3) 0.133 us | i915_gem_object_pin();
1421 3) | i915_gem_object_set_to_gtt_domain() {
1422 3) 0.065 us | i915_gem_object_flush_gpu_write_domain();
1423 3) 0.065 us | i915_gem_object_wait_rendering();
1424 3) 0.062 us | i915_gem_object_flush_cpu_write_domain();
1425 3) 1.612 us | }
1426 3) | i915_gem_object_put_fence() {
1427 3) 0.097 us | i915_gem_object_flush_fence.constprop.36();
1428 3) 0.645 us | }
1429 3) 0.070 us | add_preempt_count();
1430 3) 0.070 us | sub_preempt_count();
1431 3) 0.073 us | i915_gem_object_unpin();
1432 3) 0.068 us | mutex_unlock();
1433 3) 9.924 us | }
1434 3) + 11.236 us | }
1435 3) + 11.770 us | }
1436 3) + 13.784 us | }
1437 3) | sys_ioctl() {
1438
1439As you can see, the function_graph display is much easier
1440to follow. Also note that in addition to the function calls and
1441associated braces, other events such as scheduler events are displayed
1442in context. In fact, you can freely include any tracepoint available in
1443the trace events subsystem described in the next section by simply
1444enabling those events, and they'll appear in context in the function
1445graph display. Quite a powerful tool for understanding kernel dynamics.
1446
1447Also notice that there are various annotations on the left hand side of
1448the display. For example if the total time it took for a given function
1449to execute is above a certain threshold, an exclamation point or plus
1450sign appears on the left hand side. Please see the ftrace documentation
1451for details on all these fields.
1452
1453The 'trace events' Subsystem
1454----------------------------
1455
1456One especially important directory contained within the
1457/sys/kernel/debug/tracing directory is the 'events' subdirectory, which
1458contains representations of every tracepoint in the system. Listing out
1459the contents of the 'events' subdirectory, we see mainly another set of
1460subdirectories: ::
1461
1462 root@sugarbay:/sys/kernel/debug/tracing# cd events
1463 root@sugarbay:/sys/kernel/debug/tracing/events# ls -al
1464 drwxr-xr-x 38 root root 0 Nov 14 23:19 .
1465 drwxr-xr-x 5 root root 0 Nov 14 23:19 ..
1466 drwxr-xr-x 19 root root 0 Nov 14 23:19 block
1467 drwxr-xr-x 32 root root 0 Nov 14 23:19 btrfs
1468 drwxr-xr-x 5 root root 0 Nov 14 23:19 drm
1469 -rw-r--r-- 1 root root 0 Nov 14 23:19 enable
1470 drwxr-xr-x 40 root root 0 Nov 14 23:19 ext3
1471 drwxr-xr-x 79 root root 0 Nov 14 23:19 ext4
1472 drwxr-xr-x 14 root root 0 Nov 14 23:19 ftrace
1473 drwxr-xr-x 8 root root 0 Nov 14 23:19 hda
1474 -r--r--r-- 1 root root 0 Nov 14 23:19 header_event
1475 -r--r--r-- 1 root root 0 Nov 14 23:19 header_page
1476 drwxr-xr-x 25 root root 0 Nov 14 23:19 i915
1477 drwxr-xr-x 7 root root 0 Nov 14 23:19 irq
1478 drwxr-xr-x 12 root root 0 Nov 14 23:19 jbd
1479 drwxr-xr-x 14 root root 0 Nov 14 23:19 jbd2
1480 drwxr-xr-x 14 root root 0 Nov 14 23:19 kmem
1481 drwxr-xr-x 7 root root 0 Nov 14 23:19 module
1482 drwxr-xr-x 3 root root 0 Nov 14 23:19 napi
1483 drwxr-xr-x 6 root root 0 Nov 14 23:19 net
1484 drwxr-xr-x 3 root root 0 Nov 14 23:19 oom
1485 drwxr-xr-x 12 root root 0 Nov 14 23:19 power
1486 drwxr-xr-x 3 root root 0 Nov 14 23:19 printk
1487 drwxr-xr-x 8 root root 0 Nov 14 23:19 random
1488 drwxr-xr-x 4 root root 0 Nov 14 23:19 raw_syscalls
1489 drwxr-xr-x 3 root root 0 Nov 14 23:19 rcu
1490 drwxr-xr-x 6 root root 0 Nov 14 23:19 rpm
1491 drwxr-xr-x 20 root root 0 Nov 14 23:19 sched
1492 drwxr-xr-x 7 root root 0 Nov 14 23:19 scsi
1493 drwxr-xr-x 4 root root 0 Nov 14 23:19 signal
1494 drwxr-xr-x 5 root root 0 Nov 14 23:19 skb
1495 drwxr-xr-x 4 root root 0 Nov 14 23:19 sock
1496 drwxr-xr-x 10 root root 0 Nov 14 23:19 sunrpc
1497 drwxr-xr-x 538 root root 0 Nov 14 23:19 syscalls
1498 drwxr-xr-x 4 root root 0 Nov 14 23:19 task
1499 drwxr-xr-x 14 root root 0 Nov 14 23:19 timer
1500 drwxr-xr-x 3 root root 0 Nov 14 23:19 udp
1501 drwxr-xr-x 21 root root 0 Nov 14 23:19 vmscan
1502 drwxr-xr-x 3 root root 0 Nov 14 23:19 vsyscall
1503 drwxr-xr-x 6 root root 0 Nov 14 23:19 workqueue
1504 drwxr-xr-x 26 root root 0 Nov 14 23:19 writeback
1505
1506Each one of these subdirectories
1507corresponds to a 'subsystem' and contains yet again more subdirectories,
1508each one of those finally corresponding to a tracepoint. For example,
1509here are the contents of the 'kmem' subsystem: ::
1510
1511 root@sugarbay:/sys/kernel/debug/tracing/events# cd kmem
1512 root@sugarbay:/sys/kernel/debug/tracing/events/kmem# ls -al
1513 drwxr-xr-x 14 root root 0 Nov 14 23:19 .
1514 drwxr-xr-x 38 root root 0 Nov 14 23:19 ..
1515 -rw-r--r-- 1 root root 0 Nov 14 23:19 enable
1516 -rw-r--r-- 1 root root 0 Nov 14 23:19 filter
1517 drwxr-xr-x 2 root root 0 Nov 14 23:19 kfree
1518 drwxr-xr-x 2 root root 0 Nov 14 23:19 kmalloc
1519 drwxr-xr-x 2 root root 0 Nov 14 23:19 kmalloc_node
1520 drwxr-xr-x 2 root root 0 Nov 14 23:19 kmem_cache_alloc
1521 drwxr-xr-x 2 root root 0 Nov 14 23:19 kmem_cache_alloc_node
1522 drwxr-xr-x 2 root root 0 Nov 14 23:19 kmem_cache_free
1523 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_alloc
1524 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_alloc_extfrag
1525 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_alloc_zone_locked
1526 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_free
1527 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_free_batched
1528 drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_pcpu_drain
1529
1530Let's see what's inside the subdirectory for a
1531specific tracepoint, in this case the one for kmalloc: ::
1532
1533 root@sugarbay:/sys/kernel/debug/tracing/events/kmem# cd kmalloc
1534 root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# ls -al
1535 drwxr-xr-x 2 root root 0 Nov 14 23:19 .
1536 drwxr-xr-x 14 root root 0 Nov 14 23:19 ..
1537 -rw-r--r-- 1 root root 0 Nov 14 23:19 enable
1538 -rw-r--r-- 1 root root 0 Nov 14 23:19 filter
1539 -r--r--r-- 1 root root 0 Nov 14 23:19 format
1540 -r--r--r-- 1 root root 0 Nov 14 23:19 id
1541
1542The 'format' file for the
1543tracepoint describes the event in memory, which is used by the various
1544tracing tools that now make use of these tracepoint to parse the event
1545and make sense of it, along with a 'print fmt' field that allows tools
1546like ftrace to display the event as text. Here's what the format of the
1547kmalloc event looks like: ::
1548
1549 root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# cat format
1550 name: kmalloc
1551 ID: 313
1552 format:
1553 field:unsigned short common_type; offset:0; size:2; signed:0;
1554 field:unsigned char common_flags; offset:2; size:1; signed:0;
1555 field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
1556 field:int common_pid; offset:4; size:4; signed:1;
1557 field:int common_padding; offset:8; size:4; signed:1;
1558
1559 field:unsigned long call_site; offset:16; size:8; signed:0;
1560 field:const void * ptr; offset:24; size:8; signed:0;
1561 field:size_t bytes_req; offset:32; size:8; signed:0;
1562 field:size_t bytes_alloc; offset:40; size:8; signed:0;
1563 field:gfp_t gfp_flags; offset:48; size:4; signed:0;
1564
1565 print fmt: "call_site=%lx ptr=%p bytes_req=%zu bytes_alloc=%zu gfp_flags=%s", REC->call_site, REC->ptr, REC->bytes_req, REC->bytes_alloc,
1566 (REC->gfp_flags) ? __print_flags(REC->gfp_flags, "|", {(unsigned long)(((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | ((
1567 gfp_t)0x20000u) | (( gfp_t)0x02u) | (( gfp_t)0x08u)) | (( gfp_t)0x4000u) | (( gfp_t)0x10000u) | (( gfp_t)0x1000u) | (( gfp_t)0x200u) | ((
1568 gfp_t)0x400000u)), "GFP_TRANSHUGE"}, {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | (( gfp_t)0x20000u) | ((
1569 gfp_t)0x02u) | (( gfp_t)0x08u)), "GFP_HIGHUSER_MOVABLE"}, {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | ((
1570 gfp_t)0x20000u) | (( gfp_t)0x02u)), "GFP_HIGHUSER"}, {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | ((
1571 gfp_t)0x20000u)), "GFP_USER"}, {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u) | (( gfp_t)0x80000u)), GFP_TEMPORARY"},
1572 {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u) | (( gfp_t)0x80u)), "GFP_KERNEL"}, {(unsigned long)((( gfp_t)0x10u) | (( gfp_t)0x40u)),
1573 "GFP_NOFS"}, {(unsigned long)((( gfp_t)0x20u)), "GFP_ATOMIC"}, {(unsigned long)((( gfp_t)0x10u)), "GFP_NOIO"}, {(unsigned long)((
1574 gfp_t)0x20u), "GFP_HIGH"}, {(unsigned long)(( gfp_t)0x10u), "GFP_WAIT"}, {(unsigned long)(( gfp_t)0x40u), "GFP_IO"}, {(unsigned long)((
1575 gfp_t)0x100u), "GFP_COLD"}, {(unsigned long)(( gfp_t)0x200u), "GFP_NOWARN"}, {(unsigned long)(( gfp_t)0x400u), "GFP_REPEAT"}, {(unsigned
1576 long)(( gfp_t)0x800u), "GFP_NOFAIL"}, {(unsigned long)(( gfp_t)0x1000u), "GFP_NORETRY"}, {(unsigned long)(( gfp_t)0x4000u), "GFP_COMP"},
1577 {(unsigned long)(( gfp_t)0x8000u), "GFP_ZERO"}, {(unsigned long)(( gfp_t)0x10000u), "GFP_NOMEMALLOC"}, {(unsigned long)(( gfp_t)0x20000u),
1578 "GFP_HARDWALL"}, {(unsigned long)(( gfp_t)0x40000u), "GFP_THISNODE"}, {(unsigned long)(( gfp_t)0x80000u), "GFP_RECLAIMABLE"}, {(unsigned
1579 long)(( gfp_t)0x08u), "GFP_MOVABLE"}, {(unsigned long)(( gfp_t)0), "GFP_NOTRACK"}, {(unsigned long)(( gfp_t)0x400000u), "GFP_NO_KSWAPD"},
1580 {(unsigned long)(( gfp_t)0x800000u), "GFP_OTHER_NODE"} ) : "GFP_NOWAIT"
1581
1582The 'enable' file
1583in the tracepoint directory is what allows the user (or tools such as
1584trace-cmd) to actually turn the tracepoint on and off. When enabled, the
1585corresponding tracepoint will start appearing in the ftrace 'trace' file
1586described previously. For example, this turns on the kmalloc tracepoint: ::
1587
1588 root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 1 > enable
1589
1590At the moment, we're not interested in the function tracer or
1591some other tracer that might be in effect, so we first turn it off, but
1592if we do that, we still need to turn tracing on in order to see the
1593events in the output buffer: ::
1594
1595 root@sugarbay:/sys/kernel/debug/tracing# echo nop > current_tracer
1596 root@sugarbay:/sys/kernel/debug/tracing# echo 1 > tracing_on
1597
1598Now, if we look at the the 'trace' file, we see nothing
1599but the kmalloc events we just turned on: ::
1600
1601 root@sugarbay:/sys/kernel/debug/tracing# cat trace | less
1602 # tracer: nop
1603 #
1604 # entries-in-buffer/entries-written: 1897/1897 #P:8
1605 #
1606 # _-----=> irqs-off
1607 # / _----=> need-resched
1608 # | / _---=> hardirq/softirq
1609 # || / _--=> preempt-depth
1610 # ||| / delay
1611 # TASK-PID CPU# |||| TIMESTAMP FUNCTION
1612 # | | | |||| | |
1613 dropbear-1465 [000] ...1 18154.620753: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1614 <idle>-0 [000] ..s3 18154.621640: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1615 <idle>-0 [000] ..s3 18154.621656: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1616 matchbox-termin-1361 [001] ...1 18154.755472: kmalloc: call_site=ffffffff81614050 ptr=ffff88006d5f0e00 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_KERNEL|GFP_REPEAT
1617 Xorg-1264 [002] ...1 18154.755581: kmalloc: call_site=ffffffff8141abe8 ptr=ffff8800734f4cc0 bytes_req=168 bytes_alloc=192 gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY
1618 Xorg-1264 [002] ...1 18154.755583: kmalloc: call_site=ffffffff814192a3 ptr=ffff88001f822520 bytes_req=24 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
1619 Xorg-1264 [002] ...1 18154.755589: kmalloc: call_site=ffffffff81419edb ptr=ffff8800721a2f00 bytes_req=64 bytes_alloc=64 gfp_flags=GFP_KERNEL|GFP_ZERO
1620 matchbox-termin-1361 [001] ...1 18155.354594: kmalloc: call_site=ffffffff81614050 ptr=ffff88006db35400 bytes_req=576 bytes_alloc=1024 gfp_flags=GFP_KERNEL|GFP_REPEAT
1621 Xorg-1264 [002] ...1 18155.354703: kmalloc: call_site=ffffffff8141abe8 ptr=ffff8800734f4cc0 bytes_req=168 bytes_alloc=192 gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY
1622 Xorg-1264 [002] ...1 18155.354705: kmalloc: call_site=ffffffff814192a3 ptr=ffff88001f822520 bytes_req=24 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
1623 Xorg-1264 [002] ...1 18155.354711: kmalloc: call_site=ffffffff81419edb ptr=ffff8800721a2f00 bytes_req=64 bytes_alloc=64 gfp_flags=GFP_KERNEL|GFP_ZERO
1624 <idle>-0 [000] ..s3 18155.673319: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1625 dropbear-1465 [000] ...1 18155.673525: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1626 <idle>-0 [000] ..s3 18155.674821: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1627 <idle>-0 [000] ..s3 18155.793014: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1628 dropbear-1465 [000] ...1 18155.793219: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1629 <idle>-0 [000] ..s3 18155.794147: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1630 <idle>-0 [000] ..s3 18155.936705: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1631 dropbear-1465 [000] ...1 18155.936910: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1632 <idle>-0 [000] ..s3 18155.937869: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1633 matchbox-termin-1361 [001] ...1 18155.953667: kmalloc: call_site=ffffffff81614050 ptr=ffff88006d5f2000 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_KERNEL|GFP_REPEAT
1634 Xorg-1264 [002] ...1 18155.953775: kmalloc: call_site=ffffffff8141abe8 ptr=ffff8800734f4cc0 bytes_req=168 bytes_alloc=192 gfp_flags=GFP_KERNEL|GFP_NOWARN|GFP_NORETRY
1635 Xorg-1264 [002] ...1 18155.953777: kmalloc: call_site=ffffffff814192a3 ptr=ffff88001f822520 bytes_req=24 bytes_alloc=32 gfp_flags=GFP_KERNEL|GFP_ZERO
1636 Xorg-1264 [002] ...1 18155.953783: kmalloc: call_site=ffffffff81419edb ptr=ffff8800721a2f00 bytes_req=64 bytes_alloc=64 gfp_flags=GFP_KERNEL|GFP_ZERO
1637 <idle>-0 [000] ..s3 18156.176053: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1638 dropbear-1465 [000] ...1 18156.176257: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1639 <idle>-0 [000] ..s3 18156.177717: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1640 <idle>-0 [000] ..s3 18156.399229: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d555800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1641 dropbear-1465 [000] ...1 18156.399434: kmalloc: call_site=ffffffff816650d4 ptr=ffff8800729c3000 bytes_http://rostedt.homelinux.com/kernelshark/req=2048 bytes_alloc=2048 gfp_flags=GFP_KERNEL
1642 <idle>-0 [000] ..s3 18156.400660: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC
1643 matchbox-termin-1361 [001] ...1 18156.552800: kmalloc: call_site=ffffffff81614050 ptr=ffff88006db34800 bytes_req=576 bytes_alloc=1024 gfp_flags=GFP_KERNEL|GFP_REPEAT
1644
1645To again disable the kmalloc event, we need to send 0 to the enable file: ::
1646
1647 root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 0 > enable
1648
1649You can enable any number of events or complete subsystems (by
1650using the 'enable' file in the subsystem directory) and get an
1651arbitrarily fine-grained idea of what's going on in the system by
1652enabling as many of the appropriate tracepoints as applicable.
1653
1654A number of the tools described in this HOWTO do just that, including
1655trace-cmd and kernelshark in the next section.
1656
1657.. admonition:: Tying it Together
1658
1659 These tracepoints and their representation are used not only by
1660 ftrace, but by many of the other tools covered in this document and
1661 they form a central point of integration for the various tracers
1662 available in Linux. They form a central part of the instrumentation
1663 for the following tools: perf, lttng, ftrace, blktrace and SystemTap
1664
1665.. admonition:: Tying it Together
1666
1667 Eventually all the special-purpose tracers currently available in
1668 /sys/kernel/debug/tracing will be removed and replaced with
1669 equivalent tracers based on the 'trace events' subsystem.
1670
1671.. _trace-cmd-kernelshark:
1672
1673trace-cmd/kernelshark
1674---------------------
1675
1676trace-cmd is essentially an extensive command-line 'wrapper' interface
1677that hides the details of all the individual files in
1678/sys/kernel/debug/tracing, allowing users to specify specific particular
1679events within the /sys/kernel/debug/tracing/events/ subdirectory and to
1680collect traces and avoid having to deal with those details directly.
1681
1682As yet another layer on top of that, kernelshark provides a GUI that
1683allows users to start and stop traces and specify sets of events using
1684an intuitive interface, and view the output as both trace events and as
1685a per-CPU graphical display. It directly uses 'trace-cmd' as the
1686plumbing that accomplishes all that underneath the covers (and actually
1687displays the trace-cmd command it uses, as we'll see).
1688
1689To start a trace using kernelshark, first start kernelshark: ::
1690
1691 root@sugarbay:~# kernelshark
1692
1693Then bring up the 'Capture' dialog by
1694choosing from the kernelshark menu: ::
1695
1696 Capture | Record
1697
1698That will display the following dialog, which allows you to choose one or more
1699events (or even one or more complete subsystems) to trace:
1700
1701.. image:: figures/kernelshark-choose-events.png
1702 :align: center
1703
1704Note that these are exactly the same sets of events described in the
1705previous trace events subsystem section, and in fact is where trace-cmd
1706gets them for kernelshark.
1707
1708In the above screenshot, we've decided to explore the graphics subsystem
1709a bit and so have chosen to trace all the tracepoints contained within
1710the 'i915' and 'drm' subsystems.
1711
1712After doing that, we can start and stop the trace using the 'Run' and
1713'Stop' button on the lower right corner of the dialog (the same button
1714will turn into the 'Stop' button after the trace has started):
1715
1716.. image:: figures/kernelshark-output-display.png
1717 :align: center
1718
1719Notice that the right-hand pane shows the exact trace-cmd command-line
1720that's used to run the trace, along with the results of the trace-cmd
1721run.
1722
1723Once the 'Stop' button is pressed, the graphical view magically fills up
1724with a colorful per-cpu display of the trace data, along with the
1725detailed event listing below that:
1726
1727.. image:: figures/kernelshark-i915-display.png
1728 :align: center
1729
1730Here's another example, this time a display resulting from tracing 'all
1731events':
1732
1733.. image:: figures/kernelshark-all.png
1734 :align: center
1735
1736The tool is pretty self-explanatory, but for more detailed information
1737on navigating through the data, see the `kernelshark
1738website <http://rostedt.homelinux.com/kernelshark/>`__.
1739
1740.. _ftrace-documentation:
1741
1742ftrace Documentation
1743--------------------
1744
1745The documentation for ftrace can be found in the kernel Documentation
1746directory: ::
1747
1748 Documentation/trace/ftrace.txt
1749
1750The documentation for the trace event subsystem can also be found in the kernel
1751Documentation directory: ::
1752
1753 Documentation/trace/events.txt
1754
1755There is a nice series of articles on using ftrace and trace-cmd at LWN:
1756
1757- `Debugging the kernel using Ftrace - part
1758 1 <http://lwn.net/Articles/365835/>`__
1759
1760- `Debugging the kernel using Ftrace - part
1761 2 <http://lwn.net/Articles/366796/>`__
1762
1763- `Secrets of the Ftrace function
1764 tracer <http://lwn.net/Articles/370423/>`__
1765
1766- `trace-cmd: A front-end for
1767 Ftrace <https://lwn.net/Articles/410200/>`__
1768
1769There's more detailed documentation kernelshark usage here:
1770`KernelShark <http://rostedt.homelinux.com/kernelshark/>`__
1771
1772An amusing yet useful README (a tracing mini-HOWTO) can be found in
1773``/sys/kernel/debug/tracing/README``.
1774
1775.. _profile-manual-systemtap:
1776
1777systemtap
1778=========
1779
1780SystemTap is a system-wide script-based tracing and profiling tool.
1781
1782SystemTap scripts are C-like programs that are executed in the kernel to
1783gather/print/aggregate data extracted from the context they end up being
1784invoked under.
1785
1786For example, this probe from the `SystemTap
1787tutorial <http://sourceware.org/systemtap/tutorial/>`__ simply prints a
1788line every time any process on the system open()s a file. For each line,
1789it prints the executable name of the program that opened the file, along
1790with its PID, and the name of the file it opened (or tried to open),
1791which it extracts from the open syscall's argstr.
1792
1793.. code-block:: none
1794
1795 probe syscall.open
1796 {
1797 printf ("%s(%d) open (%s)\n", execname(), pid(), argstr)
1798 }
1799
1800 probe timer.ms(4000) # after 4 seconds
1801 {
1802 exit ()
1803 }
1804
1805Normally, to execute this
1806probe, you'd simply install systemtap on the system you want to probe,
1807and directly run the probe on that system e.g. assuming the name of the
1808file containing the above text is trace_open.stp: ::
1809
1810 # stap trace_open.stp
1811
1812What systemtap does under the covers to run this probe is 1) parse and
1813convert the probe to an equivalent 'C' form, 2) compile the 'C' form
1814into a kernel module, 3) insert the module into the kernel, which arms
1815it, and 4) collect the data generated by the probe and display it to the
1816user.
1817
1818In order to accomplish steps 1 and 2, the 'stap' program needs access to
1819the kernel build system that produced the kernel that the probed system
1820is running. In the case of a typical embedded system (the 'target'), the
1821kernel build system unfortunately isn't typically part of the image
1822running on the target. It is normally available on the 'host' system
1823that produced the target image however; in such cases, steps 1 and 2 are
1824executed on the host system, and steps 3 and 4 are executed on the
1825target system, using only the systemtap 'runtime'.
1826
1827The systemtap support in Yocto assumes that only steps 3 and 4 are run
1828on the target; it is possible to do everything on the target, but this
1829section assumes only the typical embedded use-case.
1830
1831So basically what you need to do in order to run a systemtap script on
1832the target is to 1) on the host system, compile the probe into a kernel
1833module that makes sense to the target, 2) copy the module onto the
1834target system and 3) insert the module into the target kernel, which
1835arms it, and 4) collect the data generated by the probe and display it
1836to the user.
1837
1838.. _systemtap-setup:
1839
1840systemtap Setup
1841---------------
1842
1843Those are a lot of steps and a lot of details, but fortunately Yocto
1844includes a script called 'crosstap' that will take care of those
1845details, allowing you to simply execute a systemtap script on the remote
1846target, with arguments if necessary.
1847
1848In order to do this from a remote host, however, you need to have access
1849to the build for the image you booted. The 'crosstap' script provides
1850details on how to do this if you run the script on the host without
1851having done a build: ::
1852
1853 $ crosstap root@192.168.1.88 trace_open.stp
1854
1855 Error: No target kernel build found.
1856 Did you forget to create a local build of your image?
1857
1858 'crosstap' requires a local sdk build of the target system
1859 (or a build that includes 'tools-profile') in order to build
1860 kernel modules that can probe the target system.
1861
1862 Practically speaking, that means you need to do the following:
1863 - If you're running a pre-built image, download the release
1864 and/or BSP tarballs used to build the image.
1865 - If you're working from git sources, just clone the metadata
1866 and BSP layers needed to build the image you'll be booting.
1867 - Make sure you're properly set up to build a new image (see
1868 the BSP README and/or the widely available basic documentation
1869 that discusses how to build images).
1870 - Build an -sdk version of the image e.g.:
1871 $ bitbake core-image-sato-sdk
1872 OR
1873 - Build a non-sdk image but include the profiling tools:
1874 [ edit local.conf and add 'tools-profile' to the end of
1875 the EXTRA_IMAGE_FEATURES variable ]
1876 $ bitbake core-image-sato
1877
1878 Once you've build the image on the host system, you're ready to
1879 boot it (or the equivalent pre-built image) and use 'crosstap'
1880 to probe it (you need to source the environment as usual first):
1881
1882 $ source oe-init-build-env
1883 $ cd ~/my/systemtap/scripts
1884 $ crosstap root@192.168.1.xxx myscript.stp
1885
1886.. note::
1887
1888 SystemTap, which uses 'crosstap', assumes you can establish an ssh
1889 connection to the remote target. Please refer to the crosstap wiki
1890 page for details on verifying ssh connections at
1891 . Also, the ability to ssh into the target system is not enabled by
1892 default in \*-minimal images.
1893
1894So essentially what you need to
1895do is build an SDK image or image with 'tools-profile' as detailed in
1896the ":ref:`profile-manual/profile-manual-intro:General Setup`" section of this
1897manual, and boot the resulting target image.
1898
1899.. note::
1900
1901 If you have a build directory containing multiple machines, you need
1902 to have the MACHINE you're connecting to selected in local.conf, and
1903 the kernel in that machine's build directory must match the kernel on
1904 the booted system exactly, or you'll get the above 'crosstap' message
1905 when you try to invoke a script.
1906
1907Running a Script on a Target
1908----------------------------
1909
1910Once you've done that, you should be able to run a systemtap script on
1911the target: ::
1912
1913 $ cd /path/to/yocto
1914 $ source oe-init-build-env
1915
1916 ### Shell environment set up for builds. ###
1917
1918 You can now run 'bitbake <target>'
1919
1920 Common targets are:
1921 core-image-minimal
1922 core-image-sato
1923 meta-toolchain
1924 meta-ide-support
1925
1926 You can also run generated qemu images with a command like 'runqemu qemux86-64'
1927
1928Once you've done that, you can cd to whatever
1929directory contains your scripts and use 'crosstap' to run the script: ::
1930
1931 $ cd /path/to/my/systemap/script
1932 $ crosstap root@192.168.7.2 trace_open.stp
1933
1934If you get an error connecting to the target e.g.: ::
1935
1936 $ crosstap root@192.168.7.2 trace_open.stp
1937 error establishing ssh connection on remote 'root@192.168.7.2'
1938
1939Try ssh'ing to the target and see what happens: ::
1940
1941 $ ssh root@192.168.7.2
1942
1943A lot of the time, connection
1944problems are due specifying a wrong IP address or having a 'host key
1945verification error'.
1946
1947If everything worked as planned, you should see something like this
1948(enter the password when prompted, or press enter if it's set up to use
1949no password):
1950
1951.. code-block:: none
1952
1953 $ crosstap root@192.168.7.2 trace_open.stp
1954 root@192.168.7.2's password:
1955 matchbox-termin(1036) open ("/tmp/vte3FS2LW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
1956 matchbox-termin(1036) open ("/tmp/vteJMC7LW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600)
1957
1958.. _systemtap-documentation:
1959
1960systemtap Documentation
1961-----------------------
1962
1963The SystemTap language reference can be found here: `SystemTap Language
1964Reference <http://sourceware.org/systemtap/langref/>`__
1965
1966Links to other SystemTap documents, tutorials, and examples can be found
1967here: `SystemTap documentation
1968page <http://sourceware.org/systemtap/documentation.html>`__
1969
1970.. _profile-manual-sysprof:
1971
1972Sysprof
1973=======
1974
1975Sysprof is a very easy to use system-wide profiler that consists of a
1976single window with three panes and a few buttons which allow you to
1977start, stop, and view the profile from one place.
1978
1979.. _sysprof-setup:
1980
1981Sysprof Setup
1982-------------
1983
1984For this section, we'll assume you've already performed the basic setup
1985outlined in the ":ref:`profile-manual/profile-manual-intro:General Setup`" section.
1986
1987Sysprof is a GUI-based application that runs on the target system. For
1988the rest of this document we assume you've ssh'ed to the host and will
1989be running Sysprof on the target (you can use the '-X' option to ssh and
1990have the Sysprof GUI run on the target but display remotely on the host
1991if you want).
1992
1993.. _sysprof-basic-usage:
1994
1995Basic Sysprof Usage
1996-------------------
1997
1998To start profiling the system, you simply press the 'Start' button. To
1999stop profiling and to start viewing the profile data in one easy step,
2000press the 'Profile' button.
2001
2002Once you've pressed the profile button, the three panes will fill up
2003with profiling data:
2004
2005.. image:: figures/sysprof-copy-to-user.png
2006 :align: center
2007
2008The left pane shows a list of functions and processes. Selecting one of
2009those expands that function in the right pane, showing all its callees.
2010Note that this caller-oriented display is essentially the inverse of
2011perf's default callee-oriented callchain display.
2012
2013In the screenshot above, we're focusing on ``__copy_to_user_ll()`` and
2014looking up the callchain we can see that one of the callers of
2015``__copy_to_user_ll`` is sys_read() and the complete callpath between them.
2016Notice that this is essentially a portion of the same information we saw
2017in the perf display shown in the perf section of this page.
2018
2019.. image:: figures/sysprof-copy-from-user.png
2020 :align: center
2021
2022Similarly, the above is a snapshot of the Sysprof display of a
2023copy-from-user callchain.
2024
2025Finally, looking at the third Sysprof pane in the lower left, we can see
2026a list of all the callers of a particular function selected in the top
2027left pane. In this case, the lower pane is showing all the callers of
2028``__mark_inode_dirty``:
2029
2030.. image:: figures/sysprof-callers.png
2031 :align: center
2032
2033Double-clicking on one of those functions will in turn change the focus
2034to the selected function, and so on.
2035
2036.. admonition:: Tying it Together
2037
2038 If you like sysprof's 'caller-oriented' display, you may be able to
2039 approximate it in other tools as well. For example, 'perf report' has
2040 the -g (--call-graph) option that you can experiment with; one of the
2041 options is 'caller' for an inverted caller-based callgraph display.
2042
2043.. _sysprof-documentation:
2044
2045Sysprof Documentation
2046---------------------
2047
2048There doesn't seem to be any documentation for Sysprof, but maybe that's
2049because it's pretty self-explanatory. The Sysprof website, however, is
2050here: `Sysprof, System-wide Performance Profiler for
2051Linux <http://sysprof.com/>`__
2052
2053LTTng (Linux Trace Toolkit, next generation)
2054============================================
2055
2056.. _lttng-setup:
2057
2058LTTng Setup
2059-----------
2060
2061For this section, we'll assume you've already performed the basic setup
2062outlined in the ":ref:`profile-manual/profile-manual-intro:General Setup`" section.
2063LTTng is run on the target system by ssh'ing to it.
2064
2065Collecting and Viewing Traces
2066-----------------------------
2067
2068Once you've applied the above commits and built and booted your image
2069(you need to build the core-image-sato-sdk image or use one of the other
2070methods described in the ":ref:`profile-manual/profile-manual-intro:General Setup`" section), you're ready to start
2071tracing.
2072
2073Collecting and viewing a trace on the target (inside a shell)
2074~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2075
2076First, from the host, ssh to the target: ::
2077
2078 $ ssh -l root 192.168.1.47
2079 The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
2080 RSA key fingerprint is 23:bd:c8:b1:a8:71:52:00:ee:00:4f:64:9e:10:b9:7e.
2081 Are you sure you want to continue connecting (yes/no)? yes
2082 Warning: Permanently added '192.168.1.47' (RSA) to the list of known hosts.
2083 root@192.168.1.47's password:
2084
2085Once on the target, use these steps to create a trace: ::
2086
2087 root@crownbay:~# lttng create
2088 Spawning a session daemon
2089 Session auto-20121015-232120 created.
2090 Traces will be written in /home/root/lttng-traces/auto-20121015-232120
2091
2092Enable the events you want to trace (in this case all kernel events): ::
2093
2094 root@crownbay:~# lttng enable-event --kernel --all
2095 All kernel events are enabled in channel channel0
2096
2097Start the trace: ::
2098
2099 root@crownbay:~# lttng start
2100 Tracing started for session auto-20121015-232120
2101
2102And then stop the trace after awhile or after running a particular workload that
2103you want to trace: ::
2104
2105 root@crownbay:~# lttng stop
2106 Tracing stopped for session auto-20121015-232120
2107
2108You can now view the trace in text form on the target: ::
2109
2110 root@crownbay:~# lttng view
2111 [23:21:56.989270399] (+?.?????????) sys_geteuid: { 1 }, { }
2112 [23:21:56.989278081] (+0.000007682) exit_syscall: { 1 }, { ret = 0 }
2113 [23:21:56.989286043] (+0.000007962) sys_pipe: { 1 }, { fildes = 0xB77B9E8C }
2114 [23:21:56.989321802] (+0.000035759) exit_syscall: { 1 }, { ret = 0 }
2115 [23:21:56.989329345] (+0.000007543) sys_mmap_pgoff: { 1 }, { addr = 0x0, len = 10485760, prot = 3, flags = 131362, fd = 4294967295, pgoff = 0 }
2116 [23:21:56.989351694] (+0.000022349) exit_syscall: { 1 }, { ret = -1247805440 }
2117 [23:21:56.989432989] (+0.000081295) sys_clone: { 1 }, { clone_flags = 0x411, newsp = 0xB5EFFFE4, parent_tid = 0xFFFFFFFF, child_tid = 0x0 }
2118 [23:21:56.989477129] (+0.000044140) sched_stat_runtime: { 1 }, { comm = "lttng-consumerd", tid = 1193, runtime = 681660, vruntime = 43367983388 }
2119 [23:21:56.989486697] (+0.000009568) sched_migrate_task: { 1 }, { comm = "lttng-consumerd", tid = 1193, prio = 20, orig_cpu = 1, dest_cpu = 1 }
2120 [23:21:56.989508418] (+0.000021721) hrtimer_init: { 1 }, { hrtimer = 3970832076, clockid = 1, mode = 1 }
2121 [23:21:56.989770462] (+0.000262044) hrtimer_cancel: { 1 }, { hrtimer = 3993865440 }
2122 [23:21:56.989771580] (+0.000001118) hrtimer_cancel: { 0 }, { hrtimer = 3993812192 }
2123 [23:21:56.989776957] (+0.000005377) hrtimer_expire_entry: { 1 }, { hrtimer = 3993865440, now = 79815980007057, function = 3238465232 }
2124 [23:21:56.989778145] (+0.000001188) hrtimer_expire_entry: { 0 }, { hrtimer = 3993812192, now = 79815980008174, function = 3238465232 }
2125 [23:21:56.989791695] (+0.000013550) softirq_raise: { 1 }, { vec = 1 }
2126 [23:21:56.989795396] (+0.000003701) softirq_raise: { 0 }, { vec = 1 }
2127 [23:21:56.989800635] (+0.000005239) softirq_raise: { 0 }, { vec = 9 }
2128 [23:21:56.989807130] (+0.000006495) sched_stat_runtime: { 1 }, { comm = "lttng-consumerd", tid = 1193, runtime = 330710, vruntime = 43368314098 }
2129 [23:21:56.989809993] (+0.000002863) sched_stat_runtime: { 0 }, { comm = "lttng-sessiond", tid = 1181, runtime = 1015313, vruntime = 36976733240 }
2130 [23:21:56.989818514] (+0.000008521) hrtimer_expire_exit: { 0 }, { hrtimer = 3993812192 }
2131 [23:21:56.989819631] (+0.000001117) hrtimer_expire_exit: { 1 }, { hrtimer = 3993865440 }
2132 [23:21:56.989821866] (+0.000002235) hrtimer_start: { 0 }, { hrtimer = 3993812192, function = 3238465232, expires = 79815981000000, softexpires = 79815981000000 }
2133 [23:21:56.989822984] (+0.000001118) hrtimer_start: { 1 }, { hrtimer = 3993865440, function = 3238465232, expires = 79815981000000, softexpires = 79815981000000 }
2134 [23:21:56.989832762] (+0.000009778) softirq_entry: { 1 }, { vec = 1 }
2135 [23:21:56.989833879] (+0.000001117) softirq_entry: { 0 }, { vec = 1 }
2136 [23:21:56.989838069] (+0.000004190) timer_cancel: { 1 }, { timer = 3993871956 }
2137 [23:21:56.989839187] (+0.000001118) timer_cancel: { 0 }, { timer = 3993818708 }
2138 [23:21:56.989841492] (+0.000002305) timer_expire_entry: { 1 }, { timer = 3993871956, now = 79515980, function = 3238277552 }
2139 [23:21:56.989842819] (+0.000001327) timer_expire_entry: { 0 }, { timer = 3993818708, now = 79515980, function = 3238277552 }
2140 [23:21:56.989854831] (+0.000012012) sched_stat_runtime: { 1 }, { comm = "lttng-consumerd", tid = 1193, runtime = 49237, vruntime = 43368363335 }
2141 [23:21:56.989855949] (+0.000001118) sched_stat_runtime: { 0 }, { comm = "lttng-sessiond", tid = 1181, runtime = 45121, vruntime = 36976778361 }
2142 [23:21:56.989861257] (+0.000005308) sched_stat_sleep: { 1 }, { comm = "kworker/1:1", tid = 21, delay = 9451318 }
2143 [23:21:56.989862374] (+0.000001117) sched_stat_sleep: { 0 }, { comm = "kworker/0:0", tid = 4, delay = 9958820 }
2144 [23:21:56.989868241] (+0.000005867) sched_wakeup: { 0 }, { comm = "kworker/0:0", tid = 4, prio = 120, success = 1, target_cpu = 0 }
2145 [23:21:56.989869358] (+0.000001117) sched_wakeup: { 1 }, { comm = "kworker/1:1", tid = 21, prio = 120, success = 1, target_cpu = 1 }
2146 [23:21:56.989877460] (+0.000008102) timer_expire_exit: { 1 }, { timer = 3993871956 }
2147 [23:21:56.989878577] (+0.000001117) timer_expire_exit: { 0 }, { timer = 3993818708 }
2148 .
2149 .
2150 .
2151
2152You can now safely destroy the trace
2153session (note that this doesn't delete the trace - it's still there in
2154~/lttng-traces): ::
2155
2156 root@crownbay:~# lttng destroy
2157 Session auto-20121015-232120 destroyed at /home/root
2158
2159Note that the trace is saved in a directory of the same name as returned by
2160'lttng create', under the ~/lttng-traces directory (note that you can change this by
2161supplying your own name to 'lttng create'): ::
2162
2163 root@crownbay:~# ls -al ~/lttng-traces
2164 drwxrwx--- 3 root root 1024 Oct 15 23:21 .
2165 drwxr-xr-x 5 root root 1024 Oct 15 23:57 ..
2166 drwxrwx--- 3 root root 1024 Oct 15 23:21 auto-20121015-232120
2167
2168Collecting and viewing a userspace trace on the target (inside a shell)
2169~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2170
2171For LTTng userspace tracing, you need to have a properly instrumented
2172userspace program. For this example, we'll use the 'hello' test program
2173generated by the lttng-ust build.
2174
2175The 'hello' test program isn't installed on the rootfs by the lttng-ust
2176build, so we need to copy it over manually. First cd into the build
2177directory that contains the hello executable: ::
2178
2179 $ cd build/tmp/work/core2_32-poky-linux/lttng-ust/2.0.5-r0/git/tests/hello/.libs
2180
2181Copy that over to the target machine: ::
2182
2183 $ scp hello root@192.168.1.20:
2184
2185You now have the instrumented lttng 'hello world' test program on the
2186target, ready to test.
2187
2188First, from the host, ssh to the target: ::
2189
2190 $ ssh -l root 192.168.1.47
2191 The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
2192 RSA key fingerprint is 23:bd:c8:b1:a8:71:52:00:ee:00:4f:64:9e:10:b9:7e.
2193 Are you sure you want to continue connecting (yes/no)? yes
2194 Warning: Permanently added '192.168.1.47' (RSA) to the list of known hosts.
2195 root@192.168.1.47's password:
2196
2197Once on the target, use these steps to create a trace: ::
2198
2199 root@crownbay:~# lttng create
2200 Session auto-20190303-021943 created.
2201 Traces will be written in /home/root/lttng-traces/auto-20190303-021943
2202
2203Enable the events you want to trace (in this case all userspace events): ::
2204
2205 root@crownbay:~# lttng enable-event --userspace --all
2206 All UST events are enabled in channel channel0
2207
2208Start the trace: ::
2209
2210 root@crownbay:~# lttng start
2211 Tracing started for session auto-20190303-021943
2212
2213Run the instrumented hello world program: ::
2214
2215 root@crownbay:~# ./hello
2216 Hello, World!
2217 Tracing... done.
2218
2219And then stop the trace after awhile or after running a particular workload
2220that you want to trace: ::
2221
2222 root@crownbay:~# lttng stop
2223 Tracing stopped for session auto-20190303-021943
2224
2225You can now view the trace in text form on the target: ::
2226
2227 root@crownbay:~# lttng view
2228 [02:31:14.906146544] (+?.?????????) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 0, intfield2 = 0x0, longfield = 0, netintfield = 0, netintfieldhex = 0x0, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ], arrfield2 = "test", _seqfield1_length = 4, seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ], _seqfield2_length = 4, seqfield2 = "test", stringfield = "test", floatfield = 2222, doublefield = 2, boolfield = 1 }
2229 [02:31:14.906170360] (+0.000023816) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 1, intfield2 = 0x1, longfield = 1, netintfield = 1, netintfieldhex = 0x1, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ], arrfield2 = "test", _seqfield1_length = 4, seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ], _seqfield2_length = 4, seqfield2 = "test", stringfield = "test", floatfield = 2222, doublefield = 2, boolfield = 1 }
2230 [02:31:14.906183140] (+0.000012780) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 2, intfield2 = 0x2, longfield = 2, netintfield = 2, netintfieldhex = 0x2, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ], arrfield2 = "test", _seqfield1_length = 4, seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ], _seqfield2_length = 4, seqfield2 = "test", stringfield = "test", floatfield = 2222, doublefield = 2, boolfield = 1 }
2231 [02:31:14.906194385] (+0.000011245) hello:1424 ust_tests_hello:tptest: { cpu_id = 1 }, { intfield = 3, intfield2 = 0x3, longfield = 3, netintfield = 3, netintfieldhex = 0x3, arrfield1 = [ [0] = 1, [1] = 2, [2] = 3 ], arrfield2 = "test", _seqfield1_length = 4, seqfield1 = [ [0] = 116, [1] = 101, [2] = 115, [3] = 116 ], _seqfield2_length = 4, seqfield2 = "test", stringfield = "test", floatfield = 2222, doublefield = 2, boolfield = 1 }
2232 .
2233 .
2234 .
2235
2236You can now safely destroy the trace session (note that this doesn't delete the
2237trace - it's still there in ~/lttng-traces): ::
2238
2239 root@crownbay:~# lttng destroy
2240 Session auto-20190303-021943 destroyed at /home/root
2241
2242.. _lltng-documentation:
2243
2244LTTng Documentation
2245-------------------
2246
2247You can find the primary LTTng Documentation on the `LTTng
2248Documentation <https://lttng.org/docs/>`__ site. The documentation on
2249this site is appropriate for intermediate to advanced software
2250developers who are working in a Linux environment and are interested in
2251efficient software tracing.
2252
2253For information on LTTng in general, visit the `LTTng
2254Project <http://lttng.org/lttng2.0>`__ site. You can find a "Getting
2255Started" link on this site that takes you to an LTTng Quick Start.
2256
2257.. _profile-manual-blktrace:
2258
2259blktrace
2260========
2261
2262blktrace is a tool for tracing and reporting low-level disk I/O.
2263blktrace provides the tracing half of the equation; its output can be
2264piped into the blkparse program, which renders the data in a
2265human-readable form and does some basic analysis:
2266
2267.. _blktrace-setup:
2268
2269blktrace Setup
2270--------------
2271
2272For this section, we'll assume you've already performed the basic setup
2273outlined in the ":ref:`profile-manual/profile-manual-intro:General Setup`"
2274section.
2275
2276blktrace is an application that runs on the target system. You can run
2277the entire blktrace and blkparse pipeline on the target, or you can run
2278blktrace in 'listen' mode on the target and have blktrace and blkparse
2279collect and analyze the data on the host (see the
2280":ref:`profile-manual/profile-manual-usage:Using blktrace Remotely`" section
2281below). For the rest of this section we assume you've ssh'ed to the host and
2282will be running blkrace on the target.
2283
2284.. _blktrace-basic-usage:
2285
2286Basic blktrace Usage
2287--------------------
2288
2289To record a trace, simply run the 'blktrace' command, giving it the name
2290of the block device you want to trace activity on: ::
2291
2292 root@crownbay:~# blktrace /dev/sdc
2293
2294In another shell, execute a workload you want to trace. ::
2295
2296 root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2; sync
2297 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
2298 linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA
2299
2300Press Ctrl-C in the blktrace shell to stop the trace. It
2301will display how many events were logged, along with the per-cpu file
2302sizes (blktrace records traces in per-cpu kernel buffers and simply
2303dumps them to userspace for blkparse to merge and sort later). ::
2304
2305 ^C=== sdc ===
2306 CPU 0: 7082 events, 332 KiB data
2307 CPU 1: 1578 events, 74 KiB data
2308 Total: 8660 events (dropped 0), 406 KiB data
2309
2310If you examine the files saved to disk, you see multiple files, one per CPU and
2311with the device name as the first part of the filename: ::
2312
2313 root@crownbay:~# ls -al
2314 drwxr-xr-x 6 root root 1024 Oct 27 22:39 .
2315 drwxr-sr-x 4 root root 1024 Oct 26 18:24 ..
2316 -rw-r--r-- 1 root root 339938 Oct 27 22:40 sdc.blktrace.0
2317 -rw-r--r-- 1 root root 75753 Oct 27 22:40 sdc.blktrace.1
2318
2319To view the trace events, simply invoke 'blkparse' in the directory
2320containing the trace files, giving it the device name that forms the
2321first part of the filenames: ::
2322
2323 root@crownbay:~# blkparse sdc
2324
2325 8,32 1 1 0.000000000 1225 Q WS 3417048 + 8 [jbd2/sdc-8]
2326 8,32 1 2 0.000025213 1225 G WS 3417048 + 8 [jbd2/sdc-8]
2327 8,32 1 3 0.000033384 1225 P N [jbd2/sdc-8]
2328 8,32 1 4 0.000043301 1225 I WS 3417048 + 8 [jbd2/sdc-8]
2329 8,32 1 0 0.000057270 0 m N cfq1225 insert_request
2330 8,32 1 0 0.000064813 0 m N cfq1225 add_to_rr
2331 8,32 1 5 0.000076336 1225 U N [jbd2/sdc-8] 1
2332 8,32 1 0 0.000088559 0 m N cfq workload slice:150
2333 8,32 1 0 0.000097359 0 m N cfq1225 set_active wl_prio:0 wl_type:1
2334 8,32 1 0 0.000104063 0 m N cfq1225 Not idling. st->count:1
2335 8,32 1 0 0.000112584 0 m N cfq1225 fifo= (null)
2336 8,32 1 0 0.000118730 0 m N cfq1225 dispatch_insert
2337 8,32 1 0 0.000127390 0 m N cfq1225 dispatched a request
2338 8,32 1 0 0.000133536 0 m N cfq1225 activate rq, drv=1
2339 8,32 1 6 0.000136889 1225 D WS 3417048 + 8 [jbd2/sdc-8]
2340 8,32 1 7 0.000360381 1225 Q WS 3417056 + 8 [jbd2/sdc-8]
2341 8,32 1 8 0.000377422 1225 G WS 3417056 + 8 [jbd2/sdc-8]
2342 8,32 1 9 0.000388876 1225 P N [jbd2/sdc-8]
2343 8,32 1 10 0.000397886 1225 Q WS 3417064 + 8 [jbd2/sdc-8]
2344 8,32 1 11 0.000404800 1225 M WS 3417064 + 8 [jbd2/sdc-8]
2345 8,32 1 12 0.000412343 1225 Q WS 3417072 + 8 [jbd2/sdc-8]
2346 8,32 1 13 0.000416533 1225 M WS 3417072 + 8 [jbd2/sdc-8]
2347 8,32 1 14 0.000422121 1225 Q WS 3417080 + 8 [jbd2/sdc-8]
2348 8,32 1 15 0.000425194 1225 M WS 3417080 + 8 [jbd2/sdc-8]
2349 8,32 1 16 0.000431968 1225 Q WS 3417088 + 8 [jbd2/sdc-8]
2350 8,32 1 17 0.000435251 1225 M WS 3417088 + 8 [jbd2/sdc-8]
2351 8,32 1 18 0.000440279 1225 Q WS 3417096 + 8 [jbd2/sdc-8]
2352 8,32 1 19 0.000443911 1225 M WS 3417096 + 8 [jbd2/sdc-8]
2353 8,32 1 20 0.000450336 1225 Q WS 3417104 + 8 [jbd2/sdc-8]
2354 8,32 1 21 0.000454038 1225 M WS 3417104 + 8 [jbd2/sdc-8]
2355 8,32 1 22 0.000462070 1225 Q WS 3417112 + 8 [jbd2/sdc-8]
2356 8,32 1 23 0.000465422 1225 M WS 3417112 + 8 [jbd2/sdc-8]
2357 8,32 1 24 0.000474222 1225 I WS 3417056 + 64 [jbd2/sdc-8]
2358 8,32 1 0 0.000483022 0 m N cfq1225 insert_request
2359 8,32 1 25 0.000489727 1225 U N [jbd2/sdc-8] 1
2360 8,32 1 0 0.000498457 0 m N cfq1225 Not idling. st->count:1
2361 8,32 1 0 0.000503765 0 m N cfq1225 dispatch_insert
2362 8,32 1 0 0.000512914 0 m N cfq1225 dispatched a request
2363 8,32 1 0 0.000518851 0 m N cfq1225 activate rq, drv=2
2364 .
2365 .
2366 .
2367 8,32 0 0 58.515006138 0 m N cfq3551 complete rqnoidle 1
2368 8,32 0 2024 58.516603269 3 C WS 3156992 + 16 [0]
2369 8,32 0 0 58.516626736 0 m N cfq3551 complete rqnoidle 1
2370 8,32 0 0 58.516634558 0 m N cfq3551 arm_idle: 8 group_idle: 0
2371 8,32 0 0 58.516636933 0 m N cfq schedule dispatch
2372 8,32 1 0 58.516971613 0 m N cfq3551 slice expired t=0
2373 8,32 1 0 58.516982089 0 m N cfq3551 sl_used=13 disp=6 charge=13 iops=0 sect=80
2374 8,32 1 0 58.516985511 0 m N cfq3551 del_from_rr
2375 8,32 1 0 58.516990819 0 m N cfq3551 put_queue
2376
2377 CPU0 (sdc):
2378 Reads Queued: 0, 0KiB Writes Queued: 331, 26,284KiB
2379 Read Dispatches: 0, 0KiB Write Dispatches: 485, 40,484KiB
2380 Reads Requeued: 0 Writes Requeued: 0
2381 Reads Completed: 0, 0KiB Writes Completed: 511, 41,000KiB
2382 Read Merges: 0, 0KiB Write Merges: 13, 160KiB
2383 Read depth: 0 Write depth: 2
2384 IO unplugs: 23 Timer unplugs: 0
2385 CPU1 (sdc):
2386 Reads Queued: 0, 0KiB Writes Queued: 249, 15,800KiB
2387 Read Dispatches: 0, 0KiB Write Dispatches: 42, 1,600KiB
2388 Reads Requeued: 0 Writes Requeued: 0
2389 Reads Completed: 0, 0KiB Writes Completed: 16, 1,084KiB
2390 Read Merges: 0, 0KiB Write Merges: 40, 276KiB
2391 Read depth: 0 Write depth: 2
2392 IO unplugs: 30 Timer unplugs: 1
2393
2394 Total (sdc):
2395 Reads Queued: 0, 0KiB Writes Queued: 580, 42,084KiB
2396 Read Dispatches: 0, 0KiB Write Dispatches: 527, 42,084KiB
2397 Reads Requeued: 0 Writes Requeued: 0
2398 Reads Completed: 0, 0KiB Writes Completed: 527, 42,084KiB
2399 Read Merges: 0, 0KiB Write Merges: 53, 436KiB
2400 IO unplugs: 53 Timer unplugs: 1
2401
2402 Throughput (R/W): 0KiB/s / 719KiB/s
2403 Events (sdc): 6,592 entries
2404 Skips: 0 forward (0 - 0.0%)
2405 Input file sdc.blktrace.0 added
2406 Input file sdc.blktrace.1 added
2407
2408The report shows each event that was
2409found in the blktrace data, along with a summary of the overall block
2410I/O traffic during the run. You can look at the
2411`blkparse <http://linux.die.net/man/1/blkparse>`__ manpage to learn the
2412meaning of each field displayed in the trace listing.
2413
2414.. _blktrace-live-mode:
2415
2416Live Mode
2417~~~~~~~~~
2418
2419blktrace and blkparse are designed from the ground up to be able to
2420operate together in a 'pipe mode' where the stdout of blktrace can be
2421fed directly into the stdin of blkparse: ::
2422
2423 root@crownbay:~# blktrace /dev/sdc -o - | blkparse -i -
2424
2425This enables long-lived tracing sessions
2426to run without writing anything to disk, and allows the user to look for
2427certain conditions in the trace data in 'real-time' by viewing the trace
2428output as it scrolls by on the screen or by passing it along to yet
2429another program in the pipeline such as grep which can be used to
2430identify and capture conditions of interest.
2431
2432There's actually another blktrace command that implements the above
2433pipeline as a single command, so the user doesn't have to bother typing
2434in the above command sequence: ::
2435
2436 root@crownbay:~# btrace /dev/sdc
2437
2438Using blktrace Remotely
2439~~~~~~~~~~~~~~~~~~~~~~~
2440
2441Because blktrace traces block I/O and at the same time normally writes
2442its trace data to a block device, and in general because it's not really
2443a great idea to make the device being traced the same as the device the
2444tracer writes to, blktrace provides a way to trace without perturbing
2445the traced device at all by providing native support for sending all
2446trace data over the network.
2447
2448To have blktrace operate in this mode, start blktrace on the target
2449system being traced with the -l option, along with the device to trace: ::
2450
2451 root@crownbay:~# blktrace -l /dev/sdc
2452 server: waiting for connections...
2453
2454On the host system, use the -h option to connect to the target system,
2455also passing it the device to trace: ::
2456
2457 $ blktrace -d /dev/sdc -h 192.168.1.43
2458 blktrace: connecting to 192.168.1.43
2459 blktrace: connected!
2460
2461On the target system, you should see this: ::
2462
2463 server: connection from 192.168.1.43
2464
2465In another shell, execute a workload you want to trace. ::
2466
2467 root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2; sync
2468 Connecting to downloads.yoctoproject.org (140.211.169.59:80)
2469 linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA
2470
2471When it's done, do a Ctrl-C on the host system to stop the
2472trace: ::
2473
2474 ^C=== sdc ===
2475 CPU 0: 7691 events, 361 KiB data
2476 CPU 1: 4109 events, 193 KiB data
2477 Total: 11800 events (dropped 0), 554 KiB data
2478
2479On the target system, you should also see a trace summary for the trace
2480just ended: ::
2481
2482 server: end of run for 192.168.1.43:sdc
2483 === sdc ===
2484 CPU 0: 7691 events, 361 KiB data
2485 CPU 1: 4109 events, 193 KiB data
2486 Total: 11800 events (dropped 0), 554 KiB data
2487
2488The blktrace instance on the host will
2489save the target output inside a hostname-timestamp directory: ::
2490
2491 $ ls -al
2492 drwxr-xr-x 10 root root 1024 Oct 28 02:40 .
2493 drwxr-sr-x 4 root root 1024 Oct 26 18:24 ..
2494 drwxr-xr-x 2 root root 1024 Oct 28 02:40 192.168.1.43-2012-10-28-02:40:56
2495
2496cd into that directory to see the output files: ::
2497
2498 $ ls -l
2499 -rw-r--r-- 1 root root 369193 Oct 28 02:44 sdc.blktrace.0
2500 -rw-r--r-- 1 root root 197278 Oct 28 02:44 sdc.blktrace.1
2501
2502And run blkparse on the host system using the device name: ::
2503
2504 $ blkparse sdc
2505
2506 8,32 1 1 0.000000000 1263 Q RM 6016 + 8 [ls]
2507 8,32 1 0 0.000036038 0 m N cfq1263 alloced
2508 8,32 1 2 0.000039390 1263 G RM 6016 + 8 [ls]
2509 8,32 1 3 0.000049168 1263 I RM 6016 + 8 [ls]
2510 8,32 1 0 0.000056152 0 m N cfq1263 insert_request
2511 8,32 1 0 0.000061600 0 m N cfq1263 add_to_rr
2512 8,32 1 0 0.000075498 0 m N cfq workload slice:300
2513 .
2514 .
2515 .
2516 8,32 0 0 177.266385696 0 m N cfq1267 arm_idle: 8 group_idle: 0
2517 8,32 0 0 177.266388140 0 m N cfq schedule dispatch
2518 8,32 1 0 177.266679239 0 m N cfq1267 slice expired t=0
2519 8,32 1 0 177.266689297 0 m N cfq1267 sl_used=9 disp=6 charge=9 iops=0 sect=56
2520 8,32 1 0 177.266692649 0 m N cfq1267 del_from_rr
2521 8,32 1 0 177.266696560 0 m N cfq1267 put_queue
2522
2523 CPU0 (sdc):
2524 Reads Queued: 0, 0KiB Writes Queued: 270, 21,708KiB
2525 Read Dispatches: 59, 2,628KiB Write Dispatches: 495, 39,964KiB
2526 Reads Requeued: 0 Writes Requeued: 0
2527 Reads Completed: 90, 2,752KiB Writes Completed: 543, 41,596KiB
2528 Read Merges: 0, 0KiB Write Merges: 9, 344KiB
2529 Read depth: 2 Write depth: 2
2530 IO unplugs: 20 Timer unplugs: 1
2531 CPU1 (sdc):
2532 Reads Queued: 688, 2,752KiB Writes Queued: 381, 20,652KiB
2533 Read Dispatches: 31, 124KiB Write Dispatches: 59, 2,396KiB
2534 Reads Requeued: 0 Writes Requeued: 0
2535 Reads Completed: 0, 0KiB Writes Completed: 11, 764KiB
2536 Read Merges: 598, 2,392KiB Write Merges: 88, 448KiB
2537 Read depth: 2 Write depth: 2
2538 IO unplugs: 52 Timer unplugs: 0
2539
2540 Total (sdc):
2541 Reads Queued: 688, 2,752KiB Writes Queued: 651, 42,360KiB
2542 Read Dispatches: 90, 2,752KiB Write Dispatches: 554, 42,360KiB
2543 Reads Requeued: 0 Writes Requeued: 0
2544 Reads Completed: 90, 2,752KiB Writes Completed: 554, 42,360KiB
2545 Read Merges: 598, 2,392KiB Write Merges: 97, 792KiB
2546 IO unplugs: 72 Timer unplugs: 1
2547
2548 Throughput (R/W): 15KiB/s / 238KiB/s
2549 Events (sdc): 9,301 entries
2550 Skips: 0 forward (0 - 0.0%)
2551
2552You should see the trace events and summary just as you would have if you'd run
2553the same command on the target.
2554
2555Tracing Block I/O via 'ftrace'
2556~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2557
2558It's also possible to trace block I/O using only
2559:ref:`profile-manual/profile-manual-usage:The 'trace events' Subsystem`, which
2560can be useful for casual tracing if you don't want to bother dealing with the
2561userspace tools.
2562
2563To enable tracing for a given device, use /sys/block/xxx/trace/enable,
2564where xxx is the device name. This for example enables tracing for
2565/dev/sdc: ::
2566
2567 root@crownbay:/sys/kernel/debug/tracing# echo 1 > /sys/block/sdc/trace/enable
2568
2569Once you've selected the device(s) you want
2570to trace, selecting the 'blk' tracer will turn the blk tracer on: ::
2571
2572 root@crownbay:/sys/kernel/debug/tracing# cat available_tracers
2573 blk function_graph function nop
2574
2575 root@crownbay:/sys/kernel/debug/tracing# echo blk > current_tracer
2576
2577Execute the workload you're interested in: ::
2578
2579 root@crownbay:/sys/kernel/debug/tracing# cat /media/sdc/testfile.txt
2580
2581And look at the output (note here that we're using 'trace_pipe' instead of
2582trace to capture this trace - this allows us to wait around on the pipe
2583for data to appear): ::
2584
2585 root@crownbay:/sys/kernel/debug/tracing# cat trace_pipe
2586 cat-3587 [001] d..1 3023.276361: 8,32 Q R 1699848 + 8 [cat]
2587 cat-3587 [001] d..1 3023.276410: 8,32 m N cfq3587 alloced
2588 cat-3587 [001] d..1 3023.276415: 8,32 G R 1699848 + 8 [cat]
2589 cat-3587 [001] d..1 3023.276424: 8,32 P N [cat]
2590 cat-3587 [001] d..2 3023.276432: 8,32 I R 1699848 + 8 [cat]
2591 cat-3587 [001] d..1 3023.276439: 8,32 m N cfq3587 insert_request
2592 cat-3587 [001] d..1 3023.276445: 8,32 m N cfq3587 add_to_rr
2593 cat-3587 [001] d..2 3023.276454: 8,32 U N [cat] 1
2594 cat-3587 [001] d..1 3023.276464: 8,32 m N cfq workload slice:150
2595 cat-3587 [001] d..1 3023.276471: 8,32 m N cfq3587 set_active wl_prio:0 wl_type:2
2596 cat-3587 [001] d..1 3023.276478: 8,32 m N cfq3587 fifo= (null)
2597 cat-3587 [001] d..1 3023.276483: 8,32 m N cfq3587 dispatch_insert
2598 cat-3587 [001] d..1 3023.276490: 8,32 m N cfq3587 dispatched a request
2599 cat-3587 [001] d..1 3023.276497: 8,32 m N cfq3587 activate rq, drv=1
2600 cat-3587 [001] d..2 3023.276500: 8,32 D R 1699848 + 8 [cat]
2601
2602And this turns off tracing for the specified device: ::
2603
2604 root@crownbay:/sys/kernel/debug/tracing# echo 0 > /sys/block/sdc/trace/enable
2605
2606.. _blktrace-documentation:
2607
2608blktrace Documentation
2609----------------------
2610
2611Online versions of the man pages for the commands discussed in this
2612section can be found here:
2613
2614- http://linux.die.net/man/8/blktrace
2615
2616- http://linux.die.net/man/1/blkparse
2617
2618- http://linux.die.net/man/8/btrace
2619
2620The above manpages, along with manpages for the other blktrace utilities
2621(btt, blkiomon, etc) can be found in the /doc directory of the blktrace
2622tools git repo: ::
2623
2624 $ git clone git://git.kernel.dk/blktrace.git
diff --git a/documentation/profile-manual/profile-manual-usage.xml b/documentation/profile-manual/profile-manual-usage.xml
index 9a4273a0fe..3a7148cbd3 100644
--- a/documentation/profile-manual/profile-manual-usage.xml
+++ b/documentation/profile-manual/profile-manual-usage.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='profile-manual-usage'> 6<chapter id='profile-manual-usage'>
6 7
diff --git a/documentation/profile-manual/profile-manual.rst b/documentation/profile-manual/profile-manual.rst
new file mode 100644
index 0000000000..2c8fcf3e60
--- /dev/null
+++ b/documentation/profile-manual/profile-manual.rst
@@ -0,0 +1,19 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3==========================================
4Yocto Project Profiling and Tracing Manual
5==========================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 profile-manual-intro
14 profile-manual-arch
15 profile-manual-usage
16 profile-manual-examples
17 history
18
19.. include:: /boilerplate.rst
diff --git a/documentation/profile-manual/profile-manual.xml b/documentation/profile-manual/profile-manual.xml
index 39cdb817d9..48bfba5b8f 100755
--- a/documentation/profile-manual/profile-manual.xml
+++ b/documentation/profile-manual/profile-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='profile-manual' lang='en' 6<book id='profile-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -102,28 +103,8 @@
102 </revision> 103 </revision>
103 <revision> 104 <revision>
104 <revnumber>3.1</revnumber> 105 <revnumber>3.1</revnumber>
105 <date>April 2020</date>
106 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
107 </revision>
108 <revision>
109 <revnumber>3.1.1</revnumber>
110 <date>June 2020</date>
111 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
112 </revision>
113 <revision>
114 <revnumber>3.1.2</revnumber>
115 <date>August 2020</date>
116 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
117 </revision>
118 <revision>
119 <revnumber>3.1.3</revnumber>
120 <date>October 2020</date>
121 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
122 </revision>
123 <revision>
124 <revnumber>3.1.4</revnumber>
125 <date>&REL_MONTH_YEAR;</date> 106 <date>&REL_MONTH_YEAR;</date>
126 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 107 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
127 </revision> 108 </revision>
128 </revhistory> 109 </revhistory>
129 110
diff --git a/documentation/ref-manual/examples/hello-autotools/hello_2.10.bb b/documentation/ref-manual/examples/hello-autotools/hello_2.10.bb
new file mode 100644
index 0000000000..aa2beb9a9b
--- /dev/null
+++ b/documentation/ref-manual/examples/hello-autotools/hello_2.10.bb
@@ -0,0 +1,9 @@
1DESCRIPTION = "GNU Helloworld application"
2SECTION = "examples"
3LICENSE = "GPLv3"
4LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
5
6SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"
7SRC_URI[sha256sum] = "31e066137a962676e89f69d1b65382de95a7ef7d914b8cb956f41ea72e0f516b"
8
9inherit autotools-brokensep gettext
diff --git a/documentation/ref-manual/examples/hello-autotools/hello_2.3.bb b/documentation/ref-manual/examples/hello-autotools/hello_2.3.bb
deleted file mode 100644
index 5dfb0b30cf..0000000000
--- a/documentation/ref-manual/examples/hello-autotools/hello_2.3.bb
+++ /dev/null
@@ -1,8 +0,0 @@
1DESCRIPTION = "GNU Helloworld application"
2SECTION = "examples"
3LICENSE = "GPLv3"
4LIC_FILES_CHKSUM = "file://COPYING;md5=adefda309052235aa5d1e99ce7557010"
5
6SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.bz2"
7
8inherit autotools
diff --git a/documentation/ref-manual/examples/libxpm/libxpm_3.5.6.bb b/documentation/ref-manual/examples/libxpm/libxpm_3.5.6.bb
index b58d4d7bd1..c0c8986405 100644
--- a/documentation/ref-manual/examples/libxpm/libxpm_3.5.6.bb
+++ b/documentation/ref-manual/examples/libxpm/libxpm_3.5.6.bb
@@ -1,4 +1,4 @@
1require xorg-lib-common.inc 1require recipes-graphics/xorg-lib/xorg-lib-common.inc
2 2
3DESCRIPTION = "X11 Pixmap library" 3DESCRIPTION = "X11 Pixmap library"
4LICENSE = "X-BSD" 4LICENSE = "X-BSD"
diff --git a/documentation/ref-manual/faq.rst b/documentation/ref-manual/faq.rst
new file mode 100644
index 0000000000..2d2aaad0a9
--- /dev/null
+++ b/documentation/ref-manual/faq.rst
@@ -0,0 +1,451 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***
4FAQ
5***
6
7**Q:** How does Poky differ from `OpenEmbedded <http://www.openembedded.org/>`__?
8
9**A:** The term ``Poky`` refers to the specific reference build
10system that the Yocto Project provides. Poky is based on
11:term:`OpenEmbedded-Core (OE-Core)` and :term:`BitBake`. Thus, the
12generic term used here for the build system is the "OpenEmbedded build
13system." Development in the Yocto Project using Poky is closely tied to
14OpenEmbedded, with changes always being merged to OE-Core or BitBake
15first before being pulled back into Poky. This practice benefits both
16projects immediately.
17
18**Q:** My development system does not meet the required Git, tar, and
19Python versions. In particular, I do not have Python 3.5.0 or greater.
20Can I still use the Yocto Project?
21
22**A:** You can get the required tools on your host development system a
23couple different ways (i.e. building a tarball or downloading a
24tarball). See the "`Required Git, tar, Python and gcc
25Versions <#required-git-tar-python-and-gcc-versions>`__" section for
26steps on how to update your build tools.
27
28**Q:** How can you claim Poky / OpenEmbedded-Core is stable?
29
30**A:** There are three areas that help with stability;
31
32- The Yocto Project team keeps :term:`OpenEmbedded-Core (OE-Core)` small and
33 focused, containing around 830 recipes as opposed to the thousands
34 available in other OpenEmbedded community layers. Keeping it small
35 makes it easy to test and maintain.
36
37- The Yocto Project team runs manual and automated tests using a small,
38 fixed set of reference hardware as well as emulated targets.
39
40- The Yocto Project uses an autobuilder, which provides continuous
41 build and integration tests.
42
43**Q:** How do I get support for my board added to the Yocto Project?
44
45**A:** Support for an additional board is added by creating a Board
46Support Package (BSP) layer for it. For more information on how to
47create a BSP layer, see the
48":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
49section in the Yocto Project Development Tasks Manual and the
50:doc:`../bsp-guide/bsp-guide`.
51
52Usually, if the board is not completely exotic, adding support in the
53Yocto Project is fairly straightforward.
54
55**Q:** Are there any products built using the OpenEmbedded build system?
56
57**A:** The software running on the `Vernier
58LabQuest <http://vernier.com/labquest/>`__ is built using the
59OpenEmbedded build system. See the `Vernier
60LabQuest <http://www.vernier.com/products/interfaces/labq/>`__ website
61for more information. There are a number of pre-production devices using
62the OpenEmbedded build system and the Yocto Project team announces them
63as soon as they are released.
64
65**Q:** What does the OpenEmbedded build system produce as output?
66
67**A:** Because you can use the same set of recipes to create output of
68various formats, the output of an OpenEmbedded build depends on how you
69start it. Usually, the output is a flashable image ready for the target
70device.
71
72**Q:** How do I add my package to the Yocto Project?
73
74**A:** To add a package, you need to create a BitBake recipe. For
75information on how to create a BitBake recipe, see the
76":ref:`dev-manual/dev-manual-common-tasks:writing a new recipe`"
77section in the Yocto Project Development Tasks Manual.
78
79**Q:** Do I have to reflash my entire board with a new Yocto Project
80image when recompiling a package?
81
82**A:** The OpenEmbedded build system can build packages in various
83formats such as IPK for OPKG, Debian package (``.deb``), or RPM. You can
84then upgrade the packages using the package tools on the device, much
85like on a desktop distribution such as Ubuntu or Fedora. However,
86package management on the target is entirely optional.
87
88**Q:** I see the error
89'``chmod: XXXXX new permissions are r-xrwxrwx, not r-xr-xr-x``'. What is
90wrong?
91
92**A:** You are probably running the build on an NTFS filesystem. Use
93``ext2``, ``ext3``, or ``ext4`` instead.
94
95**Q:** I see lots of 404 responses for files when the OpenEmbedded build
96system is trying to download sources. Is something wrong?
97
98**A:** Nothing is wrong. The OpenEmbedded build system checks any
99configured source mirrors before downloading from the upstream sources.
100The build system does this searching for both source archives and
101pre-checked out versions of SCM-managed software. These checks help in
102large installations because it can reduce load on the SCM servers
103themselves. The address above is one of the default mirrors configured
104into the build system. Consequently, if an upstream source disappears,
105the team can place sources there so builds continue to work.
106
107**Q:** I have machine-specific data in a package for one machine only
108but the package is being marked as machine-specific in all cases, how do
109I prevent this?
110
111**A:** Set ``SRC_URI_OVERRIDES_PACKAGE_ARCH`` = "0" in the ``.bb`` file
112but make sure the package is manually marked as machine-specific for the
113case that needs it. The code that handles
114``SRC_URI_OVERRIDES_PACKAGE_ARCH`` is in the
115``meta/classes/base.bbclass`` file.
116
117**Q:** I'm behind a firewall and need to use a proxy server. How do I do
118that?
119
120**A:** Most source fetching by the OpenEmbedded build system is done by
121``wget`` and you therefore need to specify the proxy settings in a
122``.wgetrc`` file, which can be in your home directory if you are a
123single user or can be in ``/usr/local/etc/wgetrc`` as a global user
124file.
125
126Following is the applicable code for setting various proxy types in the
127``.wgetrc`` file. By default, these settings are disabled with comments.
128To use them, remove the comments: ::
129
130 # You can set the default proxies for Wget to use for http, https, and ftp.
131 # They will override the value in the environment.
132 #https_proxy = http://proxy.yoyodyne.com:18023/
133 #http_proxy = http://proxy.yoyodyne.com:18023/
134 #ftp_proxy = http://proxy.yoyodyne.com:18023/
135
136 # If you do not want to use proxy at all, set this to off.
137 #use_proxy = on
138
139The Yocto Project also includes a
140``meta-poky/conf/site.conf.sample`` file that shows how to configure CVS
141and Git proxy servers if needed. For more information on setting up
142various proxy types and configuring proxy servers, see the
143":yocto_wiki:`Working Behind a Network Proxy </wiki/Working_Behind_a_Network_Proxy>`"
144Wiki page.
145
146**Q:** What's the difference between target and target\ ``-native``?
147
148**A:** The ``*-native`` targets are designed to run on the system being
149used for the build. These are usually tools that are needed to assist
150the build in some way such as ``quilt-native``, which is used to apply
151patches. The non-native version is the one that runs on the target
152device.
153
154**Q:** I'm seeing random build failures. Help?!
155
156**A:** If the same build is failing in totally different and random
157ways, the most likely explanation is:
158
159- The hardware you are running the build on has some problem.
160
161- You are running the build under virtualization, in which case the
162 virtualization probably has bugs.
163
164The OpenEmbedded build system processes a massive amount of data that
165causes lots of network, disk and CPU activity and is sensitive to even
166single-bit failures in any of these areas. True random failures have
167always been traced back to hardware or virtualization issues.
168
169**Q:** When I try to build a native recipe, the build fails with
170``iconv.h`` problems.
171
172**A:** If you get an error message that indicates GNU ``libiconv`` is
173not in use but ``iconv.h`` has been included from ``libiconv``, you need
174to check to see if you have a previously installed version of the header
175file in ``/usr/local/include``.
176::
177
178 #error GNU libiconv not in use but included iconv.h is from libiconv
179
180If you find a previously installed
181file, you should either uninstall it or temporarily rename it and try
182the build again.
183
184This issue is just a single manifestation of "system leakage" issues
185caused when the OpenEmbedded build system finds and uses previously
186installed files during a native build. This type of issue might not be
187limited to ``iconv.h``. Be sure that leakage cannot occur from
188``/usr/local/include`` and ``/opt`` locations.
189
190**Q:** What do we need to ship for license compliance?
191
192**A:** This is a difficult question and you need to consult your lawyer
193for the answer for your specific case. It is worth bearing in mind that
194for GPL compliance, there needs to be enough information shipped to
195allow someone else to rebuild and produce the same end result you are
196shipping. This means sharing the source code, any patches applied to it,
197and also any configuration information about how that package was
198configured and built.
199
200You can find more information on licensing in the
201":ref:`overview-manual/overview-manual-development-environment:licensing`"
202section in the Yocto
203Project Overview and Concepts Manual and also in the
204":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
205section in the Yocto Project Development Tasks Manual.
206
207**Q:** How do I disable the cursor on my touchscreen device?
208
209**A:** You need to create a form factor file as described in the
210":ref:`bsp-filelayout-misc-recipes`" section in
211the Yocto Project Board Support Packages (BSP) Developer's Guide. Set
212the ``HAVE_TOUCHSCREEN`` variable equal to one as follows:
213::
214
215 HAVE_TOUCHSCREEN=1
216
217**Q:** How do I make sure connected network interfaces are brought up by
218default?
219
220**A:** The default interfaces file provided by the netbase recipe does
221not automatically bring up network interfaces. Therefore, you will need
222to add a BSP-specific netbase that includes an interfaces file. See the
223":ref:`bsp-filelayout-misc-recipes`" section in
224the Yocto Project Board Support Packages (BSP) Developer's Guide for
225information on creating these types of miscellaneous recipe files.
226
227For example, add the following files to your layer: ::
228
229 meta-MACHINE/recipes-bsp/netbase/netbase/MACHINE/interfaces
230 meta-MACHINE/recipes-bsp/netbase/netbase_5.0.bbappend
231
232**Q:** How do I create images with more free space?
233
234**A:** By default, the OpenEmbedded build system creates images that are
2351.3 times the size of the populated root filesystem. To affect the image
236size, you need to set various configurations:
237
238- *Image Size:* The OpenEmbedded build system uses the
239 :term:`IMAGE_ROOTFS_SIZE` variable to define
240 the size of the image in Kbytes. The build system determines the size
241 by taking into account the initial root filesystem size before any
242 modifications such as requested size for the image and any requested
243 additional free disk space to be added to the image.
244
245- *Overhead:* Use the
246 :term:`IMAGE_OVERHEAD_FACTOR` variable
247 to define the multiplier that the build system applies to the initial
248 image size, which is 1.3 by default.
249
250- *Additional Free Space:* Use the
251 :term:`IMAGE_ROOTFS_EXTRA_SPACE`
252 variable to add additional free space to the image. The build system
253 adds this space to the image after it determines its
254 ``IMAGE_ROOTFS_SIZE``.
255
256**Q:** Why don't you support directories with spaces in the pathnames?
257
258**A:** The Yocto Project team has tried to do this before but too many
259of the tools the OpenEmbedded build system depends on, such as
260``autoconf``, break when they find spaces in pathnames. Until that
261situation changes, the team will not support spaces in pathnames.
262
263**Q:** How do I use an external toolchain?
264
265**A:** The toolchain configuration is very flexible and customizable. It
266is primarily controlled with the ``TCMODE`` variable. This variable
267controls which ``tcmode-*.inc`` file to include from the
268``meta/conf/distro/include`` directory within the :term:`Source Directory`.
269
270The default value of ``TCMODE`` is "default", which tells the
271OpenEmbedded build system to use its internally built toolchain (i.e.
272``tcmode-default.inc``). However, other patterns are accepted. In
273particular, "external-\*" refers to external toolchains. One example is
274the Sourcery G++ Toolchain. The support for this toolchain resides in
275the separate ``meta-sourcery`` layer at
276http://github.com/MentorEmbedded/meta-sourcery/.
277
278In addition to the toolchain configuration, you also need a
279corresponding toolchain recipe file. This recipe file needs to package
280up any pre-built objects in the toolchain such as ``libgcc``,
281``libstdcc++``, any locales, and ``libc``.
282
283**Q:** How does the OpenEmbedded build system obtain source code and
284will it work behind my firewall or proxy server?
285
286**A:** The way the build system obtains source code is highly
287configurable. You can setup the build system to get source code in most
288environments if HTTP transport is available.
289
290When the build system searches for source code, it first tries the local
291download directory. If that location fails, Poky tries
292:term:`PREMIRRORS`, the upstream source, and then
293:term:`MIRRORS` in that order.
294
295Assuming your distribution is "poky", the OpenEmbedded build system uses
296the Yocto Project source ``PREMIRRORS`` by default for SCM-based
297sources, upstreams for normal tarballs, and then falls back to a number
298of other mirrors including the Yocto Project source mirror if those
299fail.
300
301As an example, you could add a specific server for the build system to
302attempt before any others by adding something like the following to the
303``local.conf`` configuration file: ::
304
305 PREMIRRORS_prepend = "\
306 git://.*/.* http://www.yoctoproject.org/sources/ \n \
307 ftp://.*/.* http://www.yoctoproject.org/sources/ \n \
308 http://.*/.* http://www.yoctoproject.org/sources/ \n \
309 https://.*/.* http://www.yoctoproject.org/sources/ \n"
310
311These changes cause the build system to intercept Git, FTP, HTTP, and
312HTTPS requests and direct them to the ``http://`` sources mirror. You
313can use ``file://`` URLs to point to local directories or network shares
314as well.
315
316Aside from the previous technique, these options also exist:
317::
318
319 BB_NO_NETWORK = "1"
320
321This statement tells BitBake to issue an error
322instead of trying to access the Internet. This technique is useful if
323you want to ensure code builds only from local sources.
324
325Here is another technique:
326::
327
328 BB_FETCH_PREMIRRORONLY = "1"
329
330This statement
331limits the build system to pulling source from the ``PREMIRRORS`` only.
332Again, this technique is useful for reproducing builds.
333
334Here is another technique:
335::
336
337 BB_GENERATE_MIRROR_TARBALLS = "1"
338
339This
340statement tells the build system to generate mirror tarballs. This
341technique is useful if you want to create a mirror server. If not,
342however, the technique can simply waste time during the build.
343
344Finally, consider an example where you are behind an HTTP-only firewall.
345You could make the following changes to the ``local.conf`` configuration
346file as long as the ``PREMIRRORS`` server is current: ::
347
348 PREMIRRORS_prepend = "\
349 ftp://.*/.* http://www.yoctoproject.org/sources/ \n \
350 http://.*/.* http://www.yoctoproject.org/sources/ \n \
351 https://.*/.* http://www.yoctoproject.org/sources/ \n"
352 BB_FETCH_PREMIRRORONLY = "1"
353
354These changes would cause the build system to successfully fetch source
355over HTTP and any network accesses to anything other than the
356``PREMIRRORS`` would fail.
357
358The build system also honors the standard shell environment variables
359``http_proxy``, ``ftp_proxy``, ``https_proxy``, and ``all_proxy`` to
360redirect requests through proxy servers.
361
362.. note::
363
364 You can find more information on the
365 ":yocto_wiki:`Working Behind a Network Proxy </wiki/Working_Behind_a_Network_Proxy>`"
366 Wiki page.
367
368**Q:** Can I get rid of build output so I can start over?
369
370**A:** Yes - you can easily do this. When you use BitBake to build an
371image, all the build output goes into the directory created when you run
372the build environment setup script (i.e.
373````` <#structure-core-script>`__). By default, this :term:`Build Directory`
374is named ``build`` but can be named
375anything you want.
376
377Within the Build Directory, is the ``tmp`` directory. To remove all the
378build output yet preserve any source code or downloaded files from
379previous builds, simply remove the ``tmp`` directory.
380
381**Q:** Why do ``${bindir}`` and ``${libdir}`` have strange values for
382``-native`` recipes?
383
384**A:** Executables and libraries might need to be used from a directory
385other than the directory into which they were initially installed.
386Complicating this situation is the fact that sometimes these executables
387and libraries are compiled with the expectation of being run from that
388initial installation target directory. If this is the case, moving them
389causes problems.
390
391This scenario is a fundamental problem for package maintainers of
392mainstream Linux distributions as well as for the OpenEmbedded build
393system. As such, a well-established solution exists. Makefiles,
394Autotools configuration scripts, and other build systems are expected to
395respect environment variables such as ``bindir``, ``libdir``, and
396``sysconfdir`` that indicate where executables, libraries, and data
397reside when a program is actually run. They are also expected to respect
398a ``DESTDIR`` environment variable, which is prepended to all the other
399variables when the build system actually installs the files. It is
400understood that the program does not actually run from within
401``DESTDIR``.
402
403When the OpenEmbedded build system uses a recipe to build a
404target-architecture program (i.e. one that is intended for inclusion on
405the image being built), that program eventually runs from the root file
406system of that image. Thus, the build system provides a value of
407"/usr/bin" for ``bindir``, a value of "/usr/lib" for ``libdir``, and so
408forth.
409
410Meanwhile, ``DESTDIR`` is a path within the :term:`Build Directory`.
411However, when the recipe builds a
412native program (i.e. one that is intended to run on the build machine),
413that program is never installed directly to the build machine's root
414file system. Consequently, the build system uses paths within the Build
415Directory for ``DESTDIR``, ``bindir`` and related variables. To better
416understand this, consider the following two paths where the first is
417relatively normal and the second is not: ::
418
419 /home/maxtothemax/poky-bootchart2/build/tmp/work/i586-poky-linux/zlib/
420 1.2.8-r0/sysroot-destdir/usr/bin
421
422 /home/maxtothemax/poky-bootchart2/build/tmp/work/x86_64-linux/
423 zlib-native/1.2.8-r0/sysroot-destdir/home/maxtothemax/poky-bootchart2/
424 build/tmp/sysroots/x86_64-linux/usr/bin
425
426.. note::
427
428 Due to these lengthy examples, the paths are artificially broken
429 across lines for readability.
430
431Even if the paths look unusual,
432they both are correct - the first for a target and the second for a
433native recipe. These paths are a consequence of the ``DESTDIR``
434mechanism and while they appear strange, they are correct and in
435practice very effective.
436
437**Q:** The files provided by my ``*-native`` recipe do not appear to be
438available to other recipes. Files are missing from the native sysroot,
439my recipe is installing to the wrong place, or I am getting permissions
440errors during the do_install task in my recipe! What is wrong?
441
442**A:** This situation results when a build system does not recognize the
443environment variables supplied to it by :term:`BitBake`. The
444incident that prompted this FAQ entry involved a Makefile that used an
445environment variable named ``BINDIR`` instead of the more standard
446variable ``bindir``. The makefile's hardcoded default value of
447"/usr/bin" worked most of the time, but not for the recipe's ``-native``
448variant. For another example, permissions errors might be caused by a
449Makefile that ignores ``DESTDIR`` or uses a different name for that
450environment variable. Check the the build system to see if these kinds
451of issues exist.
diff --git a/documentation/ref-manual/faq.xml b/documentation/ref-manual/faq.xml
index d94cb32a86..2f8fcf3242 100644
--- a/documentation/ref-manual/faq.xml
+++ b/documentation/ref-manual/faq.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='faq'> 6<chapter id='faq'>
6<title>FAQ</title> 7<title>FAQ</title>
@@ -322,7 +323,7 @@
322 <qandaentry> 323 <qandaentry>
323 <question> 324 <question>
324 <para> 325 <para>
325 What’s the difference between <replaceable>target</replaceable> and <replaceable>target</replaceable><filename>-native</filename>? 326 What's the difference between <replaceable>target</replaceable> and <replaceable>target</replaceable><filename>-native</filename>?
326 </para> 327 </para>
327 </question> 328 </question>
328 <answer> 329 <answer>
diff --git a/documentation/ref-manual/history.rst b/documentation/ref-manual/history.rst
new file mode 100644
index 0000000000..e962d92978
--- /dev/null
+++ b/documentation/ref-manual/history.rst
@@ -0,0 +1,74 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 0.9
15 - November 2010
16 - The initial document released with the Yocto Project 0.9 Release
17 * - 1.0
18 - April 2011
19 - Released with the Yocto Project 1.0 Release.
20 * - 1.1
21 - October 2011
22 - Released with the Yocto Project 1.1 Release.
23 * - 1.2
24 - April 2012
25 - Released with the Yocto Project 1.2 Release.
26 * - 1.3
27 - October 2012
28 - Released with the Yocto Project 1.3 Release.
29 * - 1.4
30 - April 2013
31 - Released with the Yocto Project 1.4 Release.
32 * - 1.5
33 - October 2013
34 - Released with the Yocto Project 1.5 Release.
35 * - 1.6
36 - April 2014
37 - Released with the Yocto Project 1.6 Release.
38 * - 1.7
39 - October 2014
40 - Released with the Yocto Project 1.7 Release.
41 * - 1.8
42 - April 2015
43 - Released with the Yocto Project 1.8 Release.
44 * - 2.0
45 - October 2015
46 - Released with the Yocto Project 2.0 Release.
47 * - 2.1
48 - April 2016
49 - Released with the Yocto Project 2.1 Release.
50 * - 2.2
51 - October 2016
52 - Released with the Yocto Project 2.2 Release.
53 * - 2.3
54 - May 2017
55 - Released with the Yocto Project 2.3 Release.
56 * - 2.4
57 - October 2017
58 - Released with the Yocto Project 2.4 Release.
59 * - 2.5
60 - May 2018
61 - Released with the Yocto Project 2.5 Release.
62 * - 2.6
63 - November 2018
64 - Released with the Yocto Project 2.6 Release.
65 * - 2.7
66 - May 2019
67 - Released with the Yocto Project 2.7 Release.
68 * - 3.0
69 - October 2019
70 - Released with the Yocto Project 3.0 Release.
71 * - 3.1
72 - April 2020
73 - Released with the Yocto Project 3.1 Release.
74
diff --git a/documentation/ref-manual/migration-1.3.rst b/documentation/ref-manual/migration-1.3.rst
new file mode 100644
index 0000000000..ebbc238873
--- /dev/null
+++ b/documentation/ref-manual/migration-1.3.rst
@@ -0,0 +1,195 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3Moving to the Yocto Project 1.3 Release
4=======================================
5
6This section provides migration information for moving to the Yocto
7Project 1.3 Release from the prior release.
8
9.. _1.3-local-configuration:
10
11Local Configuration
12-------------------
13
14Differences include changes for
15:term:`SSTATE_MIRRORS` and ``bblayers.conf``.
16
17.. _migration-1.3-sstate-mirrors:
18
19SSTATE_MIRRORS
20~~~~~~~~~~~~~~
21
22The shared state cache (sstate-cache), as pointed to by
23:term:`SSTATE_DIR`, by default now has two-character
24subdirectories to prevent issues arising from too many files in the same
25directory. Also, native sstate-cache packages, which are built to run on
26the host system, will go into a subdirectory named using the distro ID
27string. If you copy the newly structured sstate-cache to a mirror
28location (either local or remote) and then point to it in
29:term:`SSTATE_MIRRORS`, you need to append "PATH"
30to the end of the mirror URL so that the path used by BitBake before the
31mirror substitution is appended to the path used to access the mirror.
32Here is an example: ::
33
34 SSTATE_MIRRORS = "file://.* http://someserver.tld/share/sstate/PATH"
35
36.. _migration-1.3-bblayers-conf:
37
38bblayers.conf
39~~~~~~~~~~~~~
40
41The ``meta-yocto`` layer consists of two parts that correspond to the
42Poky reference distribution and the reference hardware Board Support
43Packages (BSPs), respectively: ``meta-yocto`` and ``meta-yocto-bsp``.
44When running BitBake for the first time after upgrading, your
45``conf/bblayers.conf`` file will be updated to handle this change and
46you will be asked to re-run or restart for the changes to take effect.
47
48.. _1.3-recipes:
49
50Recipes
51-------
52
53Differences include changes for the following:
54
55.. _migration-1.3-python-function-whitespace:
56
57Python Function Whitespace
58~~~~~~~~~~~~~~~~~~~~~~~~~~
59
60All Python functions must now use four spaces for indentation.
61Previously, an inconsistent mix of spaces and tabs existed, which made
62extending these functions using ``_append`` or ``_prepend`` complicated
63given that Python treats whitespace as syntactically significant. If you
64are defining or extending any Python functions (e.g.
65``populate_packages``, ``do_unpack``, ``do_patch`` and so forth) in
66custom recipes or classes, you need to ensure you are using consistent
67four-space indentation.
68
69.. _migration-1.3-proto=-in-src-uri:
70
71proto= in SRC_URI
72~~~~~~~~~~~~~~~~~
73
74Any use of ``proto=`` in :term:`SRC_URI` needs to be
75changed to ``protocol=``. In particular, this applies to the following
76URIs:
77
78- ``svn://``
79
80- ``bzr://``
81
82- ``hg://``
83
84- ``osc://``
85
86Other URIs were already using ``protocol=``. This change improves
87consistency.
88
89.. _migration-1.3-nativesdk:
90
91nativesdk
92~~~~~~~~~
93
94The suffix ``nativesdk`` is now implemented as a prefix, which
95simplifies a lot of the packaging code for ``nativesdk`` recipes. All
96custom ``nativesdk`` recipes, which are relocatable packages that are
97native to :term:`SDK_ARCH`, and any references need to
98be updated to use ``nativesdk-*`` instead of ``*-nativesdk``.
99
100.. _migration-1.3-task-recipes:
101
102Task Recipes
103~~~~~~~~~~~~
104
105"Task" recipes are now known as "Package groups" and have been renamed
106from ``task-*.bb`` to ``packagegroup-*.bb``. Existing references to the
107previous ``task-*`` names should work in most cases as there is an
108automatic upgrade path for most packages. However, you should update
109references in your own recipes and configurations as they could be
110removed in future releases. You should also rename any custom ``task-*``
111recipes to ``packagegroup-*``, and change them to inherit
112``packagegroup`` instead of ``task``, as well as taking the opportunity
113to remove anything now handled by ``packagegroup.bbclass``, such as
114providing ``-dev`` and ``-dbg`` packages, setting
115:term:`LIC_FILES_CHKSUM`, and so forth. See the
116":ref:`packagegroup.bbclass <ref-classes-packagegroup>`" section for
117further details.
118
119.. _migration-1.3-image-features:
120
121IMAGE_FEATURES
122~~~~~~~~~~~~~~
123
124Image recipes that previously included "apps-console-core" in
125:term:`IMAGE_FEATURES` should now include "splash"
126instead to enable the boot-up splash screen. Retaining
127"apps-console-core" will still include the splash screen but generates a
128warning. The "apps-x11-core" and "apps-x11-games" ``IMAGE_FEATURES``
129features have been removed.
130
131.. _migration-1.3-removed-recipes:
132
133Removed Recipes
134~~~~~~~~~~~~~~~
135
136The following recipes have been removed. For most of them, it is
137unlikely that you would have any references to them in your own
138:term:`Metadata`. However, you should check your metadata
139against this list to be sure:
140
141- ``libx11-trim``: Replaced by ``libx11``, which has a negligible
142 size difference with modern Xorg.
143
144- ``xserver-xorg-lite``: Use ``xserver-xorg``, which has a negligible
145 size difference when DRI and GLX modules are not installed.
146
147- ``xserver-kdrive``: Effectively unmaintained for many years.
148
149- ``mesa-xlib``: No longer serves any purpose.
150
151- ``galago``: Replaced by telepathy.
152
153- ``gail``: Functionality was integrated into GTK+ 2.13.
154
155- ``eggdbus``: No longer needed.
156
157- ``gcc-*-intermediate``: The build has been restructured to avoid
158 the need for this step.
159
160- ``libgsmd``: Unmaintained for many years. Functionality now
161 provided by ``ofono`` instead.
162
163- *contacts, dates, tasks, eds-tools*: Largely unmaintained PIM
164 application suite. It has been moved to ``meta-gnome`` in
165 ``meta-openembedded``.
166
167In addition to the previously listed changes, the ``meta-demoapps``
168directory has also been removed because the recipes in it were not being
169maintained and many had become obsolete or broken. Additionally, these
170recipes were not parsed in the default configuration. Many of these
171recipes are already provided in an updated and maintained form within
172the OpenEmbedded community layers such as ``meta-oe`` and
173``meta-gnome``. For the remainder, you can now find them in the
174``meta-extras`` repository, which is in the
175:yocto_git:`Source Repositories <>` at
176http://git.yoctoproject.org/cgit/cgit.cgi/meta-extras/.
177
178.. _1.3-linux-kernel-naming:
179
180Linux Kernel Naming
181-------------------
182
183The naming scheme for kernel output binaries has been changed to now
184include :term:`PE` as part of the filename:
185::
186
187 KERNEL_IMAGE_BASE_NAME ?= "${KERNEL_IMAGETYPE}-${PE}-${PV}-${PR}-${MACHINE}-${DATETIME}"
188
189Because the ``PE`` variable is not set by default, these binary files
190could result with names that include two dash characters. Here is an
191example: ::
192
193 bzImage--3.10.9+git0+cd502a8814_7144bcc4b8-r0-qemux86-64-20130830085431.bin
194
195
diff --git a/documentation/ref-manual/migration-1.4.rst b/documentation/ref-manual/migration-1.4.rst
new file mode 100644
index 0000000000..a658bdff68
--- /dev/null
+++ b/documentation/ref-manual/migration-1.4.rst
@@ -0,0 +1,237 @@
1Moving to the Yocto Project 1.4 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 1.4 Release from the prior release.
6
7.. _migration-1.4-bitbake:
8
9BitBake
10-------
11
12Differences include the following:
13
14- *Comment Continuation:* If a comment ends with a line continuation
15 (\) character, then the next line must also be a comment. Any
16 instance where this is not the case, now triggers a warning. You must
17 either remove the continuation character, or be sure the next line is
18 a comment.
19
20- *Package Name Overrides:* The runtime package specific variables
21 :term:`RDEPENDS`,
22 :term:`RRECOMMENDS`,
23 :term:`RSUGGESTS`,
24 :term:`RPROVIDES`,
25 :term:`RCONFLICTS`,
26 :term:`RREPLACES`, :term:`FILES`,
27 :term:`ALLOW_EMPTY`, and the pre, post, install,
28 and uninstall script functions ``pkg_preinst``, ``pkg_postinst``,
29 ``pkg_prerm``, and ``pkg_postrm`` should always have a package name
30 override. For example, use ``RDEPENDS_${PN}`` for the main package
31 instead of ``RDEPENDS``. BitBake uses more strict checks when it
32 parses recipes.
33
34.. _migration-1.4-build-behavior:
35
36Build Behavior
37--------------
38
39Differences include the following:
40
41- *Shared State Code:* The shared state code has been optimized to
42 avoid running unnecessary tasks. For example, the following no longer
43 populates the target sysroot since that is not necessary:
44 ::
45
46 $ bitbake -c rootfs some-image
47
48 Instead, the system just needs to extract the
49 output package contents, re-create the packages, and construct the
50 root filesystem. This change is unlikely to cause any problems unless
51 you have missing declared dependencies.
52
53- *Scanning Directory Names:* When scanning for files in
54 :term:`SRC_URI`, the build system now uses
55 :term:`FILESOVERRIDES` instead of
56 :term:`OVERRIDES` for the directory names. In
57 general, the values previously in ``OVERRIDES`` are now in
58 ``FILESOVERRIDES`` as well. However, if you relied upon an additional
59 value you previously added to ``OVERRIDES``, you might now need to
60 add it to ``FILESOVERRIDES`` unless you are already adding it through
61 the :term:`MACHINEOVERRIDES` or
62 :term:`DISTROOVERRIDES` variables, as
63 appropriate. For more related changes, see the
64 "`Variables <#migration-1.4-variables>`__" section.
65
66.. _migration-1.4-proxies-and-fetching-source:
67
68Proxies and Fetching Source
69---------------------------
70
71A new ``oe-git-proxy`` script has been added to replace previous methods
72of handling proxies and fetching source from Git. See the
73``meta-yocto/conf/site.conf.sample`` file for information on how to use
74this script.
75
76.. _migration-1.4-custom-interfaces-file-netbase-change:
77
78Custom Interfaces File (netbase change)
79---------------------------------------
80
81If you have created your own custom ``etc/network/interfaces`` file by
82creating an append file for the ``netbase`` recipe, you now need to
83create an append file for the ``init-ifupdown`` recipe instead, which
84you can find in the :term:`Source Directory` at
85``meta/recipes-core/init-ifupdown``. For information on how to use
86append files, see the
87":ref:`dev-manual/dev-manual-common-tasks:using .bbappend files in your layer`"
88section in the Yocto Project Development Tasks Manual.
89
90.. _migration-1.4-remote-debugging:
91
92Remote Debugging
93----------------
94
95Support for remote debugging with the Eclipse IDE is now separated into
96an image feature (``eclipse-debug``) that corresponds to the
97``packagegroup-core-eclipse-debug`` package group. Previously, the
98debugging feature was included through the ``tools-debug`` image
99feature, which corresponds to the ``packagegroup-core-tools-debug``
100package group.
101
102.. _migration-1.4-variables:
103
104Variables
105---------
106
107The following variables have changed:
108
109- ``SANITY_TESTED_DISTROS``: This variable now uses a distribution
110 ID, which is composed of the host distributor ID followed by the
111 release. Previously,
112 :term:`SANITY_TESTED_DISTROS` was
113 composed of the description field. For example, "Ubuntu 12.10"
114 becomes "Ubuntu-12.10". You do not need to worry about this change if
115 you are not specifically setting this variable, or if you are
116 specifically setting it to "".
117
118- ``SRC_URI``: The ``${``\ :term:`PN`\ ``}``,
119 ``${``\ :term:`PF`\ ``}``,
120 ``${``\ :term:`P`\ ``}``, and ``FILE_DIRNAME`` directories
121 have been dropped from the default value of the
122 :term:`FILESPATH` variable, which is used as the
123 search path for finding files referred to in
124 :term:`SRC_URI`. If you have a recipe that relied upon
125 these directories, which would be unusual, then you will need to add
126 the appropriate paths within the recipe or, alternatively, rearrange
127 the files. The most common locations are still covered by ``${BP}``,
128 ``${BPN}``, and "files", which all remain in the default value of
129 :term:`FILESPATH`.
130
131.. _migration-target-package-management-with-rpm:
132
133Target Package Management with RPM
134----------------------------------
135
136If runtime package management is enabled and the RPM backend is
137selected, Smart is now installed for package download, dependency
138resolution, and upgrades instead of Zypper. For more information on how
139to use Smart, run the following command on the target:
140::
141
142 smart --help
143
144.. _migration-1.4-recipes-moved:
145
146Recipes Moved
147-------------
148
149The following recipes were moved from their previous locations because
150they are no longer used by anything in the OpenEmbedded-Core:
151
152- ``clutter-box2d``: Now resides in the ``meta-oe`` layer.
153
154- ``evolution-data-server``: Now resides in the ``meta-gnome`` layer.
155
156- ``gthumb``: Now resides in the ``meta-gnome`` layer.
157
158- ``gtkhtml2``: Now resides in the ``meta-oe`` layer.
159
160- ``gupnp``: Now resides in the ``meta-multimedia`` layer.
161
162- ``gypsy``: Now resides in the ``meta-oe`` layer.
163
164- ``libcanberra``: Now resides in the ``meta-gnome`` layer.
165
166- ``libgdata``: Now resides in the ``meta-gnome`` layer.
167
168- ``libmusicbrainz``: Now resides in the ``meta-multimedia`` layer.
169
170- ``metacity``: Now resides in the ``meta-gnome`` layer.
171
172- ``polkit``: Now resides in the ``meta-oe`` layer.
173
174- ``zeroconf``: Now resides in the ``meta-networking`` layer.
175
176.. _migration-1.4-removals-and-renames:
177
178Removals and Renames
179--------------------
180
181The following list shows what has been removed or renamed:
182
183- ``evieext``: Removed because it has been removed from ``xserver``
184 since 2008.
185
186- *Gtk+ DirectFB:* Removed support because upstream Gtk+ no longer
187 supports it as of version 2.18.
188
189- ``libxfontcache / xfontcacheproto``: Removed because they were
190 removed from the Xorg server in 2008.
191
192- ``libxp / libxprintapputil / libxprintutil / printproto``: Removed
193 because the XPrint server was removed from Xorg in 2008.
194
195- ``libxtrap / xtrapproto``: Removed because their functionality was
196 broken upstream.
197
198- *linux-yocto 3.0 kernel:* Removed with linux-yocto 3.8 kernel being
199 added. The linux-yocto 3.2 and linux-yocto 3.4 kernels remain as part
200 of the release.
201
202- ``lsbsetup``: Removed with functionality now provided by
203 ``lsbtest``.
204
205- ``matchbox-stroke``: Removed because it was never more than a
206 proof-of-concept.
207
208- ``matchbox-wm-2 / matchbox-theme-sato-2``: Removed because they are
209 not maintained. However, ``matchbox-wm`` and ``matchbox-theme-sato``
210 are still provided.
211
212- ``mesa-dri``: Renamed to ``mesa``.
213
214- ``mesa-xlib``: Removed because it was no longer useful.
215
216- ``mutter``: Removed because nothing ever uses it and the recipe is
217 very old.
218
219- ``orinoco-conf``: Removed because it has become obsolete.
220
221- ``update-modules``: Removed because it is no longer used. The
222 kernel module ``postinstall`` and ``postrm`` scripts can now do the
223 same task without the use of this script.
224
225- ``web``: Removed because it is not maintained. Superseded by
226 ``web-webkit``.
227
228- ``xf86bigfontproto``: Removed because upstream it has been disabled
229 by default since 2007. Nothing uses ``xf86bigfontproto``.
230
231- ``xf86rushproto``: Removed because its dependency in ``xserver``
232 was spurious and it was removed in 2005.
233
234- ``zypper / libzypp / sat-solver``: Removed and been functionally
235 replaced with Smart (``python-smartpm``) when RPM packaging is used
236 and package management is enabled on the target.
237
diff --git a/documentation/ref-manual/migration-1.5.rst b/documentation/ref-manual/migration-1.5.rst
new file mode 100644
index 0000000000..fa6ff92f10
--- /dev/null
+++ b/documentation/ref-manual/migration-1.5.rst
@@ -0,0 +1,355 @@
1Moving to the Yocto Project 1.5 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 1.5 Release from the prior release.
6
7.. _migration-1.5-host-dependency-changes:
8
9Host Dependency Changes
10-----------------------
11
12The OpenEmbedded build system now has some additional requirements on
13the host system:
14
15- Python 2.7.3+
16
17- Tar 1.24+
18
19- Git 1.7.8+
20
21- Patched version of Make if you are using 3.82. Most distributions
22 that provide Make 3.82 use the patched version.
23
24If the Linux distribution you are using on your build host does not
25provide packages for these, you can install and use the Buildtools
26tarball, which provides an SDK-like environment containing them.
27
28For more information on this requirement, see the "`Required Git, tar,
29Python and gcc Versions <#required-git-tar-python-and-gcc-versions>`__"
30section.
31
32.. _migration-1.5-atom-pc-bsp:
33
34``atom-pc`` Board Support Package (BSP)
35---------------------------------------
36
37The ``atom-pc`` hardware reference BSP has been replaced by a
38``genericx86`` BSP. This BSP is not necessarily guaranteed to work on
39all x86 hardware, but it will run on a wider range of systems than the
40``atom-pc`` did.
41
42.. note::
43
44 Additionally, a
45 genericx86-64
46 BSP has been added for 64-bit Atom systems.
47
48.. _migration-1.5-bitbake:
49
50BitBake
51-------
52
53The following changes have been made that relate to BitBake:
54
55- BitBake now supports a ``_remove`` operator. The addition of this
56 operator means you will have to rename any items in recipe space
57 (functions, variables) whose names currently contain ``_remove_`` or
58 end with ``_remove`` to avoid unexpected behavior.
59
60- BitBake's global method pool has been removed. This method is not
61 particularly useful and led to clashes between recipes containing
62 functions that had the same name.
63
64- The "none" server backend has been removed. The "process" server
65 backend has been serving well as the default for a long time now.
66
67- The ``bitbake-runtask`` script has been removed.
68
69- ``${``\ :term:`P`\ ``}`` and
70 ``${``\ :term:`PF`\ ``}`` are no longer added to
71 :term:`PROVIDES` by default in ``bitbake.conf``.
72 These version-specific ``PROVIDES`` items were seldom used.
73 Attempting to use them could result in two versions being built
74 simultaneously rather than just one version due to the way BitBake
75 resolves dependencies.
76
77.. _migration-1.5-qa-warnings:
78
79QA Warnings
80-----------
81
82The following changes have been made to the package QA checks:
83
84- If you have customized :term:`ERROR_QA` or
85 :term:`WARN_QA` values in your configuration, check
86 that they contain all of the issues that you wish to be reported.
87 Previous Yocto Project versions contained a bug that meant that any
88 item not mentioned in ``ERROR_QA`` or ``WARN_QA`` would be treated as
89 a warning. Consequently, several important items were not already in
90 the default value of ``WARN_QA``. All of the possible QA checks are
91 now documented in the ":ref:`insane.bbclass <ref-classes-insane>`"
92 section.
93
94- An additional QA check has been added to check if
95 ``/usr/share/info/dir`` is being installed. Your recipe should delete
96 this file within :ref:`ref-tasks-install` if "make
97 install" is installing it.
98
99- If you are using the buildhistory class, the check for the package
100 version going backwards is now controlled using a standard QA check.
101 Thus, if you have customized your ``ERROR_QA`` or ``WARN_QA`` values
102 and still wish to have this check performed, you should add
103 "version-going-backwards" to your value for one or the other
104 variables depending on how you wish it to be handled. See the
105 documented QA checks in the
106 ":ref:`insane.bbclass <ref-classes-insane>`" section.
107
108.. _migration-1.5-directory-layout-changes:
109
110Directory Layout Changes
111------------------------
112
113The following directory changes exist:
114
115- Output SDK installer files are now named to include the image name
116 and tuning architecture through the :term:`SDK_NAME`
117 variable.
118
119- Images and related files are now installed into a directory that is
120 specific to the machine, instead of a parent directory containing
121 output files for multiple machines. The
122 :term:`DEPLOY_DIR_IMAGE` variable continues
123 to point to the directory containing images for the current
124 :term:`MACHINE` and should be used anywhere there is a
125 need to refer to this directory. The ``runqemu`` script now uses this
126 variable to find images and kernel binaries and will use BitBake to
127 determine the directory. Alternatively, you can set the
128 ``DEPLOY_DIR_IMAGE`` variable in the external environment.
129
130- When buildhistory is enabled, its output is now written under the
131 :term:`Build Directory` rather than
132 :term:`TMPDIR`. Doing so makes it easier to delete
133 ``TMPDIR`` and preserve the build history. Additionally, data for
134 produced SDKs is now split by :term:`IMAGE_NAME`.
135
136- The ``pkgdata`` directory produced as part of the packaging process
137 has been collapsed into a single machine-specific directory. This
138 directory is located under ``sysroots`` and uses a machine-specific
139 name (i.e. ``tmp/sysroots/machine/pkgdata``).
140
141.. _migration-1.5-shortened-git-srcrev-values:
142
143Shortened Git ``SRCREV`` Values
144-------------------------------
145
146BitBake will now shorten revisions from Git repositories from the normal
14740 characters down to 10 characters within :term:`SRCPV`
148for improved usability in path and file names. This change should be
149safe within contexts where these revisions are used because the chances
150of spatially close collisions is very low. Distant collisions are not a
151major issue in the way the values are used.
152
153.. _migration-1.5-image-features:
154
155``IMAGE_FEATURES``
156------------------
157
158The following changes have been made that relate to
159:term:`IMAGE_FEATURES`:
160
161- The value of ``IMAGE_FEATURES`` is now validated to ensure invalid
162 feature items are not added. Some users mistakenly add package names
163 to this variable instead of using
164 :term:`IMAGE_INSTALL` in order to have the
165 package added to the image, which does not work. This change is
166 intended to catch those kinds of situations. Valid ``IMAGE_FEATURES``
167 are drawn from ``PACKAGE_GROUP`` definitions,
168 :term:`COMPLEMENTARY_GLOB` and a new
169 "validitems" varflag on ``IMAGE_FEATURES``. The "validitems" varflag
170 change allows additional features to be added if they are not
171 provided using the previous two mechanisms.
172
173- The previously deprecated "apps-console-core" ``IMAGE_FEATURES`` item
174 is no longer supported. Add "splash" to ``IMAGE_FEATURES`` if you
175 wish to have the splash screen enabled, since this is all that
176 apps-console-core was doing.
177
178.. _migration-1.5-run:
179
180``/run``
181--------
182
183The ``/run`` directory from the Filesystem Hierarchy Standard 3.0 has
184been introduced. You can find some of the implications for this change
185`here <http://cgit.openembedded.org/openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873>`__.
186The change also means that recipes that install files to ``/var/run``
187must be changed. You can find a guide on how to make these changes
188`here <https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg31649.html>`__.
189
190.. _migration-1.5-removal-of-package-manager-database-within-image-recipes:
191
192Removal of Package Manager Database Within Image Recipes
193--------------------------------------------------------
194
195The image ``core-image-minimal`` no longer adds
196``remove_packaging_data_files`` to
197:term:`ROOTFS_POSTPROCESS_COMMAND`.
198This addition is now handled automatically when "package-management" is
199not in :term:`IMAGE_FEATURES`. If you have custom
200image recipes that make this addition, you should remove the lines, as
201they are not needed and might interfere with correct operation of
202postinstall scripts.
203
204.. _migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time:
205
206Images Now Rebuild Only on Changes Instead of Every Time
207--------------------------------------------------------
208
209The :ref:`ref-tasks-rootfs` and other related image
210construction tasks are no longer marked as "nostamp". Consequently, they
211will only be re-executed when their inputs have changed. Previous
212versions of the OpenEmbedded build system always rebuilt the image when
213requested rather when necessary.
214
215.. _migration-1.5-task-recipes:
216
217Task Recipes
218------------
219
220The previously deprecated ``task.bbclass`` has now been dropped. For
221recipes that previously inherited from this class, you should rename
222them from ``task-*`` to ``packagegroup-*`` and inherit packagegroup
223instead.
224
225For more information, see the
226":ref:`packagegroup.bbclass <ref-classes-packagegroup>`" section.
227
228.. _migration-1.5-busybox:
229
230BusyBox
231-------
232
233By default, we now split BusyBox into two binaries: one that is suid
234root for those components that need it, and another for the rest of the
235components. Splitting BusyBox allows for optimization that eliminates
236the ``tinylogin`` recipe as recommended by upstream. You can disable
237this split by setting
238:term:`BUSYBOX_SPLIT_SUID` to "0".
239
240.. _migration-1.5-automated-image-testing:
241
242Automated Image Testing
243-----------------------
244
245A new automated image testing framework has been added through the
246:ref:`testimage.bbclass <ref-classes-testimage*>` class. This
247framework replaces the older ``imagetest-qemu`` framework.
248
249You can learn more about performing automated image tests in the
250":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
251section in the Yocto Project Development Tasks Manual.
252
253.. _migration-1.5-build-history:
254
255Build History
256-------------
257
258Following are changes to Build History:
259
260- Installed package sizes: ``installed-package-sizes.txt`` for an image
261 now records the size of the files installed by each package instead
262 of the size of each compressed package archive file.
263
264- The dependency graphs (``depends*.dot``) now use the actual package
265 names instead of replacing dashes, dots and plus signs with
266 underscores.
267
268- The ``buildhistory-diff`` and ``buildhistory-collect-srcrevs``
269 utilities have improved command-line handling. Use the ``--help``
270 option for each utility for more information on the new syntax.
271
272For more information on Build History, see the
273":ref:`dev-manual/dev-manual-common-tasks:maintaining build output quality`"
274section in the Yocto Project Development Tasks Manual.
275
276.. _migration-1.5-udev:
277
278``udev``
279--------
280
281Following are changes to ``udev``:
282
283- ``udev`` no longer brings in ``udev-extraconf`` automatically through
284 :term:`RRECOMMENDS`, since this was originally
285 intended to be optional. If you need the extra rules, then add
286 ``udev-extraconf`` to your image.
287
288- ``udev`` no longer brings in ``pciutils-ids`` or ``usbutils-ids``
289 through ``RRECOMMENDS``. These are not needed by ``udev`` itself and
290 removing them saves around 350KB.
291
292.. _migration-1.5-removed-renamed-recipes:
293
294Removed and Renamed Recipes
295---------------------------
296
297- The ``linux-yocto`` 3.2 kernel has been removed.
298
299- ``libtool-nativesdk`` has been renamed to ``nativesdk-libtool``.
300
301- ``tinylogin`` has been removed. It has been replaced by a suid
302 portion of Busybox. See the "`BusyBox <#migration-1.5-busybox>`__"
303 section for more information.
304
305- ``external-python-tarball`` has been renamed to
306 ``buildtools-tarball``.
307
308- ``web-webkit`` has been removed. It has been functionally replaced by
309 ``midori``.
310
311- ``imake`` has been removed. It is no longer needed by any other
312 recipe.
313
314- ``transfig-native`` has been removed. It is no longer needed by any
315 other recipe.
316
317- ``anjuta-remote-run`` has been removed. Anjuta IDE integration has
318 not been officially supported for several releases.
319
320.. _migration-1.5-other-changes:
321
322Other Changes
323-------------
324
325Following is a list of short entries describing other changes:
326
327- ``run-postinsts``: Make this generic.
328
329- ``base-files``: Remove the unnecessary ``media/``\ xxx directories.
330
331- ``alsa-state``: Provide an empty ``asound.conf`` by default.
332
333- ``classes/image``: Ensure
334 :term:`BAD_RECOMMENDATIONS` supports
335 pre-renamed package names.
336
337- ``classes/rootfs_rpm``: Implement ``BAD_RECOMMENDATIONS`` for RPM.
338
339- ``systemd``: Remove ``systemd_unitdir`` if ``systemd`` is not in
340 :term:`DISTRO_FEATURES`.
341
342- ``systemd``: Remove ``init.d`` dir if ``systemd`` unit file is
343 present and ``sysvinit`` is not a distro feature.
344
345- ``libpam``: Deny all services for the ``OTHER`` entries.
346
347- ``image.bbclass``: Move ``runtime_mapping_rename`` to avoid conflict
348 with ``multilib``. See
349 `YOCTO #4993 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=4993>`_
350 in Bugzilla for more information.
351
352- ``linux-dtb``: Use kernel build system to generate the ``dtb`` files.
353
354- ``kern-tools``: Switch from guilt to new ``kgit-s2q`` tool.
355
diff --git a/documentation/ref-manual/migration-1.6.rst b/documentation/ref-manual/migration-1.6.rst
new file mode 100644
index 0000000000..b55be46e55
--- /dev/null
+++ b/documentation/ref-manual/migration-1.6.rst
@@ -0,0 +1,417 @@
1Moving to the Yocto Project 1.6 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 1.6 Release from the prior release.
6
7.. _migration-1.6-archiver-class:
8
9``archiver`` Class
10------------------
11
12The :ref:`archiver <ref-classes-archiver>` class has been rewritten
13and its configuration has been simplified. For more details on the
14source archiver, see the
15":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
16section in the Yocto Project Development Tasks Manual.
17
18.. _migration-1.6-packaging-changes:
19
20Packaging Changes
21-----------------
22
23The following packaging changes have been made:
24
25- The ``binutils`` recipe no longer produces a ``binutils-symlinks``
26 package. ``update-alternatives`` is now used to handle the preferred
27 ``binutils`` variant on the target instead.
28
29- The tc (traffic control) utilities have been split out of the main
30 ``iproute2`` package and put into the ``iproute2-tc`` package.
31
32- The ``gtk-engines`` schemas have been moved to a dedicated
33 ``gtk-engines-schemas`` package.
34
35- The ``armv7a`` with thumb package architecture suffix has changed.
36 The suffix for these packages with the thumb optimization enabled is
37 "t2" as it should be. Use of this suffix was not the case in the 1.5
38 release. Architecture names will change within package feeds as a
39 result.
40
41.. _migration-1.6-bitbake:
42
43BitBake
44-------
45
46The following changes have been made to :term:`BitBake`.
47
48.. _migration-1.6-matching-branch-requirement-for-git-fetching:
49
50Matching Branch Requirement for Git Fetching
51~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
52
53When fetching source from a Git repository using
54:term:`SRC_URI`, BitBake will now validate the
55:term:`SRCREV` value against the branch. You can specify
56the branch using the following form: SRC_URI =
57"git://server.name/repository;branch=branchname" If you do not specify a
58branch, BitBake looks in the default "master" branch.
59
60Alternatively, if you need to bypass this check (e.g. if you are
61fetching a revision corresponding to a tag that is not on any branch),
62you can add ";nobranch=1" to the end of the URL within ``SRC_URI``.
63
64.. _migration-1.6-bitbake-deps:
65
66Python Definition substitutions
67~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
68
69BitBake had some previously deprecated Python definitions within its
70``bb`` module removed. You should use their sub-module counterparts
71instead:
72
73- ``bb.MalformedUrl``: Use ``bb.fetch.MalformedUrl``.
74
75- ``bb.encodeurl``: Use ``bb.fetch.encodeurl``.
76
77- ``bb.decodeurl``: Use ``bb.fetch.decodeurl``
78
79- ``bb.mkdirhier``: Use ``bb.utils.mkdirhier``.
80
81- ``bb.movefile``: Use ``bb.utils.movefile``.
82
83- ``bb.copyfile``: Use ``bb.utils.copyfile``.
84
85- ``bb.which``: Use ``bb.utils.which``.
86
87- ``bb.vercmp_string``: Use ``bb.utils.vercmp_string``.
88
89- ``bb.vercmp``: Use ``bb.utils.vercmp``.
90
91.. _migration-1.6-bitbake-fetcher:
92
93SVK Fetcher
94~~~~~~~~~~~
95
96The SVK fetcher has been removed from BitBake.
97
98.. _migration-1.6-bitbake-console-output:
99
100Console Output Error Redirection
101~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
102
103The BitBake console UI will now output errors to ``stderr`` instead of
104``stdout``. Consequently, if you are piping or redirecting the output of
105``bitbake`` to somewhere else, and you wish to retain the errors, you
106will need to add ``2>&1`` (or something similar) to the end of your
107``bitbake`` command line.
108
109.. _migration-1.6-task-taskname-overrides:
110
111``task-``\ taskname Overrides
112~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
113
114``task-``\ taskname overrides have been adjusted so that tasks whose
115names contain underscores have the underscores replaced by hyphens for
116the override so that they now function properly. For example, the task
117override for :ref:`ref-tasks-populate_sdk` is
118``task-populate-sdk``.
119
120.. _migration-1.6-variable-changes:
121
122Changes to Variables
123--------------------
124
125The following variables have changed. For information on the
126OpenEmbedded build system variables, see the "`Variables
127Glossary <#ref-variables-glos>`__" Chapter.
128
129.. _migration-1.6-variable-changes-TMPDIR:
130
131``TMPDIR``
132~~~~~~~~~~
133
134:term:`TMPDIR` can no longer be on an NFS mount. NFS does
135not offer full POSIX locking and inode consistency and can cause
136unexpected issues if used to store ``TMPDIR``.
137
138The check for this occurs on startup. If ``TMPDIR`` is detected on an
139NFS mount, an error occurs.
140
141.. _migration-1.6-variable-changes-PRINC:
142
143``PRINC``
144~~~~~~~~~
145
146The ``PRINC`` variable has been deprecated and triggers a warning if
147detected during a build. For :term:`PR` increments on changes,
148use the PR service instead. You can find out more about this service in
149the ":ref:`dev-manual/dev-manual-common-tasks:working with a pr service`"
150section in the Yocto Project Development Tasks Manual.
151
152.. _migration-1.6-variable-changes-IMAGE_TYPES:
153
154``IMAGE_TYPES``
155~~~~~~~~~~~~~~~
156
157The "sum.jffs2" option for :term:`IMAGE_TYPES` has
158been replaced by the "jffs2.sum" option, which fits the processing
159order.
160
161.. _migration-1.6-variable-changes-COPY_LIC_MANIFEST:
162
163``COPY_LIC_MANIFEST``
164~~~~~~~~~~~~~~~~~~~~~
165
166The :term:`COPY_LIC_MANIFEST` variable must now
167be set to "1" rather than any value in order to enable it.
168
169.. _migration-1.6-variable-changes-COPY_LIC_DIRS:
170
171``COPY_LIC_DIRS``
172~~~~~~~~~~~~~~~~~
173
174The :term:`COPY_LIC_DIRS` variable must now be set
175to "1" rather than any value in order to enable it.
176
177.. _migration-1.6-variable-changes-PACKAGE_GROUP:
178
179``PACKAGE_GROUP``
180~~~~~~~~~~~~~~~~~
181
182The ``PACKAGE_GROUP`` variable has been renamed to
183:term:`FEATURE_PACKAGES` to more accurately
184reflect its purpose. You can still use ``PACKAGE_GROUP`` but the
185OpenEmbedded build system produces a warning message when it encounters
186the variable.
187
188.. _migration-1.6-variable-changes-variable-entry-behavior:
189
190Preprocess and Post Process Command Variable Behavior
191~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
192
193The following variables now expect a semicolon separated list of
194functions to call and not arbitrary shell commands:
195
196 - :term:`ROOTFS_PREPROCESS_COMMAND`
197 - :term:`ROOTFS_POSTPROCESS_COMMAND`
198 - :term:`SDK_POSTPROCESS_COMMAND`
199 - :term:`POPULATE_SDK_POST_TARGET_COMMAND`
200 - :term:`POPULATE_SDK_POST_HOST_COMMAND`
201 - :term:`IMAGE_POSTPROCESS_COMMAND`
202 - :term:`IMAGE_PREPROCESS_COMMAND`
203 - :term:`ROOTFS_POSTUNINSTALL_COMMAND`
204 - :term:`ROOTFS_POSTINSTALL_COMMAND`
205
206For
207migration purposes, you can simply wrap shell commands in a shell
208function and then call the function. Here is an example: ::
209
210 my_postprocess_function() {
211 echo "hello" > ${IMAGE_ROOTFS}/hello.txt
212 }
213 ROOTFS_POSTPROCESS_COMMAND += "my_postprocess_function; "
214
215.. _migration-1.6-package-test-ptest:
216
217Package Test (ptest)
218--------------------
219
220Package Tests (ptest) are built but not installed by default. For
221information on using Package Tests, see the
222":ref:`dev-manual/dev-manual-common-tasks:testing packages with ptest`"
223section in the Yocto Project Development Tasks Manual. For information on the
224``ptest`` class, see the ":ref:`ptest.bbclass <ref-classes-ptest>`"
225section.
226
227.. _migration-1.6-build-changes:
228
229Build Changes
230-------------
231
232Separate build and source directories have been enabled by default for
233selected recipes where it is known to work (a whitelist) and for all
234recipes that inherit the :ref:`cmake <ref-classes-cmake>` class. In
235future releases the :ref:`autotools <ref-classes-autotools>` class
236will enable a separate build directory by default as well. Recipes
237building Autotools-based software that fails to build with a separate
238build directory should be changed to inherit from the
239:ref:`autotools-brokensep <ref-classes-autotools>` class instead of
240the ``autotools`` or ``autotools_stage``\ classes.
241
242.. _migration-1.6-building-qemu-native:
243
244``qemu-native``
245---------------
246
247``qemu-native`` now builds without SDL-based graphical output support by
248default. The following additional lines are needed in your
249``local.conf`` to enable it:
250::
251
252 PACKAGECONFIG_pn-qemu-native = "sdl"
253 ASSUME_PROVIDED += "libsdl-native"
254
255.. note::
256
257 The default
258 local.conf
259 contains these statements. Consequently, if you are building a
260 headless system and using a default
261 local.conf
262 file, you will need comment these two lines out.
263
264.. _migration-1.6-core-image-basic:
265
266``core-image-basic``
267--------------------
268
269``core-image-basic`` has been renamed to ``core-image-full-cmdline``.
270
271In addition to ``core-image-basic`` being renamed,
272``packagegroup-core-basic`` has been renamed to
273``packagegroup-core-full-cmdline`` to match.
274
275.. _migration-1.6-licensing:
276
277Licensing
278---------
279
280The top-level ``LICENSE`` file has been changed to better describe the
281license of the various components of :term:`OpenEmbedded-Core (OE-Core)`. However,
282the licensing itself remains unchanged.
283
284Normally, this change would not cause any side-effects. However, some
285recipes point to this file within
286:term:`LIC_FILES_CHKSUM` (as
287``${COREBASE}/LICENSE``) and thus the accompanying checksum must be
288changed from 3f40d7994397109285ec7b81fdeb3b58 to
2894d92cd373abda3937c2bc47fbc49d690. A better alternative is to have
290``LIC_FILES_CHKSUM`` point to a file describing the license that is
291distributed with the source that the recipe is building, if possible,
292rather than pointing to ``${COREBASE}/LICENSE``.
293
294.. _migration-1.6-cflags-options:
295
296``CFLAGS`` Options
297------------------
298
299The "-fpermissive" option has been removed from the default
300:term:`CFLAGS` value. You need to take action on
301individual recipes that fail when building with this option. You need to
302either patch the recipes to fix the issues reported by the compiler, or
303you need to add "-fpermissive" to ``CFLAGS`` in the recipes.
304
305.. _migration-1.6-custom-images:
306
307Custom Image Output Types
308-------------------------
309
310Custom image output types, as selected using
311:term:`IMAGE_FSTYPES`, must declare their
312dependencies on other image types (if any) using a new
313:term:`IMAGE_TYPEDEP` variable.
314
315.. _migration-1.6-do-package-write-task:
316
317Tasks
318-----
319
320The ``do_package_write`` task has been removed. The task is no longer
321needed.
322
323.. _migration-1.6-update-alternatives-provider:
324
325``update-alternative`` Provider
326-------------------------------
327
328The default ``update-alternatives`` provider has been changed from
329``opkg`` to ``opkg-utils``. This change resolves some troublesome
330circular dependencies. The runtime package has also been renamed from
331``update-alternatives-cworth`` to ``update-alternatives-opkg``.
332
333.. _migration-1.6-virtclass-overrides:
334
335``virtclass`` Overrides
336-----------------------
337
338The ``virtclass`` overrides are now deprecated. Use the equivalent class
339overrides instead (e.g. ``virtclass-native`` becomes ``class-native``.)
340
341.. _migration-1.6-removed-renamed-recipes:
342
343Removed and Renamed Recipes
344---------------------------
345
346The following recipes have been removed:
347
348- ``packagegroup-toolset-native`` - This recipe is largely unused.
349
350- ``linux-yocto-3.8`` - Support for the Linux yocto 3.8 kernel has been
351 dropped. Support for the 3.10 and 3.14 kernels have been added with
352 the ``linux-yocto-3.10`` and ``linux-yocto-3.14`` recipes.
353
354- ``ocf-linux`` - This recipe has been functionally replaced using
355 ``cryptodev-linux``.
356
357- ``genext2fs`` - ``genext2fs`` is no longer used by the build system
358 and is unmaintained upstream.
359
360- ``js`` - This provided an ancient version of Mozilla's javascript
361 engine that is no longer needed.
362
363- ``zaurusd`` - The recipe has been moved to the ``meta-handheld``
364 layer.
365
366- ``eglibc 2.17`` - Replaced by the ``eglibc 2.19`` recipe.
367
368- ``gcc 4.7.2`` - Replaced by the now stable ``gcc 4.8.2``.
369
370- ``external-sourcery-toolchain`` - this recipe is now maintained in
371 the ``meta-sourcery`` layer.
372
373- ``linux-libc-headers-yocto 3.4+git`` - Now using version 3.10 of the
374 ``linux-libc-headers`` by default.
375
376- ``meta-toolchain-gmae`` - This recipe is obsolete.
377
378- ``packagegroup-core-sdk-gmae`` - This recipe is obsolete.
379
380- ``packagegroup-core-standalone-gmae-sdk-target`` - This recipe is
381 obsolete.
382
383.. _migration-1.6-removed-classes:
384
385Removed Classes
386---------------
387
388The following classes have become obsolete and have been removed:
389
390- ``module_strip``
391
392- ``pkg_metainfo``
393
394- ``pkg_distribute``
395
396- ``image-empty``
397
398.. _migration-1.6-reference-bsps:
399
400Reference Board Support Packages (BSPs)
401---------------------------------------
402
403The following reference BSPs changes occurred:
404
405- The BeagleBoard (``beagleboard``) ARM reference hardware has been
406 replaced by the BeagleBone (``beaglebone``) hardware.
407
408- The RouterStation Pro (``routerstationpro``) MIPS reference hardware
409 has been replaced by the EdgeRouter Lite (``edgerouter``) hardware.
410
411The previous reference BSPs for the ``beagleboard`` and
412``routerstationpro`` machines are still available in a new
413``meta-yocto-bsp-old`` layer in the
414:yocto_git:`Source Repositories <>` at
415http://git.yoctoproject.org/cgit/cgit.cgi/meta-yocto-bsp-old/.
416
417
diff --git a/documentation/ref-manual/migration-1.7.rst b/documentation/ref-manual/migration-1.7.rst
new file mode 100644
index 0000000000..82fd37d3a9
--- /dev/null
+++ b/documentation/ref-manual/migration-1.7.rst
@@ -0,0 +1,225 @@
1Moving to the Yocto Project 1.7 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 1.7 Release from the prior release.
6
7.. _migration-1.7-changes-to-setting-qemu-packageconfig-options:
8
9Changes to Setting QEMU ``PACKAGECONFIG`` Options in ``local.conf``
10-------------------------------------------------------------------
11
12The QEMU recipe now uses a number of
13:term:`PACKAGECONFIG` options to enable various
14optional features. The method used to set defaults for these options
15means that existing ``local.conf`` files will need to be be modified to
16append to ``PACKAGECONFIG`` for ``qemu-native`` and ``nativesdk-qemu``
17instead of setting it. In other words, to enable graphical output for
18QEMU, you should now have these lines in ``local.conf``:
19::
20
21 PACKAGECONFIG_append_pn-qemu-native = " sdl"
22 PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
23
24.. _migration-1.7-minimum-git-version:
25
26Minimum Git version
27-------------------
28
29The minimum :ref:`overview-manual/overview-manual-development-environment:git`
30version required on the
31build host is now 1.7.8 because the ``--list`` option is now required by
32BitBake's Git fetcher. As always, if your host distribution does not
33provide a version of Git that meets this requirement, you can use the
34``buildtools-tarball`` that does. See the "`Required Git, tar, Python
35and gcc Versions <#required-git-tar-python-and-gcc-versions>`__" section
36for more information.
37
38.. _migration-1.7-autotools-class-changes:
39
40Autotools Class Changes
41-----------------------
42
43The following :ref:`autotools <ref-classes-autotools>` class changes
44occurred:
45
46- *A separate build directory is now used by default:* The
47 ``autotools`` class has been changed to use a directory for building
48 (:term:`B`), which is separate from the source directory
49 (:term:`S`). This is commonly referred to as ``B != S``, or
50 an out-of-tree build.
51
52 If the software being built is already capable of building in a
53 directory separate from the source, you do not need to do anything.
54 However, if the software is not capable of being built in this
55 manner, you will need to either patch the software so that it can
56 build separately, or you will need to change the recipe to inherit
57 the :ref:`autotools-brokensep <ref-classes-autotools>` class
58 instead of the ``autotools`` or ``autotools_stage`` classes.
59
60- The ``--foreign`` option is no longer passed to ``automake`` when
61 running ``autoconf``: This option tells ``automake`` that a
62 particular software package does not follow the GNU standards and
63 therefore should not be expected to distribute certain files such as
64 ``ChangeLog``, ``AUTHORS``, and so forth. Because the majority of
65 upstream software packages already tell ``automake`` to enable
66 foreign mode themselves, the option is mostly superfluous. However,
67 some recipes will need patches for this change. You can easily make
68 the change by patching ``configure.ac`` so that it passes "foreign"
69 to ``AM_INIT_AUTOMAKE()``. See `this
70 commit <http://cgit.openembedded.org/openembedded-core/commit/?id=01943188f85ce6411717fb5bf702d609f55813f2>`__
71 for an example showing how to make the patch.
72
73.. _migration-1.7-binary-configuration-scripts-disabled:
74
75Binary Configuration Scripts Disabled
76-------------------------------------
77
78Some of the core recipes that package binary configuration scripts now
79disable the scripts due to the scripts previously requiring error-prone
80path substitution. Software that links against these libraries using
81these scripts should use the much more robust ``pkg-config`` instead.
82The list of recipes changed in this version (and their configuration
83scripts) is as follows:
84::
85
86 directfb (directfb-config)
87 freetype (freetype-config)
88 gpgme (gpgme-config)
89 libassuan (libassuan-config)
90 libcroco (croco-6.0-config)
91 libgcrypt (libgcrypt-config)
92 libgpg-error (gpg-error-config)
93 libksba (ksba-config)
94 libpcap (pcap-config)
95 libpcre (pcre-config)
96 libpng (libpng-config, libpng16-config)
97 libsdl (sdl-config)
98 libusb-compat (libusb-config)
99 libxml2 (xml2-config)
100 libxslt (xslt-config)
101 ncurses (ncurses-config)
102 neon (neon-config)
103 npth (npth-config)
104 pth (pth-config)
105 taglib (taglib-config)
106
107Additionally, support for ``pkg-config`` has been added to some recipes in the
108previous list in the rare cases where the upstream software package does
109not already provide it.
110
111.. _migration-1.7-glibc-replaces-eglibc:
112
113``eglibc 2.19`` Replaced with ``glibc 2.20``
114--------------------------------------------
115
116Because ``eglibc`` and ``glibc`` were already fairly close, this
117replacement should not require any significant changes to other software
118that links to ``eglibc``. However, there were a number of minor changes
119in ``glibc 2.20`` upstream that could require patching some software
120(e.g. the removal of the ``_BSD_SOURCE`` feature test macro).
121
122``glibc 2.20`` requires version 2.6.32 or greater of the Linux kernel.
123Thus, older kernels will no longer be usable in conjunction with it.
124
125For full details on the changes in ``glibc 2.20``, see the upstream
126release notes
127`here <https://sourceware.org/ml/libc-alpha/2014-09/msg00088.html>`__.
128
129.. _migration-1.7-kernel-module-autoloading:
130
131Kernel Module Autoloading
132-------------------------
133
134The :term:`module_autoload_* <module_autoload>` variable is now
135deprecated and a new
136:term:`KERNEL_MODULE_AUTOLOAD` variable
137should be used instead. Also, :term:`module_conf_* <module_conf>`
138must now be used in conjunction with a new
139:term:`KERNEL_MODULE_PROBECONF` variable.
140The new variables no longer require you to specify the module name as
141part of the variable name. This change not only simplifies usage but
142also allows the values of these variables to be appropriately
143incorporated into task signatures and thus trigger the appropriate tasks
144to re-execute when changed. You should replace any references to
145``module_autoload_*`` with ``KERNEL_MODULE_AUTOLOAD``, and add any
146modules for which ``module_conf_*`` is specified to
147``KERNEL_MODULE_PROBECONF``.
148
149.. _migration-1.7-qa-check-changes:
150
151QA Check Changes
152----------------
153
154The following changes have occurred to the QA check process:
155
156- Additional QA checks ``file-rdeps`` and ``build-deps`` have been
157 added in order to verify that file dependencies are satisfied (e.g.
158 package contains a script requiring ``/bin/bash``) and build-time
159 dependencies are declared, respectively. For more information, please
160 see the "`QA Error and Warning Messages <#ref-qa-checks>`__" chapter.
161
162- Package QA checks are now performed during a new
163 :ref:`ref-tasks-package_qa` task rather than being
164 part of the :ref:`ref-tasks-package` task. This allows
165 more parallel execution. This change is unlikely to be an issue
166 except for highly customized recipes that disable packaging tasks
167 themselves by marking them as ``noexec``. For those packages, you
168 will need to disable the ``do_package_qa`` task as well.
169
170- Files being overwritten during the
171 :ref:`ref-tasks-populate_sysroot` task now
172 trigger an error instead of a warning. Recipes should not be
173 overwriting files written to the sysroot by other recipes. If you
174 have these types of recipes, you need to alter them so that they do
175 not overwrite these files.
176
177 You might now receive this error after changes in configuration or
178 metadata resulting in orphaned files being left in the sysroot. If
179 you do receive this error, the way to resolve the issue is to delete
180 your :term:`TMPDIR` or to move it out of the way and
181 then re-start the build. Anything that has been fully built up to
182 that point and does not need rebuilding will be restored from the
183 shared state cache and the rest of the build will be able to proceed
184 as normal.
185
186.. _migration-1.7-removed-recipes:
187
188Removed Recipes
189---------------
190
191The following recipes have been removed:
192
193- ``x-load``: This recipe has been superseded by U-boot SPL for all
194 Cortex-based TI SoCs. For legacy boards, the ``meta-ti`` layer, which
195 contains a maintained recipe, should be used instead.
196
197- ``ubootchart``: This recipe is obsolete. A ``bootchart2`` recipe has
198 been added to functionally replace it.
199
200- ``linux-yocto 3.4``: Support for the linux-yocto 3.4 kernel has been
201 dropped. Support for the 3.10 and 3.14 kernels remains, while support
202 for version 3.17 has been added.
203
204- ``eglibc`` has been removed in favor of ``glibc``. See the
205 "```eglibc 2.19`` Replaced with
206 ``glibc 2.20`` <#migration-1.7-glibc-replaces-eglibc>`__" section for
207 more information.
208
209.. _migration-1.7-miscellaneous-changes:
210
211Miscellaneous Changes
212---------------------
213
214The following miscellaneous change occurred:
215
216- The build history feature now writes ``build-id.txt`` instead of
217 ``build-id``. Additionally, ``build-id.txt`` now contains the full
218 build header as printed by BitBake upon starting the build. You
219 should manually remove old "build-id" files from your existing build
220 history repositories to avoid confusion. For information on the build
221 history feature, see the
222 ":ref:`dev-manual/dev-manual-common-tasks:maintaining build output quality`"
223 section in the Yocto Project Development Tasks Manual.
224
225
diff --git a/documentation/ref-manual/migration-1.8.rst b/documentation/ref-manual/migration-1.8.rst
new file mode 100644
index 0000000000..d601e6b63b
--- /dev/null
+++ b/documentation/ref-manual/migration-1.8.rst
@@ -0,0 +1,183 @@
1Moving to the Yocto Project 1.8 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 1.8 Release from the prior release.
6
7.. _migration-1.8-removed-recipes:
8
9Removed Recipes
10---------------
11
12The following recipes have been removed:
13
14- ``owl-video``: Functionality replaced by ``gst-player``.
15
16- ``gaku``: Functionality replaced by ``gst-player``.
17
18- ``gnome-desktop``: This recipe is now available in ``meta-gnome`` and
19 is no longer needed.
20
21- ``gsettings-desktop-schemas``: This recipe is now available in
22 ``meta-gnome`` and is no longer needed.
23
24- ``python-argparse``: The ``argparse`` module is already provided in
25 the default Python distribution in a package named
26 ``python-argparse``. Consequently, the separate ``python-argparse``
27 recipe is no longer needed.
28
29- ``telepathy-python, libtelepathy, telepathy-glib, telepathy-idle, telepathy-mission-control``:
30 All these recipes have moved to ``meta-oe`` and are consequently no
31 longer needed by any recipes in OpenEmbedded-Core.
32
33- ``linux-yocto_3.10`` and ``linux-yocto_3.17``: Support for the
34 linux-yocto 3.10 and 3.17 kernels has been dropped. Support for the
35 3.14 kernel remains, while support for 3.19 kernel has been added.
36
37- ``poky-feed-config-opkg``: This recipe has become obsolete and is no
38 longer needed. Use ``distro-feed-config`` from ``meta-oe`` instead.
39
40- ``libav 0.8.x``: ``libav 9.x`` is now used.
41
42- ``sed-native``: No longer needed. A working version of ``sed`` is
43 expected to be provided by the host distribution.
44
45.. _migration-1.8-bluez:
46
47BlueZ 4.x / 5.x Selection
48-------------------------
49
50Proper built-in support for selecting BlueZ 5.x in preference to the
51default of 4.x now exists. To use BlueZ 5.x, simply add "bluez5" to your
52:term:`DISTRO_FEATURES` value. If you had
53previously added append files (``*.bbappend``) to make this selection,
54you can now remove them.
55
56Additionally, a ``bluetooth`` class has been added to make selection of
57the appropriate bluetooth support within a recipe a little easier. If
58you wish to make use of this class in a recipe, add something such as
59the following: ::
60
61 inherit bluetooth
62 PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', '${BLUEZ}', '', d)}"
63 PACKAGECONFIG[bluez4] = "--enable-bluetooth,--disable-bluetooth,bluez4"
64 PACKAGECONFIG[bluez5] = "--enable-bluez5,--disable-bluez5,bluez5"
65
66.. _migration-1.8-kernel-build-changes:
67
68Kernel Build Changes
69--------------------
70
71The kernel build process was changed to place the source in a common
72shared work area and to place build artifacts separately in the source
73code tree. In theory, migration paths have been provided for most common
74usages in kernel recipes but this might not work in all cases. In
75particular, users need to ensure that ``${S}`` (source files) and
76``${B}`` (build artifacts) are used correctly in functions such as
77:ref:`ref-tasks-configure` and
78:ref:`ref-tasks-install`. For kernel recipes that do not
79inherit from ``kernel-yocto`` or include ``linux-yocto.inc``, you might
80wish to refer to the ``linux.inc`` file in the ``meta-oe`` layer for the
81kinds of changes you need to make. For reference, here is the
82`commit <http://cgit.openembedded.org/meta-openembedded/commit/meta-oe/recipes-kernel/linux/linux.inc?id=fc7132ede27ac67669448d3d2845ce7d46c6a1ee>`__
83where the ``linux.inc`` file in ``meta-oe`` was updated.
84
85Recipes that rely on the kernel source code and do not inherit the
86module classes might need to add explicit dependencies on the
87``do_shared_workdir`` kernel task, for example: ::
88
89 do_configure[depends] += "virtual/kernel:do_shared_workdir"
90
91.. _migration-1.8-ssl:
92
93SSL 3.0 is Now Disabled in OpenSSL
94----------------------------------
95
96SSL 3.0 is now disabled when building OpenSSL. Disabling SSL 3.0 avoids
97any lingering instances of the POODLE vulnerability. If you feel you
98must re-enable SSL 3.0, then you can add an append file (``*.bbappend``)
99for the ``openssl`` recipe to remove "-no-ssl3" from
100:term:`EXTRA_OECONF`.
101
102.. _migration-1.8-default-sysroot-poisoning:
103
104Default Sysroot Poisoning
105-------------------------
106
107``gcc's`` default sysroot and include directories are now "poisoned". In
108other words, the sysroot and include directories are being redirected to
109a non-existent location in order to catch when host directories are
110being used due to the correct options not being passed. This poisoning
111applies both to the cross-compiler used within the build and to the
112cross-compiler produced in the SDK.
113
114If this change causes something in the build to fail, it almost
115certainly means the various compiler flags and commands are not being
116passed correctly to the underlying piece of software. In such cases, you
117need to take corrective steps.
118
119.. _migration-1.8-rebuild-improvements:
120
121Rebuild Improvements
122--------------------
123
124Changes have been made to the :ref:`base <ref-classes-base>`,
125:ref:`autotools <ref-classes-autotools>`, and
126:ref:`cmake <ref-classes-cmake>` classes to clean out generated files
127when the :ref:`ref-tasks-configure` task needs to be
128re-executed.
129
130One of the improvements is to attempt to run "make clean" during the
131``do_configure`` task if a ``Makefile`` exists. Some software packages
132do not provide a working clean target within their make files. If you
133have such recipes, you need to set
134:term:`CLEANBROKEN` to "1" within the recipe, for example: ::
135
136 CLEANBROKEN = "1"
137
138.. _migration-1.8-qa-check-and-validation-changes:
139
140QA Check and Validation Changes
141-------------------------------
142
143The following QA Check and Validation Changes have occurred:
144
145- Usage of ``PRINC`` previously triggered a warning. It now triggers an
146 error. You should remove any remaining usage of ``PRINC`` in any
147 recipe or append file.
148
149- An additional QA check has been added to detect usage of ``${D}`` in
150 :term:`FILES` values where :term:`D` values
151 should not be used at all. The same check ensures that ``$D`` is used
152 in ``pkg_preinst/pkg_postinst/pkg_prerm/pkg_postrm`` functions
153 instead of ``${D}``.
154
155- :term:`S` now needs to be set to a valid value within a
156 recipe. If ``S`` is not set in the recipe, the directory is not
157 automatically created. If ``S`` does not point to a directory that
158 exists at the time the :ref:`ref-tasks-unpack` task
159 finishes, a warning will be shown.
160
161- :term:`LICENSE` is now validated for correct
162 formatting of multiple licenses. If the format is invalid (e.g.
163 multiple licenses are specified with no operators to specify how the
164 multiple licenses interact), then a warning will be shown.
165
166.. _migration-1.8-miscellaneous-changes:
167
168Miscellaneous Changes
169---------------------
170
171The following miscellaneous changes have occurred:
172
173- The ``send-error-report`` script now expects a "-s" option to be
174 specified before the server address. This assumes a server address is
175 being specified.
176
177- The ``oe-pkgdata-util`` script now expects a "-p" option to be
178 specified before the ``pkgdata`` directory, which is now optional. If
179 the ``pkgdata`` directory is not specified, the script will run
180 BitBake to query :term:`PKGDATA_DIR` from the
181 build environment.
182
183
diff --git a/documentation/ref-manual/migration-2.0.rst b/documentation/ref-manual/migration-2.0.rst
new file mode 100644
index 0000000000..570486ba00
--- /dev/null
+++ b/documentation/ref-manual/migration-2.0.rst
@@ -0,0 +1,281 @@
1Moving to the Yocto Project 2.0 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.0 Release from the prior release.
6
7.. _migration-2.0-gcc-5:
8
9GCC 5
10-----
11
12The default compiler is now GCC 5.2. This change has required fixes for
13compilation errors in a number of other recipes.
14
15One important example is a fix for when the Linux kernel freezes at boot
16time on ARM when built with GCC 5. If you are using your own kernel
17recipe or source tree and building for ARM, you will likely need to
18apply this
19`patch <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a077224fd35b2f7fbc93f14cf67074fc792fbac2>`__.
20The standard ``linux-yocto`` kernel source tree already has a workaround
21for the same issue.
22
23For further details, see https://gcc.gnu.org/gcc-5/changes.html
24and the porting guide at
25https://gcc.gnu.org/gcc-5/porting_to.html.
26
27Alternatively, you can switch back to GCC 4.9 or 4.8 by setting
28``GCCVERSION`` in your configuration, as follows:
29::
30
31 GCCVERSION = "4.9%"
32
33.. _migration-2.0-Gstreamer-0.10-removed:
34
35Gstreamer 0.10 Removed
36----------------------
37
38Gstreamer 0.10 has been removed in favor of Gstreamer 1.x. As part of
39the change, recipes for Gstreamer 0.10 and related software are now
40located in ``meta-multimedia``. This change results in Qt4 having Phonon
41and Gstreamer support in QtWebkit disabled by default.
42
43.. _migration-2.0-removed-recipes:
44
45Removed Recipes
46---------------
47
48The following recipes have been moved or removed:
49
50- ``bluez4``: The recipe is obsolete and has been moved due to
51 ``bluez5`` becoming fully integrated. The ``bluez4`` recipe now
52 resides in ``meta-oe``.
53
54- ``gamin``: The recipe is obsolete and has been removed.
55
56- ``gnome-icon-theme``: The recipe's functionally has been replaced by
57 ``adwaita-icon-theme``.
58
59- Gstreamer 0.10 Recipes: Recipes for Gstreamer 0.10 have been removed
60 in favor of the recipes for Gstreamer 1.x.
61
62- ``insserv``: The recipe is obsolete and has been removed.
63
64- ``libunique``: The recipe is no longer used and has been moved to
65 ``meta-oe``.
66
67- ``midori``: The recipe's functionally has been replaced by
68 ``epiphany``.
69
70- ``python-gst``: The recipe is obsolete and has been removed since it
71 only contains bindings for Gstreamer 0.10.
72
73- ``qt-mobility``: The recipe is obsolete and has been removed since it
74 requires ``Gstreamer 0.10``, which has been replaced.
75
76- ``subversion``: All 1.6.x versions of this recipe have been removed.
77
78- ``webkit-gtk``: The older 1.8.3 version of this recipe has been
79 removed in favor of ``webkitgtk``.
80
81.. _migration-2.0-bitbake-datastore-improvements:
82
83BitBake datastore improvements
84------------------------------
85
86The method by which BitBake's datastore handles overrides has changed.
87Overrides are now applied dynamically and ``bb.data.update_data()`` is
88now a no-op. Thus, ``bb.data.update_data()`` is no longer required in
89order to apply the correct overrides. In practice, this change is
90unlikely to require any changes to Metadata. However, these minor
91changes in behavior exist:
92
93- All potential overrides are now visible in the variable history as
94 seen when you run the following:
95 ::
96
97 $ bitbake -e
98
99- ``d.delVar('``\ VARNAME\ ``')`` and
100 ``d.setVar('``\ VARNAME\ ``', None)`` result in the variable and all
101 of its overrides being cleared out. Before the change, only the
102 non-overridden values were cleared.
103
104.. _migration-2.0-shell-message-function-changes:
105
106Shell Message Function Changes
107------------------------------
108
109The shell versions of the BitBake message functions (i.e. ``bbdebug``,
110``bbnote``, ``bbwarn``, ``bbplain``, ``bberror``, and ``bbfatal``) are
111now connected through to their BitBake equivalents ``bb.debug()``,
112``bb.note()``, ``bb.warn()``, ``bb.plain()``, ``bb.error()``, and
113``bb.fatal()``, respectively. Thus, those message functions that you
114would expect to be printed by the BitBake UI are now actually printed.
115In practice, this change means two things:
116
117- If you now see messages on the console that you did not previously
118 see as a result of this change, you might need to clean up the calls
119 to ``bbwarn``, ``bberror``, and so forth. Or, you might want to
120 simply remove the calls.
121
122- The ``bbfatal`` message function now suppresses the full error log in
123 the UI, which means any calls to ``bbfatal`` where you still wish to
124 see the full error log should be replaced by ``die`` or
125 ``bbfatal_log``.
126
127.. _migration-2.0-extra-development-debug-package-cleanup:
128
129Extra Development/Debug Package Cleanup
130---------------------------------------
131
132The following recipes have had extra ``dev/dbg`` packages removed:
133
134- ``acl``
135
136- ``apmd``
137
138- ``aspell``
139
140- ``attr``
141
142- ``augeas``
143
144- ``bzip2``
145
146- ``cogl``
147
148- ``curl``
149
150- ``elfutils``
151
152- ``gcc-target``
153
154- ``libgcc``
155
156- ``libtool``
157
158- ``libxmu``
159
160- ``opkg``
161
162- ``pciutils``
163
164- ``rpm``
165
166- ``sysfsutils``
167
168- ``tiff``
169
170- ``xz``
171
172All of the above recipes now conform to the standard packaging scheme
173where a single ``-dev``, ``-dbg``, and ``-staticdev`` package exists per
174recipe.
175
176.. _migration-2.0-recipe-maintenance-tracking-data-moved-to-oe-core:
177
178Recipe Maintenance Tracking Data Moved to OE-Core
179-------------------------------------------------
180
181Maintenance tracking data for recipes that was previously part of
182``meta-yocto`` has been moved to :term:`OpenEmbedded-Core (OE-Core)`. The change
183includes ``package_regex.inc`` and ``distro_alias.inc``, which are
184typically enabled when using the ``distrodata`` class. Additionally, the
185contents of ``upstream_tracking.inc`` has now been split out to the
186relevant recipes.
187
188.. _migration-2.0-automatic-stale-sysroot-file-cleanup:
189
190Automatic Stale Sysroot File Cleanup
191------------------------------------
192
193Stale files from recipes that no longer exist in the current
194configuration are now automatically removed from sysroot as well as
195removed from any other place managed by shared state. This automatic
196cleanup means that the build system now properly handles situations such
197as renaming the build system side of recipes, removal of layers from
198``bblayers.conf``, and :term:`DISTRO_FEATURES`
199changes.
200
201Additionally, work directories for old versions of recipes are now
202pruned. If you wish to disable pruning old work directories, you can set
203the following variable in your configuration:
204::
205
206 SSTATE_PRUNE_OBSOLETEWORKDIR = "0"
207
208.. _migration-2.0-linux-yocto-kernel-metadata-repository-now-split-from-source:
209
210``linux-yocto`` Kernel Metadata Repository Now Split from Source
211----------------------------------------------------------------
212
213The ``linux-yocto`` tree has up to now been a combined set of kernel
214changes and configuration (meta) data carried in a single tree. While
215this format is effective at keeping kernel configuration and source
216modifications synchronized, it is not always obvious to developers how
217to manipulate the Metadata as compared to the source.
218
219Metadata processing has now been removed from the
220:ref:`kernel-yocto <ref-classes-kernel-yocto>` class and the external
221Metadata repository ``yocto-kernel-cache``, which has always been used
222to seed the ``linux-yocto`` "meta" branch. This separate ``linux-yocto``
223cache repository is now the primary location for this data. Due to this
224change, ``linux-yocto`` is no longer able to process combined trees.
225Thus, if you need to have your own combined kernel repository, you must
226do the split there as well and update your recipes accordingly. See the
227``meta/recipes-kernel/linux/linux-yocto_4.1.bb`` recipe for an example.
228
229.. _migration-2.0-additional-qa-checks:
230
231Additional QA checks
232--------------------
233
234The following QA checks have been added:
235
236- Added a "host-user-contaminated" check for ownership issues for
237 packaged files outside of ``/home``. The check looks for files that
238 are incorrectly owned by the user that ran BitBake instead of owned
239 by a valid user in the target system.
240
241- Added an "invalid-chars" check for invalid (non-UTF8) characters in
242 recipe metadata variable values (i.e.
243 :term:`DESCRIPTION`,
244 :term:`SUMMARY`, :term:`LICENSE`, and
245 :term:`SECTION`). Some package managers do not support
246 these characters.
247
248- Added an "invalid-packageconfig" check for any options specified in
249 :term:`PACKAGECONFIG` that do not match any
250 ``PACKAGECONFIG`` option defined for the recipe.
251
252.. _migration-2.0-miscellaneous:
253
254Miscellaneous Changes
255---------------------
256
257These additional changes exist:
258
259- ``gtk-update-icon-cache`` has been renamed to ``gtk-icon-utils``.
260
261- The ``tools-profile`` :term:`IMAGE_FEATURES`
262 item as well as its corresponding packagegroup and
263 ``packagegroup-core-tools-profile`` no longer bring in ``oprofile``.
264 Bringing in ``oprofile`` was originally added to aid compilation on
265 resource-constrained targets. However, this aid has not been widely
266 used and is not likely to be used going forward due to the more
267 powerful target platforms and the existence of better
268 cross-compilation tools.
269
270- The :term:`IMAGE_FSTYPES` variable's default
271 value now specifies ``ext4`` instead of ``ext3``.
272
273- All support for the ``PRINC`` variable has been removed.
274
275- The ``packagegroup-core-full-cmdline`` packagegroup no longer brings
276 in ``lighttpd`` due to the fact that bringing in ``lighttpd`` is not
277 really in line with the packagegroup's purpose, which is to add full
278 versions of command-line tools that by default are provided by
279 ``busybox``.
280
281
diff --git a/documentation/ref-manual/migration-2.1.rst b/documentation/ref-manual/migration-2.1.rst
new file mode 100644
index 0000000000..a1fd3ea81d
--- /dev/null
+++ b/documentation/ref-manual/migration-2.1.rst
@@ -0,0 +1,434 @@
1Moving to the Yocto Project 2.1 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.1 Release from the prior release.
6
7.. _migration-2.1-variable-expansion-in-python-functions:
8
9Variable Expansion in Python Functions
10--------------------------------------
11
12Variable expressions, such as ``${``\ VARNAME\ ``}`` no longer expand
13automatically within Python functions. Suppressing expansion was done to
14allow Python functions to construct shell scripts or other code for
15situations in which you do not want such expressions expanded. For any
16existing code that relies on these expansions, you need to change the
17expansions to expand the value of individual variables through
18``d.getVar()``. To alternatively expand more complex expressions, use
19``d.expand()``.
20
21.. _migration-2.1-overrides-must-now-be-lower-case:
22
23Overrides Must Now be Lower-Case
24--------------------------------
25
26The convention for overrides has always been for them to be lower-case
27characters. This practice is now a requirement as BitBake's datastore
28now assumes lower-case characters in order to give a slight performance
29boost during parsing. In practical terms, this requirement means that
30anything that ends up in :term:`OVERRIDES` must now
31appear in lower-case characters (e.g. values for ``MACHINE``,
32``TARGET_ARCH``, ``DISTRO``, and also recipe names if
33``_pn-``\ recipename overrides are to be effective).
34
35.. _migration-2.1-expand-parameter-to-getvar-and-getvarflag-now-mandatory:
36
37Expand Parameter to ``getVar()`` and ``getVarFlag()`` is Now Mandatory
38----------------------------------------------------------------------
39
40The expand parameter to ``getVar()`` and ``getVarFlag()`` previously
41defaulted to False if not specified. Now, however, no default exists so
42one must be specified. You must change any ``getVar()`` calls that do
43not specify the final expand parameter to calls that do specify the
44parameter. You can run the following ``sed`` command at the base of a
45layer to make this change:
46::
47
48 sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
49 sed -e 's:\(\.getVarFlag([^,()]*,[^,()]*\)):\1, False):g' -i `grep -ril getVarFlag *`
50
51.. note::
52
53 The reason for this change is that it prepares the way for changing
54 the default to True in a future Yocto Project release. This future
55 change is a much more sensible default than False. However, the
56 change needs to be made gradually as a sudden change of the default
57 would potentially cause side-effects that would be difficult to
58 detect.
59
60.. _migration-2.1-makefile-environment-changes:
61
62Makefile Environment Changes
63----------------------------
64
65:term:`EXTRA_OEMAKE` now defaults to "" instead of
66"-e MAKEFLAGS=". Setting ``EXTRA_OEMAKE`` to "-e MAKEFLAGS=" by default
67was a historical accident that has required many classes (e.g.
68``autotools``, ``module``) and recipes to override this default in order
69to work with sensible build systems. When upgrading to the release, you
70must edit any recipe that relies upon this old default by either setting
71``EXTRA_OEMAKE`` back to "-e MAKEFLAGS=" or by explicitly setting any
72required variable value overrides using ``EXTRA_OEMAKE``, which is
73typically only needed when a Makefile sets a default value for a
74variable that is inappropriate for cross-compilation using the "="
75operator rather than the "?=" operator.
76
77.. _migration-2.1-libexecdir-reverted-to-prefix-libexec:
78
79``libexecdir`` Reverted to ``${prefix}/libexec``
80------------------------------------------------
81
82The use of ``${libdir}/${BPN}`` as ``libexecdir`` is different as
83compared to all other mainstream distributions, which either uses
84``${prefix}/libexec`` or ``${libdir}``. The use is also contrary to the
85GNU Coding Standards (i.e.
86https://www.gnu.org/prep/standards/html_node/Directory-Variables.html)
87that suggest ``${prefix}/libexec`` and also notes that any
88package-specific nesting should be done by the package itself. Finally,
89having ``libexecdir`` change between recipes makes it very difficult for
90different recipes to invoke binaries that have been installed into
91``libexecdir``. The Filesystem Hierarchy Standard (i.e.
92http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html) now
93recognizes the use of ``${prefix}/libexec/``, giving distributions the
94choice between ``${prefix}/lib`` or ``${prefix}/libexec`` without
95breaking FHS.
96
97.. _migration-2.1-ac-cv-sizeof-off-t-no-longer-cached-in-site-files:
98
99``ac_cv_sizeof_off_t`` is No Longer Cached in Site Files
100--------------------------------------------------------
101
102For recipes inheriting the :ref:`autotools <ref-classes-autotools>`
103class, ``ac_cv_sizeof_off_t`` is no longer cached in the site files for
104``autoconf``. The reason for this change is because the
105``ac_cv_sizeof_off_t`` value is not necessarily static per architecture
106as was previously assumed. Rather, the value changes based on whether
107large file support is enabled. For most software that uses ``autoconf``,
108this change should not be a problem. However, if you have a recipe that
109bypasses the standard :ref:`ref-tasks-configure` task
110from the ``autotools`` class and the software the recipe is building
111uses a very old version of ``autoconf``, the recipe might be incapable
112of determining the correct size of ``off_t`` during ``do_configure``.
113
114The best course of action is to patch the software as necessary to allow
115the default implementation from the ``autotools`` class to work such
116that ``autoreconf`` succeeds and produces a working configure script,
117and to remove the overridden ``do_configure`` task such that the default
118implementation does get used.
119
120.. _migration-2.1-image-generation-split-out-from-filesystem-generation:
121
122Image Generation is Now Split Out from Filesystem Generation
123------------------------------------------------------------
124
125Previously, for image recipes the :ref:`ref-tasks-rootfs`
126task assembled the filesystem and then from that filesystem generated
127images. With this Yocto Project release, image generation is split into
128separate ```do_image_*`` <#ref-tasks-image>`__ tasks for clarity both in
129operation and in the code.
130
131For most cases, this change does not present any problems. However, if
132you have made customizations that directly modify the ``do_rootfs`` task
133or that mention ``do_rootfs``, you might need to update those changes.
134In particular, if you had added any tasks after ``do_rootfs``, you
135should make edits so that those tasks are after the
136```do_image_complete`` <#ref-tasks-image-complete>`__ task rather than
137after ``do_rootfs`` so that the your added tasks run at the correct
138time.
139
140A minor part of this restructuring is that the post-processing
141definitions and functions have been moved from the
142:ref:`image <ref-classes-image>` class to the
143:ref:`rootfs-postcommands <ref-classes-rootfs*>` class. Functionally,
144however, they remain unchanged.
145
146.. _migration-2.1-removed-recipes:
147
148Removed Recipes
149---------------
150
151The following recipes have been removed in the 2.1 release:
152
153- ``gcc`` version 4.8: Versions 4.9 and 5.3 remain.
154
155- ``qt4``: All support for Qt 4.x has been moved out to a separate
156 ``meta-qt4`` layer because Qt 4 is no longer supported upstream.
157
158- ``x11vnc``: Moved to the ``meta-oe`` layer.
159
160- ``linux-yocto-3.14``: No longer supported.
161
162- ``linux-yocto-3.19``: No longer supported.
163
164- ``libjpeg``: Replaced by the ``libjpeg-turbo`` recipe.
165
166- ``pth``: Became obsolete.
167
168- ``liboil``: Recipe is no longer needed and has been moved to the
169 ``meta-multimedia`` layer.
170
171- ``gtk-theme-torturer``: Recipe is no longer needed and has been moved
172 to the ``meta-gnome`` layer.
173
174- ``gnome-mime-data``: Recipe is no longer needed and has been moved to
175 the ``meta-gnome`` layer.
176
177- ``udev``: Replaced by the ``eudev`` recipe for compatibility when
178 using ``sysvinit`` with newer kernels.
179
180- ``python-pygtk``: Recipe became obsolete.
181
182- ``adt-installer``: Recipe became obsolete. See the "`ADT
183 Removed <#migration-2.1-adt-removed>`__" section for more
184 information.
185
186.. _migration-2.1-class-changes:
187
188Class Changes
189-------------
190
191The following classes have changed:
192
193- ``autotools_stage``: Removed because the
194 :ref:`autotools <ref-classes-autotools>` class now provides its
195 functionality. Recipes that inherited from ``autotools_stage`` should
196 now inherit from ``autotools`` instead.
197
198- ``boot-directdisk``: Merged into the ``image-vm`` class. The
199 ``boot-directdisk`` class was rarely directly used. Consequently,
200 this change should not cause any issues.
201
202- ``bootimg``: Merged into the
203 :ref:`image-live <ref-classes-image-live>` class. The ``bootimg``
204 class was rarely directly used. Consequently, this change should not
205 cause any issues.
206
207- ``packageinfo``: Removed due to its limited use by the Hob UI, which
208 has itself been removed.
209
210.. _migration-2.1-build-system-ui-changes:
211
212Build System User Interface Changes
213-----------------------------------
214
215The following changes have been made to the build system user interface:
216
217- *Hob GTK+-based UI*: Removed because it is unmaintained and based on
218 the outdated GTK+ 2 library. The Toaster web-based UI is much more
219 capable and is actively maintained. See the
220 ":ref:`toaster-manual/toaster-manual-setup-and-use:using the toaster web interface`"
221 section in the Toaster User Manual for more information on this
222 interface.
223
224- *"puccho" BitBake UI*: Removed because is unmaintained and no longer
225 useful.
226
227.. _migration-2.1-adt-removed:
228
229ADT Removed
230-----------
231
232The Application Development Toolkit (ADT) has been removed because its
233functionality almost completely overlapped with the :ref:`standard
234SDK <sdk-manual/sdk-using:using the standard sdk>` and the
235:ref:`extensible SDK <sdk-manual/sdk-extensible:using the extensible sdk>`. For
236information on these SDKs and how to build and use them, see the
237:doc:`../sdk-manual/sdk-manual` manual.
238
239.. note::
240
241 The Yocto Project Eclipse IDE Plug-in is still supported and is not
242 affected by this change.
243
244.. _migration-2.1-poky-reference-distribution-changes:
245
246Poky Reference Distribution Changes
247-----------------------------------
248
249The following changes have been made for the Poky distribution:
250
251- The ``meta-yocto`` layer has been renamed to ``meta-poky`` to better
252 match its purpose, which is to provide the Poky reference
253 distribution. The ``meta-yocto-bsp`` layer retains its original name
254 since it provides reference machines for the Yocto Project and it is
255 otherwise unrelated to Poky. References to ``meta-yocto`` in your
256 ``conf/bblayers.conf`` should automatically be updated, so you should
257 not need to change anything unless you are relying on this naming
258 elsewhere.
259
260- The :ref:`uninative <ref-classes-uninative>` class is now enabled
261 by default in Poky. This class attempts to isolate the build system
262 from the host distribution's C library and makes re-use of native
263 shared state artifacts across different host distributions practical.
264 With this class enabled, a tarball containing a pre-built C library
265 is downloaded at the start of the build.
266
267 The ``uninative`` class is enabled through the
268 ``meta/conf/distro/include/yocto-uninative.inc`` file, which for
269 those not using the Poky distribution, can include to easily enable
270 the same functionality.
271
272 Alternatively, if you wish to build your own ``uninative`` tarball,
273 you can do so by building the ``uninative-tarball`` recipe, making it
274 available to your build machines (e.g. over HTTP/HTTPS) and setting a
275 similar configuration as the one set by ``yocto-uninative.inc``.
276
277- Static library generation, for most cases, is now disabled by default
278 in the Poky distribution. Disabling this generation saves some build
279 time as well as the size used for build output artifacts.
280
281 Disabling this library generation is accomplished through a
282 ``meta/conf/distro/include/no-static-libs.inc``, which for those not
283 using the Poky distribution can easily include to enable the same
284 functionality.
285
286 Any recipe that needs to opt-out of having the "--disable-static"
287 option specified on the configure command line either because it is
288 not a supported option for the configure script or because static
289 libraries are needed should set the following variable:
290 DISABLE_STATIC = ""
291
292- The separate ``poky-tiny`` distribution now uses the musl C library
293 instead of a heavily pared down ``glibc``. Using musl results in a
294 smaller distribution and facilitates much greater maintainability
295 because musl is designed to have a small footprint.
296
297 If you have used ``poky-tiny`` and have customized the ``glibc``
298 configuration you will need to redo those customizations with musl
299 when upgrading to the new release.
300
301.. _migration-2.1-packaging-changes:
302
303Packaging Changes
304-----------------
305
306The following changes have been made to packaging:
307
308- The ``runuser`` and ``mountpoint`` binaries, which were previously in
309 the main ``util-linux`` package, have been split out into the
310 ``util-linux-runuser`` and ``util-linux-mountpoint`` packages,
311 respectively.
312
313- The ``python-elementtree`` package has been merged into the
314 ``python-xml`` package.
315
316.. _migration-2.1-tuning-file-changes:
317
318Tuning File Changes
319-------------------
320
321The following changes have been made to the tuning files:
322
323- The "no-thumb-interwork" tuning feature has been dropped from the ARM
324 tune include files. Because interworking is required for ARM EABI,
325 attempting to disable it through a tuning feature no longer makes
326 sense.
327
328 .. note::
329
330 Support for ARM OABI was deprecated in gcc 4.7.
331
332- The ``tune-cortexm*.inc`` and ``tune-cortexr4.inc`` files have been
333 removed because they are poorly tested. Until the OpenEmbedded build
334 system officially gains support for CPUs without an MMU, these tuning
335 files would probably be better maintained in a separate layer if
336 needed.
337
338.. _migration-2.1-supporting-gobject-introspection:
339
340Supporting GObject Introspection
341--------------------------------
342
343This release supports generation of GLib Introspective Repository (GIR)
344files through GObject introspection, which is the standard mechanism for
345accessing GObject-based software from runtime environments. You can
346enable, disable, and test the generation of this data. See the
347":ref:`dev-manual/dev-manual-common-tasks:enabling gobject introspection support`"
348section in the Yocto Project Development Tasks Manual for more
349information.
350
351.. _migration-2.1-miscellaneous-changes:
352
353Miscellaneous Changes
354---------------------
355
356These additional changes exist:
357
358- The minimum Git version has been increased to 1.8.3.1. If your host
359 distribution does not provide a sufficiently recent version, you can
360 install the buildtools, which will provide it. See the "`Required
361 Git, tar, Python and gcc
362 Versions <#required-git-tar-python-and-gcc-versions>`__" section for
363 more information on the buildtools tarball.
364
365- The buggy and incomplete support for the RPM version 4 package
366 manager has been removed. The well-tested and maintained support for
367 RPM version 5 remains.
368
369- Previously, the following list of packages were removed if
370 package-management was not in
371 :term:`IMAGE_FEATURES`, regardless of any
372 dependencies:
373 ::
374
375 update-rc.d
376 base-passwd
377 shadow
378 update-alternatives
379
380 run-postinsts With the Yocto Project 2.1 release, these packages are
381 only removed if "read-only-rootfs" is in ``IMAGE_FEATURES``, since
382 they might still be needed for a read-write image even in the absence
383 of a package manager (e.g. if users need to be added, modified, or
384 removed at runtime).
385
386- The
387 :ref:`devtool modify <sdk-manual/sdk-extensible:use \`\`devtool modify\`\` to modify the source of an existing component>`
388 command now defaults to extracting the source since that is most
389 commonly expected. The "-x" or "--extract" options are now no-ops. If
390 you wish to provide your own existing source tree, you will now need
391 to specify either the "-n" or "--no-extract" options when running
392 ``devtool modify``.
393
394- If the formfactor for a machine is either not supplied or does not
395 specify whether a keyboard is attached, then the default is to assume
396 a keyboard is attached rather than assume no keyboard. This change
397 primarily affects the Sato UI.
398
399- The ``.debug`` directory packaging is now automatic. If your recipe
400 builds software that installs binaries into directories other than
401 the standard ones, you no longer need to take care of setting
402 ``FILES_${PN}-dbg`` to pick up the resulting ``.debug`` directories
403 as these directories are automatically found and added.
404
405- Inaccurate disk and CPU percentage data has been dropped from
406 ``buildstats`` output. This data has been replaced with
407 ``getrusage()`` data and corrected IO statistics. You will probably
408 need to update any custom code that reads the ``buildstats`` data.
409
410- The ``meta/conf/distro/include/package_regex.inc`` is now deprecated.
411 The contents of this file have been moved to individual recipes.
412
413 .. note::
414
415 Because this file will likely be removed in a future Yocto Project
416 release, it is suggested that you remove any references to the
417 file that might be in your configuration.
418
419- The ``v86d/uvesafb`` has been removed from the ``genericx86`` and
420 ``genericx86-64`` reference machines, which are provided by the
421 ``meta-yocto-bsp`` layer. Most modern x86 boards do not rely on this
422 file and it only adds kernel error messages during startup. If you do
423 still need to support ``uvesafb``, you can simply add ``v86d`` to
424 your image.
425
426- Build sysroot paths are now removed from debug symbol files. Removing
427 these paths means that remote GDB using an unstripped build system
428 sysroot will no longer work (although this was never documented to
429 work). The supported method to accomplish something similar is to set
430 ``IMAGE_GEN_DEBUGFS`` to "1", which will generate a companion debug
431 image containing unstripped binaries and associated debug sources
432 alongside the image.
433
434
diff --git a/documentation/ref-manual/migration-2.2.rst b/documentation/ref-manual/migration-2.2.rst
new file mode 100644
index 0000000000..59d0eeeb9d
--- /dev/null
+++ b/documentation/ref-manual/migration-2.2.rst
@@ -0,0 +1,451 @@
1Moving to the Yocto Project 2.2 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.2 Release from the prior release.
6
7.. _migration-2.2-minimum-kernel-version:
8
9Minimum Kernel Version
10----------------------
11
12The minimum kernel version for the target system and for SDK is now
133.2.0, due to the upgrade to ``glibc 2.24``. Specifically, for
14AArch64-based targets the version is 3.14. For Nios II-based targets,
15the minimum kernel version is 3.19.
16
17.. note::
18
19 For x86 and x86_64, you can reset
20 OLDEST_KERNEL
21 to anything down to 2.6.32 if desired.
22
23.. _migration-2.2-staging-directories-in-sysroot-simplified:
24
25Staging Directories in Sysroot Has Been Simplified
26--------------------------------------------------
27
28The way directories are staged in sysroot has been simplified and
29introduces the new :term:`SYSROOT_DIRS`,
30:term:`SYSROOT_DIRS_NATIVE`, and
31:term:`SYSROOT_DIRS_BLACKLIST`. See the
32`v2 patch series on the OE-Core Mailing
33List <http://lists.openembedded.org/pipermail/openembedded-core/2016-May/121365.html>`__
34for additional information.
35
36.. _migration-2.2-removal-of-old-images-from-tmp-deploy-now-enabled:
37
38Removal of Old Images and Other Files in ``tmp/deploy`` Now Enabled
39-------------------------------------------------------------------
40
41Removal of old images and other files in ``tmp/deploy/`` is now enabled
42by default due to a new staging method used for those files. As a result
43of this change, the ``RM_OLD_IMAGE`` variable is now redundant.
44
45.. _migration-2.2-python-changes:
46
47Python Changes
48--------------
49
50The following changes for Python occurred:
51
52.. _migration-2.2-bitbake-now-requires-python-3.4:
53
54BitBake Now Requires Python 3.4+
55~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
56
57BitBake requires Python 3.4 or greater.
58
59.. _migration-2.2-utf-8-locale-required-on-build-host:
60
61UTF-8 Locale Required on Build Host
62~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
63
64A UTF-8 locale is required on the build host due to Python 3. Since
65C.UTF-8 is not a standard, the default is en_US.UTF-8.
66
67.. _migration-2.2-metadata-now-must-use-python-3-syntax:
68
69Metadata Must Now Use Python 3 Syntax
70~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
71
72The metadata is now required to use Python 3 syntax. For help preparing
73metadata, see any of the many Python 3 porting guides available.
74Alternatively, you can reference the conversion commits for Bitbake and
75you can use :term:`OpenEmbedded-Core (OE-Core)` as a guide for changes. Following are
76particular areas of interest:
77
78 - subprocess command-line pipes needing locale decoding
79
80 - the syntax for octal values changed
81
82 - the ``iter*()`` functions changed name \* iterators now return views, not lists
83
84 - changed names for Python modules
85
86.. _migration-2.2-target-python-recipes-switched-to-python-3:
87
88Target Python Recipes Switched to Python 3
89~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
90
91Most target Python recipes have now been switched to Python 3.
92Unfortunately, systems using RPM as a package manager and providing
93online package-manager support through SMART still require Python 2.
94
95.. note::
96
97 Python 2 and recipes that use it can still be built for the target as
98 with previous versions.
99
100.. _migration-2.2-buildtools-tarball-includes-python-3:
101
102``buildtools-tarball`` Includes Python 3
103~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
104
105``buildtools-tarball`` now includes Python 3.
106
107.. _migration-2.2-uclibc-replaced-by-musl:
108
109uClibc Replaced by musl
110-----------------------
111
112uClibc has been removed in favor of musl. Musl has matured, is better
113maintained, and is compatible with a wider range of applications as
114compared to uClibc.
115
116.. _migration-2.2-B-no-longer-default-working-directory-for-tasks:
117
118``${B}`` No Longer Default Working Directory for Tasks
119------------------------------------------------------
120
121``${``\ :term:`B`\ ``}`` is no longer the default working
122directory for tasks. Consequently, any custom tasks you define now need
123to either have the
124``[``\ :ref:`dirs <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]`` flag
125set, or the task needs to change into the appropriate working directory
126manually (e.g using ``cd`` for a shell task).
127
128.. note::
129
130 The preferred method is to use the
131 [dirs]
132 flag.
133
134.. _migration-2.2-runqemu-ported-to-python:
135
136``runqemu`` Ported to Python
137----------------------------
138
139``runqemu`` has been ported to Python and has changed behavior in some
140cases. Previous usage patterns continue to be supported.
141
142The new ``runqemu`` is a Python script. Machine knowledge is no longer
143hardcoded into ``runqemu``. You can choose to use the ``qemuboot``
144configuration file to define the BSP's own arguments and to make it
145bootable with ``runqemu``. If you use a configuration file, use the
146following form:
147::
148
149 image-name-machine.qemuboot.conf
150
151The configuration file
152enables fine-grained tuning of options passed to QEMU without the
153``runqemu`` script hard-coding any knowledge about different machines.
154Using a configuration file is particularly convenient when trying to use
155QEMU with machines other than the ``qemu*`` machines in
156:term:`OpenEmbedded-Core (OE-Core)`. The ``qemuboot.conf`` file is generated by the
157``qemuboot`` class when the root filesystem is being build (i.e. build
158rootfs). QEMU boot arguments can be set in BSP's configuration file and
159the ``qemuboot`` class will save them to ``qemuboot.conf``.
160
161If you want to use ``runqemu`` without a configuration file, use the
162following command form:
163::
164
165 $ runqemu machine rootfs kernel [options]
166
167Supported machines are as follows:
168
169 - qemuarm
170 - qemuarm64
171 - qemux86
172 - qemux86-64
173 - qemuppc
174 - qemumips
175 - qemumips64
176 - qemumipsel
177 - qemumips64el
178
179Consider the
180following example, which uses the ``qemux86-64`` machine, provides a
181root filesystem, provides an image, and uses the ``nographic`` option: ::
182
183 $ runqemu qemux86-64 tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4 tmp/deploy/images/qemux86-64/bzImage nographic
184
185Following is a list of variables that can be set in configuration files
186such as ``bsp.conf`` to enable the BSP to be booted by ``runqemu``:
187
188.. note::
189
190 "QB" means "QEMU Boot".
191
192::
193
194 QB_SYSTEM_NAME: QEMU name (e.g. "qemu-system-i386")
195 QB_OPT_APPEND: Options to append to QEMU (e.g. "-show-cursor")
196 QB_DEFAULT_KERNEL: Default kernel to boot (e.g. "bzImage")
197 QB_DEFAULT_FSTYPE: Default FSTYPE to boot (e.g. "ext4")
198 QB_MEM: Memory (e.g. "-m 512")
199 QB_MACHINE: QEMU machine (e.g. "-machine virt")
200 QB_CPU: QEMU cpu (e.g. "-cpu qemu32")
201 QB_CPU_KVM: Similar to QB_CPU except used for kvm support (e.g. "-cpu kvm64")
202 QB_KERNEL_CMDLINE_APPEND: Options to append to the kernel's -append
203 option (e.g. "console=ttyS0 console=tty")
204 QB_DTB: QEMU dtb name
205 QB_AUDIO_DRV: QEMU audio driver (e.g. "alsa", set it when support audio)
206 QB_AUDIO_OPT: QEMU audio option (e.g. "-soundhw ac97,es1370"), which is used
207 when QB_AUDIO_DRV is set.
208 QB_KERNEL_ROOT: Kernel's root (e.g. /dev/vda)
209 QB_TAP_OPT: Network option for 'tap' mode (e.g.
210 "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device virtio-net-device,netdev=net0").
211 runqemu will replace "@TAP@" with the one that is used, such as tap0, tap1 ...
212 QB_SLIRP_OPT: Network option for SLIRP mode (e.g. "-netdev user,id=net0 -device virtio-net-device,netdev=net0")
213 QB_ROOTFS_OPT: Used as rootfs (e.g.
214 "-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device virtio-blk-device,drive=disk0").
215 runqemu will replace "@ROOTFS@" with the one which is used, such as
216 core-image-minimal-qemuarm64.ext4.
217 QB_SERIAL_OPT: Serial port (e.g. "-serial mon:stdio")
218 QB_TCPSERIAL_OPT: tcp serial port option (e.g.
219 " -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device virtconsole,chardev=virtcon"
220 runqemu will replace "@PORT@" with the port number which is used.
221
222To use ``runqemu``, set :term:`IMAGE_CLASSES` as
223follows and run ``runqemu``:
224
225.. note::
226
227 For command-line syntax, use
228 runqemu help
229 .
230
231::
232
233 IMAGE_CLASSES += "qemuboot"
234
235.. _migration-2.2-default-linker-hash-style-changed:
236
237Default Linker Hash Style Changed
238---------------------------------
239
240The default linker hash style for ``gcc-cross`` is now "sysv" in order
241to catch recipes that are building software without using the
242OpenEmbedded :term:`LDFLAGS`. This change could result in
243seeing some "No GNU_HASH in the elf binary" QA issues when building such
244recipes. You need to fix these recipes so that they use the expected
245``LDFLAGS``. Depending on how the software is built, the build system
246used by the software (e.g. a Makefile) might need to be patched.
247However, sometimes making this fix is as simple as adding the following
248to the recipe:
249::
250
251 TARGET_CC_ARCH += "${LDFLAGS}"
252
253.. _migration-2.2-kernel-image-base-name-no-longer-uses-kernel-imagetype:
254
255``KERNEL_IMAGE_BASE_NAME`` no Longer Uses ``KERNEL_IMAGETYPE``
256--------------------------------------------------------------
257
258The ``KERNEL_IMAGE_BASE_NAME`` variable no longer uses the
259:term:`KERNEL_IMAGETYPE` variable to create the
260image's base name. Because the OpenEmbedded build system can now build
261multiple kernel image types, this part of the kernel image base name as
262been removed leaving only the following:
263::
264
265 KERNEL_IMAGE_BASE_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}-${DATETIME}"
266
267If you have recipes or
268classes that use ``KERNEL_IMAGE_BASE_NAME`` directly, you might need to
269update the references to ensure they continue to work.
270
271.. _migration-2.2-bitbake-changes:
272
273BitBake Changes
274---------------
275
276The following changes took place for BitBake:
277
278- The "goggle" UI and standalone image-writer tool have been removed as
279 they both require GTK+ 2.0 and were not being maintained.
280
281- The Perforce fetcher now supports :term:`SRCREV` for
282 specifying the source revision to use, be it
283 ``${``\ :term:`AUTOREV`\ ``}``, changelist number,
284 p4date, or label, in preference to separate
285 :term:`SRC_URI` parameters to specify these. This
286 change is more in-line with how the other fetchers work for source
287 control systems. Recipes that fetch from Perforce will need to be
288 updated to use ``SRCREV`` in place of specifying the source revision
289 within ``SRC_URI``.
290
291- Some of BitBake's internal code structures for accessing the recipe
292 cache needed to be changed to support the new multi-configuration
293 functionality. These changes will affect external tools that use
294 BitBake's tinfoil module. For information on these changes, see the
295 changes made to the scripts supplied with OpenEmbedded-Core:
296 `1 <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=189371f8393971d00bca0fceffd67cc07784f6ee>`__
297 and
298 `2 <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=4a5aa7ea4d07c2c90a1654b174873abb018acc67>`__.
299
300- The task management code has been rewritten to avoid using ID
301 indirection in order to improve performance. This change is unlikely
302 to cause any problems for most users. However, the setscene
303 verification function as pointed to by
304 ``BB_SETSCENE_VERIFY_FUNCTION`` needed to change signature.
305 Consequently, a new variable named ``BB_SETSCENE_VERIFY_FUNCTION2``
306 has been added allowing multiple versions of BitBake to work with
307 suitably written metadata, which includes OpenEmbedded-Core and Poky.
308 Anyone with custom BitBake task scheduler code might also need to
309 update the code to handle the new structure.
310
311.. _migration-2.2-swabber-has-been-removed:
312
313Swabber has Been Removed
314------------------------
315
316Swabber, a tool that was intended to detect host contamination in the
317build process, has been removed, as it has been unmaintained and unused
318for some time and was never particularly effective. The OpenEmbedded
319build system has since incorporated a number of mechanisms including
320enhanced QA checks that mean that there is less of a need for such a
321tool.
322
323.. _migration-2.2-removed-recipes:
324
325Removed Recipes
326---------------
327
328The following recipes have been removed:
329
330- ``augeas``: No longer needed and has been moved to ``meta-oe``.
331
332- ``directfb``: Unmaintained and has been moved to ``meta-oe``.
333
334- ``gcc``: Removed 4.9 version. Versions 5.4 and 6.2 are still present.
335
336- ``gnome-doc-utils``: No longer needed.
337
338- ``gtk-doc-stub``: Replaced by ``gtk-doc``.
339
340- ``gtk-engines``: No longer needed and has been moved to
341 ``meta-gnome``.
342
343- ``gtk-sato-engine``: Became obsolete.
344
345- ``libglade``: No longer needed and has been moved to ``meta-oe``.
346
347- ``libmad``: Unmaintained and functionally replaced by ``libmpg123``.
348 ``libmad`` has been moved to ``meta-oe``.
349
350- ``libowl``: Became obsolete.
351
352- ``libxsettings-client``: No longer needed.
353
354- ``oh-puzzles``: Functionally replaced by ``puzzles``.
355
356- ``oprofileui``: Became obsolete. OProfile has been largely supplanted
357 by perf.
358
359- ``packagegroup-core-directfb.bb``: Removed.
360
361- ``core-image-directfb.bb``: Removed.
362
363- ``pointercal``: No longer needed and has been moved to ``meta-oe``.
364
365- ``python-imaging``: No longer needed and moved to ``meta-python``
366
367- ``python-pyrex``: No longer needed and moved to ``meta-python``.
368
369- ``sato-icon-theme``: Became obsolete.
370
371- ``swabber-native``: Swabber has been removed. See the `entry on
372 Swabber <#migration-2.2-swabber-has-been-removed>`__.
373
374- ``tslib``: No longer needed and has been moved to ``meta-oe``.
375
376- ``uclibc``: Removed in favor of musl.
377
378- ``xtscal``: No longer needed and moved to ``meta-oe``
379
380.. _migration-2.2-removed-classes:
381
382Removed Classes
383---------------
384
385The following classes have been removed:
386
387- ``distutils-native-base``: No longer needed.
388
389- ``distutils3-native-base``: No longer needed.
390
391- ``sdl``: Only set :term:`DEPENDS` and
392 :term:`SECTION`, which are better set within the
393 recipe instead.
394
395- ``sip``: Mostly unused.
396
397- ``swabber``: See the `entry on
398 Swabber <#migration-2.2-swabber-has-been-removed>`__.
399
400.. _migration-2.2-minor-packaging-changes:
401
402Minor Packaging Changes
403-----------------------
404
405The following minor packaging changes have occurred:
406
407- ``grub``: Split ``grub-editenv`` into its own package.
408
409- ``systemd``: Split container and vm related units into a new package,
410 systemd-container.
411
412- ``util-linux``: Moved ``prlimit`` to a separate
413 ``util-linux-prlimit`` package.
414
415.. _migration-2.2-miscellaneous-changes:
416
417Miscellaneous Changes
418---------------------
419
420The following miscellaneous changes have occurred:
421
422- ``package_regex.inc``: Removed because the definitions
423 ``package_regex.inc`` previously contained have been moved to their
424 respective recipes.
425
426- Both ``devtool add`` and ``recipetool create`` now use a fixed
427 :term:`SRCREV` by default when fetching from a Git
428 repository. You can override this in either case to use
429 ``${``\ :term:`AUTOREV`\ ``}`` instead by using the
430 ``-a`` or ``DASHDASHautorev`` command-line option
431
432- ``distcc``: GTK+ UI is now disabled by default.
433
434- ``packagegroup-core-tools-testapps``: Removed Piglit.
435
436- ``image.bbclass``: Renamed COMPRESS(ION) to CONVERSION. This change
437 means that ``COMPRESSIONTYPES``, ``COMPRESS_DEPENDS`` and
438 ``COMPRESS_CMD`` are deprecated in favor of ``CONVERSIONTYPES``,
439 ``CONVERSION_DEPENDS`` and ``CONVERSION_CMD``. The ``COMPRESS*``
440 variable names will still work in the 2.2 release but metadata that
441 does not need to be backwards-compatible should be changed to use the
442 new names as the ``COMPRESS*`` ones will be removed in a future
443 release.
444
445- ``gtk-doc``: A full version of ``gtk-doc`` is now made available.
446 However, some old software might not be capable of using the current
447 version of ``gtk-doc`` to build documentation. You need to change
448 recipes that build such software so that they explicitly disable
449 building documentation with ``gtk-doc``.
450
451
diff --git a/documentation/ref-manual/migration-2.3.rst b/documentation/ref-manual/migration-2.3.rst
new file mode 100644
index 0000000000..7f34f0cd75
--- /dev/null
+++ b/documentation/ref-manual/migration-2.3.rst
@@ -0,0 +1,530 @@
1Moving to the Yocto Project 2.3 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.3 Release from the prior release.
6
7.. _migration-2.3-recipe-specific-sysroots:
8
9Recipe-specific Sysroots
10------------------------
11
12The OpenEmbedded build system now uses one sysroot per recipe to resolve
13long-standing issues with configuration script auto-detection of
14undeclared dependencies. Consequently, you might find that some of your
15previously written custom recipes are missing declared dependencies,
16particularly those dependencies that are incidentally built earlier in a
17typical build process and thus are already likely to be present in the
18shared sysroot in previous releases.
19
20Consider the following:
21
22- *Declare Build-Time Dependencies:* Because of this new feature, you
23 must explicitly declare all build-time dependencies for your recipe.
24 If you do not declare these dependencies, they are not populated into
25 the sysroot for the recipe.
26
27- *Specify Pre-Installation and Post-Installation Native Tool
28 Dependencies:* You must specifically specify any special native tool
29 dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
30 the :term:`PACKAGE_WRITE_DEPS` variable.
31 Specifying these dependencies ensures that these tools are available
32 if these scripts need to be run on the build host during the
33 :ref:`ref-tasks-rootfs` task.
34
35 As an example, see the ``dbus`` recipe. You will see that this recipe
36 has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
37 :term:`DISTRO_FEATURES`. In the example,
38 ``systemd-systemctl-native`` is added to ``PACKAGE_WRITE_DEPS``,
39 which is also conditional on "systemd" being in ``DISTRO_FEATURES``.
40
41- Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``: You need to
42 examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
43 steps to take.
44
45 Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
46 were in previous Yocto Project releases. However, since a separate
47 sysroot is now being populated for every recipe and if existing
48 functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
49 relocation, then you will need to change these to use a
50 post-installation script that is installed by a function added to
51 :term:`SYSROOT_PREPROCESS_FUNCS`.
52
53 For an example, see the ``pixbufcache`` class in ``meta/classes/`` in
54 the :ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`.
55
56 .. note::
57
58 The
59 SSTATEPOSTINSTFUNCS
60 variable itself is now deprecated in favor of the
61 do_populate_sysroot[postfuncs]
62 task. Consequently, if you do still have any function or functions
63 that need to be called after the sysroot component is created for
64 a recipe, then you would be well advised to take steps to use a
65 post installation script as described previously. Taking these
66 steps prepares your code for when
67 SSTATEPOSTINSTFUNCS
68 is removed in a future Yocto Project release.
69
70- *Specify the Sysroot when Using Certain External Scripts:* Because
71 the shared sysroot is now gone, the scripts
72 ``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
73 such that you need to specify which recipe's
74 :term:`STAGING_DIR_NATIVE` is used.
75
76.. note::
77
78 You can find more information on how recipe-specific sysroots work in
79 the "
80 staging.bbclass
81 " section.
82
83.. _migration-2.3-path-variable:
84
85``PATH`` Variable
86-----------------
87
88Within the environment used to run build tasks, the environment variable
89``PATH`` is now sanitized such that the normal native binary paths
90(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
91directory containing symbolic links linking only to the binaries from
92the host mentioned in the :term:`HOSTTOOLS` and
93:term:`HOSTTOOLS_NONFATAL` variables is added
94to ``PATH``.
95
96Consequently, any native binaries provided by the host that you need to
97call needs to be in one of these two variables at the configuration
98level.
99
100Alternatively, you can add a native recipe (i.e. ``-native``) that
101provides the binary to the recipe's :term:`DEPENDS`
102value.
103
104.. note::
105
106 PATH
107 is not sanitized in the same way within
108 devshell
109 . If it were, you would have difficulty running host tools for
110 development and debugging within the shell.
111
112.. _migration-2.3-scripts:
113
114Changes to Scripts
115------------------
116
117The following changes to scripts took place:
118
119- ``oe-find-native-sysroot``: The usage for the
120 ``oe-find-native-sysroot`` script has changed to the following:
121 ::
122
123 $ . oe-find-native-sysroot recipe
124
125 You must now supply a recipe for recipe
126 as part of the command. Prior to the Yocto Project &DISTRO; release, it
127 was not necessary to provide the script with the command.
128
129- ``oe-run-native``: The usage for the ``oe-run-native`` script has
130 changed to the following:
131 ::
132
133 $ oe-run-native native_recipe tool
134
135 You must
136 supply the name of the native recipe and the tool you want to run as
137 part of the command. Prior to the Yocto Project DISTRO release, it
138 was not necessary to provide the native recipe with the command.
139
140- ``cleanup-workdir``: The ``cleanup-workdir`` script has been
141 removed because the script was found to be deleting files it should
142 not have, which lead to broken build trees. Rather than trying to
143 delete portions of :term:`TMPDIR` and getting it wrong,
144 it is recommended that you delete ``TMPDIR`` and have it restored
145 from shared state (sstate) on subsequent builds.
146
147- ``wipe-sysroot``: The ``wipe-sysroot`` script has been removed as
148 it is no longer needed with recipe-specific sysroots.
149
150.. _migration-2.3-functions:
151
152Changes to Functions
153--------------------
154
155The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
156and related functions have been removed in favor of ``d.getVar()``,
157``d.setVar()``, and so forth.
158
159You need to fix any references to these old functions.
160
161.. _migration-2.3-bitbake-changes:
162
163BitBake Changes
164---------------
165
166The following changes took place for BitBake:
167
168- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
169 graphical dependency explorer UI ``depexp`` was replaced by
170 ``taskexp`` ("Task Explorer"), which provides a graphical way of
171 exploring the ``task-depends.dot`` file. The data presented by Task
172 Explorer is much more accurate than the data that was presented by
173 ``depexp``. Being able to visualize the data is an often requested
174 feature as standard ``*.dot`` file viewers cannot usual cope with the
175 size of the ``task-depends.dot`` file.
176
177- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
178 ``pn-depends.dot`` files as previously generated using the
179 ``bitbake -g`` command have been removed. A ``recipe-depends.dot``
180 file is now generated as a collapsed version of ``task-depends.dot``
181 instead.
182
183 The reason for this change is because ``package-depends.dot`` and
184 ``pn-depends.dot`` largely date back to a time before task-based
185 execution and do not take into account task-level dependencies
186 between recipes, which could be misleading.
187
188- *Mirror Variable Splitting Changes:* Mirror variables including
189 :term:`MIRRORS`, :term:`PREMIRRORS`,
190 and :term:`SSTATE_MIRRORS` can now separate
191 values entirely with spaces. Consequently, you no longer need "\\n".
192 BitBake looks for pairs of values, which simplifies usage. There
193 should be no change required to existing mirror variable values
194 themselves.
195
196- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
197 "rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
198 instead of an "rsh" parameter. This new optional parameter is used
199 when the "protocol" parameter is set to "svn+ssh". You can only use
200 the new parameter to specify the ``ssh`` program used by SVN. The SVN
201 fetcher passes the new parameter through the ``SVN_SSH`` environment
202 variable during the :ref:`ref-tasks-fetch` task.
203
204 See the ":ref:`bitbake:svn-fetcher`"
205 section in the BitBake
206 User Manual for additional information.
207
208- ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
209 Removed: Because the mechanism they were part of is no longer
210 necessary with recipe-specific sysroots, the
211 ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
212 variables have been removed.
213
214.. _migration-2.3-absolute-symlinks:
215
216Absolute Symbolic Links
217-----------------------
218
219Absolute symbolic links (symlinks) within staged files are no longer
220permitted and now trigger an error. Any explicit creation of symlinks
221can use the ``lnr`` script, which is a replacement for ``ln -r``.
222
223If the build scripts in the software that the recipe is building are
224creating a number of absolute symlinks that need to be corrected, you
225can inherit ``relative_symlinks`` within the recipe to turn those
226absolute symlinks into relative symlinks.
227
228.. _migration-2.3-gplv2-and-gplv3-moves:
229
230GPLv2 Versions of GPLv3 Recipes Moved
231-------------------------------------
232
233Older GPLv2 versions of GPLv3 recipes have moved to a separate
234``meta-gplv2`` layer.
235
236If you use :term:`INCOMPATIBLE_LICENSE` to
237exclude GPLv3 or set :term:`PREFERRED_VERSION`
238to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
239``meta-gplv2`` layer to your configuration.
240
241.. note::
242
243 You can find
244 meta-gplv2
245 layer in the OpenEmbedded layer index at
246 .
247
248These relocated GPLv2 recipes do not receive the same level of
249maintenance as other core recipes. The recipes do not get security fixes
250and upstream no longer maintains them. In fact, the upstream community
251is actively hostile towards people that use the old versions of the
252recipes. Moving these recipes into a separate layer both makes the
253different needs of the recipes clearer and clearly identifies the number
254of these recipes.
255
256.. note::
257
258 The long-term solution might be to move to BSD-licensed replacements
259 of the GPLv3 components for those that need to exclude GPLv3-licensed
260 components from the target system. This solution will be investigated
261 for future Yocto Project releases.
262
263.. _migration-2.3-package-management-changes:
264
265Package Management Changes
266--------------------------
267
268The following package management changes took place:
269
270- Smart package manager is replaced by DNF package manager. Smart has
271 become unmaintained upstream, is not ported to Python 3.x.
272 Consequently, Smart needed to be replaced. DNF is the only feasible
273 candidate.
274
275 The change in functionality is that the on-target runtime package
276 management from remote package feeds is now done with a different
277 tool that has a different set of command-line options. If you have
278 scripts that call the tool directly, or use its API, they need to be
279 fixed.
280
281 For more information, see the `DNF
282 Documentation <http://dnf.readthedocs.io/en/latest/>`__.
283
284- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
285
286 - DNF is API-incompatible with Rpm 5.x and porting it and
287 maintaining the port is non-trivial.
288
289 - Rpm 5.x itself has limited maintenance upstream, and the Yocto
290 Project is one of the very few remaining users.
291
292- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
293
294 - Version 6.x of Berkeley DB has largely been rejected by the open
295 source community due to its AGPLv3 license. As a result, most
296 mainstream open source projects that require DB are still
297 developed and tested with DB 5.x.
298
299 - In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
300 Thus, no reason exists to continue carrying DB 6.x in OE-core.
301
302- ``createrepo`` is replaced with ``createrepo_c``.
303
304 ``createrepo_c`` is the current incarnation of the tool that
305 generates remote repository metadata. It is written in C as compared
306 to ``createrepo``, which is written in Python. ``createrepo_c`` is
307 faster and is maintained.
308
309- Architecture-independent RPM packages are "noarch" instead of "all".
310
311 This change was made because too many places in DNF/RPM4 stack
312 already make that assumption. Only the filenames and the architecture
313 tag has changed. Nothing else has changed in OE-core system,
314 particularly in the :ref:`allarch.bbclass <ref-classes-allarch>`
315 class.
316
317- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
318 currently supported. This issue will be fully addressed in a future
319 Yocto Project release. See `defect
320 11209 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11209>`__
321 for more information on a solution to package feed signing with RPM
322 in the Yocto Project 2.3 release.
323
324- OPKG now uses the libsolv backend for resolving package dependencies
325 by default. This is vastly superior to OPKG's internal ad-hoc solver
326 that was previously used. This change does have a small impact on
327 disk (around 500 KB) and memory footprint.
328
329 .. note::
330
331 For further details on this change, see the
332 commit message
333 .
334
335.. _migration-2.3-removed-recipes:
336
337Removed Recipes
338---------------
339
340The following recipes have been removed:
341
342- ``linux-yocto 4.8``: Version 4.8 has been removed. Versions 4.1
343 (LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
344
345- ``python-smartpm``: Functionally replaced by ``dnf``.
346
347- ``createrepo``: Replaced by the ``createrepo-c`` recipe.
348
349- ``rpmresolve``: No longer needed with the move to RPM 4 as RPM
350 itself is used instead.
351
352- ``gstreamer``: Removed the GStreamer Git version recipes as they
353 have been stale. ``1.10.``\ x recipes are still present.
354
355- ``alsa-conf-base``: Merged into ``alsa-conf`` since ``libasound``
356 depended on both. Essentially, no way existed to install only one of
357 these.
358
359- ``tremor``: Moved to ``meta-multimedia``. Fixed-integer Vorbis
360 decoding is not needed by current hardware. Thus, GStreamer's ivorbis
361 plugin has been disabled by default eliminating the need for the
362 ``tremor`` recipe in :term:`OpenEmbedded-Core (OE-Core)`.
363
364- ``gummiboot``: Replaced by ``systemd-boot``.
365
366.. _migration-2.3-wic-changes:
367
368Wic Changes
369-----------
370
371The following changes have been made to Wic:
372
373.. note::
374
375 For more information on Wic, see the "
376 Creating Partitioned Images Using Wic
377 " section in the Yocto Project Development Tasks Manual.
378
379- *Default Output Directory Changed:* Wic's default output directory is
380 now the current directory by default instead of the unusual
381 ``/var/tmp/wic``.
382
383 The "-o" and "--outdir" options remain unchanged and are used to
384 specify your preferred output directory if you do not want to use the
385 default directory.
386
387- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
388 it duplicates functionality of the rawcopy plugin.
389
390.. _migration-2.3-qa-changes:
391
392QA Changes
393----------
394
395The following QA checks have changed:
396
397- ``unsafe-references-in-binaries``: The
398 ``unsafe-references-in-binaries`` QA check, which was disabled by
399 default, has now been removed. This check was intended to detect
400 binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
401 the case where the user has ``/usr`` on a separate filesystem to
402 ``/``.
403
404 The removed QA check was buggy. Additionally, ``/usr`` residing on a
405 separate partition from ``/`` is now a rare configuration.
406 Consequently, ``unsafe-references-in-binaries`` was removed.
407
408- ``file-rdeps``: The ``file-rdeps`` QA check is now an error by
409 default instead of a warning. Because it is an error instead of a
410 warning, you need to address missing runtime dependencies.
411
412 For additional information, see the
413 :ref:`insane <ref-classes-insane>` class and the "`Errors and
414 Warnings <#qa-errors-and-warnings>`__" section.
415
416.. _migration-2.3-miscellaneous-changes:
417
418Miscellaneous Changes
419---------------------
420
421The following miscellaneous changes have occurred:
422
423- In this release, a number of recipes have been changed to ignore the
424 ``largefile`` :term:`DISTRO_FEATURES` item,
425 enabling large file support unconditionally. This feature has always
426 been enabled by default. Disabling the feature has not been widely
427 tested.
428
429 .. note::
430
431 Future releases of the Yocto Project will remove entirely the
432 ability to disable the
433 largefile
434 feature, which would make it unconditionally enabled everywhere.
435
436- If the :term:`DISTRO_VERSION` value contains
437 the value of the :term:`DATE` variable, which is the
438 default between Poky releases, the ``DATE`` value is explicitly
439 excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
440 displayed at the login prompt, in order to avoid conflicts with
441 Multilib enabled. Regardless, the ``DATE`` value is inaccurate if the
442 ``base-files`` recipe is restored from shared state (sstate) rather
443 than rebuilt.
444
445 If you need the build date recorded in ``/etc/issue*`` or anywhere
446 else in your image, a better method is to define a post-processing
447 function to do it and have the function called from
448 :term:`ROOTFS_POSTPROCESS_COMMAND`.
449 Doing so ensures the value is always up-to-date with the created
450 image.
451
452- Dropbear's ``init`` script now disables DSA host keys by default.
453 This change is in line with the systemd service file, which supports
454 RSA keys only, and with recent versions of OpenSSH, which deprecates
455 DSA host keys.
456
457- The :ref:`buildhistory <ref-classes-buildhistory>` class now
458 correctly uses tabs as separators between all columns in
459 ``installed-package-sizes.txt`` in order to aid import into other
460 tools.
461
462- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
463 ``DISTRO_FEATURES`` feature. Distributions that previously set:
464 ::
465
466 USE_LDCONFIG = "0"
467
468 should now instead use the following:
469
470 ::
471
472 DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
473
474- The default value of
475 :term:`COPYLEFT_LICENSE_INCLUDE` now
476 includes all versions of AGPL licenses in addition to GPL and LGPL.
477
478 .. note::
479
480 The default list is not intended to be guaranteed as a complete
481 safe list. You should seek legal advice based on what you are
482 distributing if you are unsure.
483
484- Kernel module packages are now suffixed with the kernel version in
485 order to allow module packages from multiple kernel versions to
486 co-exist on a target system. If you wish to return to the previous
487 naming scheme that does not include the version suffix, use the
488 following:
489 ::
490
491 KERNEL_MODULE_PACKAGE_SUFFIX to ""
492
493- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
494 ``*.la`` files are not actually needed on Linux and relocating them
495 is an unnecessary burden.
496
497 If you need to preserve these ``.la`` files (e.g. in a custom
498 distribution), you must change
499 :term:`INHERIT_DISTRO` such that
500 "remove-libtool" is not included in the value.
501
502- Extensible SDKs built for GCC 5+ now refuse to install on a
503 distribution where the host GCC version is 4.8 or 4.9. This change
504 resulted from the fact that the installation is known to fail due to
505 the way the ``uninative`` shared state (sstate) package is built. See
506 the :ref:`uninative <ref-classes-uninative>` class for additional
507 information.
508
509- All native and nativesdk recipes now use a separate
510 ``DISTRO_FEATURES`` value instead of sharing the value used by
511 recipes for the target, in order to avoid unnecessary rebuilds.
512
513 The ``DISTRO_FEATURES`` for ``native`` recipes is
514 :term:`DISTRO_FEATURES_NATIVE` added to
515 an intersection of ``DISTRO_FEATURES`` and
516 :term:`DISTRO_FEATURES_FILTER_NATIVE`.
517
518 For nativesdk recipes, the corresponding variables are
519 :term:`DISTRO_FEATURES_NATIVESDK`
520 and
521 :term:`DISTRO_FEATURES_FILTER_NATIVESDK`.
522
523- The ``FILESDIR`` variable, which was previously deprecated and rarely
524 used, has now been removed. You should change any recipes that set
525 ``FILESDIR`` to set :term:`FILESPATH` instead.
526
527- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
528 longer needed with recipe-specific sysroots.
529
530
diff --git a/documentation/ref-manual/migration-2.4.rst b/documentation/ref-manual/migration-2.4.rst
new file mode 100644
index 0000000000..260b3204b6
--- /dev/null
+++ b/documentation/ref-manual/migration-2.4.rst
@@ -0,0 +1,327 @@
1Moving to the Yocto Project 2.4 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.4 Release from the prior release.
6
7.. _migration-2.4-memory-resident-mode:
8
9Memory Resident Mode
10--------------------
11
12A persistent mode is now available in BitBake's default operation,
13replacing its previous "memory resident mode" (i.e.
14``oe-init-build-env-memres``). Now you only need to set
15:term:`BB_SERVER_TIMEOUT` to a timeout (in
16seconds) and BitBake's server stays resident for that amount of time
17between invocations. The ``oe-init-build-env-memres`` script has been
18removed since a separate environment setup script is no longer needed.
19
20.. _migration-2.4-packaging-changes:
21
22Packaging Changes
23-----------------
24
25This section provides information about packaging changes that have
26occurred:
27
28- ``python3`` Changes:
29
30 - The main "python3" package now brings in all of the standard
31 Python 3 distribution rather than a subset. This behavior matches
32 what is expected based on traditional Linux distributions. If you
33 wish to install a subset of Python 3, specify ``python-core`` plus
34 one or more of the individual packages that are still produced.
35
36 - ``python3``: The ``bz2.py``, ``lzma.py``, and
37 ``_compression.py`` scripts have been moved from the
38 ``python3-misc`` package to the ``python3-compression`` package.
39
40- ``binutils``: The ``libbfd`` library is now packaged in a separate
41 "libbfd" package. This packaging saves space when certain tools (e.g.
42 ``perf``) are installed. In such cases, the tools only need
43 ``libbfd`` rather than all the packages in ``binutils``.
44
45- ``util-linux`` Changes:
46
47 - The ``su`` program is now packaged in a separate "util-linux-su"
48 package, which is only built when "pam" is listed in the
49 :term:`DISTRO_FEATURES` variable.
50 ``util-linux`` should not be installed unless it is needed because
51 ``su`` is normally provided through the shadow file format. The
52 main ``util-linux`` package has runtime dependencies (i.e.
53 :term:`RDEPENDS`) on the ``util-linux-su`` package
54 when "pam" is in ``DISTRO_FEATURES``.
55
56 - The ``switch_root`` program is now packaged in a separate
57 "util-linux-switch-root" package for small initramfs images that
58 do not need the whole ``util-linux`` package or the busybox
59 binary, which are both much larger than ``switch_root``. The main
60 ``util-linux`` package has a recommended runtime dependency (i.e.
61 :term:`RRECOMMENDS`) on the
62 ``util-linux-switch-root`` package.
63
64 - The ``ionice`` program is now packaged in a separate
65 "util-linux-ionice" package. The main ``util-linux`` package has a
66 recommended runtime dependency (i.e. ``RRECOMMENDS``) on the
67 ``util-linux-ionice`` package.
68
69- ``initscripts``: The ``sushell`` program is now packaged in a
70 separate "initscripts-sushell" package. This packaging change allows
71 systems to pull ``sushell`` in when ``selinux`` is enabled. The
72 change also eliminates needing to pull in the entire ``initscripts``
73 package. The main ``initscripts`` package has a runtime dependency
74 (i.e. ``RDEPENDS``) on the ``sushell`` package when "selinux" is in
75 ``DISTRO_FEATURES``.
76
77- ``glib-2.0``: The ``glib-2.0`` package now has a recommended
78 runtime dependency (i.e. ``RRECOMMENDS``) on the ``shared-mime-info``
79 package, since large portions of GIO are not useful without the MIME
80 database. You can remove the dependency by using the
81 :term:`BAD_RECOMMENDATIONS` variable if
82 ``shared-mime-info`` is too large and is not required.
83
84- *Go Standard Runtime:* The Go standard runtime has been split out
85 from the main ``go`` recipe into a separate ``go-runtime`` recipe.
86
87.. _migration-2.4-removed-recipes:
88
89Removed Recipes
90---------------
91
92The following recipes have been removed:
93
94- ``acpitests``: This recipe is not maintained.
95
96- ``autogen-native``: No longer required by Grub, oe-core, or
97 meta-oe.
98
99- ``bdwgc``: Nothing in OpenEmbedded-Core requires this recipe. It
100 has moved to meta-oe.
101
102- ``byacc``: This recipe was only needed by rpm 5.x and has moved to
103 meta-oe.
104
105- ``gcc (5.4)``: The 5.4 series dropped the recipe in favor of 6.3 /
106 7.2.
107
108- ``gnome-common``: Deprecated upstream and no longer needed.
109
110- ``go-bootstrap-native``: Go 1.9 does its own bootstrapping so this
111 recipe has been removed.
112
113- ``guile``: This recipe was only needed by ``autogen-native`` and
114 ``remake``. The recipe is no longer needed by either of these
115 programs.
116
117- ``libclass-isa-perl``: This recipe was previously needed for LSB 4,
118 no longer needed.
119
120- ``libdumpvalue-perl``: This recipe was previously needed for LSB 4,
121 no longer needed.
122
123- ``libenv-perl``: This recipe was previously needed for LSB 4, no
124 longer needed.
125
126- ``libfile-checktree-perl``: This recipe was previously needed for
127 LSB 4, no longer needed.
128
129- ``libi18n-collate-perl``: This recipe was previously needed for LSB
130 4, no longer needed.
131
132- ``libiconv``: This recipe was only needed for ``uclibc``, which was
133 removed in the previous release. ``glibc`` and ``musl`` have their
134 own implementations. ``meta-mingw`` still needs ``libiconv``, so it
135 has been moved to ``meta-mingw``.
136
137- ``libpng12``: This recipe was previously needed for LSB. The
138 current ``libpng`` is 1.6.x.
139
140- ``libpod-plainer-perl``: This recipe was previously needed for LSB
141 4, no longer needed.
142
143- ``linux-yocto (4.1)``: This recipe was removed in favor of 4.4,
144 4.9, 4.10 and 4.12.
145
146- ``mailx``: This recipe was previously only needed for LSB
147 compatibility, and upstream is defunct.
148
149- ``mesa (git version only)``: The git version recipe was stale with
150 respect to the release version.
151
152- ``ofono (git version only)``: The git version recipe was stale with
153 respect to the release version.
154
155- ``portmap``: This recipe is obsolete and is superseded by
156 ``rpcbind``.
157
158- ``python3-pygpgme``: This recipe is old and unmaintained. It was
159 previously required by ``dnf``, which has switched to official
160 ``gpgme`` Python bindings.
161
162- ``python-async``: This recipe has been removed in favor of the
163 Python 3 version.
164
165- ``python-gitdb``: This recipe has been removed in favor of the
166 Python 3 version.
167
168- ``python-git``: This recipe was removed in favor of the Python 3
169 version.
170
171- ``python-mako``: This recipe was removed in favor of the Python 3
172 version.
173
174- ``python-pexpect``: This recipe was removed in favor of the Python
175 3 version.
176
177- ``python-ptyprocess``: This recipe was removed in favor of Python
178 the 3 version.
179
180- ``python-pycurl``: Nothing is using this recipe in
181 OpenEmbedded-Core (i.e. ``meta-oe``).
182
183- ``python-six``: This recipe was removed in favor of the Python 3
184 version.
185
186- ``python-smmap``: This recipe was removed in favor of the Python 3
187 version.
188
189- ``remake``: Using ``remake`` as the provider of ``virtual/make`` is
190 broken. Consequently, this recipe is not needed in OpenEmbedded-Core.
191
192.. _migration-2.4-kernel-device-tree-move:
193
194Kernel Device Tree Move
195-----------------------
196
197Kernel Device Tree support is now easier to enable in a kernel recipe.
198The Device Tree code has moved to a
199:ref:`kernel-devicetree <ref-classes-kernel-devicetree>` class.
200Functionality is automatically enabled for any recipe that inherits the
201:ref:`kernel <ref-classes-kernel>` class and sets the
202:term:`KERNEL_DEVICETREE` variable. The
203previous mechanism for doing this,
204``meta/recipes-kernel/linux/linux-dtb.inc``, is still available to avoid
205breakage, but triggers a deprecation warning. Future releases of the
206Yocto Project will remove ``meta/recipes-kernel/linux/linux-dtb.inc``.
207It is advisable to remove any ``require`` statements that request
208``meta/recipes-kernel/linux/linux-dtb.inc`` from any custom kernel
209recipes you might have. This will avoid breakage in post 2.4 releases.
210
211.. _migration-2.4-package-qa-changes:
212
213Package QA Changes
214------------------
215
216The following package QA changes took place:
217
218- The "unsafe-references-in-scripts" QA check has been removed.
219
220- If you refer to ``${COREBASE}/LICENSE`` within
221 :term:`LIC_FILES_CHKSUM` you receive a
222 warning because this file is a description of the license for
223 OE-Core. Use ``${COMMON_LICENSE_DIR}/MIT`` if your recipe is
224 MIT-licensed and you cannot use the preferred method of referring to
225 a file within the source tree.
226
227.. _migration-2.4-readme-changes:
228
229``README`` File Changes
230-----------------------
231
232The following are changes to ``README`` files:
233
234- The main Poky ``README`` file has been moved to the ``meta-poky``
235 layer and has been renamed ``README.poky``. A symlink has been
236 created so that references to the old location work.
237
238- The ``README.hardware`` file has been moved to ``meta-yocto-bsp``. A
239 symlink has been created so that references to the old location work.
240
241- A ``README.qemu`` file has been created with coverage of the
242 ``qemu*`` machines.
243
244.. _migration-2.4-miscellaneous-changes:
245
246Miscellaneous Changes
247---------------------
248
249The following are additional changes:
250
251- The ``ROOTFS_PKGMANAGE_BOOTSTRAP`` variable and any references to it
252 have been removed. You should remove this variable from any custom
253 recipes.
254
255- The ``meta-yocto`` directory has been removed.
256
257 .. note::
258
259 In the Yocto Project 2.1 release
260 meta-yocto
261 was renamed to
262 meta-poky
263 and the
264 meta-yocto
265 subdirectory remained to avoid breaking existing configurations.
266
267- The ``maintainers.inc`` file, which tracks maintainers by listing a
268 primary person responsible for each recipe in OE-Core, has been moved
269 from ``meta-poky`` to OE-Core (i.e. from
270 ``meta-poky/conf/distro/include`` to ``meta/conf/distro/include``).
271
272- The :ref:`buildhistory <ref-classes-buildhistory>` class now makes
273 a single commit per build rather than one commit per subdirectory in
274 the repository. This behavior assumes the commits are enabled with
275 :term:`BUILDHISTORY_COMMIT` = "1", which
276 is typical. Previously, the ``buildhistory`` class made one commit
277 per subdirectory in the repository in order to make it easier to see
278 the changes for a particular subdirectory. To view a particular
279 change, specify that subdirectory as the last parameter on the
280 ``git show`` or ``git diff`` commands.
281
282- The ``x86-base.inc`` file, which is included by all x86-based machine
283 configurations, now sets :term:`IMAGE_FSTYPES`
284 using ``?=`` to "live" rather than appending with ``+=``. This change
285 makes the default easier to override.
286
287- BitBake fires multiple "BuildStarted" events when multiconfig is
288 enabled (one per configuration). For more information, see the
289 ":ref:`Events <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:events>`" section in the BitBake User
290 Manual.
291
292- By default, the ``security_flags.inc`` file sets a
293 :term:`GCCPIE` variable with an option to enable
294 Position Independent Executables (PIE) within ``gcc``. Enabling PIE
295 in the GNU C Compiler (GCC), makes Return Oriented Programming (ROP)
296 attacks much more difficult to execute.
297
298- OE-Core now provides a ``bitbake-layers`` plugin that implements a
299 "create-layer" subcommand. The implementation of this subcommand has
300 resulted in the ``yocto-layer`` script being deprecated and will
301 likely be removed in the next Yocto Project release.
302
303- The ``vmdk``, ``vdi``, and ``qcow2`` image file types are now used in
304 conjunction with the "wic" image type through ``CONVERSION_CMD``.
305 Consequently, the equivalent image types are now ``wic.vmdk``,
306 ``wic.vdi``, and ``wic.qcow2``, respectively.
307
308- ``do_image_<type>[depends]`` has replaced ``IMAGE_DEPENDS_<type>``.
309 If you have your own classes that implement custom image types, then
310 you need to update them.
311
312- OpenSSL 1.1 has been introduced. However, the default is still 1.0.x
313 through the :term:`PREFERRED_VERSION`
314 variable. This preference is set is due to the remaining
315 compatibility issues with other software. The
316 :term:`PROVIDES` variable in the openssl 1.0 recipe
317 now includes "openssl10" as a marker that can be used in
318 :term:`DEPENDS` within recipes that build software
319 that still depend on OpenSSL 1.0.
320
321- To ensure consistent behavior, BitBake's "-r" and "-R" options (i.e.
322 prefile and postfile), which are used to read or post-read additional
323 configuration files from the command line, now only affect the
324 current BitBake command. Before these BitBake changes, these options
325 would "stick" for future executions.
326
327
diff --git a/documentation/ref-manual/migration-2.5.rst b/documentation/ref-manual/migration-2.5.rst
new file mode 100644
index 0000000000..a2adc17757
--- /dev/null
+++ b/documentation/ref-manual/migration-2.5.rst
@@ -0,0 +1,310 @@
1Moving to the Yocto Project 2.5 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.5 Release from the prior release.
6
7.. _migration-2.5-packaging-changes:
8
9Packaging Changes
10-----------------
11
12This section provides information about packaging changes that have
13occurred:
14
15- ``bind-libs``: The libraries packaged by the bind recipe are in a
16 separate ``bind-libs`` package.
17
18- ``libfm-gtk``: The ``libfm`` GTK+ bindings are split into a
19 separate ``libfm-gtk`` package.
20
21- ``flex-libfl``: The flex recipe splits out libfl into a separate
22 ``flex-libfl`` package to avoid too many dependencies being pulled in
23 where only the library is needed.
24
25- ``grub-efi``: The ``grub-efi`` configuration is split into a
26 separate ``grub-bootconf`` recipe. However, the dependency
27 relationship from ``grub-efi`` is through a virtual/grub-bootconf
28 provider making it possible to have your own recipe provide the
29 dependency. Alternatively, you can use a BitBake append file to bring
30 the configuration back into the ``grub-efi`` recipe.
31
32- *armv7a Legacy Package Feed Support:* Legacy support is removed for
33 transitioning from ``armv7a`` to ``armv7a-vfp-neon`` in package
34 feeds, which was previously enabled by setting
35 ``PKGARCHCOMPAT_ARMV7A``. This transition occurred in 2011 and active
36 package feeds should by now be updated to the new naming.
37
38.. _migration-2.5-removed-recipes:
39
40Removed Recipes
41---------------
42
43The following recipes have been removed:
44
45- ``gcc``: The version 6.4 recipes are replaced by 7.x.
46
47- ``gst-player``: Renamed to ``gst-examples`` as per upstream.
48
49- ``hostap-utils``: This software package is obsolete.
50
51- ``latencytop``: This recipe is no longer maintained upstream. The
52 last release was in 2009.
53
54- ``libpfm4``: The only file that requires this recipe is
55 ``oprofile``, which has been removed.
56
57- ``linux-yocto``: The version 4.4, 4.9, and 4.10 recipes have been
58 removed. Versions 4.12, 4.14, and 4.15 remain.
59
60- ``man``: This recipe has been replaced by modern ``man-db``
61
62- ``mkelfimage``: This tool has been removed in the upstream coreboot
63 project, and is no longer needed with the removal of the ELF image
64 type.
65
66- ``nativesdk-postinst-intercept``: This recipe is not maintained.
67
68- ``neon``: This software package is no longer maintained upstream
69 and is no longer needed by anything in OpenEmbedded-Core.
70
71- ``oprofile``: The functionality of this recipe is replaced by
72 ``perf`` and keeping compatibility on an ongoing basis with ``musl``
73 is difficult.
74
75- ``pax``: This software package is obsolete.
76
77- ``stat``: This software package is not maintained upstream.
78 ``coreutils`` provides a modern stat binary.
79
80- ``zisofs-tools-native``: This recipe is no longer needed because
81 the compressed ISO image feature has been removed.
82
83.. _migration-2.5-scripts-and-tools-changes:
84
85Scripts and Tools Changes
86-------------------------
87
88The following are changes to scripts and tools:
89
90- ``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer``: The
91 ``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer`` scripts
92 previously shipped with poky but not in OpenEmbedded-Core have been
93 removed. These scripts are not maintained and are outdated. In many
94 cases, they are also limited in scope. The
95 ``bitbake-layers create-layer`` command is a direct replacement for
96 ``yocto-layer``. See the documentation to create a BSP or kernel
97 recipe in the ":ref:`bsp-guide/bsp:bsp kernel recipe example`" section.
98
99- ``devtool finish``: ``devtool finish`` now exits with an error if
100 there are uncommitted changes or a rebase/am in progress in the
101 recipe's source repository. If this error occurs, there might be
102 uncommitted changes that will not be included in updates to the
103 patches applied by the recipe. A -f/--force option is provided for
104 situations that the uncommitted changes are inconsequential and you
105 want to proceed regardless.
106
107- ``scripts/oe-setup-rpmrepo`` script: The functionality of
108 ``scripts/oe-setup-rpmrepo`` is replaced by
109 ``bitbake package-index``.
110
111- ``scripts/test-dependencies.sh`` script: The script is largely made
112 obsolete by the recipe-specific sysroots functionality introduced in
113 the previous release.
114
115.. _migration-2.5-bitbake-changes:
116
117BitBake Changes
118---------------
119
120The following are BitBake changes:
121
122- The ``--runall`` option has changed. There are two different
123 behaviors people might want:
124
125 - *Behavior A:* For a given target (or set of targets) look through
126 the task graph and run task X only if it is present and will be
127 built.
128
129 - *Behavior B:* For a given target (or set of targets) look through
130 the task graph and run task X if any recipe in the taskgraph has
131 such a target, even if it is not in the original task graph.
132
133 The ``--runall`` option now performs "Behavior B". Previously
134 ``--runall`` behaved like "Behavior A". A ``--runonly`` option has
135 been added to retain the ability to perform "Behavior A".
136
137- Several explicit "run this task for all recipes in the dependency
138 tree" tasks have been removed (e.g. ``fetchall``, ``checkuriall``,
139 and the ``*all`` tasks provided by the ``distrodata`` and
140 ``archiver`` classes). There is a BitBake option to complete this for
141 any arbitrary task. For example:
142 ::
143
144 bitbake <target> -c fetchall
145
146 should now be replaced with:
147 ::
148
149 bitbake <target> --runall=fetch
150
151.. _migration-2.5-python-and-python3-changes:
152
153Python and Python 3 Changes
154---------------------------
155
156The following are auto-packaging changes to Python and Python 3:
157
158The script-managed ``python-*-manifest.inc`` files that were previously
159used to generate Python and Python 3 packages have been replaced with a
160JSON-based file that is easier to read and maintain. A new task is
161available for maintainers of the Python recipes to update the JSON file
162when upgrading to new Python versions. You can now edit the file
163directly instead of having to edit a script and run it to update the
164file.
165
166One particular change to note is that the Python recipes no longer have
167build-time provides for their packages. This assumes ``python-foo`` is
168one of the packages provided by the Python recipe. You can no longer run
169``bitbake python-foo`` or have a
170:term:`DEPENDS` on ``python-foo``,
171but doing either of the following causes the package to work as
172expected: ::
173
174 IMAGE_INSTALL_append = " python-foo"
175
176or ::
177
178 RDEPENDS_${PN} = "python-foo"
179
180The earlier build-time provides behavior was a quirk of the
181way the Python manifest file was created. For more information on this
182change please see `this
183commit <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=8d94b9db221d1def42f091b991903faa2d1651ce>`__.
184
185.. _migration-2.5-miscellaneous-changes:
186
187Miscellaneous Changes
188---------------------
189
190The following are additional changes:
191
192- The ``kernel`` class supports building packages for multiple kernels.
193 If your kernel recipe or ``.bbappend`` file mentions packaging at
194 all, you should replace references to the kernel in package names
195 with ``${KERNEL_PACKAGE_NAME}``. For example, if you disable
196 automatic installation of the kernel image using
197 ``RDEPENDS_kernel-base = ""`` you can avoid warnings using
198 ``RDEPENDS_${KERNEL_PACKAGE_NAME}-base = ""`` instead.
199
200- The ``buildhistory`` class commits changes to the repository by
201 default so you no longer need to set ``BUILDHISTORY_COMMIT = "1"``.
202 If you want to disable commits you need to set
203 ``BUILDHISTORY_COMMIT = "0"`` in your configuration.
204
205- The ``beaglebone`` reference machine has been renamed to
206 ``beaglebone-yocto``. The ``beaglebone-yocto`` BSP is a reference
207 implementation using only mainline components available in
208 OpenEmbedded-Core and ``meta-yocto-bsp``, whereas Texas Instruments
209 maintains a full-featured BSP in the ``meta-ti`` layer. This rename
210 avoids the previous name clash that existed between the two BSPs.
211
212- The ``update-alternatives`` class no longer works with SysV ``init``
213 scripts because this usage has been problematic. Also, the
214 ``sysklogd`` recipe no longer uses ``update-alternatives`` because it
215 is incompatible with other implementations.
216
217- By default, the :ref:`cmake <ref-classes-cmake>` class uses
218 ``ninja`` instead of ``make`` for building. This improves build
219 performance. If a recipe is broken with ``ninja``, then the recipe
220 can set ``OECMAKE_GENERATOR = "Unix Makefiles"`` to change back to
221 ``make``.
222
223- The previously deprecated ``base_*`` functions have been removed in
224 favor of their replacements in ``meta/lib/oe`` and
225 ``bitbake/lib/bb``. These are typically used from recipes and
226 classes. Any references to the old functions must be updated. The
227 following table shows the removed functions and their replacements:
228
229 +------------------------------+----------------------------------------------------------+
230 | *Removed* | *Replacement* |
231 +==============================+==========================================================+
232 | base_path_join() | oe.path.join() |
233 +------------------------------+----------------------------------------------------------+
234 | base_path_relative() | oe.path.relative() |
235 +------------------------------+----------------------------------------------------------+
236 | base_path_out() | oe.path.format_display() |
237 +------------------------------+----------------------------------------------------------+
238 | base_read_file() | oe.utils.read_file() |
239 +------------------------------+----------------------------------------------------------+
240 | base_ifelse() | oe.utils.ifelse() |
241 +------------------------------+----------------------------------------------------------+
242 | base_conditional() | oe.utils.conditional() |
243 +------------------------------+----------------------------------------------------------+
244 | base_less_or_equal() | oe.utils.less_or_equal() |
245 +------------------------------+----------------------------------------------------------+
246 | base_version_less_or_equal() | oe.utils.version_less_or_equal() |
247 +------------------------------+----------------------------------------------------------+
248 | base_contains() | bb.utils.contains() |
249 +------------------------------+----------------------------------------------------------+
250 | base_both_contain() | oe.utils.both_contain() |
251 +------------------------------+----------------------------------------------------------+
252 | base_prune_suffix() | oe.utils.prune_suffix() |
253 +------------------------------+----------------------------------------------------------+
254 | oe_filter() | oe.utils.str_filter() |
255 +------------------------------+----------------------------------------------------------+
256 | oe_filter_out() | oe.utils.str_filter_out() (or use the \_remove operator) |
257 +------------------------------+----------------------------------------------------------+
258
259- Using ``exit 1`` to explicitly defer a postinstall script until first
260 boot is now deprecated since it is not an obvious mechanism and can
261 mask actual errors. If you want to explicitly defer a postinstall to
262 first boot on the target rather than at ``rootfs`` creation time, use
263 ``pkg_postinst_ontarget()`` or call
264 ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
265 Any failure of a ``pkg_postinst()`` script (including ``exit 1``)
266 will trigger a warning during ``do_rootfs``.
267
268 For more information, see the
269 ":ref:`dev-manual/dev-manual-common-tasks:post-installation scripts`"
270 section in the Yocto Project Development Tasks Manual.
271
272- The ``elf`` image type has been removed. This image type was removed
273 because the ``mkelfimage`` tool that was required to create it is no
274 longer provided by coreboot upstream and required updating every time
275 ``binutils`` updated.
276
277- Support for .iso image compression (previously enabled through
278 ``COMPRESSISO = "1"``) has been removed. The userspace tools
279 (``zisofs-tools``) are unmaintained and ``squashfs`` provides better
280 performance and compression. In order to build a live image with
281 squashfs+lz4 compression enabled you should now set
282 ``LIVE_ROOTFS_TYPE = "squashfs-lz4"`` and ensure that ``live`` is in
283 ``IMAGE_FSTYPES``.
284
285- Recipes with an unconditional dependency on ``libpam`` are only
286 buildable with ``pam`` in ``DISTRO_FEATURES``. If the dependency is
287 truly optional then it is recommended that the dependency be
288 conditional upon ``pam`` being in ``DISTRO_FEATURES``.
289
290- For EFI-based machines, the bootloader (``grub-efi`` by default) is
291 installed into the image at /boot. Wic can be used to split the
292 bootloader into separate boot and rootfs partitions if necessary.
293
294- Patches whose context does not match exactly (i.e. where patch
295 reports "fuzz" when applying) will generate a warning. For an example
296 of this see `this
297 commit <http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=cc97bc08125b63821ce3f616771830f77c456f57>`__.
298
299- Layers are expected to set ``LAYERSERIES_COMPAT_layername`` to match
300 the version(s) of OpenEmbedded-Core they are compatible with. This is
301 specified as codenames using spaces to separate multiple values (e.g.
302 "rocko sumo"). If a layer does not set
303 ``LAYERSERIES_COMPAT_layername``, a warning will is shown. If a layer
304 sets a value that does not include the current version ("sumo" for
305 the 2.5 release), then an error will be produced.
306
307- The ``TZ`` environment variable is set to "UTC" within the build
308 environment in order to fix reproducibility problems in some recipes.
309
310
diff --git a/documentation/ref-manual/migration-2.6.rst b/documentation/ref-manual/migration-2.6.rst
new file mode 100644
index 0000000000..f16aaaa975
--- /dev/null
+++ b/documentation/ref-manual/migration-2.6.rst
@@ -0,0 +1,476 @@
1Moving to the Yocto Project 2.6 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.6 Release from the prior release.
6
7.. _migration-2.6-gcc-changes:
8
9GCC 8.2 is Now Used by Default
10------------------------------
11
12The GNU Compiler Collection version 8.2 is now used by default for
13compilation. For more information on what has changed in the GCC 8.x
14release, see https://gcc.gnu.org/gcc-8/changes.html.
15
16If you still need to compile with version 7.x, GCC 7.3 is also provided.
17You can select this version by setting the and can be selected by
18setting the :term:`GCCVERSION` variable to "7.%" in
19your configuration.
20
21.. _migration-2.6-removed-recipes:
22
23Removed Recipes
24---------------
25
26The following recipes have been removed:
27
28- *beecrypt*: No longer needed since moving to RPM 4.
29- *bigreqsproto*: Replaced by ``xorgproto``.
30- *calibrateproto*: Removed in favor of ``xinput``.
31- *compositeproto*: Replaced by ``xorgproto``.
32- *damageproto*: Replaced by ``xorgproto``.
33- *dmxproto*: Replaced by ``xorgproto``.
34- *dri2proto*: Replaced by ``xorgproto``.
35- *dri3proto*: Replaced by ``xorgproto``.
36- *eee-acpi-scripts*: Became obsolete.
37- *fixesproto*: Replaced by ``xorgproto``.
38- *fontsproto*: Replaced by ``xorgproto``.
39- *fstests*: Became obsolete.
40- *gccmakedep*: No longer used.
41- *glproto*: Replaced by ``xorgproto``.
42- *gnome-desktop3*: No longer needed. This recipe has moved to ``meta-oe``.
43- *icon-naming-utils*: No longer used since the Sato theme was removed in 2016.
44- *inputproto*: Replaced by ``xorgproto``.
45- *kbproto*: Replaced by ``xorgproto``.
46- *libusb-compat*: Became obsolete.
47- *libuser*: Became obsolete.
48- *libnfsidmap*: No longer an external requirement since ``nfs-utils`` 2.2.1. ``libnfsidmap`` is now integrated.
49- *libxcalibrate*: No longer needed with ``xinput``
50- *mktemp*: Became obsolete. The ``mktemp`` command is provided by both ``busybox`` and ``coreutils``.
51- *ossp-uuid*: Is not being maintained and has mostly been replaced by ``uuid.h`` in ``util-linux``.
52- *pax-utils*: No longer needed. Previous QA tests that did use this recipe are now done at build time.
53- *pcmciautils*: Became obsolete.
54- *pixz*: No longer needed. ``xz`` now supports multi-threaded compression.
55- *presentproto*: Replaced by ``xorgproto``.
56- *randrproto*: Replaced by ``xorgproto``.
57- *recordproto*: Replaced by ``xorgproto``.
58- *renderproto*: Replaced by ``xorgproto``.
59- *resourceproto*: Replaced by ``xorgproto``.
60- *scrnsaverproto*: Replaced by ``xorgproto``.
61- *trace-cmd*: Became obsolete. ``perf`` replaced this recipe's functionally.
62- *videoproto*: Replaced by ``xorgproto``.
63- *wireless-tools*: Became obsolete. Superseded by ``iw``.
64- *xcmiscproto*: Replaced by ``xorgproto``.
65- *xextproto*: Replaced by ``xorgproto``.
66- *xf86dgaproto*: Replaced by ``xorgproto``.
67- *xf86driproto*: Replaced by ``xorgproto``.
68- *xf86miscproto*: Replaced by ``xorgproto``.
69- *xf86-video-omapfb*: Became obsolete. Use kernel modesetting driver instead.
70- *xf86-video-omap*: Became obsolete. Use kernel modesetting driver instead.
71- *xf86vidmodeproto*: Replaced by ``xorgproto``.
72- *xineramaproto*: Replaced by ``xorgproto``.
73- *xproto*: Replaced by ``xorgproto``.
74- *yasm*: No longer needed since previous usages are now satisfied by ``nasm``.
75
76.. _migration-2.6-packaging-changes:
77
78Packaging Changes
79-----------------
80
81The following packaging changes have been made:
82
83- *cmake*: ``cmake.m4`` and ``toolchain`` files have been moved to
84 the main package.
85
86- *iptables*: The ``iptables`` modules have been split into
87 separate packages.
88
89- *alsa-lib*: ``libasound`` is now in the main ``alsa-lib`` package
90 instead of ``libasound``.
91
92- *glibc*: ``libnss-db`` is now in its own package along with a
93 ``/var/db/makedbs.sh`` script to update databases.
94
95- *python and python3*: The main package has been removed from
96 the recipe. You must install specific packages or ``python-modules``
97 / ``python3-modules`` for everything.
98
99- *systemtap*: Moved ``systemtap-exporter`` into its own package.
100
101.. _migration-2.6-xorg-protocol-dependencies:
102
103XOrg Protocol dependencies
104--------------------------
105
106The ``*proto`` upstream repositories have been combined into one
107"xorgproto" repository. Thus, the corresponding recipes have also been
108combined into a single ``xorgproto`` recipe. Any recipes that depend
109upon the older ``*proto`` recipes need to be changed to depend on the
110newer ``xorgproto`` recipe instead.
111
112For names of recipes removed because of this repository change, see the
113`Removed Recipes <#migration-2.6-removed-recipes>`__ section.
114
115.. _migration-2.6-distutils-distutils3-fetching-dependencies:
116
117``distutils`` and ``distutils3`` Now Prevent Fetching Dependencies During the ``do_configure`` Task
118---------------------------------------------------------------------------------------------------
119
120Previously, it was possible for Python recipes that inherited the
121:ref:`distutils <ref-classes-distutils>` and
122:ref:`distutils3 <ref-classes-distutils3>` classes to fetch code
123during the :ref:`ref-tasks-configure` task to satisfy
124dependencies mentioned in ``setup.py`` if those dependencies were not
125provided in the sysroot (i.e. recipes providing the dependencies were
126missing from :term:`DEPENDS`).
127
128.. note::
129
130 This change affects classes beyond just the two mentioned (i.e.
131 distutils
132 and
133 distutils3
134 ). Any recipe that inherits
135 distutils\*
136 classes are affected. For example, the
137 setuptools
138 and
139 setuptools3
140 recipes are affected since they inherit the
141 distutils\*
142 classes.
143
144Fetching these types of dependencies that are not provided in the
145sysroot negatively affects the ability to reproduce builds. This type of
146fetching is now explicitly disabled. Consequently, any missing
147dependencies in Python recipes that use these classes now result in an
148error during the ``do_configure`` task.
149
150.. _migration-2.6-linux-yocto-configuration-audit-issues-now-correctly-reported:
151
152``linux-yocto`` Configuration Audit Issues Now Correctly Reported
153-----------------------------------------------------------------
154
155Due to a bug, the kernel configuration audit functionality was not
156writing out any resulting warnings during the build. This issue is now
157corrected. You might notice these warnings now if you have a custom
158kernel configuration with a ``linux-yocto`` style kernel recipe.
159
160.. _migration-2.6-image-kernel-artifact-naming-changes:
161
162Image/Kernel Artifact Naming Changes
163------------------------------------
164
165The following changes have been made:
166
167- Name variables (e.g. :term:`IMAGE_NAME`) use a new
168 ``IMAGE_VERSION_SUFFIX`` variable instead of
169 :term:`DATETIME`. Using ``IMAGE_VERSION_SUFFIX``
170 allows easier and more direct changes.
171
172 The ``IMAGE_VERSION_SUFFIX`` variable is set in the ``bitbake.conf``
173 configuration file as follows:
174 ::
175
176 IMAGE_VERSION_SUFFIX = "-${DATETIME}"
177
178- Several variables have changed names for consistency:
179 ::
180
181 Old Variable Name New Variable Name
182 ========================================================
183 KERNEL_IMAGE_BASE_NAME :term:`KERNEL_IMAGE_NAME`
184 KERNEL_IMAGE_SYMLINK_NAME :term:`KERNEL_IMAGE_LINK_NAME`
185 MODULE_TARBALL_BASE_NAME :term:`MODULE_TARBALL_NAME`
186 MODULE_TARBALL_SYMLINK_NAME :term:`MODULE_TARBALL_LINK_NAME`
187 INITRAMFS_BASE_NAME :term:`INITRAMFS_NAME`
188
189- The ``MODULE_IMAGE_BASE_NAME`` variable has been removed. The module
190 tarball name is now controlled directly with the
191 :term:`MODULE_TARBALL_NAME` variable.
192
193- The :term:`KERNEL_DTB_NAME` and
194 :term:`KERNEL_DTB_LINK_NAME` variables
195 have been introduced to control kernel Device Tree Binary (DTB)
196 artifact names instead of mangling ``KERNEL_IMAGE_*`` variables.
197
198- The :term:`KERNEL_FIT_NAME` and
199 :term:`KERNEL_FIT_LINK_NAME` variables
200 have been introduced to specify the name of flattened image tree
201 (FIT) kernel images similar to other deployed artifacts.
202
203- The :term:`MODULE_TARBALL_NAME` and
204 :term:`MODULE_TARBALL_LINK_NAME`
205 variable values no longer include the "module-" prefix or ".tgz"
206 suffix. These parts are now hardcoded so that the values are
207 consistent with other artifact naming variables.
208
209- Added the :term:`INITRAMFS_LINK_NAME`
210 variable so that the symlink can be controlled similarly to other
211 artifact types.
212
213- :term:`INITRAMFS_NAME` now uses
214 "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" instead
215 of "${PV}-${PR}-${MACHINE}-${DATETIME}", which makes it consistent
216 with other variables.
217
218.. _migration-2.6-serial-console-deprecated:
219
220``SERIAL_CONSOLE`` Deprecated
221-----------------------------
222
223The :term:`SERIAL_CONSOLE` variable has been
224functionally replaced by the
225:term:`SERIAL_CONSOLES` variable for some time.
226With the Yocto Project 2.6 release, ``SERIAL_CONSOLE`` has been
227officially deprecated.
228
229``SERIAL_CONSOLE`` will continue to work as before for the 2.6 release.
230However, for the sake of future compatibility, it is recommended that
231you replace all instances of ``SERIAL_CONSOLE`` with
232``SERIAL_CONSOLES``.
233
234.. note::
235
236 The only difference in usage is that
237 SERIAL_CONSOLES
238 expects entries to be separated using semicolons as compared to
239 SERIAL_CONSOLE
240 , which expects spaces.
241
242.. _migration-2.6-poky-sets-unknown-configure-option-to-qa-error:
243
244Configure Script Reports Unknown Options as Errors
245--------------------------------------------------
246
247If the configure script reports an unknown option, this now triggers a
248QA error instead of a warning. Any recipes that previously got away with
249specifying such unknown options now need to be fixed.
250
251.. _migration-2.6-override-changes:
252
253Override Changes
254----------------
255
256The following changes have occurred:
257
258- The ``virtclass-native`` and ``virtclass-nativesdk`` Overrides Have
259 Been Removed: The ``virtclass-native`` and ``virtclass-nativesdk``
260 overrides have been deprecated since 2012 in favor of
261 ``class-native`` and ``class-nativesdk``, respectively. Both
262 ``virtclass-native`` and ``virtclass-nativesdk`` are now dropped.
263
264 .. note::
265
266 The
267 virtclass-multilib-
268 overrides for multilib are still valid.
269
270- The ``forcevariable`` Override Now Has a Higher Priority Than
271 ``libc`` Overrides: The ``forcevariable`` override is documented to
272 be the highest priority override. However, due to a long-standing
273 quirk of how :term:`OVERRIDES` is set, the ``libc``
274 overrides (e.g. ``libc-glibc``, ``libc-musl``, and so forth)
275 erroneously had a higher priority. This issue is now corrected.
276
277 It is likely this change will not cause any problems. However, it is
278 possible with some unusual configurations that you might see a change
279 in behavior if you were relying on the previous behavior. Be sure to
280 check how you use ``forcevariable`` and ``libc-*`` overrides in your
281 custom layers and configuration files to ensure they make sense.
282
283- The ``build-${BUILD_OS}`` Override Has Been Removed: The
284 ``build-${BUILD_OS}``, which is typically ``build-linux``, override
285 has been removed because building on a host operating system other
286 than a recent version of Linux is neither supported nor recommended.
287 Dropping the override avoids giving the impression that other host
288 operating systems might be supported.
289
290- The "_remove" operator now preserves whitespace. Consequently, when
291 specifying list items to remove, be aware that leading and trailing
292 whitespace resulting from the removal is retained.
293
294 See the ":ref:`bitbake:removing-override-style-syntax`"
295 section in the BitBake User Manual for a detailed example.
296
297.. _migration-2.6-systemd-configuration-now-split-out-to-system-conf:
298
299``systemd`` Configuration is Now Split Into ``systemd-conf``
300------------------------------------------------------------
301
302The configuration for the ``systemd`` recipe has been moved into a
303``system-conf`` recipe. Moving this configuration to a separate recipe
304avoids the ``systemd`` recipe from becoming machine-specific for cases
305where machine-specific configurations need to be applied (e.g. for
306``qemu*`` machines).
307
308Currently, the new recipe packages the following files:
309::
310
311 ${sysconfdir}/machine-id
312 ${sysconfdir}/systemd/coredump.conf
313 ${sysconfdir}/systemd/journald.conf
314 ${sysconfdir}/systemd/logind.conf
315 ${sysconfdir}/systemd/system.conf
316 ${sysconfdir}/systemd/user.conf
317
318If you previously used bbappend files to append the ``systemd`` recipe to
319change any of the listed files, you must do so for the ``systemd-conf``
320recipe instead.
321
322.. _migration-2.6-automatic-testing-changes:
323
324Automatic Testing Changes
325-------------------------
326
327This section provides information about automatic testing changes:
328
329- ``TEST_IMAGE`` Variable Removed: Prior to this release, you set the
330 ``TEST_IMAGE`` variable to "1" to enable automatic testing for
331 successfully built images. The ``TEST_IMAGE`` variable no longer
332 exists and has been replaced by the
333 :term:`TESTIMAGE_AUTO` variable.
334
335- Inheriting the ``testimage`` and ``testsdk`` Classes: Best
336 practices now dictate that you use the
337 :term:`IMAGE_CLASSES` variable rather than the
338 :term:`INHERIT` variable when you inherit the
339 :ref:`testimage <ref-classes-testimage*>` and
340 :ref:`testsdk <ref-classes-testsdk>` classes used for automatic
341 testing.
342
343.. _migration-2.6-openssl-changes:
344
345OpenSSL Changes
346---------------
347
348`OpenSSL <https://www.openssl.org/>`__ has been upgraded from 1.0 to
3491.1. By default, this upgrade could cause problems for recipes that have
350both versions in their dependency chains. The problem is that both
351versions cannot be installed together at build time.
352
353.. note::
354
355 It is possible to have both versions of the library at runtime.
356
357.. _migration-2.6-bitbake-changes:
358
359BitBake Changes
360---------------
361
362The server logfile ``bitbake-cookerdaemon.log`` is now always placed in
363the :term:`Build Directory` instead of the current
364directory.
365
366.. _migration-2.6-security-changes:
367
368Security Changes
369----------------
370
371The Poky distribution now uses security compiler flags by default.
372Inclusion of these flags could cause new failures due to stricter
373checking for various potential security issues in code.
374
375.. _migration-2.6-post-installation-changes:
376
377Post Installation Changes
378-------------------------
379
380You must explicitly mark post installs to defer to the target. If you
381want to explicitly defer a postinstall to first boot on the target
382rather than at rootfs creation time, use ``pkg_postinst_ontarget()`` or
383call ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
384Any failure of a ``pkg_postinst()`` script (including exit 1) triggers
385an error during the :ref:`ref-tasks-rootfs` task.
386
387For more information on post-installation behavior, see the
388":ref:`dev-manual/dev-manual-common-tasks:post-installation scripts`"
389section in the Yocto Project Development Tasks Manual.
390
391.. _migration-2.6-python-3-profile-guided-optimizations:
392
393Python 3 Profile-Guided Optimization
394------------------------------------
395
396The ``python3`` recipe now enables profile-guided optimization. Using
397this optimization requires a little extra build time in exchange for
398improved performance on the target at runtime. Additionally, the
399optimization is only enabled if the current
400:term:`MACHINE` has support for user-mode emulation in
401QEMU (i.e. "qemu-usermode" is in
402:term:`MACHINE_FEATURES`, which it is by
403default).
404
405If you wish to disable Python profile-guided optimization regardless of
406the value of ``MACHINE_FEATURES``, then ensure that
407:term:`PACKAGECONFIG` for the ``python3`` recipe
408does not contain "pgo". You could accomplish the latter using the
409following at the configuration level:
410::
411
412 PACKAGECONFIG_remove_pn-python3 = "pgo"
413
414Alternatively, you can set ``PACKAGECONFIG`` using an append file
415for the ``python3`` recipe.
416
417.. _migration-2.6-miscellaneous-changes:
418
419Miscellaneous Changes
420---------------------
421
422The following miscellaneous changes occurred:
423
424- Default to using the Thumb-2 instruction set for armv7a and above. If
425 you have any custom recipes that build software that needs to be
426 built with the ARM instruction set, change the recipe to set the
427 instruction set as follows:
428 ::
429
430 ARM_INSTRUCTION_SET = "arm"
431
432- ``run-postinsts`` no longer uses ``/etc/*-postinsts`` for
433 ``dpkg/opkg`` in favor of built-in postinst support. RPM behavior
434 remains unchanged.
435
436- The ``NOISO`` and ``NOHDD`` variables are no longer used. You now
437 control building ``*.iso`` and ``*.hddimg`` image types directly by
438 using the :term:`IMAGE_FSTYPES` variable.
439
440- The ``scripts/contrib/mkefidisk.sh`` has been removed in favor of
441 Wic.
442
443- ``kernel-modules`` has been removed from
444 :term:`RRECOMMENDS` for ``qemumips`` and
445 ``qemumips64`` machines. Removal also impacts the ``x86-base.inc``
446 file.
447
448 .. note::
449
450 genericx86
451 and
452 genericx86-64
453 retain
454 kernel-modules
455 as part of the
456 RRECOMMENDS
457 variable setting.
458
459- The ``LGPLv2_WHITELIST_GPL-3.0`` variable has been removed. If you
460 are setting this variable in your configuration, set or append it to
461 the ``WHITELIST_GPL-3.0`` variable instead.
462
463- ``${ASNEEDED}`` is now included in the
464 :term:`TARGET_LDFLAGS` variable directly. The
465 remaining definitions from ``meta/conf/distro/include/as-needed.inc``
466 have been moved to corresponding recipes.
467
468- Support for DSA host keys has been dropped from the OpenSSH recipes.
469 If you are still using DSA keys, you must switch over to a more
470 secure algorithm as recommended by OpenSSH upstream.
471
472- The ``dhcp`` recipe now uses the ``dhcpd6.conf`` configuration file
473 in ``dhcpd6.service`` for IPv6 DHCP rather than re-using
474 ``dhcpd.conf``, which is now reserved for IPv4.
475
476
diff --git a/documentation/ref-manual/migration-2.7.rst b/documentation/ref-manual/migration-2.7.rst
new file mode 100644
index 0000000000..7e628fc3ec
--- /dev/null
+++ b/documentation/ref-manual/migration-2.7.rst
@@ -0,0 +1,180 @@
1Moving to the Yocto Project 2.7 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 2.7 Release from the prior release.
6
7.. _migration-2.7-bitbake-changes:
8
9BitBake Changes
10---------------
11
12The following changes have been made to BitBake:
13
14- BitBake now checks anonymous Python functions and pure Python
15 functions (e.g. ``def funcname:``) in the metadata for tab
16 indentation. If found, BitBake produces a warning.
17
18- Bitbake now checks
19 :term:`BBFILE_COLLECTIONS` for duplicate
20 entries and triggers an error if any are found.
21
22.. _migration-2.7-eclipse-support-dropped:
23
24Eclipse Support Removed
25-----------------------
26
27Support for the Eclipse IDE has been removed. Support continues for
28those releases prior to 2.7 that did include support. The 2.7 release
29does not include the Eclipse Yocto plugin.
30
31.. _migration-2.7-qemu-native-splits-system-and-user-mode-parts:
32
33``qemu-native`` Splits the System and User-Mode Parts
34-----------------------------------------------------
35
36The system and user-mode parts of ``qemu-native`` are now split.
37``qemu-native`` provides the user-mode components and
38``qemu-system-native`` provides the system components. If you have
39recipes that depend on QEMU's system emulation functionality at build
40time, they should now depend upon ``qemu-system-native`` instead of
41``qemu-native``.
42
43.. _migration-2.7-upstream-tracking.inc-removed:
44
45The ``upstream-tracking.inc`` File Has Been Removed
46---------------------------------------------------
47
48The previously deprecated ``upstream-tracking.inc`` file is now removed.
49Any ``UPSTREAM_TRACKING*`` variables are now set in the corresponding
50recipes instead.
51
52Remove any references you have to the ``upstream-tracking.inc`` file in
53your configuration.
54
55.. _migration-2.7-distro-features-libc-removed:
56
57The ``DISTRO_FEATURES_LIBC`` Variable Has Been Removed
58------------------------------------------------------
59
60The ``DISTRO_FEATURES_LIBC`` variable is no longer used. The ability to
61configure glibc using kconfig has been removed for quite some time
62making the ``libc-*`` features set no longer effective.
63
64Remove any references you have to ``DISTRO_FEATURES_LIBC`` in your own
65layers.
66
67.. _migration-2.7-license-values:
68
69License Value Corrections
70-------------------------
71
72The following corrections have been made to the
73:term:`LICENSE` values set by recipes:
74
75- *socat*: Corrected ``LICENSE`` to be "GPLv2" rather than "GPLv2+".
76- *libgfortran*: Set license to "GPL-3.0-with-GCC-exception".
77- *elfutils*: Removed "Elfutils-Exception" and set to "GPLv2" for shared libraries
78
79.. _migration-2.7-packaging-changes:
80
81Packaging Changes
82-----------------
83
84This section provides information about packaging changes.
85
86- ``bind``: The ``nsupdate`` binary has been moved to the
87 ``bind-utils`` package.
88
89- Debug split: The default debug split has been changed to create
90 separate source packages (i.e. package_name\ ``-dbg`` and
91 package_name\ ``-src``). If you are currently using ``dbg-pkgs`` in
92 :term:`IMAGE_FEATURES` to bring in debug
93 symbols and you still need the sources, you must now also add
94 ``src-pkgs`` to ``IMAGE_FEATURES``. Source packages remain in the
95 target portion of the SDK by default, unless you have set your own
96 value for :term:`SDKIMAGE_FEATURES` that
97 does not include ``src-pkgs``.
98
99- Mount all using ``util-linux``: ``/etc/default/mountall`` has moved
100 into the -mount sub-package.
101
102- Splitting binaries using ``util-linux``: ``util-linux`` now splits
103 each binary into its own package for fine-grained control. The main
104 ``util-linux`` package pulls in the individual binary packages using
105 the :term:`RRECOMMENDS` and
106 :term:`RDEPENDS` variables. As a result, existing
107 images should not see any changes assuming
108 :term:`NO_RECOMMENDATIONS` is not set.
109
110- ``netbase/base-files``: ``/etc/hosts`` has moved from ``netbase`` to
111 ``base-files``.
112
113- ``tzdata``: The main package has been converted to an empty meta
114 package that pulls in all ``tzdata`` packages by default.
115
116- ``lrzsz``: This package has been removed from
117 ``packagegroup-self-hosted`` and
118 ``packagegroup-core-tools-testapps``. The X/Y/ZModem support is less
119 likely to be needed on modern systems. If you are relying on these
120 packagegroups to include the ``lrzsz`` package in your image, you now
121 need to explicitly add the package.
122
123.. _migration-2.7-removed-recipes:
124
125Removed Recipes
126---------------
127
128The following recipes have been removed:
129
130- *gcc*: Drop version 7.3 recipes. Version 8.3 now remains.
131- *linux-yocto*: Drop versions 4.14 and 4.18 recipes. Versions 4.19 and 5.0 remain.
132- *go*: Drop version 1.9 recipes. Versions 1.11 and 1.12 remain.
133- *xvideo-tests*: Became obsolete.
134- *libart-lgpl*: Became obsolete.
135- *gtk-icon-utils-native*: These tools are now provided by gtk+3-native
136- *gcc-cross-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
137- *gcc-crosssdk-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
138- *glibc-initial*: Removed because the benefits of having it for site_config are currently outweighed by the cost of building the recipe.
139
140.. _migration-2.7-removed-classes:
141
142Removed Classes
143---------------
144
145The following classes have been removed:
146
147- *distutils-tools*: This class was never used.
148- *bugzilla.bbclass*: Became obsolete.
149- *distrodata*: This functionally has been replaced by a more modern tinfoil-based implementation.
150
151.. _migration-2.7-miscellaneous-changes:
152
153Miscellaneous Changes
154---------------------
155
156The following miscellaneous changes occurred:
157
158- The ``distro`` subdirectory of the Poky repository has been removed
159 from the top-level ``scripts`` directory.
160
161- Perl now builds for the target using
162 `perl-cross <http://arsv.github.io/perl-cross/>`_ for better
163 maintainability and improved build performance. This change should
164 not present any problems unless you have heavily customized your Perl
165 recipe.
166
167- ``arm-tunes``: Removed the "-march" option if mcpu is already added.
168
169- ``update-alternatives``: Convert file renames to
170 :term:`PACKAGE_PREPROCESS_FUNCS`
171
172- ``base/pixbufcache``: Obsolete ``sstatecompletions`` code has been
173 removed.
174
175- :ref:`native <ref-classes-native>` class:
176 :term:`RDEPENDS` handling has been enabled.
177
178- ``inetutils``: This recipe has rsh disabled.
179
180
diff --git a/documentation/ref-manual/migration-3.0.rst b/documentation/ref-manual/migration-3.0.rst
new file mode 100644
index 0000000000..e1305dfccf
--- /dev/null
+++ b/documentation/ref-manual/migration-3.0.rst
@@ -0,0 +1,321 @@
1Moving to the Yocto Project 3.0 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 3.0 Release from the prior release.
6
7.. _migration-3.0-init-system-selection:
8
9Init System Selection
10---------------------
11
12Changing the init system manager previously required setting a number of
13different variables. You can now change the manager by setting the
14``INIT_MANAGER`` variable and the corresponding include files (i.e.
15``conf/distro/include/init-manager-*.conf``). Include files are provided
16for four values: "none", "sysvinit", "systemd", and "mdev-busybox". The
17default value, "none", for ``INIT_MANAGER`` should allow your current
18settings to continue working. However, it is advisable to explicitly set
19``INIT_MANAGER``.
20
21.. _migration-3.0-lsb-support-removed:
22
23LSB Support Removed
24-------------------
25
26Linux Standard Base (LSB) as a standard is not current, and is not well
27suited for embedded applications. Support can be continued in a separate
28layer if needed. However, presently LSB support has been removed from
29the core.
30
31As a result of this change, the ``poky-lsb`` derivative distribution
32configuration that was also used for testing alternative configurations
33has been replaced with a ``poky-altcfg`` distribution that has LSB parts
34removed.
35
36.. _migration-3.0-removed-recipes:
37
38Removed Recipes
39---------------
40
41The following recipes have been removed.
42
43- ``core-image-lsb-dev``: Part of removed LSB support.
44
45- ``core-image-lsb``: Part of removed LSB support.
46
47- ``core-image-lsb-sdk``: Part of removed LSB support.
48
49- ``cve-check-tool``: Functionally replaced by the ``cve-update-db``
50 recipe and ``cve-check`` class.
51
52- ``eglinfo``: No longer maintained. ``eglinfo`` from ``mesa-demos`` is
53 an adequate and maintained alternative.
54
55- ``gcc-8.3``: Version 8.3 removed. Replaced by 9.2.
56
57- ``gnome-themes-standard``: Only needed by gtk+ 2.x, which has been
58 removed.
59
60- ``gtk+``: GTK+ 2 is obsolete and has been replaced by gtk+3.
61
62- ``irda-utils``: Has become obsolete. IrDA support has been removed
63 from the Linux kernel in version 4.17 and later.
64
65- ``libnewt-python``: ``libnewt`` Python support merged into main
66 ``libnewt`` recipe.
67
68- ``libsdl``: Replaced by newer ``libsdl2``.
69
70- ``libx11-diet``: Became obsolete.
71
72- ``libxx86dga``: Removed obsolete client library.
73
74- ``libxx86misc``: Removed. Library is redundant.
75
76- ``linux-yocto``: Version 5.0 removed, which is now redundant (5.2 /
77 4.19 present).
78
79- ``lsbinitscripts``: Part of removed LSB support.
80
81- ``lsb``: Part of removed LSB support.
82
83- ``lsbtest``: Part of removed LSB support.
84
85- ``openssl10``: Replaced by newer ``openssl`` version 1.1.
86
87- ``packagegroup-core-lsb``: Part of removed LSB support.
88
89- ``python-nose``: Removed the Python 2.x version of the recipe.
90
91- ``python-numpy``: Removed the Python 2.x version of the recipe.
92
93- ``python-scons``: Removed the Python 2.x version of the recipe.
94
95- ``source-highlight``: No longer needed.
96
97- ``stress``: Replaced by ``stress-ng``.
98
99- ``vulkan``: Split into ``vulkan-loader``, ``vulkan-headers``, and
100 ``vulkan-tools``.
101
102- ``weston-conf``: Functionality moved to ``weston-init``.
103
104.. _migration-3.0-packaging-changes:
105
106Packaging Changes
107-----------------
108
109The following packaging changes have occurred.
110
111- The `Epiphany <https://en.wikipedia.org/wiki/GNOME_Web>`__ browser
112 has been dropped from ``packagegroup-self-hosted`` as it has not been
113 needed inside ``build-appliance-image`` for quite some time and was
114 causing resource problems.
115
116- ``libcap-ng`` Python support has been moved to a separate
117 ``libcap-ng-python`` recipe to streamline the build process when the
118 Python bindings are not needed.
119
120- ``libdrm`` now packages the file ``amdgpu.ids`` into a separate
121 ``libdrm-amdgpu`` package.
122
123- ``python3``: The ``runpy`` module is now in the ``python3-core``
124 package as it is required to support the common "python3 -m" command
125 usage.
126
127- ``distcc`` now provides separate ``distcc-client`` and
128 ``distcc-server`` packages as typically one or the other are needed,
129 rather than both.
130
131- ``python*-setuptools`` recipes now separately package the
132 ``pkg_resources`` module in a ``python-pkg-resources`` /
133 ``python3-pkg-resources`` package as the module is useful independent
134 of the rest of the setuptools package. The main ``python-setuptools``
135 / ``python3-setuptools`` package depends on this new package so you
136 should only need to update dependencies unless you want to take
137 advantage of the increased granularity.
138
139.. _migration-3.0-cve-checking:
140
141CVE Checking
142------------
143
144``cve-check-tool`` has been functionally replaced by a new
145``cve-update-db`` recipe and functionality built into the ``cve-check``
146class. The result uses NVD JSON data feeds rather than the deprecated
147XML feeds that ``cve-check-tool`` was using, supports CVSSv3 scoring,
148and makes other improvements.
149
150Additionally, the ``CVE_CHECK_CVE_WHITELIST`` variable has been replaced
151by ``CVE_CHECK_WHITELIST``.
152
153.. _migration-3.0-bitbake-changes:
154
155Bitbake Changes
156---------------
157
158The following BitBake changes have occurred.
159
160- ``addtask`` statements now properly validate dependent tasks.
161 Previously, an invalid task was silently ignored. With this change,
162 the invalid task generates a warning.
163
164- Other invalid ``addtask`` and ``deltask`` usages now trigger these
165 warnings: "multiple target tasks arguments with addtask / deltask",
166 and "multiple before/after clauses".
167
168- The "multiconfig" prefix is now shortened to "mc". "multiconfig" will
169 continue to work, however it may be removed in a future release.
170
171- The ``bitbake -g`` command no longer generates a
172 ``recipe-depends.dot`` file as the contents (i.e. a reprocessed
173 version of ``task-depends.dot``) were confusing.
174
175- The ``bb.build.FuncFailed`` exception, previously raised by
176 ``bb.build.exec_func()`` when certain other exceptions have occurred,
177 has been removed. The real underlying exceptions will be raised
178 instead. If you have calls to ``bb.build.exec_func()`` in custom
179 classes or ``tinfoil-using`` scripts, any references to
180 ``bb.build.FuncFailed`` should be cleaned up.
181
182- Additionally, the ``bb.build.exec_func()`` no longer accepts the
183 "pythonexception" parameter. The function now always raises
184 exceptions. Remove this argument in any calls to
185 ``bb.build.exec_func()`` in custom classes or scripts.
186
187- The
188 :term:`bitbake:BB_SETSCENE_VERIFY_FUNCTION2`
189 is no longer used. In the unlikely event that you have any references
190 to it, they should be removed.
191
192- The ``RunQueueExecuteScenequeue`` and ``RunQueueExecuteTasks`` events
193 have been removed since setscene tasks are now executed as part of
194 the normal runqueue. Any event handling code in custom classes or
195 scripts that handles these two events need to be updated.
196
197- The arguments passed to functions used with
198 :term:`bitbake:BB_HASHCHECK_FUNCTION`
199 have changed. If you are using your own custom hash check function,
200 see
201 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725
202 for details.
203
204- Task specifications in ``BB_TASKDEPDATA`` and class implementations
205 used in signature generator classes now use "<fn>:<task>" everywhere
206 rather than the "." delimiter that was being used in some places.
207 This change makes it consistent with all areas in the code. Custom
208 signature generator classes and code that reads ``BB_TASKDEPDATA``
209 need to be updated to use ':' as a separator rather than '.'.
210
211.. _migration-3.0-sanity-checks:
212
213Sanity Checks
214-------------
215
216The following sanity check changes occurred.
217
218- :term:`SRC_URI` is now checked for usage of two
219 problematic items:
220
221 - "${PN}" prefix/suffix use - Warnings always appear if ${PN} is
222 used. You must fix the issue regardless of whether multiconfig or
223 anything else that would cause prefixing/suffixing to happen.
224
225 - Github archive tarballs - these are not guaranteed to be stable.
226 Consequently, it is likely that the tarballs will be refreshed and
227 thus the SRC_URI checksums will fail to apply. It is recommended
228 that you fetch either an official release tarball or a specific
229 revision from the actual Git repository instead.
230
231 Either one of these items now trigger a warning by default. If you
232 wish to disable this check, remove ``src-uri-bad`` from
233 :term:`WARN_QA`.
234
235- The ``file-rdeps`` runtime dependency check no longer expands
236 :term:`RDEPENDS` recursively as there is no mechanism
237 to ensure they can be fully computed, and thus races sometimes result
238 in errors either showing up or not. Thus, you might now see errors
239 for missing runtime dependencies that were previously satisfied
240 recursively. Here is an example: package A contains a shell script
241 starting with ``#!/bin/bash`` but has no dependency on bash. However,
242 package A depends on package B, which does depend on bash. You need
243 to add the missing dependency or dependencies to resolve the warning.
244
245- Setting ``DEPENDS_${PN}`` anywhere (i.e. typically in a recipe) now
246 triggers an error. The error is triggered because
247 :term:`DEPENDS` is not a package-specific variable
248 unlike RDEPENDS. You should set ``DEPENDS`` instead.
249
250- systemd currently does not work well with the musl C library because
251 only upstream officially supports linking the library with glibc.
252 Thus, a warning is shown when building systemd in conjunction with
253 musl.
254
255.. _migration-3.0-miscellaneous-changes:
256
257Miscellaneous Changes
258---------------------
259
260The following miscellaneous changes have occurred.
261
262- The ``gnome`` class has been removed because it now does very little.
263 You should update recipes that previously inherited this class to do
264 the following: inherit gnomebase gtk-icon-cache gconf mime
265
266- The ``meta/recipes-kernel/linux/linux-dtb.inc`` file has been
267 removed. This file was previously deprecated in favor of setting
268 :term:`KERNEL_DEVICETREE` in any kernel
269 recipe and only produced a warning. Remove any ``include`` or
270 ``require`` statements pointing to this file.
271
272- :term:`TARGET_CFLAGS`,
273 :term:`TARGET_CPPFLAGS`,
274 :term:`TARGET_CXXFLAGS`, and
275 :term:`TARGET_LDFLAGS` are no longer exported
276 to the external environment. This change did not require any changes
277 to core recipes, which is a good indicator that no changes will be
278 required. However, if for some reason the software being built by one
279 of your recipes is expecting these variables to be set, then building
280 the recipe will fail. In such cases, you must either export the
281 variable or variables in the recipe or change the scripts so that
282 exporting is not necessary.
283
284- You must change the host distro identifier used in
285 :term:`NATIVELSBSTRING` to use all lowercase
286 characters even if it does not contain a version number. This change
287 is necessary only if you are not using ``uninative`` and
288 :term:`SANITY_TESTED_DISTROS`.
289
290- In the ``base-files`` recipe, writing the hostname into
291 ``/etc/hosts`` and ``/etc/hostname`` is now done within the main
292 :ref:`ref-tasks-install` function rather than in the
293 ``do_install_basefilesissue`` function. The reason for the change is
294 because ``do_install_basefilesissue`` is more easily overridden
295 without having to duplicate the hostname functionality. If you have
296 done the latter (e.g. in a ``base-files`` bbappend), then you should
297 remove it from your customized ``do_install_basefilesissue``
298 function.
299
300- The ``wic --expand`` command now uses commas to separate "key:value"
301 pairs rather than hyphens.
302
303 .. note::
304
305 The wic command-line help is not updated.
306
307 You must update any scripts or commands where you use
308 ``wic --expand`` with multiple "key:value" pairs.
309
310- UEFI image variable settings have been moved from various places to a
311 central ``conf/image-uefi.conf``. This change should not influence
312 any existing configuration as the ``meta/conf/image-uefi.conf`` in
313 the core metadata sets defaults that can be overridden in the same
314 manner as before.
315
316- ``conf/distro/include/world-broken.inc`` has been removed. For cases
317 where certain recipes need to be disabled when using the musl C
318 library, these recipes now have ``COMPATIBLE_HOST_libc-musl`` set
319 with a comment that explains why.
320
321
diff --git a/documentation/ref-manual/migration-3.1.rst b/documentation/ref-manual/migration-3.1.rst
new file mode 100644
index 0000000000..92c8c77613
--- /dev/null
+++ b/documentation/ref-manual/migration-3.1.rst
@@ -0,0 +1,276 @@
1Moving to the Yocto Project 3.1 Release
2=======================================
3
4This section provides migration information for moving to the Yocto
5Project 3.1 Release from the prior release.
6
7.. _migration-3.1-minimum-system-requirements:
8
9Minimum system requirements
10---------------------------
11
12The following versions / requirements of build host components have been
13updated:
14
15- gcc 5.0
16
17- python 3.5
18
19- tar 1.28
20
21- ``rpcgen`` is now required on the host (part of the ``libc-dev-bin``
22 package on Ubuntu, Debian and related distributions, and the
23 ``glibc`` package on RPM-based distributions).
24
25Additionally, the ``makeinfo`` and ``pod2man`` tools are *no longer*
26required on the host.
27
28.. _migration-3.1-mpc8315e-rdb-removed:
29
30mpc8315e-rdb machine removed
31----------------------------
32
33The MPC8315E-RDB machine is old/obsolete and unobtainable, thus given
34the maintenance burden the ``mpc8315e-rdb`` machine configuration that
35supported it has been removed in this release. The removal does leave a
36gap in official PowerPC reference hardware support; this may change in
37future if a suitable machine with accompanying support resources is
38found.
39
40.. _migration-3.1-python-2-removed:
41
42Python 2 removed
43----------------
44
45Due to the expiration of upstream support in January 2020, support for
46Python 2 has now been removed; it is recommended that you use Python 3
47instead. If absolutely needed there is a meta-python2 community layer
48containing Python 2, related classes and various Python 2-based modules,
49however it should not be considered as supported.
50
51.. _migration-3.1-reproducible-builds:
52
53Reproducible builds now enabled by default
54------------------------------------------
55
56In order to avoid unnecessary differences in output files (aiding binary
57reproducibility), the Poky distribution configuration
58(``DISTRO = "poky"``) now inherits the ``reproducible_build`` class by
59default.
60
61.. _migration-3.1-ptest-feature-impact:
62
63Impact of ptest feature is now more significant
64-----------------------------------------------
65
66The Poky distribution configuration (``DISTRO = "poky"``) enables ptests
67by default to enable runtime testing of various components. In this
68release, a dependency needed to be added that has resulted in a
69significant increase in the number of components that will be built just
70when building a simple image such as core-image-minimal. If you do not
71need runtime tests enabled for core components, then it is recommended
72that you remove "ptest" from
73:term:`DISTRO_FEATURES` to save a significant
74amount of build time e.g. by adding the following in your configuration:
75::
76
77 DISTRO_FEATURES_remove = "ptest"
78
79.. _migration-3.1-removed-recipes:
80
81Removed recipes
82---------------
83
84The following recipes have been removed:
85
86- ``chkconfig``: obsolete
87
88- ``console-tools``: obsolete
89
90- ``enchant``: replaced by ``enchant2``
91
92- ``foomatic-filters``: obsolete
93
94- ``libidn``: no longer needed, moved to meta-oe
95
96- ``libmodulemd``: replaced by ``libmodulemd-v1``
97
98- ``linux-yocto``: drop 4.19, 5.2 version recipes (5.4 now provided)
99
100- ``nspr``: no longer needed, moved to meta-oe
101
102- ``nss``: no longer needed, moved to meta-oe
103
104- ``python``: Python 2 removed (Python 3 preferred)
105
106- ``python-setuptools``: Python 2 version removed (python3-setuptools
107 preferred)
108
109- ``sysprof``: no longer needed, moved to meta-oe
110
111- ``texi2html``: obsolete
112
113- ``u-boot-fw-utils``: functionally replaced by ``libubootenv``
114
115.. _migration-3.1-features-check:
116
117features_check class replaces distro_features_check
118---------------------------------------------------
119
120The ``distro_features_check`` class has had its functionality expanded,
121now supporting ``ANY_OF_MACHINE_FEATURES``,
122``REQUIRED_MACHINE_FEATURES``, ``CONFLICT_MACHINE_FEATURES``,
123``ANY_OF_COMBINED_FEATURES``, ``REQUIRED_COMBINED_FEATURES``,
124``CONFLICT_COMBINED_FEATURES``. As a result the class has now been
125renamed to ``features_check``; the ``distro_features_check`` class still
126exists but generates a warning and redirects to the new class. In
127preparation for a future removal of the old class it is recommended that
128you update recipes currently inheriting ``distro_features_check`` to
129inherit ``features_check`` instead.
130
131.. _migration-3.1-removed-classes:
132
133Removed classes
134---------------
135
136The following classes have been removed:
137
138- ``distutils-base``: moved to meta-python2
139
140- ``distutils``: moved to meta-python2
141
142- ``libc-common``: merged into the glibc recipe as nothing else used
143 it.
144
145- ``python-dir``: moved to meta-python2
146
147- ``pythonnative``: moved to meta-python2
148
149- ``setuptools``: moved to meta-python2
150
151- ``tinderclient``: dropped as it was obsolete.
152
153.. _migration-3.1-src-uri-checksums:
154
155SRC_URI checksum behaviour
156--------------------------
157
158Previously, recipes by tradition included both SHA256 and MD5 checksums
159for remotely fetched files in :term:`SRC_URI`, even
160though only one is actually mandated. However, the MD5 checksum does not
161add much given its inherent weakness; thus when a checksum fails only
162the SHA256 sum will now be printed. The md5sum will still be verified if
163it is specified.
164
165.. _migration-3.1-npm:
166
167npm fetcher changes
168-------------------
169
170The npm fetcher has been completely reworked in this release. The npm
171fetcher now only fetches the package source itself and no longer the
172dependencies; there is now also an npmsw fetcher which explicitly
173fetches the shrinkwrap file and the dependencies. This removes the
174slightly awkward ``NPM_LOCKDOWN`` and ``NPM_SHRINKWRAP`` variables which
175pointed to local files; the lockdown file is no longer needed at all.
176Additionally, the package name in ``npm://`` entries in
177:term:`SRC_URI` is now specified using a ``package``
178parameter instead of the earlier ``name`` which overlapped with the
179generic ``name`` parameter. All recipes using the npm fetcher will need
180to be changed as a result.
181
182An example of the new scheme: ::
183
184 SRC_URI = "npm://registry.npmjs.org;package=array-flatten;version=1.1.1 \\
185 npmsw://${THISDIR}/npm-shrinkwrap.json"
186
187Another example where the sources are fetched from git rather than an npm repository: ::
188
189 SRC_URI = "git://github.com/foo/bar.git;protocol=https \
190 npmsw://${THISDIR}/npm-shrinkwrap.json"
191
192devtool and recipetool have also been updated to match with the npm
193fetcher changes. Other than producing working and more complete recipes
194for npm sources, there is also a minor change to the command line for
195devtool: the ``--fetch-dev`` option has been renamed to ``--npm-dev`` as
196it is npm-specific.
197
198.. _migration-3.1-packaging-changes:
199
200Packaging changes
201-----------------
202
203- ``intltool`` has been removed from ``packagegroup-core-sdk`` as it is
204 rarely needed to build modern software - gettext can do most of the
205 things it used to be needed for. ``intltool`` has also been removed
206 from ``packagegroup-core-self-hosted`` as it is not needed to for
207 standard builds.
208
209- git: ``git-am``, ``git-difftool``, ``git-submodule``, and
210 ``git-request-pull`` are no longer perl-based, so are now installed
211 with the main ``git`` package instead of within ``git-perltools``.
212
213- The ``ldconfig`` binary built as part of glibc has now been moved to
214 its own ``ldconfig`` package (note no ``glibc-`` prefix). This
215 package is in the :term:`RRECOMMENDS` of the main
216 ``glibc`` package if ``ldconfig`` is present in
217 :term:`DISTRO_FEATURES`.
218
219- ``libevent`` now splits each shared library into its own package (as
220 Debian does). Since these are shared libraries and will be pulled in
221 through the normal shared library dependency handling, there should
222 be no impact to existing configurations other than less unnecessary
223 libraries being installed in some cases.
224
225- linux-firmware now has a new package for ``bcm4366c`` and includes
226 available NVRAM config files into the ``bcm43340``, ``bcm43362``,
227 ``bcm43430`` and ``bcm4356-pcie`` packages.
228
229- ``harfbuzz`` now splits the new ``libharfbuzz-subset.so`` library
230 into its own package to reduce the main package size in cases where
231 ``libharfbuzz-subset.so`` is not needed.
232
233.. _migration-3.1-package-qa-warnings:
234
235Additional warnings
236-------------------
237
238Warnings will now be shown at ``do_package_qa`` time in the following
239circumstances:
240
241- A recipe installs ``.desktop`` files containing ``MimeType`` keys but
242 does not inherit the new ``mime-xdg`` class
243
244- A recipe installs ``.xml`` files into ``${datadir}/mime/packages``
245 but does not inherit the ``mime`` class
246
247.. _migration-3.1-x86-live-wic:
248
249``wic`` image type now used instead of ``live`` by default for x86
250------------------------------------------------------------------
251
252``conf/machine/include/x86-base.inc`` (inherited by most x86 machine
253configurations) now specifies ``wic`` instead of ``live`` by default in
254:term:`IMAGE_FSTYPES`. The ``live`` image type will
255likely be removed in a future release so it is recommended that you use
256``wic`` instead.
257
258.. _migration-3.1-misc:
259
260Miscellaneous changes
261---------------------
262
263- The undocumented ``SRC_DISTRIBUTE_LICENSES`` variable has now been
264 removed in favour of a new ``AVAILABLE_LICENSES`` variable which is
265 dynamically set based upon license files found in
266 ``${COMMON_LICENSE_DIR}`` and ``${LICENSE_PATH}``.
267
268- The tune definition for big-endian microblaze machines is now
269 ``microblaze`` instead of ``microblazeeb``.
270
271- ``newlib`` no longer has built-in syscalls. ``libgloss`` should then
272 provide the syscalls, ``crt0.o`` and other functions that are no
273 longer part of ``newlib`` itself. If you are using
274 ``TCLIBC = "newlib"`` this now means that you must link applications
275 with both ``newlib`` and ``libgloss``, whereas before ``newlib``
276 would run in many configurations by itself.
diff --git a/documentation/ref-manual/migration-general.rst b/documentation/ref-manual/migration-general.rst
new file mode 100644
index 0000000000..182482ec43
--- /dev/null
+++ b/documentation/ref-manual/migration-general.rst
@@ -0,0 +1,54 @@
1General Migration Considerations
2================================
3
4Some considerations are not tied to a specific Yocto Project release.
5This section presents information you should consider when migrating to
6any new Yocto Project release.
7
8- *Dealing with Customized Recipes*:
9
10 Issues could arise if you take
11 older recipes that contain customizations and simply copy them
12 forward expecting them to work after you migrate to new Yocto Project
13 metadata. For example, suppose you have a recipe in your layer that
14 is a customized version of a core recipe copied from the earlier
15 release, rather than through the use of an append file. When you
16 migrate to a newer version of Yocto Project, the metadata (e.g.
17 perhaps an include file used by the recipe) could have changed in a
18 way that would break the build. Say, for example, a function is
19 removed from an include file and the customized recipe tries to call
20 that function.
21
22 You could "forward-port" all your customizations in your recipe so
23 that everything works for the new release. However, this is not the
24 optimal solution as you would have to repeat this process with each
25 new release if changes occur that give rise to problems.
26
27 The better solution (where practical) is to use append files
28 (``*.bbappend``) to capture any customizations you want to make to a
29 recipe. Doing so, isolates your changes from the main recipe making
30 them much more manageable. However, sometimes it is not practical to
31 use an append file. A good example of this is when introducing a
32 newer or older version of a recipe in another layer.
33
34- *Updating Append Files*:
35
36 Since append files generally only contain
37 your customizations, they often do not need to be adjusted for new
38 releases. However, if the ``.bbappend`` file is specific to a
39 particular version of the recipe (i.e. its name does not use the %
40 wildcard) and the version of the recipe to which it is appending has
41 changed, then you will at a minimum need to rename the append file to
42 match the name of the recipe file. A mismatch between an append file
43 and its corresponding recipe file (``.bb``) will trigger an error
44 during parsing.
45
46 Depending on the type of customization the append file applies, other
47 incompatibilities might occur when you upgrade. For example, if your
48 append file applies a patch and the recipe to which it is appending
49 is updated to a newer version, the patch might no longer apply. If
50 this is the case and assuming the patch is still needed, you must
51 modify the patch file so that it does apply.
52
53
54
diff --git a/documentation/ref-manual/migration.rst b/documentation/ref-manual/migration.rst
new file mode 100644
index 0000000000..6c6119daec
--- /dev/null
+++ b/documentation/ref-manual/migration.rst
@@ -0,0 +1,30 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************************************
4Migrating to a Newer Yocto Project Release
5******************************************
6
7This chapter provides information you can use to migrate work to a newer
8Yocto Project release. You can find the same information in the release
9notes for a given release.
10
11.. toctree::
12
13 migration-general
14 migration-1.3
15 migration-1.4
16 migration-1.5
17 migration-1.6
18 migration-1.7
19 migration-1.8
20 migration-2.0
21 migration-2.1
22 migration-2.2
23 migration-2.3
24 migration-2.4
25 migration-2.5
26 migration-2.6
27 migration-2.7
28 migration-3.0
29 migration-3.1
30
diff --git a/documentation/ref-manual/migration.xml b/documentation/ref-manual/migration.xml
index affc8b90a7..d3d5b16bdd 100644
--- a/documentation/ref-manual/migration.xml
+++ b/documentation/ref-manual/migration.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='migration'> 6<chapter id='migration'>
6<title>Migrating to a Newer Yocto Project Release</title> 7<title>Migrating to a Newer Yocto Project Release</title>
diff --git a/documentation/ref-manual/ref-classes.rst b/documentation/ref-manual/ref-classes.rst
new file mode 100644
index 0000000000..60ce8efd21
--- /dev/null
+++ b/documentation/ref-manual/ref-classes.rst
@@ -0,0 +1,2963 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******
4Classes
5*******
6
7Class files are used to abstract common functionality and share it
8amongst multiple recipe (``.bb``) files. To use a class file, you simply
9make sure the recipe inherits the class. In most cases, when a recipe
10inherits a class it is enough to enable its features. There are cases,
11however, where in the recipe you might need to set variables or override
12some default behavior.
13
14Any :term:`Metadata` usually found in a recipe can also be
15placed in a class file. Class files are identified by the extension
16``.bbclass`` and are usually placed in a ``classes/`` directory beneath
17the ``meta*/`` directory found in the :term:`Source Directory`.
18Class files can also be pointed to by
19:term:`BUILDDIR` (e.g. ``build/``) in the same way as
20``.conf`` files in the ``conf`` directory. Class files are searched for
21in :term:`BBPATH` using the same method by which ``.conf``
22files are searched.
23
24This chapter discusses only the most useful and important classes. Other
25classes do exist within the ``meta/classes`` directory in the Source
26Directory. You can reference the ``.bbclass`` files directly for more
27information.
28
29.. _ref-classes-allarch:
30
31``allarch.bbclass``
32===================
33
34The ``allarch`` class is inherited by recipes that do not produce
35architecture-specific output. The class disables functionality that is
36normally needed for recipes that produce executable binaries (such as
37building the cross-compiler and a C library as pre-requisites, and
38splitting out of debug symbols during packaging).
39
40.. note::
41
42 Unlike some distro recipes (e.g. Debian), OpenEmbedded recipes that
43 produce packages that depend on tunings through use of the
44 :term:`RDEPENDS` and
45 :term:`TUNE_PKGARCH` variables, should never be
46 configured for all architectures using ``allarch``. This is the case
47 even if the recipes do not produce architecture-specific output.
48
49 Configuring such recipes for all architectures causes the
50 ```do_package_write_*`` tasks to
51 have different signatures for the machines with different tunings.
52 Additionally, unnecessary rebuilds occur every time an image for a
53 different ``MACHINE`` is built even when the recipe never changes.
54
55By default, all recipes inherit the :ref:`base <ref-classes-base>` and
56:ref:`package <ref-classes-package>` classes, which enable
57functionality needed for recipes that produce executable output. If your
58recipe, for example, only produces packages that contain configuration
59files, media files, or scripts (e.g. Python and Perl), then it should
60inherit the ``allarch`` class.
61
62.. _ref-classes-archiver:
63
64``archiver.bbclass``
65====================
66
67The ``archiver`` class supports releasing source code and other
68materials with the binaries.
69
70For more details on the source archiver, see the
71":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
72section in the Yocto Project Development Tasks Manual. You can also see
73the :term:`ARCHIVER_MODE` variable for information
74about the variable flags (varflags) that help control archive creation.
75
76.. _ref-classes-autotools:
77
78``autotools*.bbclass``
79======================
80
81The ``autotools*`` classes support Autotooled packages.
82
83The ``autoconf``, ``automake``, and ``libtool`` packages bring
84standardization. This class defines a set of tasks (e.g. ``configure``,
85``compile`` and so forth) that work for all Autotooled packages. It
86should usually be enough to define a few standard variables and then
87simply ``inherit autotools``. These classes can also work with software
88that emulates Autotools. For more information, see the
89":ref:`new-recipe-autotooled-package`" section
90in the Yocto Project Development Tasks Manual.
91
92By default, the ``autotools*`` classes use out-of-tree builds (i.e.
93``autotools.bbclass`` building with ``B != S``).
94
95If the software being built by a recipe does not support using
96out-of-tree builds, you should have the recipe inherit the
97``autotools-brokensep`` class. The ``autotools-brokensep`` class behaves
98the same as the ``autotools`` class but builds with :term:`B`
99== :term:`S`. This method is useful when out-of-tree build
100support is either not present or is broken.
101
102.. note::
103
104 It is recommended that out-of-tree support be fixed and used if at
105 all possible.
106
107It's useful to have some idea of how the tasks defined by the
108``autotools*`` classes work and what they do behind the scenes.
109
110- :ref:`ref-tasks-configure` - Regenerates the
111 configure script (using ``autoreconf``) and then launches it with a
112 standard set of arguments used during cross-compilation. You can pass
113 additional parameters to ``configure`` through the ``EXTRA_OECONF``
114 or :term:`PACKAGECONFIG_CONFARGS`
115 variables.
116
117- :ref:`ref-tasks-compile` - Runs ``make`` with
118 arguments that specify the compiler and linker. You can pass
119 additional arguments through the ``EXTRA_OEMAKE`` variable.
120
121- :ref:`ref-tasks-install` - Runs ``make install`` and
122 passes in ``${``\ :term:`D`\ ``}`` as ``DESTDIR``.
123
124.. _ref-classes-base:
125
126``base.bbclass``
127================
128
129The ``base`` class is special in that every ``.bb`` file implicitly
130inherits the class. This class contains definitions for standard basic
131tasks such as fetching, unpacking, configuring (empty by default),
132compiling (runs any ``Makefile`` present), installing (empty by default)
133and packaging (empty by default). These classes are often overridden or
134extended by other classes such as the
135:ref:`autotools <ref-classes-autotools>` class or the
136:ref:`package <ref-classes-package>` class.
137
138The class also contains some commonly used functions such as
139``oe_runmake``, which runs ``make`` with the arguments specified in
140:term:`EXTRA_OEMAKE` variable as well as the
141arguments passed directly to ``oe_runmake``.
142
143.. _ref-classes-bash-completion:
144
145``bash-completion.bbclass``
146===========================
147
148Sets up packaging and dependencies appropriate for recipes that build
149software that includes bash-completion data.
150
151.. _ref-classes-bin-package:
152
153``bin_package.bbclass``
154=======================
155
156The ``bin_package`` class is a helper class for recipes that extract the
157contents of a binary package (e.g. an RPM) and install those contents
158rather than building the binary from source. The binary package is
159extracted and new packages in the configured output package format are
160created. Extraction and installation of proprietary binaries is a good
161example use for this class.
162
163.. note::
164
165 For RPMs and other packages that do not contain a subdirectory, you
166 should specify an appropriate fetcher parameter to point to the
167 subdirectory. For example, if BitBake is using the Git fetcher (
168 git://
169 ), the "subpath" parameter limits the checkout to a specific subpath
170 of the tree. Here is an example where
171 ${BP}
172 is used so that the files are extracted into the subdirectory
173 expected by the default value of
174 S
175 :
176 ::
177
178 SRC_URI = "git://example.com/downloads/somepackage.rpm;subpath=${BP}"
179
180
181 See the "
182 Fetchers
183 " section in the BitBake User Manual for more information on
184 supported BitBake Fetchers.
185
186.. _ref-classes-binconfig:
187
188``binconfig.bbclass``
189=====================
190
191The ``binconfig`` class helps to correct paths in shell scripts.
192
193Before ``pkg-config`` had become widespread, libraries shipped shell
194scripts to give information about the libraries and include paths needed
195to build software (usually named ``LIBNAME-config``). This class assists
196any recipe using such scripts.
197
198During staging, the OpenEmbedded build system installs such scripts into
199the ``sysroots/`` directory. Inheriting this class results in all paths
200in these scripts being changed to point into the ``sysroots/`` directory
201so that all builds that use the script use the correct directories for
202the cross compiling layout. See the
203:term:`BINCONFIG_GLOB` variable for more
204information.
205
206.. _ref-classes-binconfig-disabled:
207
208``binconfig-disabled.bbclass``
209==============================
210
211An alternative version of the :ref:`binconfig <ref-classes-binconfig>`
212class, which disables binary configuration scripts by making them return
213an error in favor of using ``pkg-config`` to query the information. The
214scripts to be disabled should be specified using the
215:term:`BINCONFIG` variable within the recipe inheriting
216the class.
217
218.. _ref-classes-blacklist:
219
220``blacklist.bbclass``
221=====================
222
223The ``blacklist`` class prevents the OpenEmbedded build system from
224building specific recipes (blacklists them). To use this class, inherit
225the class globally and set :term:`PNBLACKLIST` for
226each recipe you wish to blacklist. Specify the :term:`PN`
227value as a variable flag (varflag) and provide a reason, which is
228reported, if the package is requested to be built as the value. For
229example, if you want to blacklist a recipe called "exoticware", you add
230the following to your ``local.conf`` or distribution configuration:
231::
232
233 INHERIT += "blacklist"
234 PNBLACKLIST[exoticware] = "Not supported by our organization."
235
236.. _ref-classes-buildhistory:
237
238``buildhistory.bbclass``
239========================
240
241The ``buildhistory`` class records a history of build output metadata,
242which can be used to detect possible regressions as well as used for
243analysis of the build output. For more information on using Build
244History, see the
245":ref:`dev-manual/dev-manual-common-tasks:maintaining build output quality`"
246section in the Yocto Project Development Tasks Manual.
247
248.. _ref-classes-buildstats:
249
250``buildstats.bbclass``
251======================
252
253The ``buildstats`` class records performance statistics about each task
254executed during the build (e.g. elapsed time, CPU usage, and I/O usage).
255
256When you use this class, the output goes into the
257:term:`BUILDSTATS_BASE` directory, which defaults
258to ``${TMPDIR}/buildstats/``. You can analyze the elapsed time using
259``scripts/pybootchartgui/pybootchartgui.py``, which produces a cascading
260chart of the entire build process and can be useful for highlighting
261bottlenecks.
262
263Collecting build statistics is enabled by default through the
264:term:`USER_CLASSES` variable from your
265``local.conf`` file. Consequently, you do not have to do anything to
266enable the class. However, if you want to disable the class, simply
267remove "buildstats" from the ``USER_CLASSES`` list.
268
269.. _ref-classes-buildstats-summary:
270
271``buildstats-summary.bbclass``
272==============================
273
274When inherited globally, prints statistics at the end of the build on
275sstate re-use. In order to function, this class requires the
276:ref:`buildstats <ref-classes-buildstats>` class be enabled.
277
278.. _ref-classes-ccache:
279
280``ccache.bbclass``
281==================
282
283The ``ccache`` class enables the C/C++ Compiler Cache for the build.
284This class is used to give a minor performance boost during the build.
285However, using the class can lead to unexpected side-effects. Thus, it
286is recommended that you do not use this class. See
287http://ccache.samba.org/ for information on the C/C++ Compiler
288Cache.
289
290.. _ref-classes-chrpath:
291
292``chrpath.bbclass``
293===================
294
295The ``chrpath`` class is a wrapper around the "chrpath" utility, which
296is used during the build process for ``nativesdk``, ``cross``, and
297``cross-canadian`` recipes to change ``RPATH`` records within binaries
298in order to make them relocatable.
299
300.. _ref-classes-clutter:
301
302``clutter.bbclass``
303===================
304
305The ``clutter`` class consolidates the major and minor version naming
306and other common items used by Clutter and related recipes.
307
308.. note::
309
310 Unlike some other classes related to specific libraries, recipes
311 building other software that uses Clutter do not need to inherit this
312 class unless they use the same recipe versioning scheme that the
313 Clutter and related recipes do.
314
315.. _ref-classes-cmake:
316
317``cmake.bbclass``
318=================
319
320The ``cmake`` class allows for recipes that need to build software using
321the `CMake <https://cmake.org/overview/>`__ build system. You can use
322the :term:`EXTRA_OECMAKE` variable to specify
323additional configuration options to be passed using the ``cmake``
324command line.
325
326On the occasion that you would be installing custom CMake toolchain
327files supplied by the application being built, you should install them
328to the preferred CMake Module directory: ``${D}${datadir}/cmake/``
329Modules during
330:ref:`ref-tasks-install`.
331
332.. _ref-classes-cml1:
333
334``cml1.bbclass``
335================
336
337The ``cml1`` class provides basic support for the Linux kernel style
338build configuration system.
339
340.. _ref-classes-compress_doc:
341
342``compress_doc.bbclass``
343========================
344
345Enables compression for man pages and info pages. This class is intended
346to be inherited globally. The default compression mechanism is gz (gzip)
347but you can select an alternative mechanism by setting the
348:term:`DOC_COMPRESS` variable.
349
350.. _ref-classes-copyleft_compliance:
351
352``copyleft_compliance.bbclass``
353===============================
354
355The ``copyleft_compliance`` class preserves source code for the purposes
356of license compliance. This class is an alternative to the ``archiver``
357class and is still used by some users even though it has been deprecated
358in favor of the :ref:`archiver <ref-classes-archiver>` class.
359
360.. _ref-classes-copyleft_filter:
361
362``copyleft_filter.bbclass``
363===========================
364
365A class used by the :ref:`archiver <ref-classes-archiver>` and
366:ref:`copyleft_compliance <ref-classes-copyleft_compliance>` classes
367for filtering licenses. The ``copyleft_filter`` class is an internal
368class and is not intended to be used directly.
369
370.. _ref-classes-core-image:
371
372``core-image.bbclass``
373======================
374
375The ``core-image`` class provides common definitions for the
376``core-image-*`` image recipes, such as support for additional
377:term:`IMAGE_FEATURES`.
378
379.. _ref-classes-cpan:
380
381``cpan*.bbclass``
382=================
383
384The ``cpan*`` classes support Perl modules.
385
386Recipes for Perl modules are simple. These recipes usually only need to
387point to the source's archive and then inherit the proper class file.
388Building is split into two methods depending on which method the module
389authors used.
390
391- Modules that use old ``Makefile.PL``-based build system require
392 ``cpan.bbclass`` in their recipes.
393
394- Modules that use ``Build.PL``-based build system require using
395 ``cpan_build.bbclass`` in their recipes.
396
397Both build methods inherit the ``cpan-base`` class for basic Perl
398support.
399
400.. _ref-classes-cross:
401
402``cross.bbclass``
403=================
404
405The ``cross`` class provides support for the recipes that build the
406cross-compilation tools.
407
408.. _ref-classes-cross-canadian:
409
410``cross-canadian.bbclass``
411==========================
412
413The ``cross-canadian`` class provides support for the recipes that build
414the Canadian Cross-compilation tools for SDKs. See the
415":ref:`overview-manual/overview-manual-concepts:cross-development toolchain generation`"
416section in the Yocto Project Overview and Concepts Manual for more
417discussion on these cross-compilation tools.
418
419.. _ref-classes-crosssdk:
420
421``crosssdk.bbclass``
422====================
423
424The ``crosssdk`` class provides support for the recipes that build the
425cross-compilation tools used for building SDKs. See the
426":ref:`overview-manual/overview-manual-concepts:cross-development toolchain generation`"
427section in the Yocto Project Overview and Concepts Manual for more
428discussion on these cross-compilation tools.
429
430.. _ref-classes-debian:
431
432``debian.bbclass``
433==================
434
435The ``debian`` class renames output packages so that they follow the
436Debian naming policy (i.e. ``glibc`` becomes ``libc6`` and
437``glibc-devel`` becomes ``libc6-dev``.) Renaming includes the library
438name and version as part of the package name.
439
440If a recipe creates packages for multiple libraries (shared object files
441of ``.so`` type), use the :term:`LEAD_SONAME`
442variable in the recipe to specify the library on which to apply the
443naming scheme.
444
445.. _ref-classes-deploy:
446
447``deploy.bbclass``
448==================
449
450The ``deploy`` class handles deploying files to the
451:term:`DEPLOY_DIR_IMAGE` directory. The main
452function of this class is to allow the deploy step to be accelerated by
453shared state. Recipes that inherit this class should define their own
454:ref:`ref-tasks-deploy` function to copy the files to be
455deployed to :term:`DEPLOYDIR`, and use ``addtask`` to
456add the task at the appropriate place, which is usually after
457:ref:`ref-tasks-compile` or
458:ref:`ref-tasks-install`. The class then takes care of
459staging the files from ``DEPLOYDIR`` to ``DEPLOY_DIR_IMAGE``.
460
461.. _ref-classes-devshell:
462
463``devshell.bbclass``
464====================
465
466The ``devshell`` class adds the ``do_devshell`` task. Distribution
467policy dictates whether to include this class. See the ":ref:`platdev-appdev-devshell`"
468section in the Yocto Project Development Tasks Manual for more
469information about using ``devshell``.
470
471.. _ref-classes-devupstream:
472
473``devupstream.bbclass``
474=======================
475
476The ``devupstream`` class uses
477:term:`BBCLASSEXTEND` to add a variant of the
478recipe that fetches from an alternative URI (e.g. Git) instead of a
479tarball. Following is an example:
480::
481
482 BBCLASSEXTEND = "devupstream:target"
483 SRC_URI_class-devupstream = "git://git.example.com/example"
484 SRCREV_class-devupstream = "abcd1234"
485
486Adding the above statements to your recipe creates a variant that has
487:term:`DEFAULT_PREFERENCE` set to "-1".
488Consequently, you need to select the variant of the recipe to use it.
489Any development-specific adjustments can be done by using the
490``class-devupstream`` override. Here is an example:
491::
492
493 DEPENDS_append_class-devupstream = " gperf-native"
494 do_configure_prepend_class-devupstream() {
495 touch ${S}/README
496 }
497
498The class
499currently only supports creating a development variant of the target
500recipe, not ``native`` or ``nativesdk`` variants.
501
502The ``BBCLASSEXTEND`` syntax (i.e. ``devupstream:target``) provides
503support for ``native`` and ``nativesdk`` variants. Consequently, this
504functionality can be added in a future release.
505
506Support for other version control systems such as Subversion is limited
507due to BitBake's automatic fetch dependencies (e.g.
508``subversion-native``).
509
510.. _ref-classes-distro_features_check:
511
512``distro_features_check.bbclass``
513=================================
514
515The ``distro_features_check`` class allows individual recipes to check
516for required and conflicting
517:term:`DISTRO_FEATURES`.
518
519This class provides support for the
520:term:`REQUIRED_DISTRO_FEATURES` and
521:term:`CONFLICT_DISTRO_FEATURES`
522variables. If any conditions specified in the recipe using the above
523variables are not met, the recipe will be skipped.
524
525.. _ref-classes-distutils:
526
527``distutils*.bbclass``
528======================
529
530The ``distutils*`` classes support recipes for Python version 2.x
531extensions, which are simple. These recipes usually only need to point
532to the source's archive and then inherit the proper class. Building is
533split into two methods depending on which method the module authors
534used.
535
536- Extensions that use an Autotools-based build system require Autotools
537 and the classes based on ``distutils`` in their recipes.
538
539- Extensions that use build systems based on ``distutils`` require the
540 ``distutils`` class in their recipes.
541
542- Extensions that use build systems based on ``setuptools`` require the
543 :ref:`setuptools <ref-classes-setuptools>` class in their recipes.
544
545The ``distutils-common-base`` class is required by some of the
546``distutils*`` classes to provide common Python2 support.
547
548.. _ref-classes-distutils3:
549
550``distutils3*.bbclass``
551=======================
552
553The ``distutils3*`` classes support recipes for Python version 3.x
554extensions, which are simple. These recipes usually only need to point
555to the source's archive and then inherit the proper class. Building is
556split into three methods depending on which method the module authors
557used.
558
559- Extensions that use an Autotools-based build system require Autotools
560 and ``distutils``-based classes in their recipes.
561
562- Extensions that use ``distutils``-based build systems require the
563 ``distutils`` class in their recipes.
564
565- Extensions that use build systems based on ``setuptools3`` require
566 the :ref:`setuptools3 <ref-classes-setuptools>` class in their
567 recipes.
568
569The ``distutils3*`` classes either inherit their corresponding
570``distutils*`` class or replicate them using a Python3 version instead
571(e.g. ``distutils3-base`` inherits ``distutils-common-base``, which is
572the same as ``distutils-base`` but inherits ``python3native`` instead of
573``pythonnative``).
574
575.. _ref-classes-externalsrc:
576
577``externalsrc.bbclass``
578=======================
579
580The ``externalsrc`` class supports building software from source code
581that is external to the OpenEmbedded build system. Building software
582from an external source tree means that the build system's normal fetch,
583unpack, and patch process is not used.
584
585By default, the OpenEmbedded build system uses the :term:`S`
586and :term:`B` variables to locate unpacked recipe source code
587and to build it, respectively. When your recipe inherits the
588``externalsrc`` class, you use the
589:term:`EXTERNALSRC` and
590:term:`EXTERNALSRC_BUILD` variables to
591ultimately define ``S`` and ``B``.
592
593By default, this class expects the source code to support recipe builds
594that use the :term:`B` variable to point to the directory in
595which the OpenEmbedded build system places the generated objects built
596from the recipes. By default, the ``B`` directory is set to the
597following, which is separate from the source directory (``S``):
598::
599
600 ${WORKDIR}/${BPN}/{PV}/
601
602See these variables for more information:
603:term:`WORKDIR`, :term:`BPN`, and
604:term:`PV`,
605
606For more information on the ``externalsrc`` class, see the comments in
607``meta/classes/externalsrc.bbclass`` in the :term:`Source Directory`.
608For information on how to use the
609``externalsrc`` class, see the
610":ref:`dev-manual/dev-manual-common-tasks:building software from an external source`"
611section in the Yocto Project Development Tasks Manual.
612
613.. _ref-classes-extrausers:
614
615``extrausers.bbclass``
616======================
617
618The ``extrausers`` class allows additional user and group configuration
619to be applied at the image level. Inheriting this class either globally
620or from an image recipe allows additional user and group operations to
621be performed using the
622:term:`EXTRA_USERS_PARAMS` variable.
623
624.. note::
625
626 The user and group operations added using the
627 extrausers
628 class are not tied to a specific recipe outside of the recipe for the
629 image. Thus, the operations can be performed across the image as a
630 whole. Use the
631 useradd
632 class to add user and group configuration to a specific recipe.
633
634Here is an example that uses this class in an image recipe:
635::
636
637 inherit extrausers
638 EXTRA_USERS_PARAMS = "\
639 useradd -p '' tester; \
640 groupadd developers; \
641 userdel nobody; \
642 groupdel -g video; \
643 groupmod -g 1020 developers; \
644 usermod -s /bin/sh tester; \
645 "
646
647Here is an example that adds two users named "tester-jim" and "tester-sue" and assigns
648passwords:
649::
650
651 inherit extrausers
652 EXTRA_USERS_PARAMS = "\
653 useradd -P tester01 tester-jim; \
654 useradd -P tester01 tester-sue; \
655 "
656
657Finally, here is an example that sets the root password to "1876*18":
658::
659
660 inherit extrausers
661 EXTRA_USERS_PARAMS = "\
662 usermod -P 1876*18 root; \
663 "
664
665.. _ref-classes-fontcache:
666
667``fontcache.bbclass``
668=====================
669
670The ``fontcache`` class generates the proper post-install and
671post-remove (postinst and postrm) scriptlets for font packages. These
672scriptlets call ``fc-cache`` (part of ``Fontconfig``) to add the fonts
673to the font information cache. Since the cache files are
674architecture-specific, ``fc-cache`` runs using QEMU if the postinst
675scriptlets need to be run on the build host during image creation.
676
677If the fonts being installed are in packages other than the main
678package, set :term:`FONT_PACKAGES` to specify the
679packages containing the fonts.
680
681.. _ref-classes-fs-uuid:
682
683``fs-uuid.bbclass``
684===================
685
686The ``fs-uuid`` class extracts UUID from
687``${``\ :term:`ROOTFS`\ ``}``, which must have been built
688by the time that this function gets called. The ``fs-uuid`` class only
689works on ``ext`` file systems and depends on ``tune2fs``.
690
691.. _ref-classes-gconf:
692
693``gconf.bbclass``
694=================
695
696The ``gconf`` class provides common functionality for recipes that need
697to install GConf schemas. The schemas will be put into a separate
698package (``${``\ :term:`PN`\ ``}-gconf``) that is created
699automatically when this class is inherited. This package uses the
700appropriate post-install and post-remove (postinst/postrm) scriptlets to
701register and unregister the schemas in the target image.
702
703.. _ref-classes-gettext:
704
705``gettext.bbclass``
706===================
707
708The ``gettext`` class provides support for building software that uses
709the GNU ``gettext`` internationalization and localization system. All
710recipes building software that use ``gettext`` should inherit this
711class.
712
713.. _ref-classes-gnomebase:
714
715``gnomebase.bbclass``
716=====================
717
718The ``gnomebase`` class is the base class for recipes that build
719software from the GNOME stack. This class sets
720:term:`SRC_URI` to download the source from the GNOME
721mirrors as well as extending :term:`FILES` with the typical
722GNOME installation paths.
723
724.. _ref-classes-gobject-introspection:
725
726``gobject-introspection.bbclass``
727=================================
728
729Provides support for recipes building software that supports GObject
730introspection. This functionality is only enabled if the
731"gobject-introspection-data" feature is in
732:term:`DISTRO_FEATURES` as well as
733"qemu-usermode" being in
734:term:`MACHINE_FEATURES`.
735
736.. note::
737
738 This functionality is backfilled by default and, if not applicable,
739 should be disabled through
740 DISTRO_FEATURES_BACKFILL_CONSIDERED
741 or
742 MACHINE_FEATURES_BACKFILL_CONSIDERED
743 , respectively.
744
745.. _ref-classes-grub-efi:
746
747``grub-efi.bbclass``
748====================
749
750The ``grub-efi`` class provides ``grub-efi``-specific functions for
751building bootable images.
752
753This class supports several variables:
754
755- :term:`INITRD`: Indicates list of filesystem images to
756 concatenate and use as an initial RAM disk (initrd) (optional).
757
758- :term:`ROOTFS`: Indicates a filesystem image to include
759 as the root filesystem (optional).
760
761- :term:`GRUB_GFXSERIAL`: Set this to "1" to have
762 graphics and serial in the boot menu.
763
764- :term:`LABELS`: A list of targets for the automatic
765 configuration.
766
767- :term:`APPEND`: An override list of append strings for
768 each ``LABEL``.
769
770- :term:`GRUB_OPTS`: Additional options to add to the
771 configuration (optional). Options are delimited using semi-colon
772 characters (``;``).
773
774- :term:`GRUB_TIMEOUT`: Timeout before executing
775 the default ``LABEL`` (optional).
776
777.. _ref-classes-gsettings:
778
779``gsettings.bbclass``
780=====================
781
782The ``gsettings`` class provides common functionality for recipes that
783need to install GSettings (glib) schemas. The schemas are assumed to be
784part of the main package. Appropriate post-install and post-remove
785(postinst/postrm) scriptlets are added to register and unregister the
786schemas in the target image.
787
788.. _ref-classes-gtk-doc:
789
790``gtk-doc.bbclass``
791===================
792
793The ``gtk-doc`` class is a helper class to pull in the appropriate
794``gtk-doc`` dependencies and disable ``gtk-doc``.
795
796.. _ref-classes-gtk-icon-cache:
797
798``gtk-icon-cache.bbclass``
799==========================
800
801The ``gtk-icon-cache`` class generates the proper post-install and
802post-remove (postinst/postrm) scriptlets for packages that use GTK+ and
803install icons. These scriptlets call ``gtk-update-icon-cache`` to add
804the fonts to GTK+'s icon cache. Since the cache files are
805architecture-specific, ``gtk-update-icon-cache`` is run using QEMU if
806the postinst scriptlets need to be run on the build host during image
807creation.
808
809.. _ref-classes-gtk-immodules-cache:
810
811``gtk-immodules-cache.bbclass``
812===============================
813
814The ``gtk-immodules-cache`` class generates the proper post-install and
815post-remove (postinst/postrm) scriptlets for packages that install GTK+
816input method modules for virtual keyboards. These scriptlets call
817``gtk-update-icon-cache`` to add the input method modules to the cache.
818Since the cache files are architecture-specific,
819``gtk-update-icon-cache`` is run using QEMU if the postinst scriptlets
820need to be run on the build host during image creation.
821
822If the input method modules being installed are in packages other than
823the main package, set
824:term:`GTKIMMODULES_PACKAGES` to specify
825the packages containing the modules.
826
827.. _ref-classes-gzipnative:
828
829``gzipnative.bbclass``
830======================
831
832The ``gzipnative`` class enables the use of different native versions of
833``gzip`` and ``pigz`` rather than the versions of these tools from the
834build host.
835
836.. _ref-classes-icecc:
837
838``icecc.bbclass``
839=================
840
841The ``icecc`` class supports
842`Icecream <https://github.com/icecc/icecream>`__, which facilitates
843taking compile jobs and distributing them among remote machines.
844
845The class stages directories with symlinks from ``gcc`` and ``g++`` to
846``icecc``, for both native and cross compilers. Depending on each
847configure or compile, the OpenEmbedded build system adds the directories
848at the head of the ``PATH`` list and then sets the ``ICECC_CXX`` and
849``ICEC_CC`` variables, which are the paths to the ``g++`` and ``gcc``
850compilers, respectively.
851
852For the cross compiler, the class creates a ``tar.gz`` file that
853contains the Yocto Project toolchain and sets ``ICECC_VERSION``, which
854is the version of the cross-compiler used in the cross-development
855toolchain, accordingly.
856
857The class handles all three different compile stages (i.e native
858,cross-kernel and target) and creates the necessary environment
859``tar.gz`` file to be used by the remote machines. The class also
860supports SDK generation.
861
862If :term:`ICECC_PATH` is not set in your
863``local.conf`` file, then the class tries to locate the ``icecc`` binary
864using ``which``. If :term:`ICECC_ENV_EXEC` is set
865in your ``local.conf`` file, the variable should point to the
866``icecc-create-env`` script provided by the user. If you do not point to
867a user-provided script, the build system uses the default script
868provided by the recipe ``icecc-create-env-native.bb``.
869
870.. note::
871
872 This script is a modified version and not the one that comes with
873 icecc.
874
875If you do not want the Icecream distributed compile support to apply to
876specific recipes or classes, you can effectively "blacklist" them by
877listing the recipes and classes using the
878:term:`ICECC_USER_PACKAGE_BL` and
879:term:`ICECC_USER_CLASS_BL`, variables,
880respectively, in your ``local.conf`` file. Doing so causes the
881OpenEmbedded build system to handle these compilations locally.
882
883Additionally, you can list recipes using the
884:term:`ICECC_USER_PACKAGE_WL` variable in
885your ``local.conf`` file to force ``icecc`` to be enabled for recipes
886using an empty :term:`PARALLEL_MAKE` variable.
887
888Inheriting the ``icecc`` class changes all sstate signatures.
889Consequently, if a development team has a dedicated build system that
890populates :term:`SSTATE_MIRRORS` and they want to
891reuse sstate from ``SSTATE_MIRRORS``, then all developers and the build
892system need to either inherit the ``icecc`` class or nobody should.
893
894At the distribution level, you can inherit the ``icecc`` class to be
895sure that all builders start with the same sstate signatures. After
896inheriting the class, you can then disable the feature by setting the
897:term:`ICECC_DISABLED` variable to "1" as follows:
898::
899
900 INHERIT_DISTRO_append = " icecc"
901 ICECC_DISABLED ??= "1"
902
903This practice
904makes sure everyone is using the same signatures but also requires
905individuals that do want to use Icecream to enable the feature
906individually as follows in your ``local.conf`` file:
907::
908
909 ICECC_DISABLED = ""
910
911.. _ref-classes-image:
912
913``image.bbclass``
914=================
915
916The ``image`` class helps support creating images in different formats.
917First, the root filesystem is created from packages using one of the
918``rootfs*.bbclass`` files (depending on the package format used) and
919then one or more image files are created.
920
921- The ``IMAGE_FSTYPES`` variable controls the types of images to
922 generate.
923
924- The ``IMAGE_INSTALL`` variable controls the list of packages to
925 install into the image.
926
927For information on customizing images, see the
928":ref:`usingpoky-extend-customimage`" section
929in the Yocto Project Development Tasks Manual. For information on how
930images are created, see the
931":ref:`images-dev-environment`" section in the
932Yocto Project Overview and Concpets Manual.
933
934.. _ref-classes-image-buildinfo:
935
936``image-buildinfo.bbclass``
937===========================
938
939The ``image-buildinfo`` class writes information to the target
940filesystem on ``/etc/build``.
941
942.. _ref-classes-image_types:
943
944``image_types.bbclass``
945=======================
946
947The ``image_types`` class defines all of the standard image output types
948that you can enable through the
949:term:`IMAGE_FSTYPES` variable. You can use this
950class as a reference on how to add support for custom image output
951types.
952
953By default, the :ref:`image <ref-classes-image>` class automatically
954enables the ``image_types`` class. The ``image`` class uses the
955``IMGCLASSES`` variable as follows:
956::
957
958 IMGCLASSES = "rootfs_${IMAGE_PKGTYPE} image_types ${IMAGE_CLASSES}"
959 IMGCLASSES += "${@['populate_sdk_base', 'populate_sdk_ext']['linux' in d.getVar("SDK_OS")]}"
960 IMGCLASSES += "${@bb.utils.contains_any('IMAGE_FSTYPES', 'live iso hddimg', 'image-live', '', d)}"
961 IMGCLASSES += "${@bb.utils.contains('IMAGE_FSTYPES', 'container', 'image-container', '', d)}"
962 IMGCLASSES += "image_types_wic"
963 IMGCLASSES += "rootfs-postcommands"
964 IMGCLASSES += "image-postinst-intercepts"
965 inherit ${IMGCLASSES}
966
967The ``image_types`` class also handles conversion and compression of images.
968
969.. note::
970
971 To build a VMware VMDK image, you need to add "wic.vmdk" to
972 IMAGE_FSTYPES
973 . This would also be similar for Virtual Box Virtual Disk Image
974 ("vdi") and QEMU Copy On Write Version 2 ("qcow2") images.
975
976.. _ref-classes-image-live:
977
978``image-live.bbclass``
979======================
980
981This class controls building "live" (i.e. HDDIMG and ISO) images. Live
982images contain syslinux for legacy booting, as well as the bootloader
983specified by :term:`EFI_PROVIDER` if
984:term:`MACHINE_FEATURES` contains "efi".
985
986Normally, you do not use this class directly. Instead, you add "live" to
987:term:`IMAGE_FSTYPES`.
988
989.. _ref-classes-image-mklibs:
990
991``image-mklibs.bbclass``
992========================
993
994The ``image-mklibs`` class enables the use of the ``mklibs`` utility
995during the :ref:`ref-tasks-rootfs` task, which optimizes
996the size of libraries contained in the image.
997
998By default, the class is enabled in the ``local.conf.template`` using
999the :term:`USER_CLASSES` variable as follows:
1000::
1001
1002 USER_CLASSES ?= "buildstats image-mklibs image-prelink"
1003
1004.. _ref-classes-image-prelink:
1005
1006``image-prelink.bbclass``
1007=========================
1008
1009The ``image-prelink`` class enables the use of the ``prelink`` utility
1010during the :ref:`ref-tasks-rootfs` task, which optimizes
1011the dynamic linking of shared libraries to reduce executable startup
1012time.
1013
1014By default, the class is enabled in the ``local.conf.template`` using
1015the :term:`USER_CLASSES` variable as follows:
1016::
1017
1018 USER_CLASSES ?= "buildstats image-mklibs image-prelink"
1019
1020.. _ref-classes-insane:
1021
1022``insane.bbclass``
1023==================
1024
1025The ``insane`` class adds a step to the package generation process so
1026that output quality assurance checks are generated by the OpenEmbedded
1027build system. A range of checks are performed that check the build's
1028output for common problems that show up during runtime. Distribution
1029policy usually dictates whether to include this class.
1030
1031You can configure the sanity checks so that specific test failures
1032either raise a warning or an error message. Typically, failures for new
1033tests generate a warning. Subsequent failures for the same test would
1034then generate an error message once the metadata is in a known and good
1035condition. See the "`QA Error and Warning Messages <#ref-qa-checks>`__"
1036Chapter for a list of all the warning and error messages you might
1037encounter using a default configuration.
1038
1039Use the :term:`WARN_QA` and
1040:term:`ERROR_QA` variables to control the behavior of
1041these checks at the global level (i.e. in your custom distro
1042configuration). However, to skip one or more checks in recipes, you
1043should use :term:`INSANE_SKIP`. For example, to skip
1044the check for symbolic link ``.so`` files in the main package of a
1045recipe, add the following to the recipe. You need to realize that the
1046package name override, in this example ``${PN}``, must be used:
1047::
1048
1049 INSANE_SKIP_${PN} += "dev-so"
1050
1051Please keep in mind that the QA checks
1052exist in order to detect real or potential problems in the packaged
1053output. So exercise caution when disabling these checks.
1054
1055The following list shows the tests you can list with the ``WARN_QA`` and
1056``ERROR_QA`` variables:
1057
1058- ``already-stripped:`` Checks that produced binaries have not
1059 already been stripped prior to the build system extracting debug
1060 symbols. It is common for upstream software projects to default to
1061 stripping debug symbols for output binaries. In order for debugging
1062 to work on the target using ``-dbg`` packages, this stripping must be
1063 disabled.
1064
1065- ``arch:`` Checks the Executable and Linkable Format (ELF) type, bit
1066 size, and endianness of any binaries to ensure they match the target
1067 architecture. This test fails if any binaries do not match the type
1068 since there would be an incompatibility. The test could indicate that
1069 the wrong compiler or compiler options have been used. Sometimes
1070 software, like bootloaders, might need to bypass this check.
1071
1072- ``buildpaths:`` Checks for paths to locations on the build host
1073 inside the output files. Currently, this test triggers too many false
1074 positives and thus is not normally enabled.
1075
1076- ``build-deps:`` Determines if a build-time dependency that is
1077 specified through :term:`DEPENDS`, explicit
1078 :term:`RDEPENDS`, or task-level dependencies exists
1079 to match any runtime dependency. This determination is particularly
1080 useful to discover where runtime dependencies are detected and added
1081 during packaging. If no explicit dependency has been specified within
1082 the metadata, at the packaging stage it is too late to ensure that
1083 the dependency is built, and thus you can end up with an error when
1084 the package is installed into the image during the
1085 :ref:`ref-tasks-rootfs` task because the auto-detected
1086 dependency was not satisfied. An example of this would be where the
1087 :ref:`update-rc.d <ref-classes-update-rc.d>` class automatically
1088 adds a dependency on the ``initscripts-functions`` package to
1089 packages that install an initscript that refers to
1090 ``/etc/init.d/functions``. The recipe should really have an explicit
1091 ``RDEPENDS`` for the package in question on ``initscripts-functions``
1092 so that the OpenEmbedded build system is able to ensure that the
1093 ``initscripts`` recipe is actually built and thus the
1094 ``initscripts-functions`` package is made available.
1095
1096- ``compile-host-path:`` Checks the
1097 :ref:`ref-tasks-compile` log for indications that
1098 paths to locations on the build host were used. Using such paths
1099 might result in host contamination of the build output.
1100
1101- ``debug-deps:`` Checks that all packages except ``-dbg`` packages
1102 do not depend on ``-dbg`` packages, which would cause a packaging
1103 bug.
1104
1105- ``debug-files:`` Checks for ``.debug`` directories in anything but
1106 the ``-dbg`` package. The debug files should all be in the ``-dbg``
1107 package. Thus, anything packaged elsewhere is incorrect packaging.
1108
1109- ``dep-cmp:`` Checks for invalid version comparison statements in
1110 runtime dependency relationships between packages (i.e. in
1111 :term:`RDEPENDS`,
1112 :term:`RRECOMMENDS`,
1113 :term:`RSUGGESTS`,
1114 :term:`RPROVIDES`,
1115 :term:`RREPLACES`, and
1116 :term:`RCONFLICTS` variable values). Any invalid
1117 comparisons might trigger failures or undesirable behavior when
1118 passed to the package manager.
1119
1120- ``desktop:`` Runs the ``desktop-file-validate`` program against any
1121 ``.desktop`` files to validate their contents against the
1122 specification for ``.desktop`` files.
1123
1124- ``dev-deps:`` Checks that all packages except ``-dev`` or
1125 ``-staticdev`` packages do not depend on ``-dev`` packages, which
1126 would be a packaging bug.
1127
1128- ``dev-so:`` Checks that the ``.so`` symbolic links are in the
1129 ``-dev`` package and not in any of the other packages. In general,
1130 these symlinks are only useful for development purposes. Thus, the
1131 ``-dev`` package is the correct location for them. Some very rare
1132 cases do exist for dynamically loaded modules where these symlinks
1133 are needed instead in the main package.
1134
1135- ``file-rdeps:`` Checks that file-level dependencies identified by
1136 the OpenEmbedded build system at packaging time are satisfied. For
1137 example, a shell script might start with the line ``#!/bin/bash``.
1138 This line would translate to a file dependency on ``/bin/bash``. Of
1139 the three package managers that the OpenEmbedded build system
1140 supports, only RPM directly handles file-level dependencies,
1141 resolving them automatically to packages providing the files.
1142 However, the lack of that functionality in the other two package
1143 managers does not mean the dependencies do not still need resolving.
1144 This QA check attempts to ensure that explicitly declared
1145 :term:`RDEPENDS` exist to handle any file-level
1146 dependency detected in packaged files.
1147
1148- ``files-invalid:`` Checks for :term:`FILES` variable
1149 values that contain "//", which is invalid.
1150
1151- ``host-user-contaminated:`` Checks that no package produced by the
1152 recipe contains any files outside of ``/home`` with a user or group
1153 ID that matches the user running BitBake. A match usually indicates
1154 that the files are being installed with an incorrect UID/GID, since
1155 target IDs are independent from host IDs. For additional information,
1156 see the section describing the
1157 :ref:`ref-tasks-install` task.
1158
1159- ``incompatible-license:`` Report when packages are excluded from
1160 being created due to being marked with a license that is in
1161 :term:`INCOMPATIBLE_LICENSE`.
1162
1163- ``install-host-path:`` Checks the
1164 :ref:`ref-tasks-install` log for indications that
1165 paths to locations on the build host were used. Using such paths
1166 might result in host contamination of the build output.
1167
1168- ``installed-vs-shipped:`` Reports when files have been installed
1169 within ``do_install`` but have not been included in any package by
1170 way of the :term:`FILES` variable. Files that do not
1171 appear in any package cannot be present in an image later on in the
1172 build process. Ideally, all installed files should be packaged or not
1173 installed at all. These files can be deleted at the end of
1174 ``do_install`` if the files are not needed in any package.
1175
1176- ``invalid-chars:`` Checks that the recipe metadata variables
1177 :term:`DESCRIPTION`,
1178 :term:`SUMMARY`, :term:`LICENSE`, and
1179 :term:`SECTION` do not contain non-UTF-8 characters.
1180 Some package managers do not support such characters.
1181
1182- ``invalid-packageconfig:`` Checks that no undefined features are
1183 being added to :term:`PACKAGECONFIG`. For
1184 example, any name "foo" for which the following form does not exist:
1185 ::
1186
1187 PACKAGECONFIG[foo] = "..."
1188
1189- ``la:`` Checks ``.la`` files for any ``TMPDIR`` paths. Any ``.la``
1190 file containing these paths is incorrect since ``libtool`` adds the
1191 correct sysroot prefix when using the files automatically itself.
1192
1193- ``ldflags:`` Ensures that the binaries were linked with the
1194 :term:`LDFLAGS` options provided by the build system.
1195 If this test fails, check that the ``LDFLAGS`` variable is being
1196 passed to the linker command.
1197
1198- ``libdir:`` Checks for libraries being installed into incorrect
1199 (possibly hardcoded) installation paths. For example, this test will
1200 catch recipes that install ``/lib/bar.so`` when ``${base_libdir}`` is
1201 "lib32". Another example is when recipes install
1202 ``/usr/lib64/foo.so`` when ``${libdir}`` is "/usr/lib".
1203
1204- ``libexec:`` Checks if a package contains files in
1205 ``/usr/libexec``. This check is not performed if the ``libexecdir``
1206 variable has been set explicitly to ``/usr/libexec``.
1207
1208- ``packages-list:`` Checks for the same package being listed
1209 multiple times through the :term:`PACKAGES` variable
1210 value. Installing the package in this manner can cause errors during
1211 packaging.
1212
1213- ``perm-config:`` Reports lines in ``fs-perms.txt`` that have an
1214 invalid format.
1215
1216- ``perm-line:`` Reports lines in ``fs-perms.txt`` that have an
1217 invalid format.
1218
1219- ``perm-link:`` Reports lines in ``fs-perms.txt`` that specify
1220 'link' where the specified target already exists.
1221
1222- ``perms:`` Currently, this check is unused but reserved.
1223
1224- ``pkgconfig:`` Checks ``.pc`` files for any
1225 :term:`TMPDIR`/:term:`WORKDIR` paths.
1226 Any ``.pc`` file containing these paths is incorrect since
1227 ``pkg-config`` itself adds the correct sysroot prefix when the files
1228 are accessed.
1229
1230- ``pkgname:`` Checks that all packages in
1231 :term:`PACKAGES` have names that do not contain
1232 invalid characters (i.e. characters other than 0-9, a-z, ., +, and
1233 -).
1234
1235- ``pkgv-undefined:`` Checks to see if the ``PKGV`` variable is
1236 undefined during :ref:`ref-tasks-package`.
1237
1238- ``pkgvarcheck:`` Checks through the variables
1239 :term:`RDEPENDS`,
1240 :term:`RRECOMMENDS`,
1241 :term:`RSUGGESTS`,
1242 :term:`RCONFLICTS`,
1243 :term:`RPROVIDES`,
1244 :term:`RREPLACES`, :term:`FILES`,
1245 :term:`ALLOW_EMPTY`, ``pkg_preinst``,
1246 ``pkg_postinst``, ``pkg_prerm`` and ``pkg_postrm``, and reports if
1247 there are variable sets that are not package-specific. Using these
1248 variables without a package suffix is bad practice, and might
1249 unnecessarily complicate dependencies of other packages within the
1250 same recipe or have other unintended consequences.
1251
1252- ``pn-overrides:`` Checks that a recipe does not have a name
1253 (:term:`PN`) value that appears in
1254 :term:`OVERRIDES`. If a recipe is named such that
1255 its ``PN`` value matches something already in ``OVERRIDES`` (e.g.
1256 ``PN`` happens to be the same as :term:`MACHINE` or
1257 :term:`DISTRO`), it can have unexpected consequences.
1258 For example, assignments such as ``FILES_${PN} = "xyz"`` effectively
1259 turn into ``FILES = "xyz"``.
1260
1261- ``rpaths:`` Checks for rpaths in the binaries that contain build
1262 system paths such as ``TMPDIR``. If this test fails, bad ``-rpath``
1263 options are being passed to the linker commands and your binaries
1264 have potential security issues.
1265
1266- ``split-strip:`` Reports that splitting or stripping debug symbols
1267 from binaries has failed.
1268
1269- ``staticdev:`` Checks for static library files (``*.a``) in
1270 non-``staticdev`` packages.
1271
1272- ``symlink-to-sysroot:`` Checks for symlinks in packages that point
1273 into :term:`TMPDIR` on the host. Such symlinks will
1274 work on the host, but are clearly invalid when running on the target.
1275
1276- ``textrel:`` Checks for ELF binaries that contain relocations in
1277 their ``.text`` sections, which can result in a performance impact at
1278 runtime. See the explanation for the
1279 ```ELF binary`` <#qa-issue-textrel>`__ message for more information
1280 regarding runtime performance issues.
1281
1282- ``unlisted-pkg-lics:`` Checks that all declared licenses applying
1283 for a package are also declared on the recipe level (i.e. any license
1284 in ``LICENSE_*`` should appear in :term:`LICENSE`).
1285
1286- ``useless-rpaths:`` Checks for dynamic library load paths (rpaths)
1287 in the binaries that by default on a standard system are searched by
1288 the linker (e.g. ``/lib`` and ``/usr/lib``). While these paths will
1289 not cause any breakage, they do waste space and are unnecessary.
1290
1291- ``var-undefined:`` Reports when variables fundamental to packaging
1292 (i.e. :term:`WORKDIR`,
1293 :term:`DEPLOY_DIR`, :term:`D`,
1294 :term:`PN`, and :term:`PKGD`) are undefined
1295 during :ref:`ref-tasks-package`.
1296
1297- ``version-going-backwards:`` If Build History is enabled, reports
1298 when a package being written out has a lower version than the
1299 previously written package under the same name. If you are placing
1300 output packages into a feed and upgrading packages on a target system
1301 using that feed, the version of a package going backwards can result
1302 in the target system not correctly upgrading to the "new" version of
1303 the package.
1304
1305 .. note::
1306
1307 If you are not using runtime package management on your target
1308 system, then you do not need to worry about this situation.
1309
1310- ``xorg-driver-abi:`` Checks that all packages containing Xorg
1311 drivers have ABI dependencies. The ``xserver-xorg`` recipe provides
1312 driver ABI names. All drivers should depend on the ABI versions that
1313 they have been built against. Driver recipes that include
1314 ``xorg-driver-input.inc`` or ``xorg-driver-video.inc`` will
1315 automatically get these versions. Consequently, you should only need
1316 to explicitly add dependencies to binary driver recipes.
1317
1318.. _ref-classes-insserv:
1319
1320``insserv.bbclass``
1321===================
1322
1323The ``insserv`` class uses the ``insserv`` utility to update the order
1324of symbolic links in ``/etc/rc?.d/`` within an image based on
1325dependencies specified by LSB headers in the ``init.d`` scripts
1326themselves.
1327
1328.. _ref-classes-kernel:
1329
1330``kernel.bbclass``
1331==================
1332
1333The ``kernel`` class handles building Linux kernels. The class contains
1334code to build all kernel trees. All needed headers are staged into the
1335``STAGING_KERNEL_DIR`` directory to allow out-of-tree module builds
1336using the :ref:`module <ref-classes-module>` class.
1337
1338This means that each built kernel module is packaged separately and
1339inter-module dependencies are created by parsing the ``modinfo`` output.
1340If all modules are required, then installing the ``kernel-modules``
1341package installs all packages with modules and various other kernel
1342packages such as ``kernel-vmlinux``.
1343
1344The ``kernel`` class contains logic that allows you to embed an initial
1345RAM filesystem (initramfs) image when you build the kernel image. For
1346information on how to build an initramfs, see the
1347":ref:`building-an-initramfs-image`" section in
1348the Yocto Project Development Tasks Manual.
1349
1350Various other classes are used by the ``kernel`` and ``module`` classes
1351internally including the :ref:`kernel-arch <ref-classes-kernel-arch>`,
1352:ref:`module-base <ref-classes-module-base>`, and
1353:ref:`linux-kernel-base <ref-classes-linux-kernel-base>` classes.
1354
1355.. _ref-classes-kernel-arch:
1356
1357``kernel-arch.bbclass``
1358=======================
1359
1360The ``kernel-arch`` class sets the ``ARCH`` environment variable for
1361Linux kernel compilation (including modules).
1362
1363.. _ref-classes-kernel-devicetree:
1364
1365``kernel-devicetree.bbclass``
1366=============================
1367
1368The ``kernel-devicetree`` class, which is inherited by the
1369:ref:`kernel <ref-classes-kernel>` class, supports device tree
1370generation.
1371
1372.. _ref-classes-kernel-fitimage:
1373
1374``kernel-fitimage.bbclass``
1375===========================
1376
1377The ``kernel-fitimage`` class provides support to pack a kernel Image,
1378device trees and a RAM disk into a single FIT image. In theory, a FIT
1379image can support any number of kernels, RAM disks and device-trees.
1380However, ``kernel-fitimage`` currently only supports
1381limited usescases: just one kernel image, an optional RAM disk, and
1382any number of device tree.
1383
1384To create a FIT image, it is required that :term:`KERNEL_CLASSES`
1385is set to "kernel-fitimage" and :term:`KERNEL_IMAGETYPE`
1386is set to "fitImage".
1387
1388The options for the device tree compiler passed to mkimage -D feature
1389when creating the FIT image are specified using the
1390:term:`UBOOT_MKIMAGE_DTCOPTS` variable.
1391
1392Only a single kernel can be added to the FIT image created by
1393``kernel-fitimage`` and the kernel image in FIT is mandatory. The
1394address where the kernel image is to be loaded by U-boot is
1395specified by :term:`UBOOT_LOADADDRESS` and the entrypoint by
1396:term:`UBOOT_ENTRYPOINT`.
1397
1398Multiple device trees can be added to the FIT image created by
1399``kernel-fitimage`` and the device tree is optional.
1400The address where the device tree is to be loaded by U-boot is
1401specified by :term:`UBOOT_DTBO_LOADADDRESS` for device tree overlays
1402and by `:term:`UBOOT_DTB_LOADADDRESS` for device tree binaries.
1403
1404Only a single RAM disk can be added to the FIT image created by
1405``kernel-fitimage`` and the RAM disk in FIT is optional.
1406The address where the RAM disk image is to be loaded by U-boot
1407is specified by :term:`UBOOT_RD_LOADADDRESS` and the entrypoint by
1408:term:`UBOOT_RD_ENTRYPOINT`. The ramdisk is added to FIT image when
1409:term:`INITRAMFS_IMAGE` is specified.
1410
1411The FIT image generated by ``kernel-fitimage`` class is signed when the
1412variables :term:`UBOOT_SIGN_ENABLE`, :term:`UBOOT_MKIMAGE_DTCOPTS`,
1413:term:`UBOOT_SIGN_KEYDIR` and :term:`UBOOT_SIGN_KEYNAME` are set
1414appropriately. The default values used for :term:`FIT_HASH_ALG` and
1415:term:`FIT_SIGN_ALG` in ``kernel-fitimage`` are "sha256" and
1416"rsa2048" respectively.
1417
1418
1419.. _ref-classes-kernel-grub:
1420
1421``kernel-grub.bbclass``
1422=======================
1423
1424The ``kernel-grub`` class updates the boot area and the boot menu with
1425the kernel as the priority boot mechanism while installing a RPM to
1426update the kernel on a deployed target.
1427
1428.. _ref-classes-kernel-module-split:
1429
1430``kernel-module-split.bbclass``
1431===============================
1432
1433The ``kernel-module-split`` class provides common functionality for
1434splitting Linux kernel modules into separate packages.
1435
1436.. _ref-classes-kernel-uboot:
1437
1438``kernel-uboot.bbclass``
1439========================
1440
1441The ``kernel-uboot`` class provides support for building from
1442vmlinux-style kernel sources.
1443
1444.. _ref-classes-kernel-uimage:
1445
1446``kernel-uimage.bbclass``
1447=========================
1448
1449The ``kernel-uimage`` class provides support to pack uImage.
1450
1451.. _ref-classes-kernel-yocto:
1452
1453``kernel-yocto.bbclass``
1454========================
1455
1456The ``kernel-yocto`` class provides common functionality for building
1457from linux-yocto style kernel source repositories.
1458
1459.. _ref-classes-kernelsrc:
1460
1461``kernelsrc.bbclass``
1462=====================
1463
1464The ``kernelsrc`` class sets the Linux kernel source and version.
1465
1466.. _ref-classes-lib_package:
1467
1468``lib_package.bbclass``
1469=======================
1470
1471The ``lib_package`` class supports recipes that build libraries and
1472produce executable binaries, where those binaries should not be
1473installed by default along with the library. Instead, the binaries are
1474added to a separate ``${``\ :term:`PN`\ ``}-bin`` package to
1475make their installation optional.
1476
1477.. _ref-classes-libc*:
1478
1479``libc*.bbclass``
1480=================
1481
1482The ``libc*`` classes support recipes that build packages with ``libc``:
1483
1484- The ``libc-common`` class provides common support for building with
1485 ``libc``.
1486
1487- The ``libc-package`` class supports packaging up ``glibc`` and
1488 ``eglibc``.
1489
1490.. _ref-classes-license:
1491
1492``license.bbclass``
1493===================
1494
1495The ``license`` class provides license manifest creation and license
1496exclusion. This class is enabled by default using the default value for
1497the :term:`INHERIT_DISTRO` variable.
1498
1499.. _ref-classes-linux-kernel-base:
1500
1501``linux-kernel-base.bbclass``
1502=============================
1503
1504The ``linux-kernel-base`` class provides common functionality for
1505recipes that build out of the Linux kernel source tree. These builds
1506goes beyond the kernel itself. For example, the Perf recipe also
1507inherits this class.
1508
1509.. _ref-classes-linuxloader:
1510
1511``linuxloader.bbclass``
1512=======================
1513
1514Provides the function ``linuxloader()``, which gives the value of the
1515dynamic loader/linker provided on the platform. This value is used by a
1516number of other classes.
1517
1518.. _ref-classes-logging:
1519
1520``logging.bbclass``
1521===================
1522
1523The ``logging`` class provides the standard shell functions used to log
1524messages for various BitBake severity levels (i.e. ``bbplain``,
1525``bbnote``, ``bbwarn``, ``bberror``, ``bbfatal``, and ``bbdebug``).
1526
1527This class is enabled by default since it is inherited by the ``base``
1528class.
1529
1530.. _ref-classes-meta:
1531
1532``meta.bbclass``
1533================
1534
1535The ``meta`` class is inherited by recipes that do not build any output
1536packages themselves, but act as a "meta" target for building other
1537recipes.
1538
1539.. _ref-classes-metadata_scm:
1540
1541``metadata_scm.bbclass``
1542========================
1543
1544The ``metadata_scm`` class provides functionality for querying the
1545branch and revision of a Source Code Manager (SCM) repository.
1546
1547The :ref:`base <ref-classes-base>` class uses this class to print the
1548revisions of each layer before starting every build. The
1549``metadata_scm`` class is enabled by default because it is inherited by
1550the ``base`` class.
1551
1552.. _ref-classes-migrate_localcount:
1553
1554``migrate_localcount.bbclass``
1555==============================
1556
1557The ``migrate_localcount`` class verifies a recipe's localcount data and
1558increments it appropriately.
1559
1560.. _ref-classes-mime:
1561
1562``mime.bbclass``
1563================
1564
1565The ``mime`` class generates the proper post-install and post-remove
1566(postinst/postrm) scriptlets for packages that install MIME type files.
1567These scriptlets call ``update-mime-database`` to add the MIME types to
1568the shared database.
1569
1570.. _ref-classes-mirrors:
1571
1572``mirrors.bbclass``
1573===================
1574
1575The ``mirrors`` class sets up some standard
1576:term:`MIRRORS` entries for source code mirrors. These
1577mirrors provide a fall-back path in case the upstream source specified
1578in :term:`SRC_URI` within recipes is unavailable.
1579
1580This class is enabled by default since it is inherited by the
1581:ref:`base <ref-classes-base>` class.
1582
1583.. _ref-classes-module:
1584
1585``module.bbclass``
1586==================
1587
1588The ``module`` class provides support for building out-of-tree Linux
1589kernel modules. The class inherits the
1590:ref:`module-base <ref-classes-module-base>` and
1591:ref:`kernel-module-split <ref-classes-kernel-module-split>` classes,
1592and implements the :ref:`ref-tasks-compile` and
1593:ref:`ref-tasks-install` tasks. The class provides
1594everything needed to build and package a kernel module.
1595
1596For general information on out-of-tree Linux kernel modules, see the
1597":ref:`kernel-dev/kernel-dev-common:incorporating out-of-tree modules`"
1598section in the Yocto Project Linux Kernel Development Manual.
1599
1600.. _ref-classes-module-base:
1601
1602``module-base.bbclass``
1603=======================
1604
1605The ``module-base`` class provides the base functionality for building
1606Linux kernel modules. Typically, a recipe that builds software that
1607includes one or more kernel modules and has its own means of building
1608the module inherits this class as opposed to inheriting the
1609:ref:`module <ref-classes-module>` class.
1610
1611.. _ref-classes-multilib*:
1612
1613``multilib*.bbclass``
1614=====================
1615
1616The ``multilib*`` classes provide support for building libraries with
1617different target optimizations or target architectures and installing
1618them side-by-side in the same image.
1619
1620For more information on using the Multilib feature, see the
1621":ref:`combining-multiple-versions-library-files-into-one-image`"
1622section in the Yocto Project Development Tasks Manual.
1623
1624.. _ref-classes-native:
1625
1626``native.bbclass``
1627==================
1628
1629The ``native`` class provides common functionality for recipes that
1630build tools to run on the `build host <#hardware-build-system-term>`__
1631(i.e. tools that use the compiler or other tools from the build host).
1632
1633You can create a recipe that builds tools that run natively on the host
1634a couple different ways:
1635
1636- Create a myrecipe\ ``-native.bb`` recipe that inherits the ``native``
1637 class. If you use this method, you must order the inherit statement
1638 in the recipe after all other inherit statements so that the
1639 ``native`` class is inherited last.
1640
1641 .. note::
1642
1643 When creating a recipe this way, the recipe name must follow this
1644 naming convention:
1645 ::
1646
1647 myrecipe-native.bb
1648
1649
1650 Not using this naming convention can lead to subtle problems
1651 caused by existing code that depends on that naming convention.
1652
1653- Create or modify a target recipe that contains the following:
1654 ::
1655
1656 BBCLASSEXTEND = "native"
1657
1658 Inside the
1659 recipe, use ``_class-native`` and ``_class-target`` overrides to
1660 specify any functionality specific to the respective native or target
1661 case.
1662
1663Although applied differently, the ``native`` class is used with both
1664methods. The advantage of the second method is that you do not need to
1665have two separate recipes (assuming you need both) for native and
1666target. All common parts of the recipe are automatically shared.
1667
1668.. _ref-classes-nativesdk:
1669
1670``nativesdk.bbclass``
1671=====================
1672
1673The ``nativesdk`` class provides common functionality for recipes that
1674wish to build tools to run as part of an SDK (i.e. tools that run on
1675:term:`SDKMACHINE`).
1676
1677You can create a recipe that builds tools that run on the SDK machine a
1678couple different ways:
1679
1680- Create a ``nativesdk-``\ myrecipe\ ``.bb`` recipe that inherits the
1681 ``nativesdk`` class. If you use this method, you must order the
1682 inherit statement in the recipe after all other inherit statements so
1683 that the ``nativesdk`` class is inherited last.
1684
1685- Create a ``nativesdk`` variant of any recipe by adding the following:
1686 ::
1687
1688 BBCLASSEXTEND = "nativesdk"
1689
1690 Inside the
1691 recipe, use ``_class-nativesdk`` and ``_class-target`` overrides to
1692 specify any functionality specific to the respective SDK machine or
1693 target case.
1694
1695.. note::
1696
1697 When creating a recipe, you must follow this naming convention:
1698 ::
1699
1700 nativesdk-myrecipe.bb
1701
1702
1703 Not doing so can lead to subtle problems because code exists that
1704 depends on the naming convention.
1705
1706Although applied differently, the ``nativesdk`` class is used with both
1707methods. The advantage of the second method is that you do not need to
1708have two separate recipes (assuming you need both) for the SDK machine
1709and the target. All common parts of the recipe are automatically shared.
1710
1711.. _ref-classes-nopackages:
1712
1713``nopackages.bbclass``
1714======================
1715
1716Disables packaging tasks for those recipes and classes where packaging
1717is not needed.
1718
1719.. _ref-classes-npm:
1720
1721``npm.bbclass``
1722===============
1723
1724Provides support for building Node.js software fetched using the `node
1725package manager (NPM) <https://en.wikipedia.org/wiki/Npm_(software)>`__.
1726
1727.. note::
1728
1729 Currently, recipes inheriting this class must use the
1730 npm://
1731 fetcher to have dependencies fetched and packaged automatically.
1732
1733For information on how to create NPM packages, see the
1734":ref:`dev-manual/dev-manual-common-tasks:creating node package manager (npm) packages`"
1735section in the Yocto Project Development Tasks Manual.
1736
1737.. _ref-classes-oelint:
1738
1739``oelint.bbclass``
1740==================
1741
1742The ``oelint`` class is an obsolete lint checking tool that exists in
1743``meta/classes`` in the :term:`Source Directory`.
1744
1745A number of classes exist that could be generally useful in OE-Core but
1746are never actually used within OE-Core itself. The ``oelint`` class is
1747one such example. However, being aware of this class can reduce the
1748proliferation of different versions of similar classes across multiple
1749layers.
1750
1751.. _ref-classes-own-mirrors:
1752
1753``own-mirrors.bbclass``
1754=======================
1755
1756The ``own-mirrors`` class makes it easier to set up your own
1757:term:`PREMIRRORS` from which to first fetch source
1758before attempting to fetch it from the upstream specified in
1759:term:`SRC_URI` within each recipe.
1760
1761To use this class, inherit it globally and specify
1762:term:`SOURCE_MIRROR_URL`. Here is an example:
1763::
1764
1765 INHERIT += "own-mirrors"
1766 SOURCE_MIRROR_URL = "http://example.com/my-source-mirror"
1767
1768You can specify only a single URL
1769in ``SOURCE_MIRROR_URL``.
1770
1771.. _ref-classes-package:
1772
1773``package.bbclass``
1774===================
1775
1776The ``package`` class supports generating packages from a build's
1777output. The core generic functionality is in ``package.bbclass``. The
1778code specific to particular package types resides in these
1779package-specific classes:
1780:ref:`package_deb <ref-classes-package_deb>`,
1781:ref:`package_rpm <ref-classes-package_rpm>`,
1782:ref:`package_ipk <ref-classes-package_ipk>`, and
1783:ref:`package_tar <ref-classes-package_tar>`.
1784
1785.. note::
1786
1787 The
1788 package_tar
1789 class is broken and not supported. It is recommended that you do not
1790 use this class.
1791
1792You can control the list of resulting package formats by using the
1793``PACKAGE_CLASSES`` variable defined in your ``conf/local.conf``
1794configuration file, which is located in the :term:`Build Directory`.
1795When defining the variable, you can
1796specify one or more package types. Since images are generated from
1797packages, a packaging class is needed to enable image generation. The
1798first class listed in this variable is used for image generation.
1799
1800If you take the optional step to set up a repository (package feed) on
1801the development host that can be used by DNF, you can install packages
1802from the feed while you are running the image on the target (i.e.
1803runtime installation of packages). For more information, see the
1804":ref:`dev-manual/dev-manual-common-tasks:using runtime package management`"
1805section in the Yocto Project Development Tasks Manual.
1806
1807The package-specific class you choose can affect build-time performance
1808and has space ramifications. In general, building a package with IPK
1809takes about thirty percent less time as compared to using RPM to build
1810the same or similar package. This comparison takes into account a
1811complete build of the package with all dependencies previously built.
1812The reason for this discrepancy is because the RPM package manager
1813creates and processes more :term:`Metadata` than the IPK package
1814manager. Consequently, you might consider setting ``PACKAGE_CLASSES`` to
1815"package_ipk" if you are building smaller systems.
1816
1817Before making your package manager decision, however, you should
1818consider some further things about using RPM:
1819
1820- RPM starts to provide more abilities than IPK due to the fact that it
1821 processes more Metadata. For example, this information includes
1822 individual file types, file checksum generation and evaluation on
1823 install, sparse file support, conflict detection and resolution for
1824 Multilib systems, ACID style upgrade, and repackaging abilities for
1825 rollbacks.
1826
1827- For smaller systems, the extra space used for the Berkeley Database
1828 and the amount of metadata when using RPM can affect your ability to
1829 perform on-device upgrades.
1830
1831You can find additional information on the effects of the package class
1832at these two Yocto Project mailing list links:
1833
1834- https://lists.yoctoproject.org/pipermail/poky/2011-May/006362.html
1835
1836- https://lists.yoctoproject.org/pipermail/poky/2011-May/006363.html
1837
1838.. _ref-classes-package_deb:
1839
1840``package_deb.bbclass``
1841=======================
1842
1843The ``package_deb`` class provides support for creating packages that
1844use the Debian (i.e. ``.deb``) file format. The class ensures the
1845packages are written out in a ``.deb`` file format to the
1846``${``\ :term:`DEPLOY_DIR_DEB`\ ``}`` directory.
1847
1848This class inherits the :ref:`package <ref-classes-package>` class and
1849is enabled through the :term:`PACKAGE_CLASSES`
1850variable in the ``local.conf`` file.
1851
1852.. _ref-classes-package_ipk:
1853
1854``package_ipk.bbclass``
1855=======================
1856
1857The ``package_ipk`` class provides support for creating packages that
1858use the IPK (i.e. ``.ipk``) file format. The class ensures the packages
1859are written out in a ``.ipk`` file format to the
1860``${``\ :term:`DEPLOY_DIR_IPK`\ ``}`` directory.
1861
1862This class inherits the :ref:`package <ref-classes-package>` class and
1863is enabled through the :term:`PACKAGE_CLASSES`
1864variable in the ``local.conf`` file.
1865
1866.. _ref-classes-package_rpm:
1867
1868``package_rpm.bbclass``
1869=======================
1870
1871The ``package_rpm`` class provides support for creating packages that
1872use the RPM (i.e. ``.rpm``) file format. The class ensures the packages
1873are written out in a ``.rpm`` file format to the
1874``${``\ :term:`DEPLOY_DIR_RPM`\ ``}`` directory.
1875
1876This class inherits the :ref:`package <ref-classes-package>` class and
1877is enabled through the :term:`PACKAGE_CLASSES`
1878variable in the ``local.conf`` file.
1879
1880.. _ref-classes-package_tar:
1881
1882``package_tar.bbclass``
1883=======================
1884
1885The ``package_tar`` class provides support for creating tarballs. The
1886class ensures the packages are written out in a tarball format to the
1887``${``\ :term:`DEPLOY_DIR_TAR`\ ``}`` directory.
1888
1889This class inherits the :ref:`package <ref-classes-package>` class and
1890is enabled through the :term:`PACKAGE_CLASSES`
1891variable in the ``local.conf`` file.
1892
1893.. note::
1894
1895 You cannot specify the
1896 package_tar
1897 class first using the
1898 PACKAGE_CLASSES
1899 variable. You must use
1900 .deb
1901 ,
1902 .ipk
1903 , or
1904 .rpm
1905 file formats for your image or SDK.
1906
1907.. _ref-classes-packagedata:
1908
1909``packagedata.bbclass``
1910=======================
1911
1912The ``packagedata`` class provides common functionality for reading
1913``pkgdata`` files found in :term:`PKGDATA_DIR`. These
1914files contain information about each output package produced by the
1915OpenEmbedded build system.
1916
1917This class is enabled by default because it is inherited by the
1918:ref:`package <ref-classes-package>` class.
1919
1920.. _ref-classes-packagegroup:
1921
1922``packagegroup.bbclass``
1923========================
1924
1925The ``packagegroup`` class sets default values appropriate for package
1926group recipes (e.g. ``PACKAGES``, ``PACKAGE_ARCH``, ``ALLOW_EMPTY``, and
1927so forth). It is highly recommended that all package group recipes
1928inherit this class.
1929
1930For information on how to use this class, see the
1931":ref:`usingpoky-extend-customimage-customtasks`"
1932section in the Yocto Project Development Tasks Manual.
1933
1934Previously, this class was called the ``task`` class.
1935
1936.. _ref-classes-patch:
1937
1938``patch.bbclass``
1939=================
1940
1941The ``patch`` class provides all functionality for applying patches
1942during the :ref:`ref-tasks-patch` task.
1943
1944This class is enabled by default because it is inherited by the
1945:ref:`base <ref-classes-base>` class.
1946
1947.. _ref-classes-perlnative:
1948
1949``perlnative.bbclass``
1950======================
1951
1952When inherited by a recipe, the ``perlnative`` class supports using the
1953native version of Perl built by the build system rather than using the
1954version provided by the build host.
1955
1956.. _ref-classes-pixbufcache:
1957
1958``pixbufcache.bbclass``
1959=======================
1960
1961The ``pixbufcache`` class generates the proper post-install and
1962post-remove (postinst/postrm) scriptlets for packages that install
1963pixbuf loaders, which are used with ``gdk-pixbuf``. These scriptlets
1964call ``update_pixbuf_cache`` to add the pixbuf loaders to the cache.
1965Since the cache files are architecture-specific, ``update_pixbuf_cache``
1966is run using QEMU if the postinst scriptlets need to be run on the build
1967host during image creation.
1968
1969If the pixbuf loaders being installed are in packages other than the
1970recipe's main package, set
1971:term:`PIXBUF_PACKAGES` to specify the packages
1972containing the loaders.
1973
1974.. _ref-classes-pkgconfig:
1975
1976``pkgconfig.bbclass``
1977=====================
1978
1979The ``pkgconfig`` class provides a standard way to get header and
1980library information by using ``pkg-config``. This class aims to smooth
1981integration of ``pkg-config`` into libraries that use it.
1982
1983During staging, BitBake installs ``pkg-config`` data into the
1984``sysroots/`` directory. By making use of sysroot functionality within
1985``pkg-config``, the ``pkgconfig`` class no longer has to manipulate the
1986files.
1987
1988.. _ref-classes-populate-sdk:
1989
1990``populate_sdk.bbclass``
1991========================
1992
1993The ``populate_sdk`` class provides support for SDK-only recipes. For
1994information on advantages gained when building a cross-development
1995toolchain using the :ref:`ref-tasks-populate_sdk`
1996task, see the ":ref:`sdk-manual/sdk-appendix-obtain:building an sdk installer`"
1997section in the Yocto Project Application Development and the Extensible
1998Software Development Kit (eSDK) manual.
1999
2000.. _ref-classes-populate-sdk-*:
2001
2002``populate_sdk_*.bbclass``
2003==========================
2004
2005The ``populate_sdk_*`` classes support SDK creation and consist of the
2006following classes:
2007
2008- ``populate_sdk_base``: The base class supporting SDK creation under
2009 all package managers (i.e. DEB, RPM, and opkg).
2010
2011- ``populate_sdk_deb``: Supports creation of the SDK given the Debian
2012 package manager.
2013
2014- ``populate_sdk_rpm``: Supports creation of the SDK given the RPM
2015 package manager.
2016
2017- ``populate_sdk_ipk``: Supports creation of the SDK given the opkg
2018 (IPK format) package manager.
2019
2020- ``populate_sdk_ext``: Supports extensible SDK creation under all
2021 package managers.
2022
2023The ``populate_sdk_base`` class inherits the appropriate
2024``populate_sdk_*`` (i.e. ``deb``, ``rpm``, and ``ipk``) based on
2025:term:`IMAGE_PKGTYPE`.
2026
2027The base class ensures all source and destination directories are
2028established and then populates the SDK. After populating the SDK, the
2029``populate_sdk_base`` class constructs two sysroots:
2030``${``\ :term:`SDK_ARCH`\ ``}-nativesdk``, which
2031contains the cross-compiler and associated tooling, and the target,
2032which contains a target root filesystem that is configured for the SDK
2033usage. These two images reside in :term:`SDK_OUTPUT`,
2034which consists of the following:
2035::
2036
2037 ${SDK_OUTPUT}/${SDK_ARCH}-nativesdk-pkgs
2038 ${SDK_OUTPUT}/${SDKTARGETSYSROOT}/target-pkgs
2039
2040Finally, the base populate SDK class creates the toolchain environment
2041setup script, the tarball of the SDK, and the installer.
2042
2043The respective ``populate_sdk_deb``, ``populate_sdk_rpm``, and
2044``populate_sdk_ipk`` classes each support the specific type of SDK.
2045These classes are inherited by and used with the ``populate_sdk_base``
2046class.
2047
2048For more information on the cross-development toolchain generation, see
2049the ":ref:`overview-manual/overview-manual-concepts:cross-development toolchain generation`"
2050section in the Yocto Project Overview and Concepts Manual. For
2051information on advantages gained when building a cross-development
2052toolchain using the :ref:`ref-tasks-populate_sdk`
2053task, see the
2054":ref:`sdk-manual/sdk-appendix-obtain:building an sdk installer`"
2055section in the Yocto Project Application Development and the Extensible
2056Software Development Kit (eSDK) manual.
2057
2058.. _ref-classes-prexport:
2059
2060``prexport.bbclass``
2061====================
2062
2063The ``prexport`` class provides functionality for exporting
2064:term:`PR` values.
2065
2066.. note::
2067
2068 This class is not intended to be used directly. Rather, it is enabled
2069 when using "
2070 bitbake-prserv-tool export
2071 ".
2072
2073.. _ref-classes-primport:
2074
2075``primport.bbclass``
2076====================
2077
2078The ``primport`` class provides functionality for importing
2079:term:`PR` values.
2080
2081.. note::
2082
2083 This class is not intended to be used directly. Rather, it is enabled
2084 when using "
2085 bitbake-prserv-tool import
2086 ".
2087
2088.. _ref-classes-prserv:
2089
2090``prserv.bbclass``
2091==================
2092
2093The ``prserv`` class provides functionality for using a :ref:`PR
2094service <dev-manual/dev-manual-common-tasks:working with a pr service>` in order to
2095automatically manage the incrementing of the :term:`PR`
2096variable for each recipe.
2097
2098This class is enabled by default because it is inherited by the
2099:ref:`package <ref-classes-package>` class. However, the OpenEmbedded
2100build system will not enable the functionality of this class unless
2101:term:`PRSERV_HOST` has been set.
2102
2103.. _ref-classes-ptest:
2104
2105``ptest.bbclass``
2106=================
2107
2108The ``ptest`` class provides functionality for packaging and installing
2109runtime tests for recipes that build software that provides these tests.
2110
2111This class is intended to be inherited by individual recipes. However,
2112the class' functionality is largely disabled unless "ptest" appears in
2113:term:`DISTRO_FEATURES`. See the
2114":ref:`dev-manual/dev-manual-common-tasks:testing packages with ptest`"
2115section in the Yocto Project Development Tasks Manual for more information
2116on ptest.
2117
2118.. _ref-classes-ptest-gnome:
2119
2120``ptest-gnome.bbclass``
2121=======================
2122
2123Enables package tests (ptests) specifically for GNOME packages, which
2124have tests intended to be executed with ``gnome-desktop-testing``.
2125
2126For information on setting up and running ptests, see the
2127":ref:`dev-manual/dev-manual-common-tasks:testing packages with ptest`"
2128section in the Yocto Project Development Tasks Manual.
2129
2130.. _ref-classes-python-dir:
2131
2132``python-dir.bbclass``
2133======================
2134
2135The ``python-dir`` class provides the base version, location, and site
2136package location for Python.
2137
2138.. _ref-classes-python3native:
2139
2140``python3native.bbclass``
2141=========================
2142
2143The ``python3native`` class supports using the native version of Python
21443 built by the build system rather than support of the version provided
2145by the build host.
2146
2147.. _ref-classes-pythonnative:
2148
2149``pythonnative.bbclass``
2150========================
2151
2152When inherited by a recipe, the ``pythonnative`` class supports using
2153the native version of Python built by the build system rather than using
2154the version provided by the build host.
2155
2156.. _ref-classes-qemu:
2157
2158``qemu.bbclass``
2159================
2160
2161The ``qemu`` class provides functionality for recipes that either need
2162QEMU or test for the existence of QEMU. Typically, this class is used to
2163run programs for a target system on the build host using QEMU's
2164application emulation mode.
2165
2166.. _ref-classes-recipe_sanity:
2167
2168``recipe_sanity.bbclass``
2169=========================
2170
2171The ``recipe_sanity`` class checks for the presence of any host system
2172recipe prerequisites that might affect the build (e.g. variables that
2173are set or software that is present).
2174
2175.. _ref-classes-relocatable:
2176
2177``relocatable.bbclass``
2178=======================
2179
2180The ``relocatable`` class enables relocation of binaries when they are
2181installed into the sysroot.
2182
2183This class makes use of the :ref:`chrpath <ref-classes-chrpath>` class
2184and is used by both the :ref:`cross <ref-classes-cross>` and
2185:ref:`native <ref-classes-native>` classes.
2186
2187.. _ref-classes-remove-libtool:
2188
2189``remove-libtool.bbclass``
2190==========================
2191
2192The ``remove-libtool`` class adds a post function to the
2193:ref:`ref-tasks-install` task to remove all ``.la`` files
2194installed by ``libtool``. Removing these files results in them being
2195absent from both the sysroot and target packages.
2196
2197If a recipe needs the ``.la`` files to be installed, then the recipe can
2198override the removal by setting ``REMOVE_LIBTOOL_LA`` to "0" as follows:
2199::
2200
2201 REMOVE_LIBTOOL_LA = "0"
2202
2203.. note::
2204
2205 The
2206 remove-libtool
2207 class is not enabled by default.
2208
2209.. _ref-classes-report-error:
2210
2211``report-error.bbclass``
2212========================
2213
2214The ``report-error`` class supports enabling the :ref:`error reporting
2215tool <dev-manual/dev-manual-common-tasks:using the error reporting tool>`",
2216which allows you to submit build error information to a central database.
2217
2218The class collects debug information for recipe, recipe version, task,
2219machine, distro, build system, target system, host distro, branch,
2220commit, and log. From the information, report files using a JSON format
2221are created and stored in
2222``${``\ :term:`LOG_DIR`\ ``}/error-report``.
2223
2224.. _ref-classes-rm-work:
2225
2226``rm_work.bbclass``
2227===================
2228
2229The ``rm_work`` class supports deletion of temporary workspace, which
2230can ease your hard drive demands during builds.
2231
2232The OpenEmbedded build system can use a substantial amount of disk space
2233during the build process. A portion of this space is the work files
2234under the ``${TMPDIR}/work`` directory for each recipe. Once the build
2235system generates the packages for a recipe, the work files for that
2236recipe are no longer needed. However, by default, the build system
2237preserves these files for inspection and possible debugging purposes. If
2238you would rather have these files deleted to save disk space as the
2239build progresses, you can enable ``rm_work`` by adding the following to
2240your ``local.conf`` file, which is found in the :term:`Build Directory`.
2241::
2242
2243 INHERIT += "rm_work"
2244
2245If you are
2246modifying and building source code out of the work directory for a
2247recipe, enabling ``rm_work`` will potentially result in your changes to
2248the source being lost. To exclude some recipes from having their work
2249directories deleted by ``rm_work``, you can add the names of the recipe
2250or recipes you are working on to the ``RM_WORK_EXCLUDE`` variable, which
2251can also be set in your ``local.conf`` file. Here is an example:
2252::
2253
2254 RM_WORK_EXCLUDE += "busybox glibc"
2255
2256.. _ref-classes-rootfs*:
2257
2258``rootfs*.bbclass``
2259===================
2260
2261The ``rootfs*`` classes support creating the root filesystem for an
2262image and consist of the following classes:
2263
2264- The ``rootfs-postcommands`` class, which defines filesystem
2265 post-processing functions for image recipes.
2266
2267- The ``rootfs_deb`` class, which supports creation of root filesystems
2268 for images built using ``.deb`` packages.
2269
2270- The ``rootfs_rpm`` class, which supports creation of root filesystems
2271 for images built using ``.rpm`` packages.
2272
2273- The ``rootfs_ipk`` class, which supports creation of root filesystems
2274 for images built using ``.ipk`` packages.
2275
2276- The ``rootfsdebugfiles`` class, which installs additional files found
2277 on the build host directly into the root filesystem.
2278
2279The root filesystem is created from packages using one of the
2280``rootfs*.bbclass`` files as determined by the
2281:term:`PACKAGE_CLASSES` variable.
2282
2283For information on how root filesystem images are created, see the
2284:ref:`image-generation-dev-environment`"
2285section in the Yocto Project Overview and Concepts Manual.
2286
2287.. _ref-classes-sanity:
2288
2289``sanity.bbclass``
2290==================
2291
2292The ``sanity`` class checks to see if prerequisite software is present
2293on the host system so that users can be notified of potential problems
2294that might affect their build. The class also performs basic user
2295configuration checks from the ``local.conf`` configuration file to
2296prevent common mistakes that cause build failures. Distribution policy
2297usually determines whether to include this class.
2298
2299.. _ref-classes-scons:
2300
2301``scons.bbclass``
2302=================
2303
2304The ``scons`` class supports recipes that need to build software that
2305uses the SCons build system. You can use the
2306:term:`EXTRA_OESCONS` variable to specify
2307additional configuration options you want to pass SCons command line.
2308
2309.. _ref-classes-sdl:
2310
2311``sdl.bbclass``
2312===============
2313
2314The ``sdl`` class supports recipes that need to build software that uses
2315the Simple DirectMedia Layer (SDL) library.
2316
2317.. _ref-classes-setuptools:
2318
2319``setuptools.bbclass``
2320======================
2321
2322The ``setuptools`` class supports Python version 2.x extensions that use
2323build systems based on ``setuptools``. If your recipe uses these build
2324systems, the recipe needs to inherit the ``setuptools`` class.
2325
2326.. _ref-classes-setuptools3:
2327
2328``setuptools3.bbclass``
2329=======================
2330
2331The ``setuptools3`` class supports Python version 3.x extensions that
2332use build systems based on ``setuptools3``. If your recipe uses these
2333build systems, the recipe needs to inherit the ``setuptools3`` class.
2334
2335.. _ref-classes-sign_rpm:
2336
2337``sign_rpm.bbclass``
2338====================
2339
2340The ``sign_rpm`` class supports generating signed RPM packages.
2341
2342.. _ref-classes-sip:
2343
2344``sip.bbclass``
2345===============
2346
2347The ``sip`` class supports recipes that build or package SIP-based
2348Python bindings.
2349
2350.. _ref-classes-siteconfig:
2351
2352``siteconfig.bbclass``
2353======================
2354
2355The ``siteconfig`` class provides functionality for handling site
2356configuration. The class is used by the
2357:ref:`autotools <ref-classes-autotools>` class to accelerate the
2358:ref:`ref-tasks-configure` task.
2359
2360.. _ref-classes-siteinfo:
2361
2362``siteinfo.bbclass``
2363====================
2364
2365The ``siteinfo`` class provides information about the targets that might
2366be needed by other classes or recipes.
2367
2368As an example, consider Autotools, which can require tests that must
2369execute on the target hardware. Since this is not possible in general
2370when cross compiling, site information is used to provide cached test
2371results so these tests can be skipped over but still make the correct
2372values available. The ``meta/site directory`` contains test results
2373sorted into different categories such as architecture, endianness, and
2374the ``libc`` used. Site information provides a list of files containing
2375data relevant to the current build in the ``CONFIG_SITE`` variable that
2376Autotools automatically picks up.
2377
2378The class also provides variables like ``SITEINFO_ENDIANNESS`` and
2379``SITEINFO_BITS`` that can be used elsewhere in the metadata.
2380
2381.. _ref-classes-spdx:
2382
2383``spdx.bbclass``
2384================
2385
2386The ``spdx`` class integrates real-time license scanning, generation of
2387SPDX standard output, and verification of license information during the
2388build.
2389
2390.. note::
2391
2392 This class is currently at the prototype stage in the 1.6 release.
2393
2394.. _ref-classes-sstate:
2395
2396``sstate.bbclass``
2397==================
2398
2399The ``sstate`` class provides support for Shared State (sstate). By
2400default, the class is enabled through the
2401:term:`INHERIT_DISTRO` variable's default value.
2402
2403For more information on sstate, see the
2404":ref:`overview-manual/overview-manual-concepts:shared state cache`"
2405section in the Yocto Project Overview and Concepts Manual.
2406
2407.. _ref-classes-staging:
2408
2409``staging.bbclass``
2410===================
2411
2412The ``staging`` class installs files into individual recipe work
2413directories for sysroots. The class contains the following key tasks:
2414
2415- The :ref:`ref-tasks-populate_sysroot` task,
2416 which is responsible for handing the files that end up in the recipe
2417 sysroots.
2418
2419- The
2420 :ref:`ref-tasks-prepare_recipe_sysroot`
2421 task (a "partner" task to the ``populate_sysroot`` task), which
2422 installs the files into the individual recipe work directories (i.e.
2423 :term:`WORKDIR`).
2424
2425The code in the ``staging`` class is complex and basically works in two
2426stages:
2427
2428- *Stage One:* The first stage addresses recipes that have files they
2429 want to share with other recipes that have dependencies on the
2430 originating recipe. Normally these dependencies are installed through
2431 the :ref:`ref-tasks-install` task into
2432 ``${``\ :term:`D`\ ``}``. The ``do_populate_sysroot`` task
2433 copies a subset of these files into ``${SYSROOT_DESTDIR}``. This
2434 subset of files is controlled by the
2435 :term:`SYSROOT_DIRS`,
2436 :term:`SYSROOT_DIRS_NATIVE`, and
2437 :term:`SYSROOT_DIRS_BLACKLIST`
2438 variables.
2439
2440 .. note::
2441
2442 Additionally, a recipe can customize the files further by
2443 declaring a processing function in the
2444 SYSROOT_PREPROCESS_FUNCS
2445 variable.
2446
2447 A shared state (sstate) object is built from these files and the
2448 files are placed into a subdirectory of
2449 ```tmp/sysroots-components/`` <#structure-build-tmp-sysroots-components>`__.
2450 The files are scanned for hardcoded paths to the original
2451 installation location. If the location is found in text files, the
2452 hardcoded locations are replaced by tokens and a list of the files
2453 needing such replacements is created. These adjustments are referred
2454 to as "FIXMEs". The list of files that are scanned for paths is
2455 controlled by the :term:`SSTATE_SCAN_FILES`
2456 variable.
2457
2458- *Stage Two:* The second stage addresses recipes that want to use
2459 something from another recipe and declare a dependency on that recipe
2460 through the :term:`DEPENDS` variable. The recipe will
2461 have a
2462 :ref:`ref-tasks-prepare_recipe_sysroot`
2463 task and when this task executes, it creates the ``recipe-sysroot``
2464 and ``recipe-sysroot-native`` in the recipe work directory (i.e.
2465 :term:`WORKDIR`). The OpenEmbedded build system
2466 creates hard links to copies of the relevant files from
2467 ``sysroots-components`` into the recipe work directory.
2468
2469 .. note::
2470
2471 If hard links are not possible, the build system uses actual
2472 copies.
2473
2474 The build system then addresses any "FIXMEs" to paths as defined from
2475 the list created in the first stage.
2476
2477 Finally, any files in ``${bindir}`` within the sysroot that have the
2478 prefix "``postinst-``" are executed.
2479
2480 .. note::
2481
2482 Although such sysroot post installation scripts are not
2483 recommended for general use, the files do allow some issues such
2484 as user creation and module indexes to be addressed.
2485
2486 Because recipes can have other dependencies outside of ``DEPENDS``
2487 (e.g. ``do_unpack[depends] += "tar-native:do_populate_sysroot"``),
2488 the sysroot creation function ``extend_recipe_sysroot`` is also added
2489 as a pre-function for those tasks whose dependencies are not through
2490 ``DEPENDS`` but operate similarly.
2491
2492 When installing dependencies into the sysroot, the code traverses the
2493 dependency graph and processes dependencies in exactly the same way
2494 as the dependencies would or would not be when installed from sstate.
2495 This processing means, for example, a native tool would have its
2496 native dependencies added but a target library would not have its
2497 dependencies traversed or installed. The same sstate dependency code
2498 is used so that builds should be identical regardless of whether
2499 sstate was used or not. For a closer look, see the
2500 ``setscene_depvalid()`` function in the
2501 :ref:`sstate <ref-classes-sstate>` class.
2502
2503 The build system is careful to maintain manifests of the files it
2504 installs so that any given dependency can be installed as needed. The
2505 sstate hash of the installed item is also stored so that if it
2506 changes, the build system can reinstall it.
2507
2508.. _ref-classes-syslinux:
2509
2510``syslinux.bbclass``
2511====================
2512
2513The ``syslinux`` class provides syslinux-specific functions for building
2514bootable images.
2515
2516The class supports the following variables:
2517
2518- :term:`INITRD`: Indicates list of filesystem images to
2519 concatenate and use as an initial RAM disk (initrd). This variable is
2520 optional.
2521
2522- :term:`ROOTFS`: Indicates a filesystem image to include
2523 as the root filesystem. This variable is optional.
2524
2525- :term:`AUTO_SYSLINUXMENU`: Enables creating
2526 an automatic menu when set to "1".
2527
2528- :term:`LABELS`: Lists targets for automatic
2529 configuration.
2530
2531- :term:`APPEND`: Lists append string overrides for each
2532 label.
2533
2534- :term:`SYSLINUX_OPTS`: Lists additional options
2535 to add to the syslinux file. Semicolon characters separate multiple
2536 options.
2537
2538- :term:`SYSLINUX_SPLASH`: Lists a background
2539 for the VGA boot menu when you are using the boot menu.
2540
2541- :term:`SYSLINUX_DEFAULT_CONSOLE`: Set
2542 to "console=ttyX" to change kernel boot default console.
2543
2544- :term:`SYSLINUX_SERIAL`: Sets an alternate
2545 serial port. Or, turns off serial when the variable is set with an
2546 empty string.
2547
2548- :term:`SYSLINUX_SERIAL_TTY`: Sets an
2549 alternate "console=tty..." kernel boot argument.
2550
2551.. _ref-classes-systemd:
2552
2553``systemd.bbclass``
2554===================
2555
2556The ``systemd`` class provides support for recipes that install systemd
2557unit files.
2558
2559The functionality for this class is disabled unless you have "systemd"
2560in :term:`DISTRO_FEATURES`.
2561
2562Under this class, the recipe or Makefile (i.e. whatever the recipe is
2563calling during the :ref:`ref-tasks-install` task)
2564installs unit files into
2565``${``\ :term:`D`\ ``}${systemd_unitdir}/system``. If the unit
2566files being installed go into packages other than the main package, you
2567need to set :term:`SYSTEMD_PACKAGES` in your
2568recipe to identify the packages in which the files will be installed.
2569
2570You should set :term:`SYSTEMD_SERVICE` to the
2571name of the service file. You should also use a package name override to
2572indicate the package to which the value applies. If the value applies to
2573the recipe's main package, use ``${``\ :term:`PN`\ ``}``. Here
2574is an example from the connman recipe:
2575::
2576
2577 SYSTEMD_SERVICE_${PN} = "connman.service"
2578
2579Services are set up to start on boot automatically
2580unless you have set
2581:term:`SYSTEMD_AUTO_ENABLE` to "disable".
2582
2583For more information on ``systemd``, see the
2584":ref:`dev-manual/dev-manual-common-tasks:selecting an initialization manager`"
2585section in the Yocto Project Development Tasks Manual.
2586
2587.. _ref-classes-systemd-boot:
2588
2589``systemd-boot.bbclass``
2590========================
2591
2592The ``systemd-boot`` class provides functions specific to the
2593systemd-boot bootloader for building bootable images. This is an
2594internal class and is not intended to be used directly.
2595
2596.. note::
2597
2598 The
2599 systemd-boot
2600 class is a result from merging the
2601 gummiboot
2602 class used in previous Yocto Project releases with the
2603 systemd
2604 project.
2605
2606Set the :term:`EFI_PROVIDER` variable to
2607"systemd-boot" to use this class. Doing so creates a standalone EFI
2608bootloader that is not dependent on systemd.
2609
2610For information on more variables used and supported in this class, see
2611the :term:`SYSTEMD_BOOT_CFG`,
2612:term:`SYSTEMD_BOOT_ENTRIES`, and
2613:term:`SYSTEMD_BOOT_TIMEOUT` variables.
2614
2615You can also see the `Systemd-boot
2616documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__
2617for more information.
2618
2619.. _ref-classes-terminal:
2620
2621``terminal.bbclass``
2622====================
2623
2624The ``terminal`` class provides support for starting a terminal session.
2625The :term:`OE_TERMINAL` variable controls which
2626terminal emulator is used for the session.
2627
2628Other classes use the ``terminal`` class anywhere a separate terminal
2629session needs to be started. For example, the
2630:ref:`patch <ref-classes-patch>` class assuming
2631:term:`PATCHRESOLVE` is set to "user", the
2632:ref:`cml1 <ref-classes-cml1>` class, and the
2633:ref:`devshell <ref-classes-devshell>` class all use the ``terminal``
2634class.
2635
2636.. _ref-classes-testimage*:
2637
2638``testimage*.bbclass``
2639======================
2640
2641The ``testimage*`` classes support running automated tests against
2642images using QEMU and on actual hardware. The classes handle loading the
2643tests and starting the image. To use the classes, you need to perform
2644steps to set up the environment.
2645
2646.. note::
2647
2648 Best practices include using
2649 IMAGE_CLASSES
2650 rather than
2651 INHERIT
2652 to inherit the
2653 testimage
2654 class for automated image testing.
2655
2656The tests are commands that run on the target system over ``ssh``. Each
2657test is written in Python and makes use of the ``unittest`` module.
2658
2659The ``testimage.bbclass`` runs tests on an image when called using the
2660following:
2661::
2662
2663 $ bitbake -c testimage image
2664
2665The ``testimage-auto`` class
2666runs tests on an image after the image is constructed (i.e.
2667:term:`TESTIMAGE_AUTO` must be set to "1").
2668
2669For information on how to enable, run, and create new tests, see the
2670":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
2671section in the Yocto Project Development Tasks Manual.
2672
2673.. _ref-classes-testsdk:
2674
2675``testsdk.bbclass``
2676===================
2677
2678This class supports running automated tests against software development
2679kits (SDKs). The ``testsdk`` class runs tests on an SDK when called
2680using the following:
2681::
2682
2683 $ bitbake -c testsdk image
2684
2685.. note::
2686
2687 Best practices include using
2688 IMAGE_CLASSES
2689 rather than
2690 INHERIT
2691 to inherit the
2692 testsdk
2693 class for automated SDK testing.
2694
2695.. _ref-classes-texinfo:
2696
2697``texinfo.bbclass``
2698===================
2699
2700This class should be inherited by recipes whose upstream packages invoke
2701the ``texinfo`` utilities at build-time. Native and cross recipes are
2702made to use the dummy scripts provided by ``texinfo-dummy-native``, for
2703improved performance. Target architecture recipes use the genuine
2704Texinfo utilities. By default, they use the Texinfo utilities on the
2705host system.
2706
2707.. note::
2708
2709 If you want to use the Texinfo recipe shipped with the build system,
2710 you can remove "texinfo-native" from
2711 ASSUME_PROVIDED
2712 and makeinfo from
2713 SANITY_REQUIRED_UTILITIES
2714 .
2715
2716.. _ref-classes-tinderclient:
2717
2718``tinderclient.bbclass``
2719========================
2720
2721The ``tinderclient`` class submits build results to an external
2722Tinderbox instance.
2723
2724.. note::
2725
2726 This class is currently unmaintained.
2727
2728.. _ref-classes-toaster:
2729
2730``toaster.bbclass``
2731===================
2732
2733The ``toaster`` class collects information about packages and images and
2734sends them as events that the BitBake user interface can receive. The
2735class is enabled when the Toaster user interface is running.
2736
2737This class is not intended to be used directly.
2738
2739.. _ref-classes-toolchain-scripts:
2740
2741``toolchain-scripts.bbclass``
2742=============================
2743
2744The ``toolchain-scripts`` class provides the scripts used for setting up
2745the environment for installed SDKs.
2746
2747.. _ref-classes-typecheck:
2748
2749``typecheck.bbclass``
2750=====================
2751
2752The ``typecheck`` class provides support for validating the values of
2753variables set at the configuration level against their defined types.
2754The OpenEmbedded build system allows you to define the type of a
2755variable using the "type" varflag. Here is an example:
2756::
2757
2758 IMAGE_FEATURES[type] = "list"
2759
2760.. _ref-classes-uboot-config:
2761
2762``uboot-config.bbclass``
2763========================
2764
2765The ``uboot-config`` class provides support for U-Boot configuration for
2766a machine. Specify the machine in your recipe as follows:
2767::
2768
2769 UBOOT_CONFIG ??= <default>
2770 UBOOT_CONFIG[foo] = "config,images"
2771
2772You can also specify the machine using this method:
2773::
2774
2775 UBOOT_MACHINE = "config"
2776
2777See the :term:`UBOOT_CONFIG` and :term:`UBOOT_MACHINE` variables for additional
2778information.
2779
2780.. _ref-classes-uninative:
2781
2782``uninative.bbclass``
2783=====================
2784
2785Attempts to isolate the build system from the host distribution's C
2786library in order to make re-use of native shared state artifacts across
2787different host distributions practical. With this class enabled, a
2788tarball containing a pre-built C library is downloaded at the start of
2789the build. In the Poky reference distribution this is enabled by default
2790through ``meta/conf/distro/include/yocto-uninative.inc``. Other
2791distributions that do not derive from poky can also
2792"``require conf/distro/include/yocto-uninative.inc``" to use this.
2793Alternatively if you prefer, you can build the uninative-tarball recipe
2794yourself, publish the resulting tarball (e.g. via HTTP) and set
2795``UNINATIVE_URL`` and ``UNINATIVE_CHECKSUM`` appropriately. For an
2796example, see the ``meta/conf/distro/include/yocto-uninative.inc``.
2797
2798The ``uninative`` class is also used unconditionally by the extensible
2799SDK. When building the extensible SDK, ``uninative-tarball`` is built
2800and the resulting tarball is included within the SDK.
2801
2802.. _ref-classes-update-alternatives:
2803
2804``update-alternatives.bbclass``
2805===============================
2806
2807The ``update-alternatives`` class helps the alternatives system when
2808multiple sources provide the same command. This situation occurs when
2809several programs that have the same or similar function are installed
2810with the same name. For example, the ``ar`` command is available from
2811the ``busybox``, ``binutils`` and ``elfutils`` packages. The
2812``update-alternatives`` class handles renaming the binaries so that
2813multiple packages can be installed without conflicts. The ``ar`` command
2814still works regardless of which packages are installed or subsequently
2815removed. The class renames the conflicting binary in each package and
2816symlinks the highest priority binary during installation or removal of
2817packages.
2818
2819To use this class, you need to define a number of variables:
2820
2821- :term:`ALTERNATIVE`
2822
2823- :term:`ALTERNATIVE_LINK_NAME`
2824
2825- :term:`ALTERNATIVE_TARGET`
2826
2827- :term:`ALTERNATIVE_PRIORITY`
2828
2829These variables list alternative commands needed by a package, provide
2830pathnames for links, default links for targets, and so forth. For
2831details on how to use this class, see the comments in the
2832:yocto_git:`update-alternatives.bbclass </cgit/cgit.cgi/poky/tree/meta/classes/update-alternatives.bbclass>`
2833file.
2834
2835.. note::
2836
2837 You can use the
2838 update-alternatives
2839 command directly in your recipes. However, this class simplifies
2840 things in most cases.
2841
2842.. _ref-classes-update-rc.d:
2843
2844``update-rc.d.bbclass``
2845=======================
2846
2847The ``update-rc.d`` class uses ``update-rc.d`` to safely install an
2848initialization script on behalf of the package. The OpenEmbedded build
2849system takes care of details such as making sure the script is stopped
2850before a package is removed and started when the package is installed.
2851
2852Three variables control this class: ``INITSCRIPT_PACKAGES``,
2853``INITSCRIPT_NAME`` and ``INITSCRIPT_PARAMS``. See the variable links
2854for details.
2855
2856.. _ref-classes-useradd:
2857
2858``useradd*.bbclass``
2859====================
2860
2861The ``useradd*`` classes support the addition of users or groups for
2862usage by the package on the target. For example, if you have packages
2863that contain system services that should be run under their own user or
2864group, you can use these classes to enable creation of the user or
2865group. The ``meta-skeleton/recipes-skeleton/useradd/useradd-example.bb``
2866recipe in the :term:`Source Directory` provides a simple
2867example that shows how to add three users and groups to two packages.
2868See the ``useradd-example.bb`` recipe for more information on how to use
2869these classes.
2870
2871The ``useradd_base`` class provides basic functionality for user or
2872groups settings.
2873
2874The ``useradd*`` classes support the
2875:term:`USERADD_PACKAGES`,
2876:term:`USERADD_PARAM`,
2877:term:`GROUPADD_PARAM`, and
2878:term:`GROUPMEMS_PARAM` variables.
2879
2880The ``useradd-staticids`` class supports the addition of users or groups
2881that have static user identification (``uid``) and group identification
2882(``gid``) values.
2883
2884The default behavior of the OpenEmbedded build system for assigning
2885``uid`` and ``gid`` values when packages add users and groups during
2886package install time is to add them dynamically. This works fine for
2887programs that do not care what the values of the resulting users and
2888groups become. In these cases, the order of the installation determines
2889the final ``uid`` and ``gid`` values. However, if non-deterministic
2890``uid`` and ``gid`` values are a problem, you can override the default,
2891dynamic application of these values by setting static values. When you
2892set static values, the OpenEmbedded build system looks in
2893:term:`BBPATH` for ``files/passwd`` and ``files/group``
2894files for the values.
2895
2896To use static ``uid`` and ``gid`` values, you need to set some
2897variables. See the :term:`USERADDEXTENSION`,
2898:term:`USERADD_UID_TABLES`,
2899:term:`USERADD_GID_TABLES`, and
2900:term:`USERADD_ERROR_DYNAMIC` variables.
2901You can also see the :ref:`useradd <ref-classes-useradd>` class for
2902additional information.
2903
2904.. note::
2905
2906 You do not use the
2907 useradd-staticids
2908 class directly. You either enable or disable the class by setting the
2909 USERADDEXTENSION
2910 variable. If you enable or disable the class in a configured system,
2911 TMPDIR
2912 might contain incorrect
2913 uid
2914 and
2915 gid
2916 values. Deleting the
2917 TMPDIR
2918 directory will correct this condition.
2919
2920.. _ref-classes-utility-tasks:
2921
2922``utility-tasks.bbclass``
2923=========================
2924
2925The ``utility-tasks`` class provides support for various "utility" type
2926tasks that are applicable to all recipes, such as
2927:ref:`ref-tasks-clean` and
2928:ref:`ref-tasks-listtasks`.
2929
2930This class is enabled by default because it is inherited by the
2931:ref:`base <ref-classes-base>` class.
2932
2933.. _ref-classes-utils:
2934
2935``utils.bbclass``
2936=================
2937
2938The ``utils`` class provides some useful Python functions that are
2939typically used in inline Python expressions (e.g. ``${@...}``). One
2940example use is for ``bb.utils.contains()``.
2941
2942This class is enabled by default because it is inherited by the
2943:ref:`base <ref-classes-base>` class.
2944
2945.. _ref-classes-vala:
2946
2947``vala.bbclass``
2948================
2949
2950The ``vala`` class supports recipes that need to build software written
2951using the Vala programming language.
2952
2953.. _ref-classes-waf:
2954
2955``waf.bbclass``
2956===============
2957
2958The ``waf`` class supports recipes that need to build software that uses
2959the Waf build system. You can use the
2960:term:`EXTRA_OECONF` or
2961:term:`PACKAGECONFIG_CONFARGS` variables
2962to specify additional configuration options to be passed on the Waf
2963command line.
diff --git a/documentation/ref-manual/ref-classes.xml b/documentation/ref-manual/ref-classes.xml
index f9bbddd724..1dcd5fdd03 100644
--- a/documentation/ref-manual/ref-classes.xml
+++ b/documentation/ref-manual/ref-classes.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-classes'> 6<chapter id='ref-classes'>
6<title>Classes</title> 7<title>Classes</title>
@@ -1742,6 +1743,12 @@ This check was removed for YP 2.3 release
1742 </note> 1743 </note>
1743 </para></listitem> 1744 </para></listitem>
1744--> 1745-->
1746 <listitem><para><emphasis><filename>unlisted-pkg-lics:</filename></emphasis>
1747 Checks that all declared licenses applying for a package are also
1748 declared on the recipe level (i.e. any license in
1749 <filename>LICENSE_*</filename> should appear in
1750 <link linkend='var-LICENSE'><filename>LICENSE</filename></link>).
1751 </para></listitem>
1745 <listitem><para><emphasis><filename>useless-rpaths:</filename></emphasis> 1752 <listitem><para><emphasis><filename>useless-rpaths:</filename></emphasis>
1746 Checks for dynamic library load paths (rpaths) in the binaries that 1753 Checks for dynamic library load paths (rpaths) in the binaries that
1747 by default on a standard system are searched by the linker (e.g. 1754 by default on a standard system are searched by the linker (e.g.
@@ -1873,8 +1880,82 @@ This check was removed for YP 2.3 release
1873 1880
1874 <para> 1881 <para>
1875 The <filename>kernel-fitimage</filename> class provides support to 1882 The <filename>kernel-fitimage</filename> class provides support to
1876 pack zImages. 1883 pack a kernel Image, device trees and a RAM disk into a single
1884 FIT image. In theory, a FIT image can support any number of kernels,
1885 RAM disks and device-trees.
1886 However, <filename>kernel-fitimage</filename> currently only supports
1887 limited usescases: just one kernel image, an optional RAM disk, and
1888 any number of device tree.
1889 </para>
1890
1891 <para>
1892 To create a FIT image, it is required that
1893 <filename><link linkend='var-KERNEL_CLASSES'>KERNEL_CLASSES</link></filename>
1894 is set to "kernel-fitimage" and
1895 <filename><link linkend='var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</link></filename>
1896 is set to "fitImage".
1897 </para>
1898
1899 <para>
1900 The options for the device tree compiler passed to mkimage -D feature
1901 when creating the FIT image are specified using the
1902 <filename><link linkend='var-UBOOT_MKIMAGE_DTCOPTS'>UBOOT_MKIMAGE_DTCOPTS</link></filename>
1903 variable.
1904 </para>
1905
1906 <para>
1907 Only a single kernel can be added to the FIT image created by
1908 <filename>kernel-fitimage</filename> and the kernel image in FIT is
1909 mandatory.
1910 The address where the kernel image is to be loaded by U-boot is
1911 specified by
1912 <filename><link linkend='var-UBOOT_LOADADDRESS'>UBOOT_LOADADDRESS</link></filename>
1913 and the entrypoint by
1914 <filename><link linkend='var-UBOOT_ENTRYPOINT'>UBOOT_ENTRYPOINT</link></filename>.
1915 </para>
1916
1917 <para>
1918 Multiple device trees can be added to the FIT image created by
1919 <filename>kernel-fitimage</filename> and the device tree is optional.
1920 The address where the device tree is to be loaded by U-boot is
1921 specified by
1922 <filename><link linkend='var-UBOOT_DTBO_LOADADDRESS'>UBOOT_DTBO_LOADADDRESS</link></filename>
1923 for device tree overlays and by
1924 <filename><link linkend='var-UBOOT_DTB_LOADADDRESS'>UBOOT_DTB_LOADADDRESS</link></filename>
1925 for device tree binaries.
1926 </para>
1927
1928 <para>
1929 Only a single RAM disk can be added to the FIT image created by
1930 <filename>kernel-fitimage</filename> and the RAM disk in FIT is
1931 optional.
1932 The address where the RAM disk image is to be loaded by U-boot
1933 is specified by
1934 <filename><link linkend='var-UBOOT_RD_LOADADDRESS'>UBOOT_RD_LOADADDRESS</link></filename>
1935 and the entrypoint by
1936 <filename><link linkend='var-UBOOT_RD_ENTRYPOINT'>UBOOT_RD_ENTRYPOINT</link></filename>.
1937 The ramdisk is added to FIT image when
1938 <filename><link linkend='var-INITRAMFS_IMAGE'>INITRAMFS_IMAGE</link></filename>
1939 is specified.
1940 </para>
1941
1942 <para>
1943 The FIT image generated by <filename>kernel-fitimage</filename> class
1944 is signed when the variables
1945 <filename><link linkend='var-UBOOT_SIGN_ENABLE'>UBOOT_SIGN_ENABLE</link></filename>,
1946 <filename><link linkend='var-UBOOT_MKIMAGE_DTCOPTS'>UBOOT_MKIMAGE_DTCOPTS</link></filename>,
1947 <filename><link linkend='var-UBOOT_SIGN_KEYDIR'>UBOOT_SIGN_KEYDIR</link></filename>
1948 and
1949 <filename><link linkend='var-UBOOT_SIGN_KEYNAME'>UBOOT_SIGN_KEYNAME</link></filename>
1950 are set appropriately.
1951 The default values used for
1952 <filename><link linkend='var-FIT_HASH_ALG'>FIT_HASH_ALG</link></filename>
1953 and
1954 <filename><link linkend='var-FIT_SIGN_ALG'>FIT_SIGN_ALG</link></filename>
1955 in <filename>kernel-fitimage</filename> are "sha256" and "rsa2048"
1956 respectively.
1877 </para> 1957 </para>
1958
1878</section> 1959</section>
1879 1960
1880<section id='ref-classes-kernel-grub'> 1961<section id='ref-classes-kernel-grub'>
diff --git a/documentation/ref-manual/ref-devtool-reference.rst b/documentation/ref-manual/ref-devtool-reference.rst
new file mode 100644
index 0000000000..eaca45ae25
--- /dev/null
+++ b/documentation/ref-manual/ref-devtool-reference.rst
@@ -0,0 +1,625 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***************************
4``devtool`` Quick Reference
5***************************
6
7The ``devtool`` command-line tool provides a number of features that
8help you build, test, and package software. This command is available
9alongside the ``bitbake`` command. Additionally, the ``devtool`` command
10is a key part of the extensible SDK.
11
12This chapter provides a Quick Reference for the ``devtool`` command. For
13more information on how to apply the command when using the extensible
14SDK, see the ":doc:`../sdk-manual/sdk-extensible`" chapter in the Yocto
15Project Application Development and the Extensible Software Development
16Kit (eSDK) manual.
17
18.. _devtool-getting-help:
19
20Getting Help
21============
22
23The ``devtool`` command line is organized similarly to Git in that it
24has a number of sub-commands for each function. You can run
25``devtool --help`` to see all the commands:
26::
27
28 $ devtool -h
29 NOTE: Starting bitbake server...
30 usage: devtool [--basepath BASEPATH] [--bbpath BBPATH] [-d] [-q] [--color COLOR] [-h] <subcommand> ...
31
32 OpenEmbedded development tool
33
34 options:
35 --basepath BASEPATH Base directory of SDK / build directory
36 --bbpath BBPATH Explicitly specify the BBPATH, rather than getting it from the metadata
37 -d, --debug Enable debug output
38 -q, --quiet Print only errors
39 --color COLOR Colorize output (where COLOR is auto, always, never)
40 -h, --help show this help message and exit
41
42 subcommands:
43 Beginning work on a recipe:
44 add Add a new recipe
45 modify Modify the source for an existing recipe
46 upgrade Upgrade an existing recipe
47 Getting information:
48 status Show workspace status
49 latest-version Report the latest version of an existing recipe
50 check-upgrade-status Report upgradability for multiple (or all) recipes
51 search Search available recipes
52 Working on a recipe in the workspace:
53 build Build a recipe
54 rename Rename a recipe file in the workspace
55 edit-recipe Edit a recipe file
56 find-recipe Find a recipe file
57 configure-help Get help on configure script options
58 update-recipe Apply changes from external source tree to recipe
59 reset Remove a recipe from your workspace
60 finish Finish working on a recipe in your workspace
61 Testing changes on target:
62 deploy-target Deploy recipe output files to live target machine
63 undeploy-target Undeploy recipe output files in live target machine
64 build-image Build image including workspace recipe packages
65 Advanced:
66 create-workspace Set up workspace in an alternative location
67 extract Extract the source for an existing recipe
68 sync Synchronize the source tree for an existing recipe
69 menuconfig Alter build-time configuration for a recipe
70 import Import exported tar archive into workspace
71 export Export workspace into a tar archive
72 other:
73 selftest-reverse Reverse value (for selftest)
74 pluginfile Print the filename of this plugin
75 bbdir Print the BBPATH directory of this plugin
76 count How many times have this plugin been registered.
77 multiloaded How many times have this plugin been initialized
78 Use devtool <subcommand> --help to get help on a specific command
79
80As directed in the general help output, you can
81get more syntax on a specific command by providing the command name and
82using "--help":
83::
84
85 $ devtool add --help
86 NOTE: Starting bitbake server...
87 usage: devtool add [-h] [--same-dir | --no-same-dir] [--fetch URI] [--npm-dev] [--version VERSION] [--no-git] [--srcrev SRCREV | --autorev] [--srcbranch SRCBRANCH] [--binary] [--also-native] [--src-subdir SUBDIR] [--mirrors]
88 [--provides PROVIDES]
89 [recipename] [srctree] [fetchuri]
90
91 Adds a new recipe to the workspace to build a specified source tree. Can optionally fetch a remote URI and unpack it to create the source tree.
92
93 arguments:
94 recipename Name for new recipe to add (just name - no version, path or extension). If not specified, will attempt to auto-detect it.
95 srctree Path to external source tree. If not specified, a subdirectory of /media/build1/poky/build/workspace/sources will be used.
96 fetchuri Fetch the specified URI and extract it to create the source tree
97
98 options:
99 -h, --help show this help message and exit
100 --same-dir, -s Build in same directory as source
101 --no-same-dir Force build in a separate build directory
102 --fetch URI, -f URI Fetch the specified URI and extract it to create the source tree (deprecated - pass as positional argument instead)
103 --npm-dev For npm, also fetch devDependencies
104 --version VERSION, -V VERSION
105 Version to use within recipe (PV)
106 --no-git, -g If fetching source, do not set up source tree as a git repository
107 --srcrev SRCREV, -S SRCREV
108 Source revision to fetch if fetching from an SCM such as git (default latest)
109 --autorev, -a When fetching from a git repository, set SRCREV in the recipe to a floating revision instead of fixed
110 --srcbranch SRCBRANCH, -B SRCBRANCH
111 Branch in source repository if fetching from an SCM such as git (default master)
112 --binary, -b Treat the source tree as something that should be installed verbatim (no compilation, same directory structure). Useful with binary packages e.g. RPMs.
113 --also-native Also add native variant (i.e. support building recipe for the build host as well as the target machine)
114 --src-subdir SUBDIR Specify subdirectory within source tree to use
115 --mirrors Enable PREMIRRORS and MIRRORS for source tree fetching (disable by default).
116 --provides PROVIDES, -p PROVIDES
117 Specify an alias for the item provided by the recipe. E.g. virtual/libgl
118
119.. _devtool-the-workspace-layer-structure:
120
121The Workspace Layer Structure
122=============================
123
124``devtool`` uses a "Workspace" layer in which to accomplish builds. This
125layer is not specific to any single ``devtool`` command but is rather a
126common working area used across the tool.
127
128The following figure shows the workspace structure:
129
130.. image:: figures/build-workspace-directory.png
131 :align: center
132 :scale: 70%
133
134::
135
136 attic - A directory created if devtool believes it must preserve
137 anything when you run "devtool reset". For example, if you
138 run "devtool add", make changes to the recipe, and then
139 run "devtool reset", devtool takes notice that the file has
140 been changed and moves it into the attic should you still
141 want the recipe.
142
143 README - Provides information on what is in workspace layer and how to
144 manage it.
145
146 .devtool_md5 - A checksum file used by devtool.
147
148 appends - A directory that contains *.bbappend files, which point to
149 external source.
150
151 conf - A configuration directory that contains the layer.conf file.
152
153 recipes - A directory containing recipes. This directory contains a
154 folder for each directory added whose name matches that of the
155 added recipe. devtool places the recipe.bb file
156 within that sub-directory.
157
158 sources - A directory containing a working copy of the source files used
159 when building the recipe. This is the default directory used
160 as the location of the source tree when you do not provide a
161 source tree path. This directory contains a folder for each
162 set of source files matched to a corresponding recipe.
163
164.. _devtool-adding-a-new-recipe-to-the-workspace:
165
166Adding a New Recipe to the Workspace Layer
167==========================================
168
169Use the ``devtool add`` command to add a new recipe to the workspace
170layer. The recipe you add should not exist - ``devtool`` creates it for
171you. The source files the recipe uses should exist in an external area.
172
173The following example creates and adds a new recipe named ``jackson`` to
174a workspace layer the tool creates. The source code built by the recipes
175resides in ``/home/user/sources/jackson``:
176::
177
178 $ devtool add jackson /home/user/sources/jackson
179
180If you add a recipe and the workspace layer does not exist, the command
181creates the layer and populates it as described in "`The Workspace Layer
182Structure <#devtool-the-workspace-layer-structure>`__" section.
183
184Running ``devtool add`` when the workspace layer exists causes the tool
185to add the recipe, append files, and source files into the existing
186workspace layer. The ``.bbappend`` file is created to point to the
187external source tree.
188
189.. note::
190
191 If your recipe has runtime dependencies defined, you must be sure
192 that these packages exist on the target hardware before attempting to
193 run your application. If dependent packages (e.g. libraries) do not
194 exist on the target, your application, when run, will fail to find
195 those functions. For more information, see the
196 ":ref:`ref-manual/ref-devtool-reference:deploying your software on the target machine`"
197 section.
198
199By default, ``devtool add`` uses the latest revision (i.e. master) when
200unpacking files from a remote URI. In some cases, you might want to
201specify a source revision by branch, tag, or commit hash. You can
202specify these options when using the ``devtool add`` command:
203
204- To specify a source branch, use the ``--srcbranch`` option:
205 ::
206
207 $ devtool add --srcbranch DISTRO_NAME_NO_CAP jackson /home/user/sources/jackson
208
209 In the previous example, you are checking out the DISTRO_NAME_NO_CAP
210 branch.
211
212- To specify a specific tag or commit hash, use the ``--srcrev``
213 option:
214 ::
215
216 $ devtool add --srcrev DISTRO_REL_TAG jackson /home/user/sources/jackson
217 $ devtool add --srcrev some_commit_hash /home/user/sources/jackson
218
219 The previous examples check out the
220 DISTRO_REL_TAG tag and the commit associated with the
221 some_commit_hash hash.
222
223.. note::
224
225 If you prefer to use the latest revision every time the recipe is
226 built, use the options --autorev or -a.
227
228.. _devtool-extracting-the-source-for-an-existing-recipe:
229
230Extracting the Source for an Existing Recipe
231============================================
232
233Use the ``devtool extract`` command to extract the source for an
234existing recipe. When you use this command, you must supply the root
235name of the recipe (i.e. no version, paths, or extensions), and you must
236supply the directory to which you want the source extracted.
237
238Additional command options let you control the name of a development
239branch into which you can checkout the source and whether or not to keep
240a temporary directory, which is useful for debugging.
241
242.. _devtool-synchronizing-a-recipes-extracted-source-tree:
243
244Synchronizing a Recipe's Extracted Source Tree
245==============================================
246
247Use the ``devtool sync`` command to synchronize a previously extracted
248source tree for an existing recipe. When you use this command, you must
249supply the root name of the recipe (i.e. no version, paths, or
250extensions), and you must supply the directory to which you want the
251source extracted.
252
253Additional command options let you control the name of a development
254branch into which you can checkout the source and whether or not to keep
255a temporary directory, which is useful for debugging.
256
257.. _devtool-modifying-a-recipe:
258
259Modifying an Existing Recipe
260============================
261
262Use the ``devtool modify`` command to begin modifying the source of an
263existing recipe. This command is very similar to the
264```add`` <#devtool-adding-a-new-recipe-to-the-workspace>`__ command
265except that it does not physically create the recipe in the workspace
266layer because the recipe already exists in an another layer.
267
268The ``devtool modify`` command extracts the source for a recipe, sets it
269up as a Git repository if the source had not already been fetched from
270Git, checks out a branch for development, and applies any patches from
271the recipe as commits on top. You can use the following command to
272checkout the source files:
273::
274
275 $ devtool modify recipe
276
277Using the above command form, ``devtool`` uses the existing recipe's
278:term:`SRC_URI` statement to locate the upstream source,
279extracts the source into the default sources location in the workspace.
280The default development branch used is "devtool".
281
282.. _devtool-edit-an-existing-recipe:
283
284Edit an Existing Recipe
285=======================
286
287Use the ``devtool edit-recipe`` command to run the default editor, which
288is identified using the ``EDITOR`` variable, on the specified recipe.
289
290When you use the ``devtool edit-recipe`` command, you must supply the
291root name of the recipe (i.e. no version, paths, or extensions). Also,
292the recipe file itself must reside in the workspace as a result of the
293``devtool add`` or ``devtool upgrade`` commands. However, you can
294override that requirement by using the "-a" or "--any-recipe" option.
295Using either of these options allows you to edit any recipe regardless
296of its location.
297
298.. _devtool-updating-a-recipe:
299
300Updating a Recipe
301=================
302
303Use the ``devtool update-recipe`` command to update your recipe with
304patches that reflect changes you make to the source files. For example,
305if you know you are going to work on some code, you could first use the
306```devtool modify`` <#devtool-modifying-a-recipe>`__ command to extract
307the code and set up the workspace. After which, you could modify,
308compile, and test the code.
309
310When you are satisfied with the results and you have committed your
311changes to the Git repository, you can then run the
312``devtool update-recipe`` to create the patches and update the recipe:
313::
314
315 $ devtool update-recipe recipe
316
317If you run the ``devtool update-recipe``
318without committing your changes, the command ignores the changes.
319
320Often, you might want to apply customizations made to your software in
321your own layer rather than apply them to the original recipe. If so, you
322can use the ``-a`` or ``--append`` option with the
323``devtool update-recipe`` command. These options allow you to specify
324the layer into which to write an append file:
325::
326
327 $ devtool update-recipe recipe -a base-layer-directory
328
329The ``*.bbappend`` file is created at the
330appropriate path within the specified layer directory, which may or may
331not be in your ``bblayers.conf`` file. If an append file already exists,
332the command updates it appropriately.
333
334.. _devtool-checking-on-the-upgrade-status-of-a-recipe:
335
336Checking on the Upgrade Status of a Recipe
337==========================================
338
339Upstream recipes change over time. Consequently, you might find that you
340need to determine if you can upgrade a recipe to a newer version.
341
342To check on the upgrade status of a recipe, use the
343``devtool check-upgrade-status`` command. The command displays a table
344of your current recipe versions, the latest upstream versions, the email
345address of the recipe's maintainer, and any additional information such
346as commit hash strings and reasons you might not be able to upgrade a
347particular recipe.
348
349.. note::
350
351 - For the ``oe-core`` layer, recipe maintainers come from the
352 `maintainers.inc <http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/distro/include/maintainers.inc>`_
353 file.
354
355 - If the recipe is using the :ref:`bitbake:git-fetcher`
356 rather than a
357 tarball, the commit hash points to the commit that matches the
358 recipe's latest version tag.
359
360As with all ``devtool`` commands, you can get help on the individual
361command:
362::
363
364 $ devtool check-upgrade-status -h
365 NOTE: Starting bitbake server...
366 usage: devtool check-upgrade-status [-h] [--all] [recipe [recipe ...]]
367
368 Prints a table of recipes together with versions currently provided by recipes, and latest upstream versions, when there is a later version available
369
370 arguments:
371 recipe Name of the recipe to report (omit to report upgrade info for all recipes)
372
373 options:
374 -h, --help show this help message and exit
375 --all, -a Show all recipes, not just recipes needing upgrade
376
377Unless you provide a specific recipe name on the command line, the
378command checks all recipes in all configured layers.
379
380Following is a partial example table that reports on all the recipes.
381Notice the reported reason for not upgrading the ``base-passwd`` recipe.
382In this example, while a new version is available upstream, you do not
383want to use it because the dependency on ``cdebconf`` is not easily
384satisfied.
385
386.. note::
387
388 When a reason for not upgrading displays, the reason is usually
389 written into the recipe using the RECIPE_NO_UPDATE_REASON
390 variable. See the base-passwd.bb recipe for an example.
391
392::
393
394 $ devtool check-upgrade-status ...
395 NOTE: acpid 2.0.30 2.0.31 Ross Burton <ross.burton@intel.com>
396 NOTE: u-boot-fw-utils 2018.11 2019.01 Marek Vasut <marek.vasut@gmail.com> d3689267f92c5956e09cc7d1baa4700141662bff
397 NOTE: u-boot-tools 2018.11 2019.01 Marek Vasut <marek.vasut@gmail.com> d3689267f92c5956e09cc7d1baa4700141662bff . . .
398 NOTE: base-passwd 3.5.29 3.5.45 Anuj Mittal <anuj.mittal@intel.com> cannot be updated due to: Version 3.5.38 requires cdebconf for update-passwd utility
399 NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
400 NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
401
402.. _devtool-upgrading-a-recipe:
403
404Upgrading a Recipe
405==================
406
407As software matures, upstream recipes are upgraded to newer versions. As
408a developer, you need to keep your local recipes up-to-date with the
409upstream version releases. Several methods exist by which you can
410upgrade recipes. You can read about them in the ":ref:`gs-upgrading-recipes`"
411section of the Yocto Project Development Tasks Manual. This section
412overviews the ``devtool upgrade`` command.
413
414Before you upgrade a recipe, you can check on its upgrade status. See
415the ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" section
416for more information.
417
418The ``devtool upgrade`` command upgrades an existing recipe to a more
419recent version of the recipe upstream. The command puts the upgraded
420recipe file along with any associated files into a "workspace" and, if
421necessary, extracts the source tree to a specified location. During the
422upgrade, patches associated with the recipe are rebased or added as
423needed.
424
425When you use the ``devtool upgrade`` command, you must supply the root
426name of the recipe (i.e. no version, paths, or extensions), and you must
427supply the directory to which you want the source extracted. Additional
428command options let you control things such as the version number to
429which you want to upgrade (i.e. the :term:`PV`), the source
430revision to which you want to upgrade (i.e. the
431:term:`SRCREV`), whether or not to apply patches, and so
432forth.
433
434You can read more on the ``devtool upgrade`` workflow in the
435":ref:`sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software`"
436section in the Yocto Project Application Development and the Extensible
437Software Development Kit (eSDK) manual. You can also see an example of
438how to use ``devtool upgrade`` in the ":ref:`gs-using-devtool-upgrade`"
439section in the Yocto Project Development Tasks Manual.
440
441.. _devtool-resetting-a-recipe:
442
443Resetting a Recipe
444==================
445
446Use the ``devtool reset`` command to remove a recipe and its
447configuration (e.g. the corresponding ``.bbappend`` file) from the
448workspace layer. Realize that this command deletes the recipe and the
449append file. The command does not physically move them for you.
450Consequently, you must be sure to physically relocate your updated
451recipe and the append file outside of the workspace layer before running
452the ``devtool reset`` command.
453
454If the ``devtool reset`` command detects that the recipe or the append
455files have been modified, the command preserves the modified files in a
456separate "attic" subdirectory under the workspace layer.
457
458Here is an example that resets the workspace directory that contains the
459``mtr`` recipe:
460::
461
462 $ devtool reset mtr
463 NOTE: Cleaning sysroot for recipe mtr...
464 NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/mtr as-is; if you no longer need it then please delete it manually
465 $
466
467.. _devtool-building-your-recipe:
468
469Building Your Recipe
470====================
471
472Use the ``devtool build`` command to build your recipe. The
473``devtool build`` command is equivalent to the
474``bitbake -c populate_sysroot`` command.
475
476When you use the ``devtool build`` command, you must supply the root
477name of the recipe (i.e. do not provide versions, paths, or extensions).
478You can use either the "-s" or the "--disable-parallel-make" options to
479disable parallel makes during the build. Here is an example:
480::
481
482 $ devtool build recipe
483
484.. _devtool-building-your-image:
485
486Building Your Image
487===================
488
489Use the ``devtool build-image`` command to build an image, extending it
490to include packages from recipes in the workspace. Using this command is
491useful when you want an image that ready for immediate deployment onto a
492device for testing. For proper integration into a final image, you need
493to edit your custom image recipe appropriately.
494
495When you use the ``devtool build-image`` command, you must supply the
496name of the image. This command has no command line options:
497::
498
499 $ devtool build-image image
500
501.. _devtool-deploying-your-software-on-the-target-machine:
502
503Deploying Your Software on the Target Machine
504=============================================
505
506Use the ``devtool deploy-target`` command to deploy the recipe's build
507output to the live target machine:
508::
509
510 $ devtool deploy-target recipe target
511
512The target is the address of the target machine, which must be running
513an SSH server (i.e. ``user@hostname[:destdir]``).
514
515This command deploys all files installed during the
516:ref:`ref-tasks-install` task. Furthermore, you do not
517need to have package management enabled within the target machine. If
518you do, the package manager is bypassed.
519
520.. note::
521
522 The ``deploy-target`` functionality is for development only. You
523 should never use it to update an image that will be used in
524 production.
525
526Some conditions exist that could prevent a deployed application from
527behaving as expected. When both of the following conditions exist, your
528application has the potential to not behave correctly when run on the
529target:
530
531- You are deploying a new application to the target and the recipe you
532 used to build the application had correctly defined runtime
533 dependencies.
534
535- The target does not physically have the packages on which the
536 application depends installed.
537
538If both of these conditions exist, your application will not behave as
539expected. The reason for this misbehavior is because the
540``devtool deploy-target`` command does not deploy the packages (e.g.
541libraries) on which your new application depends. The assumption is that
542the packages are already on the target. Consequently, when a runtime
543call is made in the application for a dependent function (e.g. a library
544call), the function cannot be found.
545
546To be sure you have all the dependencies local to the target, you need
547to be sure that the packages are pre-deployed (installed) on the target
548before attempting to run your application.
549
550.. _devtool-removing-your-software-from-the-target-machine:
551
552Removing Your Software from the Target Machine
553==============================================
554
555Use the ``devtool undeploy-target`` command to remove deployed build
556output from the target machine. For the ``devtool undeploy-target``
557command to work, you must have previously used the
558":ref:`devtool deploy-target <ref-manual/ref-devtool-reference:deploying your software on the target machine>`"
559command.
560::
561
562 $ devtool undeploy-target recipe target
563
564The target is the
565address of the target machine, which must be running an SSH server (i.e.
566``user@hostname``).
567
568.. _devtool-creating-the-workspace:
569
570Creating the Workspace Layer in an Alternative Location
571=======================================================
572
573Use the ``devtool create-workspace`` command to create a new workspace
574layer in your :term:`Build Directory`. When you create a
575new workspace layer, it is populated with the ``README`` file and the
576``conf`` directory only.
577
578The following example creates a new workspace layer in your current
579working and by default names the workspace layer "workspace":
580::
581
582 $ devtool create-workspace
583
584You can create a workspace layer anywhere by supplying a pathname with
585the command. The following command creates a new workspace layer named
586"new-workspace":
587::
588
589 $ devtool create-workspace /home/scottrif/new-workspace
590
591.. _devtool-get-the-status-of-the-recipes-in-your-workspace:
592
593Get the Status of the Recipes in Your Workspace
594===============================================
595
596Use the ``devtool status`` command to list the recipes currently in your
597workspace. Information includes the paths to their respective external
598source trees.
599
600The ``devtool status`` command has no command-line options:
601::
602
603 $ devtool status
604
605Following is sample output after using
606:ref:`devtool add <ref-manual/ref-devtool-reference:adding a new recipe to the workspace layer>`
607to create and add the ``mtr_0.86.bb`` recipe to the ``workspace`` directory:
608::
609
610 $ devtool status mtr
611 :/home/scottrif/poky/build/workspace/sources/mtr (/home/scottrif/poky/build/workspace/recipes/mtr/mtr_0.86.bb)
612 $
613
614.. _devtool-search-for-available-target-recipes:
615
616Search for Available Target Recipes
617===================================
618
619Use the ``devtool search`` command to search for available target
620recipes. The command matches the recipe name, package name, description,
621and installed files. The command displays the recipe name as a result of
622a match.
623
624When you use the ``devtool search`` command, you must supply a keyword.
625The command uses the keyword when searching for a match.
diff --git a/documentation/ref-manual/ref-devtool-reference.xml b/documentation/ref-manual/ref-devtool-reference.xml
index 11f7399c5a..6c3ccc3034 100644
--- a/documentation/ref-manual/ref-devtool-reference.xml
+++ b/documentation/ref-manual/ref-devtool-reference.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-devtool-reference'> 6<chapter id='ref-devtool-reference'>
6 <title><filename>devtool</filename> Quick Reference</title> 7 <title><filename>devtool</filename> Quick Reference</title>
diff --git a/documentation/ref-manual/ref-features.rst b/documentation/ref-manual/ref-features.rst
new file mode 100644
index 0000000000..ae5a0e3b24
--- /dev/null
+++ b/documentation/ref-manual/ref-features.rst
@@ -0,0 +1,353 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3********
4Features
5********
6
7This chapter provides a reference of shipped machine and distro features
8you can include as part of your image, a reference on image features you
9can select, and a reference on feature backfilling.
10
11Features provide a mechanism for working out which packages should be
12included in the generated images. Distributions can select which
13features they want to support through the ``DISTRO_FEATURES`` variable,
14which is set or appended to in a distribution's configuration file such
15as ``poky.conf``, ``poky-tiny.conf``, ``poky-lsb.conf`` and so forth.
16Machine features are set in the ``MACHINE_FEATURES`` variable, which is
17set in the machine configuration file and specifies the hardware
18features for a given machine.
19
20These two variables combine to work out which kernel modules, utilities,
21and other packages to include. A given distribution can support a
22selected subset of features so some machine features might not be
23included if the distribution itself does not support them.
24
25One method you can use to determine which recipes are checking to see if
26a particular feature is contained or not is to ``grep`` through the
27:term:`Metadata` for the feature. Here is an example that
28discovers the recipes whose build is potentially changed based on a
29given feature:
30::
31
32 $ cd poky
33 $ git grep 'contains.*MACHINE_FEATURES.*feature'
34
35.. _ref-features-machine:
36
37Machine Features
38================
39
40The items below are features you can use with
41:term:`MACHINE_FEATURES`. Features do not have a
42one-to-one correspondence to packages, and they can go beyond simply
43controlling the installation of a package or packages. Sometimes a
44feature can influence how certain recipes are built. For example, a
45feature might determine whether a particular configure option is
46specified within the :ref:`ref-tasks-configure` task
47for a particular recipe.
48
49This feature list only represents features as shipped with the Yocto
50Project metadata:
51
52- *acpi:* Hardware has ACPI (x86/x86_64 only)
53
54- *alsa:* Hardware has ALSA audio drivers
55
56- *apm:* Hardware uses APM (or APM emulation)
57
58- *bluetooth:* Hardware has integrated BT
59
60- *efi:* Support for booting through EFI
61
62- *ext2:* Hardware HDD or Microdrive
63
64- *keyboard:* Hardware has a keyboard
65
66- *pcbios:* Support for booting through BIOS
67
68- *pci:* Hardware has a PCI bus
69
70- *pcmcia:* Hardware has PCMCIA or CompactFlash sockets
71
72- *phone:* Mobile phone (voice) support
73
74- *qvga:* Machine has a QVGA (320x240) display
75
76- *rtc:* Machine has a Real-Time Clock
77
78- *screen:* Hardware has a screen
79
80- *serial:* Hardware has serial support (usually RS232)
81
82- *touchscreen:* Hardware has a touchscreen
83
84- *usbgadget:* Hardware is USB gadget device capable
85
86- *usbhost:* Hardware is USB Host capable
87
88- *vfat:* FAT file system support
89
90- *wifi:* Hardware has integrated WiFi
91
92.. _ref-features-distro:
93
94Distro Features
95===============
96
97The items below are features you can use with
98:term:`DISTRO_FEATURES` to enable features across
99your distribution. Features do not have a one-to-one correspondence to
100packages, and they can go beyond simply controlling the installation of
101a package or packages. In most cases, the presence or absence of a
102feature translates to the appropriate option supplied to the configure
103script during the :ref:`ref-tasks-configure` task for
104the recipes that optionally support the feature.
105
106Some distro features are also machine features. These select features
107make sense to be controlled both at the machine and distribution
108configuration level. See the
109:term:`COMBINED_FEATURES` variable for more
110information.
111
112This list only represents features as shipped with the Yocto Project
113metadata:
114
115- *alsa:* Include ALSA support (OSS compatibility kernel modules
116 installed if available).
117
118- *api-documentation:* Enables generation of API documentation during
119 recipe builds. The resulting documentation is added to SDK tarballs
120 when the ``bitbake -c populate_sdk`` command is used. See the
121 ":ref:`sdk-manual/sdk-appendix-customizing-standard:adding api documentation to the standard sdk`"
122 section in the Yocto Project Application Development and the
123 Extensible Software Development Kit (eSDK) manual.
124
125- *bluetooth:* Include bluetooth support (integrated BT only).
126
127- *cramfs:* Include CramFS support.
128
129- *directfb:* Include DirectFB support.
130
131- *ext2:* Include tools for supporting for devices with internal
132 HDD/Microdrive for storing files (instead of Flash only devices).
133
134- *ipsec:* Include IPSec support.
135
136- *ipv6:* Include IPv6 support.
137
138- *keyboard:* Include keyboard support (e.g. keymaps will be loaded
139 during boot).
140
141- *ldconfig:* Include support for ldconfig and ``ld.so.conf`` on the
142 target.
143
144- *nfs:* Include NFS client support (for mounting NFS exports on
145 device).
146
147- *opengl:* Include the Open Graphics Library, which is a
148 cross-language, multi-platform application programming interface used
149 for rendering two and three-dimensional graphics.
150
151- *pci:* Include PCI bus support.
152
153- *pcmcia:* Include PCMCIA/CompactFlash support.
154
155- *ppp:* Include PPP dialup support.
156
157- *ptest:* Enables building the package tests where supported by
158 individual recipes. For more information on package tests, see the
159 ":ref:`dev-manual/dev-manual-common-tasks:testing packages with ptest`" section
160 in the Yocto Project Development Tasks Manual.
161
162- *smbfs:* Include SMB networks client support (for mounting
163 Samba/Microsoft Windows shares on device).
164
165- *systemd:* Include support for this ``init`` manager, which is a full
166 replacement of for ``init`` with parallel starting of services,
167 reduced shell overhead, and other features. This ``init`` manager is
168 used by many distributions.
169
170- *usbgadget:* Include USB Gadget Device support (for USB
171 networking/serial/storage).
172
173- *usbhost:* Include USB Host support (allows to connect external
174 keyboard, mouse, storage, network etc).
175
176- *usrmerge:* Merges the ``/bin``, ``/sbin``, ``/lib``, and ``/lib64``
177 directories into their respective counterparts in the ``/usr``
178 directory to provide better package and application compatibility.
179
180- *wayland:* Include the Wayland display server protocol and the
181 library that supports it.
182
183- *wifi:* Include WiFi support (integrated only).
184
185- *x11:* Include the X server and libraries.
186
187.. _ref-features-image:
188
189Image Features
190==============
191
192The contents of images generated by the OpenEmbedded build system can be
193controlled by the :term:`IMAGE_FEATURES` and
194:term:`EXTRA_IMAGE_FEATURES` variables that
195you typically configure in your image recipes. Through these variables,
196you can add several different predefined packages such as development
197utilities or packages with debug information needed to investigate
198application problems or profile applications.
199
200The following image features are available for all images:
201
202- *allow-empty-password:* Allows Dropbear and OpenSSH to accept root
203 logins and logins from accounts having an empty password string.
204
205- *dbg-pkgs:* Installs debug symbol packages for all packages installed
206 in a given image.
207
208- *debug-tweaks:* Makes an image suitable for development (e.g. allows
209 root logins without passwords and enables post-installation logging).
210 See the 'allow-empty-password', 'empty-root-password', and
211 'post-install-logging' features in this list for additional
212 information.
213
214- *dev-pkgs:* Installs development packages (headers and extra library
215 links) for all packages installed in a given image.
216
217- *doc-pkgs:* Installs documentation packages for all packages
218 installed in a given image.
219
220- *empty-root-password:* Sets the root password to an empty string,
221 which allows logins with a blank password.
222
223- *package-management:* Installs package management tools and preserves
224 the package manager database.
225
226- *post-install-logging:* Enables logging postinstall script runs to
227 the ``/var/log/postinstall.log`` file on first boot of the image on
228 the target system.
229
230 .. note::
231
232 To make the
233 /var/log
234 directory on the target persistent, use the
235 VOLATILE_LOG_DIR
236 variable by setting it to "no".
237
238- *ptest-pkgs:* Installs ptest packages for all ptest-enabled recipes.
239
240- *read-only-rootfs:* Creates an image whose root filesystem is
241 read-only. See the
242 ":ref:`dev-manual/dev-manual-common-tasks:creating a read-only root filesystem`"
243 section in the Yocto Project Development Tasks Manual for more
244 information.
245
246- *splash:* Enables showing a splash screen during boot. By default,
247 this screen is provided by ``psplash``, which does allow
248 customization. If you prefer to use an alternative splash screen
249 package, you can do so by setting the ``SPLASH`` variable to a
250 different package name (or names) within the image recipe or at the
251 distro configuration level.
252
253- *staticdev-pkgs:* Installs static development packages, which are
254 static libraries (i.e. ``*.a`` files), for all packages installed in
255 a given image.
256
257Some image features are available only when you inherit the
258:ref:`core-image <ref-classes-core-image>` class. The current list of
259these valid features is as follows:
260
261- *hwcodecs:* Installs hardware acceleration codecs.
262
263- *nfs-server:* Installs an NFS server.
264
265- *perf:* Installs profiling tools such as ``perf``, ``systemtap``, and
266 ``LTTng``. For general information on user-space tools, see the
267 :doc:`../sdk-manual/sdk-manual` manual.
268
269- *ssh-server-dropbear:* Installs the Dropbear minimal SSH server.
270
271- *ssh-server-openssh:* Installs the OpenSSH SSH server, which is more
272 full-featured than Dropbear. Note that if both the OpenSSH SSH server
273 and the Dropbear minimal SSH server are present in
274 ``IMAGE_FEATURES``, then OpenSSH will take precedence and Dropbear
275 will not be installed.
276
277- *tools-debug:* Installs debugging tools such as ``strace`` and
278 ``gdb``. For information on GDB, see the
279 ":ref:`platdev-gdb-remotedebug`" section
280 in the Yocto Project Development Tasks Manual. For information on
281 tracing and profiling, see the :doc:`../profile-manual/profile-manual`.
282
283- *tools-sdk:* Installs a full SDK that runs on the device.
284
285- *tools-testapps:* Installs device testing tools (e.g. touchscreen
286 debugging).
287
288- *x11:* Installs the X server.
289
290- *x11-base:* Installs the X server with a minimal environment.
291
292- *x11-sato:* Installs the OpenedHand Sato environment.
293
294.. _ref-features-backfill:
295
296Feature Backfilling
297===================
298
299Sometimes it is necessary in the OpenEmbedded build system to extend
300:term:`MACHINE_FEATURES` or
301:term:`DISTRO_FEATURES` to control functionality
302that was previously enabled and not able to be disabled. For these
303cases, we need to add an additional feature item to appear in one of
304these variables, but we do not want to force developers who have
305existing values of the variables in their configuration to add the new
306feature in order to retain the same overall level of functionality.
307Thus, the OpenEmbedded build system has a mechanism to automatically
308"backfill" these added features into existing distro or machine
309configurations. You can see the list of features for which this is done
310by finding the
311:term:`DISTRO_FEATURES_BACKFILL` and
312:term:`MACHINE_FEATURES_BACKFILL`
313variables in the ``meta/conf/bitbake.conf`` file.
314
315Because such features are backfilled by default into all configurations
316as described in the previous paragraph, developers who wish to disable
317the new features need to be able to selectively prevent the backfilling
318from occurring. They can do this by adding the undesired feature or
319features to the
320:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
321or
322:term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`
323variables for distro features and machine features respectively.
324
325Here are two examples to help illustrate feature backfilling:
326
327- *The "pulseaudio" distro feature option*: Previously, PulseAudio
328 support was enabled within the Qt and GStreamer frameworks. Because
329 of this, the feature is backfilled and thus enabled for all distros
330 through the ``DISTRO_FEATURES_BACKFILL`` variable in the
331 ``meta/conf/bitbake.conf`` file. However, your distro needs to
332 disable the feature. You can disable the feature without affecting
333 other existing distro configurations that need PulseAudio support by
334 adding "pulseaudio" to ``DISTRO_FEATURES_BACKFILL_CONSIDERED`` in
335 your distro's ``.conf`` file. Adding the feature to this variable
336 when it also exists in the ``DISTRO_FEATURES_BACKFILL`` variable
337 prevents the build system from adding the feature to your
338 configuration's ``DISTRO_FEATURES``, effectively disabling the
339 feature for that particular distro.
340
341- *The "rtc" machine feature option*: Previously, real time clock (RTC)
342 support was enabled for all target devices. Because of this, the
343 feature is backfilled and thus enabled for all machines through the
344 ``MACHINE_FEATURES_BACKFILL`` variable in the
345 ``meta/conf/bitbake.conf`` file. However, your target device does not
346 have this capability. You can disable RTC support for your device
347 without affecting other machines that need RTC support by adding the
348 feature to your machine's ``MACHINE_FEATURES_BACKFILL_CONSIDERED``
349 list in the machine's ``.conf`` file. Adding the feature to this
350 variable when it also exists in the ``MACHINE_FEATURES_BACKFILL``
351 variable prevents the build system from adding the feature to your
352 configuration's ``MACHINE_FEATURES``, effectively disabling RTC
353 support for that particular machine.
diff --git a/documentation/ref-manual/ref-features.xml b/documentation/ref-manual/ref-features.xml
index 294b297c20..8cab5ec3a8 100644
--- a/documentation/ref-manual/ref-features.xml
+++ b/documentation/ref-manual/ref-features.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-features'> 6<chapter id='ref-features'>
6 <title>Features</title> 7 <title>Features</title>
diff --git a/documentation/ref-manual/ref-images.rst b/documentation/ref-manual/ref-images.rst
new file mode 100644
index 0000000000..c88d4d75ca
--- /dev/null
+++ b/documentation/ref-manual/ref-images.rst
@@ -0,0 +1,139 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******
4Images
5******
6
7The OpenEmbedded build system provides several example images to satisfy
8different needs. When you issue the ``bitbake`` command you provide a
9"top-level" recipe that essentially begins the build for the type of
10image you want.
11
12.. note::
13
14 Building an image without GNU General Public License Version 3
15 (GPLv3), GNU Lesser General Public License Version 3 (LGPLv3), and
16 the GNU Affero General Public License Version 3 (AGPL-3.0) components
17 is only supported for minimal and base images. Furthermore, if you
18 are going to build an image using non-GPLv3 and similarly licensed
19 components, you must make the following changes in the
20 local.conf
21 file before using the BitBake command to build the minimal or base
22 image:
23 ::
24
25 1. Comment out the EXTRA_IMAGE_FEATURES line
26 2. Set INCOMPATIBLE_LICENSE = "GPL-3.0 LGPL-3.0 AGPL-3.0"
27
28
29From within the ``poky`` Git repository, you can use the following
30command to display the list of directories within the :term:`Source Directory`
31that contain image recipe files: ::
32
33 $ ls meta*/recipes*/images/*.bb
34
35Following is a list of supported recipes:
36
37- ``build-appliance-image``: An example virtual machine that contains
38 all the pieces required to run builds using the build system as well
39 as the build system itself. You can boot and run the image using
40 either the `VMware
41 Player <http://www.vmware.com/products/player/overview.html>`__ or
42 `VMware
43 Workstation <http://www.vmware.com/products/workstation/overview.html>`__.
44 For more information on this image, see the :yocto_home:`Build
45 Appliance </software-item/build-appliance>` page
46 on the Yocto Project website.
47
48- ``core-image-base``: A console-only image that fully supports the
49 target device hardware.
50
51- ``core-image-clutter``: An image with support for the Open GL-based
52 toolkit Clutter, which enables development of rich and animated
53 graphical user interfaces.
54
55- ``core-image-full-cmdline``: A console-only image with more
56 full-featured Linux system functionality installed.
57
58- ``core-image-lsb``: An image that conforms to the Linux Standard Base
59 (LSB) specification. This image requires a distribution configuration
60 that enables LSB compliance (e.g. ``poky-lsb``). If you build
61 ``core-image-lsb`` without that configuration, the image will not be
62 LSB-compliant.
63
64- ``core-image-lsb-dev``: A ``core-image-lsb`` image that is suitable
65 for development work using the host. The image includes headers and
66 libraries you can use in a host development environment. This image
67 requires a distribution configuration that enables LSB compliance
68 (e.g. ``poky-lsb``). If you build ``core-image-lsb-dev`` without that
69 configuration, the image will not be LSB-compliant.
70
71- ``core-image-lsb-sdk``: A ``core-image-lsb`` that includes everything
72 in the cross-toolchain but also includes development headers and
73 libraries to form a complete standalone SDK. This image requires a
74 distribution configuration that enables LSB compliance (e.g.
75 ``poky-lsb``). If you build ``core-image-lsb-sdk`` without that
76 configuration, the image will not be LSB-compliant. This image is
77 suitable for development using the target.
78
79- ``core-image-minimal``: A small image just capable of allowing a
80 device to boot.
81
82- ``core-image-minimal-dev``: A ``core-image-minimal`` image suitable
83 for development work using the host. The image includes headers and
84 libraries you can use in a host development environment.
85
86- ``core-image-minimal-initramfs``: A ``core-image-minimal`` image that
87 has the Minimal RAM-based Initial Root Filesystem (initramfs) as part
88 of the kernel, which allows the system to find the first "init"
89 program more efficiently. See the
90 :term:`PACKAGE_INSTALL` variable for
91 additional information helpful when working with initramfs images.
92
93- ``core-image-minimal-mtdutils``: A ``core-image-minimal`` image that
94 has support for the Minimal MTD Utilities, which let the user
95 interact with the MTD subsystem in the kernel to perform operations
96 on flash devices.
97
98- ``core-image-rt``: A ``core-image-minimal`` image plus a real-time
99 test suite and tools appropriate for real-time use.
100
101- ``core-image-rt-sdk``: A ``core-image-rt`` image that includes
102 everything in the cross-toolchain. The image also includes
103 development headers and libraries to form a complete stand-alone SDK
104 and is suitable for development using the target.
105
106- ``core-image-sato``: An image with Sato support, a mobile environment
107 and visual style that works well with mobile devices. The image
108 supports X11 with a Sato theme and applications such as a terminal,
109 editor, file manager, media player, and so forth.
110
111- ``core-image-sato-dev``: A ``core-image-sato`` image suitable for
112 development using the host. The image includes libraries needed to
113 build applications on the device itself, testing and profiling tools,
114 and debug symbols. This image was formerly ``core-image-sdk``.
115
116- ``core-image-sato-sdk``: A ``core-image-sato`` image that includes
117 everything in the cross-toolchain. The image also includes
118 development headers and libraries to form a complete standalone SDK
119 and is suitable for development using the target.
120
121- ``core-image-testmaster``: A "master" image designed to be used for
122 automated runtime testing. Provides a "known good" image that is
123 deployed to a separate partition so that you can boot into it and use
124 it to deploy a second image to be tested. You can find more
125 information about runtime testing in the
126 ":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
127 section in the Yocto Project Development Tasks Manual.
128
129- ``core-image-testmaster-initramfs``: A RAM-based Initial Root
130 Filesystem (initramfs) image tailored for use with the
131 ``core-image-testmaster`` image.
132
133- ``core-image-weston``: A very basic Wayland image with a terminal.
134 This image provides the Wayland protocol libraries and the reference
135 Weston compositor. For more information, see the
136 ":ref:`dev-manual/dev-manual-common-tasks:using wayland and weston`"
137 section in the Yocto Project Development Tasks Manual.
138
139- ``core-image-x11``: A very basic X11 image with a terminal.
diff --git a/documentation/ref-manual/ref-images.xml b/documentation/ref-manual/ref-images.xml
index 1f96186c6e..6f10a6fd2a 100644
--- a/documentation/ref-manual/ref-images.xml
+++ b/documentation/ref-manual/ref-images.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-images'> 6<chapter id='ref-images'>
6 <title>Images</title> 7 <title>Images</title>
@@ -8,7 +9,7 @@
8 <para> 9 <para>
9 The OpenEmbedded build system provides several example 10 The OpenEmbedded build system provides several example
10 images to satisfy different needs. 11 images to satisfy different needs.
11 When you issue the <filename>bitbake</filename> command you provide a top-level recipe 12 When you issue the <filename>bitbake</filename> command you provide a "top-level" recipe
12 that essentially begins the build for the type of image you want. 13 that essentially begins the build for the type of image you want.
13 </para> 14 </para>
14 15
@@ -99,7 +100,7 @@
99 <listitem><para id='images-core-image-minimal-initramfs'><filename>core-image-minimal-initramfs</filename>: 100 <listitem><para id='images-core-image-minimal-initramfs'><filename>core-image-minimal-initramfs</filename>:
100 A <filename>core-image-minimal</filename> image that has the Minimal RAM-based 101 A <filename>core-image-minimal</filename> image that has the Minimal RAM-based
101 Initial Root Filesystem (initramfs) as part of the kernel, 102 Initial Root Filesystem (initramfs) as part of the kernel,
102 which allows the system to find the first init program more efficiently. 103 which allows the system to find the first "init" program more efficiently.
103 See the 104 See the
104 <link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link> 105 <link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link>
105 variable for additional information helpful when working with 106 variable for additional information helpful when working with
diff --git a/documentation/ref-manual/ref-kickstart.rst b/documentation/ref-manual/ref-kickstart.rst
new file mode 100644
index 0000000000..45222de05b
--- /dev/null
+++ b/documentation/ref-manual/ref-kickstart.rst
@@ -0,0 +1,212 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************************************
4OpenEmbedded Kickstart (``.wks``) Reference
5*******************************************
6
7.. _openembedded-kickstart-wks-reference:
8
9Introduction
10============
11
12The current Wic implementation supports only the basic kickstart
13partitioning commands: ``partition`` (or ``part`` for short) and
14``bootloader``.
15
16.. note::
17
18 Future updates will implement more commands and options. If you use
19 anything that is not specifically supported, results can be
20 unpredictable.
21
22This chapter provides a reference on the available kickstart commands.
23The information lists the commands, their syntax, and meanings.
24Kickstart commands are based on the Fedora kickstart versions but with
25modifications to reflect Wic capabilities. You can see the original
26documentation for those commands at the following link:
27http://pykickstart.readthedocs.io/en/latest/kickstart-docs.html
28
29Command: part or partition
30==========================
31
32Either of these commands creates a partition on the system and uses the
33following syntax:
34::
35
36 part [mntpoint]
37 partition [mntpoint]
38
39If you do not
40provide mntpoint, Wic creates a partition but does not mount it.
41
42The ``mntpoint`` is where the partition is mounted and must be in one of
43the following forms:
44
45- ``/path``: For example, "/", "/usr", or "/home"
46
47- ``swap``: The created partition is used as swap space
48
49Specifying a mntpoint causes the partition to automatically be mounted.
50Wic achieves this by adding entries to the filesystem table (fstab)
51during image generation. In order for Wic to generate a valid fstab, you
52must also provide one of the ``--ondrive``, ``--ondisk``, or
53``--use-uuid`` partition options as part of the command.
54
55.. note::
56
57 The mount program must understand the PARTUUID syntax you use with
58 --use-uuid
59 and non-root
60 mountpoint
61 , including swap. The busybox versions of these application are
62 currently excluded.
63
64Here is an example that uses "/" as the mountpoint. The command uses
65``--ondisk`` to force the partition onto the ``sdb`` disk: part /
66--source rootfs --ondisk sdb --fstype=ext3 --label platform --align 1024
67
68Here is a list that describes other supported options you can use with
69the ``part`` and ``partition`` commands:
70
71- ``--size``: The minimum partition size in MBytes. Specify an
72 integer value such as 500. Do not append the number with "MB". You do
73 not need this option if you use ``--source``.
74
75- ``--fixed-size``: The exact partition size in MBytes. You cannot
76 specify with ``--size``. An error occurs when assembling the disk
77 image if the partition data is larger than ``--fixed-size``.
78
79- ``--source``: This option is a Wic-specific option that names the
80 source of the data that populates the partition. The most common
81 value for this option is "rootfs", but you can use any value that
82 maps to a valid source plugin. For information on the source plugins,
83 see the ":ref:`dev-manual/dev-manual-common-tasks:using the wic plugin interface`"
84 section in the Yocto Project Development Tasks Manual.
85
86 If you use ``--source rootfs``, Wic creates a partition as large as
87 needed and fills it with the contents of the root filesystem pointed
88 to by the ``-r`` command-line option or the equivalent rootfs derived
89 from the ``-e`` command-line option. The filesystem type used to
90 create the partition is driven by the value of the ``--fstype``
91 option specified for the partition. See the entry on ``--fstype``
92 that follows for more information.
93
94 If you use ``--source plugin-name``, Wic creates a partition as large
95 as needed and fills it with the contents of the partition that is
96 generated by the specified plugin name using the data pointed to by
97 the ``-r`` command-line option or the equivalent rootfs derived from
98 the ``-e`` command-line option. Exactly what those contents are and
99 filesystem type used are dependent on the given plugin
100 implementation.
101
102 If you do not use the ``--source`` option, the ``wic`` command
103 creates an empty partition. Consequently, you must use the ``--size``
104 option to specify the size of the empty partition.
105
106- ``--ondisk`` or ``--ondrive``: Forces the partition to be created
107 on a particular disk.
108
109- ``--fstype``: Sets the file system type for the partition. Valid
110 values are:
111
112 - ``ext4``
113
114 - ``ext3``
115
116 - ``ext2``
117
118 - ``btrfs``
119
120 - ``squashfs``
121
122 - ``swap``
123
124- ``--fsoptions``: Specifies a free-form string of options to be used
125 when mounting the filesystem. This string is copied into the
126 ``/etc/fstab`` file of the installed system and should be enclosed in
127 quotes. If not specified, the default string is "defaults".
128
129- ``--label label``: Specifies the label to give to the filesystem to
130 be made on the partition. If the given label is already in use by
131 another filesystem, a new label is created for the partition.
132
133- ``--active``: Marks the partition as active.
134
135- ``--align (in KBytes)``: This option is a Wic-specific option that
136 says to start partitions on boundaries given x KBytes.
137
138- ``--no-table``: This option is a Wic-specific option. Using the
139 option reserves space for the partition and causes it to become
140 populated. However, the partition is not added to the partition
141 table.
142
143- ``--exclude-path``: This option is a Wic-specific option that
144 excludes the given relative path from the resulting image. This
145 option is only effective with the rootfs source plugin.
146
147- ``--extra-space``: This option is a Wic-specific option that adds
148 extra space after the space filled by the content of the partition.
149 The final size can exceed the size specified by the ``--size``
150 option. The default value is 10 Mbytes.
151
152- ``--overhead-factor``: This option is a Wic-specific option that
153 multiplies the size of the partition by the option's value. You must
154 supply a value greater than or equal to "1". The default value is
155 "1.3".
156
157- ``--part-name``: This option is a Wic-specific option that
158 specifies a name for GPT partitions.
159
160- ``--part-type``: This option is a Wic-specific option that
161 specifies the partition type globally unique identifier (GUID) for
162 GPT partitions. You can find the list of partition type GUIDs at
163 http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs.
164
165- ``--use-uuid``: This option is a Wic-specific option that causes
166 Wic to generate a random GUID for the partition. The generated
167 identifier is used in the bootloader configuration to specify the
168 root partition.
169
170- ``--uuid``: This option is a Wic-specific option that specifies the
171 partition UUID.
172
173- ``--fsuuid``: This option is a Wic-specific option that specifies
174 the filesystem UUID. You can generate or modify
175 :term:`WKS_FILE` with this option if a preconfigured
176 filesystem UUID is added to the kernel command line in the bootloader
177 configuration before you run Wic.
178
179- ``--system-id``: This option is a Wic-specific option that
180 specifies the partition system ID, which is a one byte long,
181 hexadecimal parameter with or without the 0x prefix.
182
183- ``--mkfs-extraopts``: This option specifies additional options to
184 pass to the ``mkfs`` utility. Some default options for certain
185 filesystems do not take effect. See Wic's help on kickstart (i.e.
186 ``wic help kickstart``).
187
188Command: bootloader
189===================
190
191This command specifies how the bootloader should be configured and
192supports the following options:
193
194.. note::
195
196 Bootloader functionality and boot partitions are implemented by the
197 various
198 --source
199 plugins that implement bootloader functionality. The bootloader
200 command essentially provides a means of modifying bootloader
201 configuration.
202
203- ``--timeout``: Specifies the number of seconds before the
204 bootloader times out and boots the default option.
205
206- ``--append``: Specifies kernel parameters. These parameters will be
207 added to the syslinux ``APPEND`` or ``grub`` kernel command line.
208
209- ``--configfile``: Specifies a user-defined configuration file for
210 the bootloader. You can provide a full pathname for the file or a
211 file that exists in the ``canned-wks`` folder. This option overrides
212 all other bootloader options.
diff --git a/documentation/ref-manual/ref-kickstart.xml b/documentation/ref-manual/ref-kickstart.xml
index 1128bd50d0..45db1c0ff8 100644
--- a/documentation/ref-manual/ref-kickstart.xml
+++ b/documentation/ref-manual/ref-kickstart.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-kickstart'> 6<chapter id='ref-kickstart'>
6<title>OpenEmbedded Kickstart (<filename>.wks</filename>) Reference</title> 7<title>OpenEmbedded Kickstart (<filename>.wks</filename>) Reference</title>
diff --git a/documentation/ref-manual/ref-manual-customization.xsl b/documentation/ref-manual/ref-manual-customization.xsl
index c58dd905b9..3181f618e2 100644
--- a/documentation/ref-manual/ref-manual-customization.xsl
+++ b/documentation/ref-manual/ref-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/ref-manual/ref-manual.rst b/documentation/ref-manual/ref-manual.rst
new file mode 100644
index 0000000000..a106af21d8
--- /dev/null
+++ b/documentation/ref-manual/ref-manual.rst
@@ -0,0 +1,31 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3==============================
4Yocto Project Reference Manual
5==============================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 ref-system-requirements
14 ref-terms
15 ref-release-process
16 migration
17 ref-structure
18 ref-classes
19 ref-tasks
20 ref-devtool-reference
21 ref-kickstart
22 ref-qa-checks
23 ref-images
24 ref-features
25 ref-variables
26 ref-varlocality
27 faq
28 resources
29 history
30
31.. include:: /boilerplate.rst
diff --git a/documentation/ref-manual/ref-manual.xml b/documentation/ref-manual/ref-manual.xml
index 1b82a41a7d..9a914f19cf 100755
--- a/documentation/ref-manual/ref-manual.xml
+++ b/documentation/ref-manual/ref-manual.xml
@@ -128,28 +128,8 @@
128 </revision> 128 </revision>
129 <revision> 129 <revision>
130 <revnumber>3.1</revnumber> 130 <revnumber>3.1</revnumber>
131 <date>April 2020</date>
132 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
133 </revision>
134 <revision>
135 <revnumber>3.1.1</revnumber>
136 <date>June 2020</date>
137 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
138 </revision>
139 <revision>
140 <revnumber>3.1.2</revnumber>
141 <date>August 2020</date>
142 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
143 </revision>
144 <revision>
145 <revnumber>3.1.3</revnumber>
146 <date>October 2020</date>
147 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
148 </revision>
149 <revision>
150 <revnumber>3.1.4</revnumber>
151 <date>&REL_MONTH_YEAR;</date> 131 <date>&REL_MONTH_YEAR;</date>
152 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 132 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
153 </revision> 133 </revision>
154 </revhistory> 134 </revhistory>
155 135
diff --git a/documentation/ref-manual/ref-qa-checks.rst b/documentation/ref-manual/ref-qa-checks.rst
new file mode 100644
index 0000000000..3e76ac1509
--- /dev/null
+++ b/documentation/ref-manual/ref-qa-checks.rst
@@ -0,0 +1,533 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************
4QA Error and Warning Messages
5*****************************
6
7.. _qa-introduction:
8
9Introduction
10============
11
12When building a recipe, the OpenEmbedded build system performs various
13QA checks on the output to ensure that common issues are detected and
14reported. Sometimes when you create a new recipe to build new software,
15it will build with no problems. When this is not the case, or when you
16have QA issues building any software, it could take a little time to
17resolve them.
18
19While it is tempting to ignore a QA message or even to disable QA
20checks, it is best to try and resolve any reported QA issues. This
21chapter provides a list of the QA messages and brief explanations of the
22issues you could encounter so that you can properly resolve problems.
23
24The next section provides a list of all QA error and warning messages
25based on a default configuration. Each entry provides the message or
26error form along with an explanation.
27
28.. note::
29
30 - At the end of each message, the name of the associated QA test (as
31 listed in the ":ref:`insane.bbclass <ref-classes-insane>`"
32 section) appears within square brackets.
33
34 - As mentioned, this list of error and warning messages is for QA
35 checks only. The list does not cover all possible build errors or
36 warnings you could encounter.
37
38 - Because some QA checks are disabled by default, this list does not
39 include all possible QA check errors and warnings.
40
41.. _qa-errors-and-warnings:
42
43Errors and Warnings
44===================
45
46- ``<packagename>: <path> is using libexec please relocate to <libexecdir> [libexec]``
47
48 The specified package contains files in ``/usr/libexec`` when the
49 distro configuration uses a different path for ``<libexecdir>`` By
50 default, ``<libexecdir>`` is ``$prefix/libexec``. However, this
51 default can be changed (e.g. ``${libdir}``).
52
53  
54
55- ``package <packagename> contains bad RPATH <rpath> in file <file> [rpaths]``
56
57 The specified binary produced by the recipe contains dynamic library
58 load paths (rpaths) that contain build system paths such as
59 :term:`TMPDIR`, which are incorrect for the target and
60 could potentially be a security issue. Check for bad ``-rpath``
61 options being passed to the linker in your
62 :ref:`ref-tasks-compile` log. Depending on the build
63 system used by the software being built, there might be a configure
64 option to disable rpath usage completely within the build of the
65 software.
66
67  
68
69- ``<packagename>: <file> contains probably-redundant RPATH <rpath> [useless-rpaths]``
70
71 The specified binary produced by the recipe contains dynamic library
72 load paths (rpaths) that on a standard system are searched by default
73 by the linker (e.g. ``/lib`` and ``/usr/lib``). While these paths
74 will not cause any breakage, they do waste space and are unnecessary.
75 Depending on the build system used by the software being built, there
76 might be a configure option to disable rpath usage completely within
77 the build of the software.
78
79  
80
81- ``<packagename> requires <files>, but no providers in its RDEPENDS [file-rdeps]``
82
83 A file-level dependency has been identified from the specified
84 package on the specified files, but there is no explicit
85 corresponding entry in :term:`RDEPENDS`. If
86 particular files are required at runtime then ``RDEPENDS`` should be
87 declared in the recipe to ensure the packages providing them are
88 built.
89
90  
91
92- ``<packagename1> rdepends on <packagename2>, but it isn't a build dependency? [build-deps]``
93
94 A runtime dependency exists between the two specified packages, but
95 there is nothing explicit within the recipe to enable the
96 OpenEmbedded build system to ensure that dependency is satisfied.
97 This condition is usually triggered by an
98 :term:`RDEPENDS` value being added at the packaging
99 stage rather than up front, which is usually automatic based on the
100 contents of the package. In most cases, you should change the recipe
101 to add an explicit ``RDEPENDS`` for the dependency.
102
103  
104
105- ``non -dev/-dbg/nativesdk- package contains symlink .so: <packagename> path '<path>' [dev-so]``
106
107 Symlink ``.so`` files are for development only, and should therefore
108 go into the ``-dev`` package. This situation might occur if you add
109 ``*.so*`` rather than ``*.so.*`` to a non-dev package. Change
110 :term:`FILES` (and possibly
111 :term:`PACKAGES`) such that the specified ``.so``
112 file goes into an appropriate ``-dev`` package.
113
114  
115
116- ``non -staticdev package contains static .a library: <packagename> path '<path>' [staticdev]``
117
118 Static ``.a`` library files should go into a ``-staticdev`` package.
119 Change :term:`FILES` (and possibly
120 :term:`PACKAGES`) such that the specified ``.a`` file
121 goes into an appropriate ``-staticdev`` package.
122
123  
124
125- ``<packagename>: found library in wrong location [libdir]``
126
127 The specified file may have been installed into an incorrect
128 (possibly hardcoded) installation path. For example, this test will
129 catch recipes that install ``/lib/bar.so`` when ``${base_libdir}`` is
130 "lib32". Another example is when recipes install
131 ``/usr/lib64/foo.so`` when ``${libdir}`` is "/usr/lib". False
132 positives occasionally exist. For these cases add "libdir" to
133 :term:`INSANE_SKIP` for the package.
134
135  
136
137- ``non debug package contains .debug directory: <packagename> path <path> [debug-files]``
138
139 The specified package contains a ``.debug`` directory, which should
140 not appear in anything but the ``-dbg`` package. This situation might
141 occur if you add a path which contains a ``.debug`` directory and do
142 not explicitly add the ``.debug`` directory to the ``-dbg`` package.
143 If this is the case, add the ``.debug`` directory explicitly to
144 ``FILES_${PN}-dbg``. See :term:`FILES` for additional
145 information on ``FILES``.
146
147  
148
149- ``Architecture did not match (<machine_arch> to <file_arch>) on <file> [arch]``
150
151 By default, the OpenEmbedded build system checks the Executable and
152 Linkable Format (ELF) type, bit size, and endianness of any binaries
153 to ensure they match the target architecture. This test fails if any
154 binaries do not match the type since there would be an
155 incompatibility. The test could indicate that the wrong compiler or
156 compiler options have been used. Sometimes software, like
157 bootloaders, might need to bypass this check. If the file you receive
158 the error for is firmware that is not intended to be executed within
159 the target operating system or is intended to run on a separate
160 processor within the device, you can add "arch" to
161 :term:`INSANE_SKIP` for the package. Another
162 option is to check the :ref:`ref-tasks-compile` log
163 and verify that the compiler options being used are correct.
164
165  
166
167- ``Bit size did not match (<machine_bits> to <file_bits>) <recipe> on <file> [arch]``
168
169 By default, the OpenEmbedded build system checks the Executable and
170 Linkable Format (ELF) type, bit size, and endianness of any binaries
171 to ensure they match the target architecture. This test fails if any
172 binaries do not match the type since there would be an
173 incompatibility. The test could indicate that the wrong compiler or
174 compiler options have been used. Sometimes software, like
175 bootloaders, might need to bypass this check. If the file you receive
176 the error for is firmware that is not intended to be executed within
177 the target operating system or is intended to run on a separate
178 processor within the device, you can add "arch" to
179 :term:`INSANE_SKIP` for the package. Another
180 option is to check the :ref:`ref-tasks-compile` log
181 and verify that the compiler options being used are correct.
182
183  
184
185- ``Endianness did not match (<machine_endianness> to <file_endianness>) on <file> [arch]``
186
187 By default, the OpenEmbedded build system checks the Executable and
188 Linkable Format (ELF) type, bit size, and endianness of any binaries
189 to ensure they match the target architecture. This test fails if any
190 binaries do not match the type since there would be an
191 incompatibility. The test could indicate that the wrong compiler or
192 compiler options have been used. Sometimes software, like
193 bootloaders, might need to bypass this check. If the file you receive
194 the error for is firmware that is not intended to be executed within
195 the target operating system or is intended to run on a separate
196 processor within the device, you can add "arch" to
197 :term:`INSANE_SKIP` for the package. Another
198 option is to check the :ref:`ref-tasks-compile` log
199 and verify that the compiler options being used are correct.
200
201  
202
203- ``ELF binary '<file>' has relocations in .text [textrel]``
204
205 The specified ELF binary contains relocations in its ``.text``
206 sections. This situation can result in a performance impact at
207 runtime.
208
209 Typically, the way to solve this performance issue is to add "-fPIC"
210 or "-fpic" to the compiler command-line options. For example, given
211 software that reads :term:`CFLAGS` when you build it,
212 you could add the following to your recipe:
213 ::
214
215 CFLAGS_append = " -fPIC "
216
217 For more information on text relocations at runtime, see
218 http://www.akkadia.org/drepper/textrelocs.html.
219
220  
221
222- ``No GNU_HASH in the elf binary: '<file>' [ldflags]``
223
224 This indicates that binaries produced when building the recipe have
225 not been linked with the :term:`LDFLAGS` options
226 provided by the build system. Check to be sure that the ``LDFLAGS``
227 variable is being passed to the linker command. A common workaround
228 for this situation is to pass in ``LDFLAGS`` using
229 :term:`TARGET_CC_ARCH` within the recipe as
230 follows:
231 ::
232
233 TARGET_CC_ARCH += "${LDFLAGS}"
234
235  
236
237- ``Package <packagename> contains Xorg driver (<driver>) but no xorg-abi- dependencies [xorg-driver-abi]``
238
239 The specified package contains an Xorg driver, but does not have a
240 corresponding ABI package dependency. The xserver-xorg recipe
241 provides driver ABI names. All drivers should depend on the ABI
242 versions that they have been built against. Driver recipes that
243 include ``xorg-driver-input.inc`` or ``xorg-driver-video.inc`` will
244 automatically get these versions. Consequently, you should only need
245 to explicitly add dependencies to binary driver recipes.
246
247  
248
249- ``The /usr/share/info/dir file is not meant to be shipped in a particular package. [infodir]``
250
251 The ``/usr/share/info/dir`` should not be packaged. Add the following
252 line to your :ref:`ref-tasks-install` task or to your
253 ``do_install_append`` within the recipe as follows:
254 ::
255
256 rm ${D}${infodir}/dir
257  
258
259- ``Symlink <path> in <packagename> points to TMPDIR [symlink-to-sysroot]``
260
261 The specified symlink points into :term:`TMPDIR` on the
262 host. Such symlinks will work on the host. However, they are clearly
263 invalid when running on the target. You should either correct the
264 symlink to use a relative path or remove the symlink.
265
266  
267
268- ``<file> failed sanity test (workdir) in path <path> [la]``
269
270 The specified ``.la`` file contains :term:`TMPDIR`
271 paths. Any ``.la`` file containing these paths is incorrect since
272 ``libtool`` adds the correct sysroot prefix when using the files
273 automatically itself.
274
275  
276
277- ``<file> failed sanity test (tmpdir) in path <path> [pkgconfig]``
278
279 The specified ``.pc`` file contains
280 :term:`TMPDIR`\ ``/``\ :term:`WORKDIR`
281 paths. Any ``.pc`` file containing these paths is incorrect since
282 ``pkg-config`` itself adds the correct sysroot prefix when the files
283 are accessed.
284
285  
286
287- ``<packagename> rdepends on <debug_packagename> [debug-deps]``
288
289 A dependency exists between the specified non-dbg package (i.e. a
290 package whose name does not end in ``-dbg``) and a package that is a
291 ``dbg`` package. The ``dbg`` packages contain debug symbols and are
292 brought in using several different methods:
293
294 - Using the ``dbg-pkgs``
295 :term:`IMAGE_FEATURES` value.
296
297 - Using :term:`IMAGE_INSTALL`.
298
299 - As a dependency of another ``dbg`` package that was brought in
300 using one of the above methods.
301
302 The dependency might have been automatically added because the
303 ``dbg`` package erroneously contains files that it should not contain
304 (e.g. a non-symlink ``.so`` file) or it might have been added
305 manually (e.g. by adding to :term:`RDEPENDS`).
306
307  
308
309- ``<packagename> rdepends on <dev_packagename> [dev-deps]``
310
311 A dependency exists between the specified non-dev package (a package
312 whose name does not end in ``-dev``) and a package that is a ``dev``
313 package. The ``dev`` packages contain development headers and are
314 usually brought in using several different methods:
315
316 - Using the ``dev-pkgs``
317 :term:`IMAGE_FEATURES` value.
318
319 - Using :term:`IMAGE_INSTALL`.
320
321 - As a dependency of another ``dev`` package that was brought in
322 using one of the above methods.
323
324 The dependency might have been automatically added (because the
325 ``dev`` package erroneously contains files that it should not have
326 (e.g. a non-symlink ``.so`` file) or it might have been added
327 manually (e.g. by adding to :term:`RDEPENDS`).
328
329  
330
331- ``<var>_<packagename> is invalid: <comparison> (<value>) only comparisons <, =, >, <=, and >= are allowed [dep-cmp]``
332
333 If you are adding a versioned dependency relationship to one of the
334 dependency variables (:term:`RDEPENDS`,
335 :term:`RRECOMMENDS`,
336 :term:`RSUGGESTS`,
337 :term:`RPROVIDES`,
338 :term:`RREPLACES`, or
339 :term:`RCONFLICTS`), you must only use the named
340 comparison operators. Change the versioned dependency values you are
341 adding to match those listed in the message.
342
343  
344
345- ``<recipename>: The compile log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [compile-host-path]``
346
347 The log for the :ref:`ref-tasks-compile` task
348 indicates that paths on the host were searched for files, which is
349 not appropriate when cross-compiling. Look for "is unsafe for
350 cross-compilation" or "CROSS COMPILE Badness" in the specified log
351 file.
352
353  
354
355- ``<recipename>: The install log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [install-host-path]``
356
357 The log for the :ref:`ref-tasks-install` task
358 indicates that paths on the host were searched for files, which is
359 not appropriate when cross-compiling. Look for "is unsafe for
360 cross-compilation" or "CROSS COMPILE Badness" in the specified log
361 file.
362
363  
364
365- ``This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. Rerun configure task after fixing this. The path was '<path>'``
366
367 The log for the :ref:`ref-tasks-configure` task
368 indicates that paths on the host were searched for files, which is
369 not appropriate when cross-compiling. Look for "is unsafe for
370 cross-compilation" or "CROSS COMPILE Badness" in the specified log
371 file.
372
373  
374
375- ``<packagename> doesn't match the [a-z0-9.+-]+ regex [pkgname]``
376
377 The convention within the OpenEmbedded build system (sometimes
378 enforced by the package manager itself) is to require that package
379 names are all lower case and to allow a restricted set of characters.
380 If your recipe name does not match this, or you add packages to
381 :term:`PACKAGES` that do not conform to the
382 convention, then you will receive this error. Rename your recipe. Or,
383 if you have added a non-conforming package name to ``PACKAGES``,
384 change the package name appropriately.
385
386  
387
388- ``<recipe>: configure was passed unrecognized options: <options> [unknown-configure-option]``
389
390 The configure script is reporting that the specified options are
391 unrecognized. This situation could be because the options were
392 previously valid but have been removed from the configure script. Or,
393 there was a mistake when the options were added and there is another
394 option that should be used instead. If you are unsure, consult the
395 upstream build documentation, the ``./configure --help`` output, and
396 the upstream change log or release notes. Once you have worked out
397 what the appropriate change is, you can update
398 :term:`EXTRA_OECONF`,
399 :term:`PACKAGECONFIG_CONFARGS`, or the
400 individual :term:`PACKAGECONFIG` option values
401 accordingly.
402
403  
404
405- ``Recipe <recipefile> has PN of "<recipename>" which is in OVERRIDES, this can result in unexpected behavior. [pn-overrides]``
406
407 The specified recipe has a name (:term:`PN`) value that
408 appears in :term:`OVERRIDES`. If a recipe is named
409 such that its ``PN`` value matches something already in ``OVERRIDES``
410 (e.g. ``PN`` happens to be the same as :term:`MACHINE`
411 or :term:`DISTRO`), it can have unexpected
412 consequences. For example, assignments such as
413 ``FILES_${PN} = "xyz"`` effectively turn into ``FILES = "xyz"``.
414 Rename your recipe (or if ``PN`` is being set explicitly, change the
415 ``PN`` value) so that the conflict does not occur. See
416 :term:`FILES` for additional information.
417
418  
419
420- ``<recipefile>: Variable <variable> is set as not being package specific, please fix this. [pkgvarcheck]``
421
422 Certain variables (:term:`RDEPENDS`,
423 :term:`RRECOMMENDS`,
424 :term:`RSUGGESTS`,
425 :term:`RCONFLICTS`,
426 :term:`RPROVIDES`,
427 :term:`RREPLACES`, :term:`FILES`,
428 ``pkg_preinst``, ``pkg_postinst``, ``pkg_prerm``, ``pkg_postrm``, and
429 :term:`ALLOW_EMPTY`) should always be set specific
430 to a package (i.e. they should be set with a package name override
431 such as ``RDEPENDS_${PN} = "value"`` rather than
432 ``RDEPENDS = "value"``). If you receive this error, correct any
433 assignments to these variables within your recipe.
434
435  
436
437- ``File '<file>' from <recipename> was already stripped, this will prevent future debugging! [already-stripped]``
438
439 Produced binaries have already been stripped prior to the build
440 system extracting debug symbols. It is common for upstream software
441 projects to default to stripping debug symbols for output binaries.
442 In order for debugging to work on the target using ``-dbg`` packages,
443 this stripping must be disabled.
444
445 Depending on the build system used by the software being built,
446 disabling this stripping could be as easy as specifying an additional
447 configure option. If not, disabling stripping might involve patching
448 the build scripts. In the latter case, look for references to "strip"
449 or "STRIP", or the "-s" or "-S" command-line options being specified
450 on the linker command line (possibly through the compiler command
451 line if preceded with "-Wl,").
452
453 .. note::
454
455 Disabling stripping here does not mean that the final packaged
456 binaries will be unstripped. Once the OpenEmbedded build system
457 splits out debug symbols to the
458 -dbg
459 package, it will then strip the symbols from the binaries.
460
461  
462
463- ``<packagename> is listed in PACKAGES multiple times, this leads to packaging errors. [packages-list]``
464
465 Package names must appear only once in the
466 :term:`PACKAGES` variable. You might receive this
467 error if you are attempting to add a package to ``PACKAGES`` that is
468 already in the variable's value.
469
470  
471
472- ``FILES variable for package <packagename> contains '//' which is invalid. Attempting to fix this but you should correct the metadata. [files-invalid]``
473
474 The string "//" is invalid in a Unix path. Correct all occurrences
475 where this string appears in a :term:`FILES` variable so
476 that there is only a single "/".
477
478  
479
480- ``<recipename>: Files/directories were installed but not shipped in any package [installed-vs-shipped]``
481
482 Files have been installed within the
483 :ref:`ref-tasks-install` task but have not been
484 included in any package by way of the :term:`FILES`
485 variable. Files that do not appear in any package cannot be present
486 in an image later on in the build process. You need to do one of the
487 following:
488
489 - Add the files to ``FILES`` for the package you want them to appear
490 in (e.g. ``FILES_${``\ :term:`PN`\ ``}`` for the main
491 package).
492
493 - Delete the files at the end of the ``do_install`` task if the
494 files are not needed in any package.
495
496  
497
498- ``<oldpackage>-<oldpkgversion> was registered as shlib provider for <library>, changing it to <newpackage>-<newpkgversion> because it was built later``
499
500 This message means that both ``<oldpackage>`` and ``<newpackage>``
501 provide the specified shared library. You can expect this message
502 when a recipe has been renamed. However, if that is not the case, the
503 message might indicate that a private version of a library is being
504 erroneously picked up as the provider for a common library. If that
505 is the case, you should add the library's ``.so`` file name to
506 :term:`PRIVATE_LIBS` in the recipe that provides
507 the private version of the library.
508
509- ``LICENSE_<packagename> includes licenses (<licenses>) that are not listed in LICENSE [unlisted-pkg-lics]``
510
511 The :term:`LICENSE` of the recipe should be a superset
512 of all the licenses of all packages produced by this recipe. In other
513 words, any license in ``LICENSE_*`` should also appear in
514 :term:`LICENSE`.
515
516  
517
518Configuring and Disabling QA Checks
519===================================
520
521You can configure the QA checks globally so that specific check failures
522either raise a warning or an error message, using the
523:term:`WARN_QA` and :term:`ERROR_QA`
524variables, respectively. You can also disable checks within a particular
525recipe using :term:`INSANE_SKIP`. For information on
526how to work with the QA checks, see the
527":ref:`insane.bbclass <ref-classes-insane>`" section.
528
529.. note::
530
531 Please keep in mind that the QA checks exist in order to detect real
532 or potential problems in the packaged output. So exercise caution
533 when disabling these checks.
diff --git a/documentation/ref-manual/ref-qa-checks.xml b/documentation/ref-manual/ref-qa-checks.xml
index 515106ae68..0071e4a55d 100644
--- a/documentation/ref-manual/ref-qa-checks.xml
+++ b/documentation/ref-manual/ref-qa-checks.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-qa-checks'> 6<chapter id='ref-qa-checks'>
6<title>QA Error and Warning Messages</title> 7<title>QA Error and Warning Messages</title>
@@ -1170,6 +1171,31 @@ can be found then it should be implemented. I can't find one at the moment.
1170 </listitem> 1171 </listitem>
1171 </itemizedlist> 1172 </itemizedlist>
1172 </para> 1173 </para>
1174
1175 <para>
1176 <itemizedlist>
1177 <listitem>
1178 <para id='qa-issue-unlisted-pkg-lics'>
1179 <code>
1180 LICENSE_&lt;packagename&gt; includes licenses (&lt;licenses&gt;) that are not listed in LICENSE [unlisted-pkg-lics]
1181 </code>
1182 </para>
1183
1184 <para>
1185 The <link linkend='var-LICENSE'><filename>LICENSE</filename></link>
1186 of the recipe should be a superset of all the licenses of
1187 all packages produced by this recipe.
1188 In other words, any license in <filename>LICENSE_*</filename>
1189 should also appear in
1190 <link linkend='var-LICENSE'><filename>LICENSE</filename></link>.
1191 </para>
1192
1193 <para>
1194 &nbsp;
1195 </para>
1196 </listitem>
1197 </itemizedlist>
1198 </para>
1173</section> 1199</section>
1174 1200
1175<section id='configuring-and-disabling-qa-checks'> 1201<section id='configuring-and-disabling-qa-checks'>
diff --git a/documentation/ref-manual/ref-release-process.rst b/documentation/ref-manual/ref-release-process.rst
new file mode 100644
index 0000000000..be041e7254
--- /dev/null
+++ b/documentation/ref-manual/ref-release-process.rst
@@ -0,0 +1,193 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************************************
4Yocto Project Releases and the Stable Release Process
5*****************************************************
6
7The Yocto Project release process is predictable and consists of both
8major and minor (point) releases. This brief chapter provides
9information on how releases are named, their life cycle, and their
10stability.
11
12Major and Minor Release Cadence
13===============================
14
15The Yocto Project delivers major releases (e.g. DISTRO) using a six
16month cadence roughly timed each April and October of the year.
17Following are examples of some major YP releases with their codenames
18also shown. See the "`Major Release
19Codenames <#major-release-codenames>`__" section for information on
20codenames used with major releases.
21
22 - 2.2 (Morty)
23 - 2.1 (Krogoth)
24 - 2.0 (Jethro)
25
26While the cadence is never perfect, this timescale facilitates
27regular releases that have strong QA cycles while not overwhelming users
28with too many new releases. The cadence is predictable and avoids many
29major holidays in various geographies.
30
31The Yocto project delivers minor (point) releases on an unscheduled
32basis and are usually driven by the accumulation of enough significant
33fixes or enhancements to the associated major release. Following are
34some example past point releases:
35
36 - 2.1.1
37 - 2.1.2
38 - 2.2.1
39
40The point release
41indicates a point in the major release branch where a full QA cycle and
42release process validates the content of the new branch.
43
44.. note::
45
46 Realize that there can be patches merged onto the stable release
47 branches as and when they become available.
48
49Major Release Codenames
50=======================
51
52Each major release receives a codename that identifies the release in
53the :ref:`overview-manual/overview-manual-development-environment:yocto project source repositories`.
54The concept is that branches of :term:`Metadata` with the same
55codename are likely to be compatible and thus work together.
56
57.. note::
58
59 Codenames are associated with major releases because a Yocto Project
60 release number (e.g. DISTRO) could conflict with a given layer or
61 company versioning scheme. Codenames are unique, interesting, and
62 easily identifiable.
63
64Releases are given a nominal release version as well but the codename is
65used in repositories for this reason. You can find information on Yocto
66Project releases and codenames at
67https://wiki.yoctoproject.org/wiki/Releases.
68
69Stable Release Process
70======================
71
72Once released, the release enters the stable release process at which
73time a person is assigned as the maintainer for that stable release.
74This maintainer monitors activity for the release by investigating and
75handling nominated patches and backport activity. Only fixes and
76enhancements that have first been applied on the "master" branch (i.e.
77the current, in-development branch) are considered for backporting to a
78stable release.
79
80.. note::
81
82 The current Yocto Project policy regarding backporting is to consider
83 bug fixes and security fixes only. Policy dictates that features are
84 not backported to a stable release. This policy means generic recipe
85 version upgrades are unlikely to be accepted for backporting. The
86 exception to this policy occurs when a strong reason exists such as
87 the fix happens to also be the preferred upstream approach.
88
89Stable release branches have strong maintenance for about a year after
90their initial release. Should significant issues be found for any
91release regardless of its age, fixes could be backported to older
92releases. For issues that are not backported given an older release,
93Community LTS trees and branches exist where community members share
94patches for older releases. However, these types of patches do not go
95through the same release process as do point releases. You can find more
96information about stable branch maintenance at
97https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance.
98
99Testing and Quality Assurance
100=============================
101
102Part of the Yocto Project development and release process is quality
103assurance through the execution of test strategies. Test strategies
104provide the Yocto Project team a way to ensure a release is validated.
105Additionally, because the test strategies are visible to you as a
106developer, you can validate your projects. This section overviews the
107available test infrastructure used in the Yocto Project. For information
108on how to run available tests on your projects, see the
109":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
110section in the Yocto Project Development Tasks Manual.
111
112The QA/testing infrastructure is woven into the project to the point
113where core developers take some of it for granted. The infrastructure
114consists of the following pieces:
115
116- ``bitbake-selftest``: A standalone command that runs unit tests on
117 key pieces of BitBake and its fetchers.
118
119- :ref:`sanity.bbclass <ref-classes-sanity>`: This automatically
120 included class checks the build environment for missing tools (e.g.
121 ``gcc``) or common misconfigurations such as
122 :term:`MACHINE` set incorrectly.
123
124- :ref:`insane.bbclass <ref-classes-insane>`: This class checks the
125 generated output from builds for sanity. For example, if building for
126 an ARM target, did the build produce ARM binaries. If, for example,
127 the build produced PPC binaries then there is a problem.
128
129- :ref:`testimage.bbclass <ref-classes-testimage*>`: This class
130 performs runtime testing of images after they are built. The tests
131 are usually used with :doc:`QEMU <../dev-manual/dev-manual-qemu>`
132 to boot the images and check the combined runtime result boot
133 operation and functions. However, the test can also use the IP
134 address of a machine to test.
135
136- :ref:`ptest <dev-manual/dev-manual-common-tasks:testing packages with ptest>`:
137 Runs tests against packages produced during the build for a given
138 piece of software. The test allows the packages to be be run within a
139 target image.
140
141- ``oe-selftest``: Tests combination BitBake invocations. These tests
142 operate outside the OpenEmbedded build system itself. The
143 ``oe-selftest`` can run all tests by default or can run selected
144 tests or test suites.
145
146 .. note::
147
148 Running
149 oe-selftest
150 requires host packages beyond the "Essential" grouping. See the "
151 Required Packages for the Build Host
152 " section for more information.
153
154Originally, much of this testing was done manually. However, significant
155effort has been made to automate the tests so that more people can use
156them and the Yocto Project development team can run them faster and more
157efficiently.
158
159The Yocto Project's main Autobuilder (https://autobuilder.yoctoproject.org/)
160publicly tests each Yocto Project release's code in the
161:term:`OpenEmbedded-Core (OE-Core)`, Poky, and BitBake repositories. The testing
162occurs for both the current state of the "master" branch and also for
163submitted patches. Testing for submitted patches usually occurs in the
164"ross/mut" branch in the ``poky-contrib`` repository (i.e. the
165master-under-test branch) or in the "master-next" branch in the ``poky``
166repository.
167
168.. note::
169
170 You can find all these branches in the Yocto Project
171 Source Repositories
172 .
173
174Testing within these public branches ensures in a publicly visible way
175that all of the main supposed architectures and recipes in OE-Core
176successfully build and behave properly.
177
178Various features such as ``multilib``, sub architectures (e.g. ``x32``,
179``poky-tiny``, ``musl``, ``no-x11`` and and so forth),
180``bitbake-selftest``, and ``oe-selftest`` are tested as part of the QA
181process of a release. Complete testing and validation for a release
182takes the Autobuilder workers several hours.
183
184.. note::
185
186 The Autobuilder workers are non-homogeneous, which means regular
187 testing across a variety of Linux distributions occurs. The
188 Autobuilder is limited to only testing QEMU-based setups and not real
189 hardware.
190
191Finally, in addition to the Autobuilder's tests, the Yocto Project QA
192team also performs testing on a variety of platforms, which includes
193actual hardware, to ensure expected results.
diff --git a/documentation/ref-manual/ref-release-process.xml b/documentation/ref-manual/ref-release-process.xml
index 5efe17417a..87f5308067 100644
--- a/documentation/ref-manual/ref-release-process.xml
+++ b/documentation/ref-manual/ref-release-process.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-release-process'> 6<chapter id='ref-release-process'>
6<title>Yocto Project Releases and the Stable Release Process</title> 7<title>Yocto Project Releases and the Stable Release Process</title>
diff --git a/documentation/ref-manual/ref-structure.rst b/documentation/ref-manual/ref-structure.rst
new file mode 100644
index 0000000000..48a443331b
--- /dev/null
+++ b/documentation/ref-manual/ref-structure.rst
@@ -0,0 +1,890 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**************************
4Source Directory Structure
5**************************
6
7The :term:`Source Directory` consists of numerous files,
8directories and subdirectories; understanding their locations and
9contents is key to using the Yocto Project effectively. This chapter
10describes the Source Directory and gives information about those files
11and directories.
12
13For information on how to establish a local Source Directory on your
14development system, see the
15":ref:`dev-manual/dev-manual-start:locating yocto project source files`"
16section in the Yocto Project Development Tasks Manual.
17
18.. note::
19
20 The OpenEmbedded build system does not support file or directory
21 names that contain spaces. Be sure that the Source Directory you use
22 does not contain these types of names.
23
24.. _structure-core:
25
26Top-Level Core Components
27=========================
28
29This section describes the top-level components of the :term:`Source Directory`.
30
31.. _structure-core-bitbake:
32
33``bitbake/``
34------------
35
36This directory includes a copy of BitBake for ease of use. The copy
37usually matches the current stable BitBake release from the BitBake
38project. BitBake, a :term:`Metadata` interpreter, reads the
39Yocto Project Metadata and runs the tasks defined by that data. Failures
40are usually caused by errors in your Metadata and not from BitBake
41itself; consequently, most users do not need to worry about BitBake.
42
43When you run the ``bitbake`` command, the main BitBake executable (which
44resides in the ``bitbake/bin/`` directory) starts. Sourcing the
45environment setup script (i.e. :ref:`structure-core-script`) places
46the ``scripts/`` and ``bitbake/bin/`` directories (in that order) into
47the shell's ``PATH`` environment variable.
48
49For more information on BitBake, see the :doc:`BitBake User Manual
50<bitbake:index>`.
51
52.. _structure-core-build:
53
54``build/``
55----------
56
57This directory contains user configuration files and the output
58generated by the OpenEmbedded build system in its standard configuration
59where the source tree is combined with the output. The :term:`Build Directory`
60is created initially when you ``source``
61the OpenEmbedded build environment setup script (i.e.
62:ref:`structure-core-script`).
63
64It is also possible to place output and configuration files in a
65directory separate from the :term:`Source Directory` by
66providing a directory name when you ``source`` the setup script. For
67information on separating output from your local Source Directory files
68(commonly described as an "out of tree" build), see the
69":ref:`structure-core-script`" section.
70
71.. _handbook:
72
73``documentation/``
74------------------
75
76This directory holds the source for the Yocto Project documentation as
77well as templates and tools that allow you to generate PDF and HTML
78versions of the manuals. Each manual is contained in its own sub-folder;
79for example, the files for this reference manual reside in the
80``ref-manual/`` directory.
81
82.. _structure-core-meta:
83
84``meta/``
85---------
86
87This directory contains the minimal, underlying OpenEmbedded-Core
88metadata. The directory holds recipes, common classes, and machine
89configuration for strictly emulated targets (``qemux86``, ``qemuarm``,
90and so forth.)
91
92.. _structure-core-meta-poky:
93
94``meta-poky/``
95--------------
96
97Designed above the ``meta/`` content, this directory adds just enough
98metadata to define the Poky reference distribution.
99
100.. _structure-core-meta-yocto-bsp:
101
102``meta-yocto-bsp/``
103-------------------
104
105This directory contains the Yocto Project reference hardware Board
106Support Packages (BSPs). For more information on BSPs, see the
107:doc:`../bsp-guide/bsp-guide`.
108
109.. _structure-meta-selftest:
110
111``meta-selftest/``
112------------------
113
114This directory adds additional recipes and append files used by the
115OpenEmbedded selftests to verify the behavior of the build system. You
116do not have to add this layer to your ``bblayers.conf`` file unless you
117want to run the selftests.
118
119.. _structure-meta-skeleton:
120
121``meta-skeleton/``
122------------------
123
124This directory contains template recipes for BSP and kernel development.
125
126.. _structure-core-scripts:
127
128``scripts/``
129------------
130
131This directory contains various integration scripts that implement extra
132functionality in the Yocto Project environment (e.g. QEMU scripts). The
133:ref:`structure-core-script` script prepends this directory to the
134shell's ``PATH`` environment variable.
135
136The ``scripts`` directory has useful scripts that assist in contributing
137back to the Yocto Project, such as ``create-pull-request`` and
138``send-pull-request``.
139
140.. _structure-core-script:
141
142``oe-init-build-env``
143---------------------
144
145This script sets up the OpenEmbedded build environment. Running this
146script with the ``source`` command in a shell makes changes to ``PATH``
147and sets other core BitBake variables based on the current working
148directory. You need to run an environment setup script before running
149BitBake commands. The script uses other scripts within the ``scripts``
150directory to do the bulk of the work.
151
152When you run this script, your Yocto Project environment is set up, a
153:term:`Build Directory` is created, your working
154directory becomes the Build Directory, and you are presented with some
155simple suggestions as to what to do next, including a list of some
156possible targets to build. Here is an example:
157::
158
159 $ source oe-init-build-env
160
161 ### Shell environment set up for builds. ###
162
163 You can now run 'bitbake <target>'
164
165 Common targets are:
166 core-image-minimal
167 core-image-sato
168 meta-toolchain
169 meta-ide-support
170
171 You can also run generated qemu images with a command like 'runqemu qemux86-64'
172
173The default output of the ``oe-init-build-env`` script is from the
174``conf-notes.txt`` file, which is found in the ``meta-poky`` directory
175within the :term:`Source Directory`. If you design a
176custom distribution, you can include your own version of this
177configuration file to mention the targets defined by your distribution.
178See the
179":ref:`dev-manual/dev-manual-common-tasks:creating a custom template configuration directory`"
180section in the Yocto Project Development Tasks Manual for more
181information.
182
183By default, running this script without a Build Directory argument
184creates the ``build/`` directory in your current working directory. If
185you provide a Build Directory argument when you ``source`` the script,
186you direct the OpenEmbedded build system to create a Build Directory of
187your choice. For example, the following command creates a Build
188Directory named ``mybuilds/`` that is outside of the :term:`Source Directory`:
189::
190
191 $ source OE_INIT_FILE ~/mybuilds
192
193The OpenEmbedded build system uses the template configuration files, which
194are found by default in the ``meta-poky/conf/`` directory in the Source
195Directory. See the
196":ref:`dev-manual/dev-manual-common-tasks:creating a custom template configuration directory`"
197section in the Yocto Project Development Tasks Manual for more
198information.
199
200.. note::
201
202 The OpenEmbedded build system does not support file or directory
203 names that contain spaces. If you attempt to run the
204 OE_INIT_FILE
205 script from a Source Directory that contains spaces in either the
206 filenames or directory names, the script returns an error indicating
207 no such file or directory. Be sure to use a Source Directory free of
208 names containing spaces.
209
210.. _structure-basic-top-level:
211
212``LICENSE, README, and README.hardware``
213----------------------------------------
214
215These files are standard top-level files.
216
217.. _structure-build:
218
219The Build Directory - ``build/``
220================================
221
222The OpenEmbedded build system creates the :term:`Build Directory`
223when you run the build environment setup
224script :ref:`structure-core-script`. If you do not give the Build
225Directory a specific name when you run the setup script, the name
226defaults to ``build/``.
227
228For subsequent parsing and processing, the name of the Build directory
229is available via the :term:`TOPDIR` variable.
230
231.. _structure-build-buildhistory:
232
233``build/buildhistory/``
234-----------------------
235
236The OpenEmbedded build system creates this directory when you enable
237build history via the ``buildhistory`` class file. The directory
238organizes build information into image, packages, and SDK
239subdirectories. For information on the build history feature, see the
240":ref:`dev-manual/dev-manual-common-tasks:maintaining build output quality`"
241section in the Yocto Project Development Tasks Manual.
242
243.. _structure-build-conf-local.conf:
244
245``build/conf/local.conf``
246-------------------------
247
248This configuration file contains all the local user configurations for
249your build environment. The ``local.conf`` file contains documentation
250on the various configuration options. Any variable set here overrides
251any variable set elsewhere within the environment unless that variable
252is hard-coded within a file (e.g. by using '=' instead of '?='). Some
253variables are hard-coded for various reasons but such variables are
254relatively rare.
255
256At a minimum, you would normally edit this file to select the target
257``MACHINE``, which package types you wish to use
258(:term:`PACKAGE_CLASSES`), and the location from
259which you want to access downloaded files (``DL_DIR``).
260
261If ``local.conf`` is not present when you start the build, the
262OpenEmbedded build system creates it from ``local.conf.sample`` when you
263``source`` the top-level build environment setup script
264:ref:`structure-core-script`.
265
266The source ``local.conf.sample`` file used depends on the
267``$TEMPLATECONF`` script variable, which defaults to ``meta-poky/conf/``
268when you are building from the Yocto Project development environment,
269and to ``meta/conf/`` when you are building from the OpenEmbedded-Core
270environment. Because the script variable points to the source of the
271``local.conf.sample`` file, this implies that you can configure your
272build environment from any layer by setting the variable in the
273top-level build environment setup script as follows:
274::
275
276 TEMPLATECONF=your_layer/conf
277
278Once the build process gets the sample
279file, it uses ``sed`` to substitute final
280``${``\ :term:`OEROOT`\ ``}`` values for all
281``##OEROOT##`` values.
282
283.. note::
284
285 You can see how the
286 TEMPLATECONF
287 variable is used by looking at the
288 scripts/oe-setup-builddir
289 script in the
290 Source Directory
291 . You can find the Yocto Project version of the
292 local.conf.sample
293 file in the
294 meta-poky/conf
295 directory.
296
297.. _structure-build-conf-bblayers.conf:
298
299``build/conf/bblayers.conf``
300----------------------------
301
302This configuration file defines
303:ref:`layers <dev-manual/dev-manual-common-tasks:understanding and creating layers>`,
304which are directory trees, traversed (or walked) by BitBake. The
305``bblayers.conf`` file uses the :term:`BBLAYERS`
306variable to list the layers BitBake tries to find.
307
308If ``bblayers.conf`` is not present when you start the build, the
309OpenEmbedded build system creates it from ``bblayers.conf.sample`` when
310you ``source`` the top-level build environment setup script (i.e.
311:ref:`structure-core-script`).
312
313As with the ``local.conf`` file, the source ``bblayers.conf.sample``
314file used depends on the ``$TEMPLATECONF`` script variable, which
315defaults to ``meta-poky/conf/`` when you are building from the Yocto
316Project development environment, and to ``meta/conf/`` when you are
317building from the OpenEmbedded-Core environment. Because the script
318variable points to the source of the ``bblayers.conf.sample`` file, this
319implies that you can base your build from any layer by setting the
320variable in the top-level build environment setup script as follows:
321::
322
323 TEMPLATECONF=your_layer/conf
324
325Once the build process gets the sample file, it uses ``sed`` to substitute final
326``${``\ :term:`OEROOT`\ ``}`` values for all ``##OEROOT##`` values.
327
328.. note::
329
330 You can see how the
331 TEMPLATECONF
332 variable
333 scripts/oe-setup-builddir
334 script in the
335 Source Directory
336 . You can find the Yocto Project version of the
337 bblayers.conf.sample
338 file in the
339 meta-poky/conf/
340 directory.
341
342.. _structure-build-conf-sanity_info:
343
344``build/cache/sanity_info``
345---------------------------
346
347This file indicates the state of the sanity checks and is created during
348the build.
349
350.. _structure-build-downloads:
351
352``build/downloads/``
353--------------------
354
355This directory contains downloaded upstream source tarballs. You can
356reuse the directory for multiple builds or move the directory to another
357location. You can control the location of this directory through the
358``DL_DIR`` variable.
359
360.. _structure-build-sstate-cache:
361
362``build/sstate-cache/``
363-----------------------
364
365This directory contains the shared state cache. You can reuse the
366directory for multiple builds or move the directory to another location.
367You can control the location of this directory through the
368``SSTATE_DIR`` variable.
369
370.. _structure-build-tmp:
371
372``build/tmp/``
373--------------
374
375The OpenEmbedded build system creates and uses this directory for all
376the build system's output. The :term:`TMPDIR` variable
377points to this directory.
378
379BitBake creates this directory if it does not exist. As a last resort,
380to clean up a build and start it from scratch (other than the
381downloads), you can remove everything in the ``tmp`` directory or get
382rid of the directory completely. If you do, you should also completely
383remove the ``build/sstate-cache`` directory.
384
385.. _structure-build-tmp-buildstats:
386
387``build/tmp/buildstats/``
388-------------------------
389
390This directory stores the build statistics.
391
392.. _structure-build-tmp-cache:
393
394``build/tmp/cache/``
395--------------------
396
397When BitBake parses the metadata (recipes and configuration files), it
398caches the results in ``build/tmp/cache/`` to speed up future builds.
399The results are stored on a per-machine basis.
400
401During subsequent builds, BitBake checks each recipe (together with, for
402example, any files included or appended to it) to see if they have been
403modified. Changes can be detected, for example, through file
404modification time (mtime) changes and hashing of file contents. If no
405changes to the file are detected, then the parsed result stored in the
406cache is reused. If the file has changed, it is reparsed.
407
408.. _structure-build-tmp-deploy:
409
410``build/tmp/deploy/``
411---------------------
412
413This directory contains any "end result" output from the OpenEmbedded
414build process. The :term:`DEPLOY_DIR` variable points
415to this directory. For more detail on the contents of the ``deploy``
416directory, see the
417":ref:`images-dev-environment`" and
418":ref:`sdk-dev-environment`" sections in the Yocto
419Project Overview and Concepts Manual.
420
421.. _structure-build-tmp-deploy-deb:
422
423``build/tmp/deploy/deb/``
424-------------------------
425
426This directory receives any ``.deb`` packages produced by the build
427process. The packages are sorted into feeds for different architecture
428types.
429
430.. _structure-build-tmp-deploy-rpm:
431
432``build/tmp/deploy/rpm/``
433-------------------------
434
435This directory receives any ``.rpm`` packages produced by the build
436process. The packages are sorted into feeds for different architecture
437types.
438
439.. _structure-build-tmp-deploy-ipk:
440
441``build/tmp/deploy/ipk/``
442-------------------------
443
444This directory receives ``.ipk`` packages produced by the build process.
445
446.. _structure-build-tmp-deploy-licenses:
447
448``build/tmp/deploy/licenses/``
449------------------------------
450
451This directory receives package licensing information. For example, the
452directory contains sub-directories for ``bash``, ``busybox``, and
453``glibc`` (among others) that in turn contain appropriate ``COPYING``
454license files with other licensing information. For information on
455licensing, see the
456":ref:`dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle`"
457section in the Yocto Project Development Tasks Manual.
458
459.. _structure-build-tmp-deploy-images:
460
461``build/tmp/deploy/images/``
462----------------------------
463
464This directory is populated with the basic output objects of the build
465(think of them as the "generated artifacts" of the build process),
466including things like the boot loader image, kernel, root filesystem and
467more. If you want to flash the resulting image from a build onto a
468device, look here for the necessary components.
469
470Be careful when deleting files in this directory. You can safely delete
471old images from this directory (e.g. ``core-image-*``). However, the
472kernel (``*zImage*``, ``*uImage*``, etc.), bootloader and other
473supplementary files might be deployed here prior to building an image.
474Because these files are not directly produced from the image, if you
475delete them they will not be automatically re-created when you build the
476image again.
477
478If you do accidentally delete files here, you will need to force them to
479be re-created. In order to do that, you will need to know the target
480that produced them. For example, these commands rebuild and re-create
481the kernel files:
482::
483
484 $ bitbake -c clean virtual/kernel
485 $ bitbake virtual/kernel
486
487.. _structure-build-tmp-deploy-sdk:
488
489``build/tmp/deploy/sdk/``
490-------------------------
491
492The OpenEmbedded build system creates this directory to hold toolchain
493installer scripts which, when executed, install the sysroot that matches
494your target hardware. You can find out more about these installers in
495the ":ref:`sdk-manual/sdk-appendix-obtain:building an sdk installer`"
496section in the Yocto Project Application Development and the Extensible
497Software Development Kit (eSDK) manual.
498
499.. _structure-build-tmp-sstate-control:
500
501``build/tmp/sstate-control/``
502-----------------------------
503
504The OpenEmbedded build system uses this directory for the shared state
505manifest files. The shared state code uses these files to record the
506files installed by each sstate task so that the files can be removed
507when cleaning the recipe or when a newer version is about to be
508installed. The build system also uses the manifests to detect and
509produce a warning when files from one task are overwriting those from
510another.
511
512.. _structure-build-tmp-sysroots-components:
513
514``build/tmp/sysroots-components/``
515----------------------------------
516
517This directory is the location of the sysroot contents that the task
518:ref:`ref-tasks-prepare_recipe_sysroot`
519links or copies into the recipe-specific sysroot for each recipe listed
520in :term:`DEPENDS`. Population of this directory is
521handled through shared state, while the path is specified by the
522:term:`COMPONENTS_DIR` variable. Apart from a few
523unusual circumstances, handling of the ``sysroots-components`` directory
524should be automatic, and recipes should not directly reference
525``build/tmp/sysroots-components``.
526
527.. _structure-build-tmp-sysroots:
528
529``build/tmp/sysroots/``
530-----------------------
531
532Previous versions of the OpenEmbedded build system used to create a
533global shared sysroot per machine along with a native sysroot. Beginning
534with the DISTRO version of the Yocto Project, sysroots exist in
535recipe-specific :term:`WORKDIR` directories. Thus, the
536``build/tmp/sysroots/`` directory is unused.
537
538.. note::
539
540 The
541 build/tmp/sysroots/
542 directory can still be populated using the
543 bitbake build-sysroots
544 command and can be used for compatibility in some cases. However, in
545 general it is not recommended to populate this directory. Individual
546 recipe-specific sysroots should be used.
547
548.. _structure-build-tmp-stamps:
549
550``build/tmp/stamps/``
551---------------------
552
553This directory holds information that BitBake uses for accounting
554purposes to track what tasks have run and when they have run. The
555directory is sub-divided by architecture, package name, and version.
556Following is an example:
557stamps/all-poky-linux/distcc-config/1.0-r0.do_build-2fdd....2do Although
558the files in the directory are empty of data, BitBake uses the filenames
559and timestamps for tracking purposes.
560
561For information on how BitBake uses stamp files to determine if a task
562should be rerun, see the
563":ref:`overview-manual/overview-manual-concepts:stamp files and the rerunning of tasks`"
564section in the Yocto Project Overview and Concepts Manual.
565
566.. _structure-build-tmp-log:
567
568``build/tmp/log/``
569------------------
570
571This directory contains general logs that are not otherwise placed using
572the package's ``WORKDIR``. Examples of logs are the output from the
573``do_check_pkg`` or ``do_distro_check`` tasks. Running a build does not
574necessarily mean this directory is created.
575
576.. _structure-build-tmp-work:
577
578``build/tmp/work/``
579-------------------
580
581This directory contains architecture-specific work sub-directories for
582packages built by BitBake. All tasks execute from the appropriate work
583directory. For example, the source for a particular package is unpacked,
584patched, configured and compiled all within its own work directory.
585Within the work directory, organization is based on the package group
586and version for which the source is being compiled as defined by the
587:term:`WORKDIR`.
588
589It is worth considering the structure of a typical work directory. As an
590example, consider ``linux-yocto-kernel-3.0`` on the machine ``qemux86``
591built within the Yocto Project. For this package, a work directory of
592``tmp/work/qemux86-poky-linux/linux-yocto/3.0+git1+<.....>``, referred
593to as the ``WORKDIR``, is created. Within this directory, the source is
594unpacked to ``linux-qemux86-standard-build`` and then patched by Quilt.
595(See the ":ref:`using-a-quilt-workflow`" section in
596the Yocto Project Development Tasks Manual for more information.) Within
597the ``linux-qemux86-standard-build`` directory, standard Quilt
598directories ``linux-3.0/patches`` and ``linux-3.0/.pc`` are created, and
599standard Quilt commands can be used.
600
601There are other directories generated within ``WORKDIR``. The most
602important directory is ``WORKDIR/temp/``, which has log files for each
603task (``log.do_*.pid``) and contains the scripts BitBake runs for each
604task (``run.do_*.pid``). The ``WORKDIR/image/`` directory is where "make
605install" places its output that is then split into sub-packages within
606``WORKDIR/packages-split/``.
607
608.. _structure-build-tmp-work-tunearch-recipename-version:
609
610``build/tmp/work/tunearch/recipename/version/``
611-----------------------------------------------
612
613The recipe work directory - ``${WORKDIR}``.
614
615As described earlier in the
616"```build/tmp/sysroots/`` <#structure-build-tmp-sysroots>`__" section,
617beginning with the DISTRO release of the Yocto Project, the OpenEmbedded
618build system builds each recipe in its own work directory (i.e.
619:term:`WORKDIR`). The path to the work directory is
620constructed using the architecture of the given build (e.g.
621:term:`TUNE_PKGARCH`,
622:term:`MACHINE_ARCH`, or "allarch"), the recipe
623name, and the version of the recipe (i.e.
624:term:`PE`\ ``:``\ :term:`PV`\ ``-``\ :term:`PR`).
625
626A number of key subdirectories exist within each recipe work directory:
627
628- ``${WORKDIR}/temp``: Contains the log files of each task executed for
629 this recipe, the "run" files for each executed task, which contain
630 the code run, and a ``log.task_order`` file, which lists the order in
631 which tasks were executed.
632
633- ``${WORKDIR}/image``: Contains the output of the
634 :ref:`ref-tasks-install` task, which corresponds to
635 the ``${``\ :term:`D`\ ``}`` variable in that task.
636
637- ``${WORKDIR}/pseudo``: Contains the pseudo database and log for any
638 tasks executed under pseudo for the recipe.
639
640- ``${WORKDIR}/sysroot-destdir``: Contains the output of the
641 :ref:`ref-tasks-populate_sysroot` task.
642
643- ``${WORKDIR}/package``: Contains the output of the
644 :ref:`ref-tasks-package` task before the output is
645 split into individual packages.
646
647- ``${WORKDIR}/packages-split``: Contains the output of the
648 ``do_package`` task after the output has been split into individual
649 packages. Subdirectories exist for each individual package created by
650 the recipe.
651
652- ``${WORKDIR}/recipe-sysroot``: A directory populated with the target
653 dependencies of the recipe. This directory looks like the target
654 filesystem and contains libraries that the recipe might need to link
655 against (e.g. the C library).
656
657- ``${WORKDIR}/recipe-sysroot-native``: A directory populated with the
658 native dependencies of the recipe. This directory contains the tools
659 the recipe needs to build (e.g. the compiler, Autoconf, libtool, and
660 so forth).
661
662- ``${WORKDIR}/build``: This subdirectory applies only to recipes that
663 support builds where the source is separate from the build artifacts.
664 The OpenEmbedded build system uses this directory as a separate build
665 directory (i.e. ``${``\ :term:`B`\ ``}``).
666
667.. _structure-build-work-shared:
668
669``build/tmp/work-shared/``
670--------------------------
671
672For efficiency, the OpenEmbedded build system creates and uses this
673directory to hold recipes that share a work directory with other
674recipes. In practice, this is only used for ``gcc`` and its variants
675(e.g. ``gcc-cross``, ``libgcc``, ``gcc-runtime``, and so forth).
676
677.. _structure-meta:
678
679The Metadata - ``meta/``
680========================
681
682As mentioned previously, :term:`Metadata` is the core of the
683Yocto Project. Metadata has several important subdivisions:
684
685.. _structure-meta-classes:
686
687``meta/classes/``
688-----------------
689
690This directory contains the ``*.bbclass`` files. Class files are used to
691abstract common code so it can be reused by multiple packages. Every
692package inherits the ``base.bbclass`` file. Examples of other important
693classes are ``autotools.bbclass``, which in theory allows any
694Autotool-enabled package to work with the Yocto Project with minimal
695effort. Another example is ``kernel.bbclass`` that contains common code
696and functions for working with the Linux kernel. Functions like image
697generation or packaging also have their specific class files such as
698``image.bbclass``, ``rootfs_*.bbclass`` and ``package*.bbclass``.
699
700For reference information on classes, see the
701":ref:`ref-manual/ref-classes:Classes`" chapter.
702
703.. _structure-meta-conf:
704
705``meta/conf/``
706--------------
707
708This directory contains the core set of configuration files that start
709from ``bitbake.conf`` and from which all other configuration files are
710included. See the include statements at the end of the ``bitbake.conf``
711file and you will note that even ``local.conf`` is loaded from there.
712While ``bitbake.conf`` sets up the defaults, you can often override
713these by using the (``local.conf``) file, machine file or the
714distribution configuration file.
715
716.. _structure-meta-conf-machine:
717
718``meta/conf/machine/``
719----------------------
720
721This directory contains all the machine configuration files. If you set
722``MACHINE = "qemux86"``, the OpenEmbedded build system looks for a
723``qemux86.conf`` file in this directory. The ``include`` directory
724contains various data common to multiple machines. If you want to add
725support for a new machine to the Yocto Project, look in this directory.
726
727.. _structure-meta-conf-distro:
728
729``meta/conf/distro/``
730---------------------
731
732The contents of this directory controls any distribution-specific
733configurations. For the Yocto Project, the ``defaultsetup.conf`` is the
734main file here. This directory includes the versions and the ``SRCDATE``
735definitions for applications that are configured here. An example of an
736alternative configuration might be ``poky-bleeding.conf``. Although this
737file mainly inherits its configuration from Poky.
738
739.. _structure-meta-conf-machine-sdk:
740
741``meta/conf/machine-sdk/``
742--------------------------
743
744The OpenEmbedded build system searches this directory for configuration
745files that correspond to the value of
746:term:`SDKMACHINE`. By default, 32-bit and 64-bit x86
747files ship with the Yocto Project that support some SDK hosts. However,
748it is possible to extend that support to other SDK hosts by adding
749additional configuration files in this subdirectory within another
750layer.
751
752.. _structure-meta-files:
753
754``meta/files/``
755---------------
756
757This directory contains common license files and several text files used
758by the build system. The text files contain minimal device information
759and lists of files and directories with known permissions.
760
761.. _structure-meta-lib:
762
763``meta/lib/``
764-------------
765
766This directory contains OpenEmbedded Python library code used during the
767build process.
768
769.. _structure-meta-recipes-bsp:
770
771``meta/recipes-bsp/``
772---------------------
773
774This directory contains anything linking to specific hardware or
775hardware configuration information such as "u-boot" and "grub".
776
777.. _structure-meta-recipes-connectivity:
778
779``meta/recipes-connectivity/``
780------------------------------
781
782This directory contains libraries and applications related to
783communication with other devices.
784
785.. _structure-meta-recipes-core:
786
787``meta/recipes-core/``
788----------------------
789
790This directory contains what is needed to build a basic working Linux
791image including commonly used dependencies.
792
793.. _structure-meta-recipes-devtools:
794
795``meta/recipes-devtools/``
796--------------------------
797
798This directory contains tools that are primarily used by the build
799system. The tools, however, can also be used on targets.
800
801.. _structure-meta-recipes-extended:
802
803``meta/recipes-extended/``
804--------------------------
805
806This directory contains non-essential applications that add features
807compared to the alternatives in core. You might need this directory for
808full tool functionality or for Linux Standard Base (LSB) compliance.
809
810.. _structure-meta-recipes-gnome:
811
812``meta/recipes-gnome/``
813-----------------------
814
815This directory contains all things related to the GTK+ application
816framework.
817
818.. _structure-meta-recipes-graphics:
819
820``meta/recipes-graphics/``
821--------------------------
822
823This directory contains X and other graphically related system
824libraries.
825
826.. _structure-meta-recipes-kernel:
827
828``meta/recipes-kernel/``
829------------------------
830
831This directory contains the kernel and generic applications and
832libraries that have strong kernel dependencies.
833
834.. _structure-meta-recipes-lsb4:
835
836``meta/recipes-lsb4/``
837----------------------
838
839This directory contains recipes specifically added to support the Linux
840Standard Base (LSB) version 4.x.
841
842.. _structure-meta-recipes-multimedia:
843
844``meta/recipes-multimedia/``
845----------------------------
846
847This directory contains codecs and support utilities for audio, images
848and video.
849
850.. _structure-meta-recipes-rt:
851
852``meta/recipes-rt/``
853--------------------
854
855This directory contains package and image recipes for using and testing
856the ``PREEMPT_RT`` kernel.
857
858.. _structure-meta-recipes-sato:
859
860``meta/recipes-sato/``
861----------------------
862
863This directory contains the Sato demo/reference UI/UX and its associated
864applications and configuration data.
865
866.. _structure-meta-recipes-support:
867
868``meta/recipes-support/``
869-------------------------
870
871This directory contains recipes used by other recipes, but that are not
872directly included in images (i.e. dependencies of other recipes).
873
874.. _structure-meta-site:
875
876``meta/site/``
877--------------
878
879This directory contains a list of cached results for various
880architectures. Because certain "autoconf" test results cannot be
881determined when cross-compiling due to the tests not able to run on a
882live system, the information in this directory is passed to "autoconf"
883for the various architectures.
884
885.. _structure-meta-recipes-txt:
886
887``meta/recipes.txt``
888--------------------
889
890This file is a description of the contents of ``recipes-*``.
diff --git a/documentation/ref-manual/ref-structure.xml b/documentation/ref-manual/ref-structure.xml
index 27f17dd919..8588e9c2dd 100644
--- a/documentation/ref-manual/ref-structure.xml
+++ b/documentation/ref-manual/ref-structure.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-structure'> 6<chapter id='ref-structure'>
6 7
diff --git a/documentation/ref-manual/ref-style.css b/documentation/ref-manual/ref-style.css
index 7077e4b70d..622ceb8f7e 100644
--- a/documentation/ref-manual/ref-style.css
+++ b/documentation/ref-manual/ref-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/ref-manual/ref-system-requirements.rst b/documentation/ref-manual/ref-system-requirements.rst
new file mode 100644
index 0000000000..56218e4ebb
--- /dev/null
+++ b/documentation/ref-manual/ref-system-requirements.rst
@@ -0,0 +1,437 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************
4System Requirements
5*******************
6
7Welcome to the Yocto Project Reference Manual! This manual provides
8reference information for the current release of the Yocto Project, and
9is most effectively used after you have an understanding of the basics
10of the Yocto Project. The manual is neither meant to be read as a
11starting point to the Yocto Project, nor read from start to finish.
12Rather, use this manual to find variable definitions, class
13descriptions, and so forth as needed during the course of using the
14Yocto Project.
15
16For introductory information on the Yocto Project, see the
17:yocto_home:`Yocto Project Website <>` and the
18":ref:`overview-manual/overview-manual-development-environment:the yocto project development environment`"
19chapter in the Yocto Project Overview and Concepts Manual.
20
21If you want to use the Yocto Project to quickly build an image without
22having to understand concepts, work through the
23:doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` document. You can find "how-to"
24information in the :doc:`../dev-manual/dev-manual`. You can find Yocto Project overview
25and conceptual information in the :doc:`../overview-manual/overview-manual`.
26
27.. note::
28
29 For more information about the Yocto Project Documentation set, see
30 the "
31 Links and Related Documentation
32 " section.
33
34.. _detailed-supported-distros:
35
36Supported Linux Distributions
37=============================
38
39Currently, the Yocto Project is supported on the following
40distributions:
41
42- Ubuntu 16.04 (LTS)
43
44- Ubuntu 18.04 (LTS)
45
46- Ubuntu 20.04
47
48- Fedora 30
49
50- Fedora 31
51
52- Fedora 32
53
54- CentOS 7.x
55
56- CentOS 8.x
57
58- Debian GNU/Linux 8.x (Jessie)
59
60- Debian GNU/Linux 9.x (Stretch)
61
62- Debian GNU/Linux 10.x (Buster)
63
64- OpenSUSE Leap 15.1
65
66
67.. note::
68
69 - While the Yocto Project Team attempts to ensure all Yocto Project
70 releases are one hundred percent compatible with each officially
71 supported Linux distribution, instances might exist where you
72 encounter a problem while using the Yocto Project on a specific
73 distribution.
74
75 - Yocto Project releases are tested against the stable Linux
76 distributions in the above list. The Yocto Project should work
77 on other distributions but validation is not performed against
78 them.
79
80 - In particular, the Yocto Project does not support and currently
81 has no plans to support rolling-releases or development
82 distributions due to their constantly changing nature. We welcome
83 patches and bug reports, but keep in mind that our priority is on
84 the supported platforms listed below.
85
86 - You may use Windows Subsystem For Linux v2 to set up a build host
87 using Windows 10, but validation is not performed against build
88 hosts using WSLv2.
89
90 - The Yocto Project is not compatible with WSLv1, it is
91 compatible but not officially supported nor validated with
92 WSLv2, if you still decide to use WSL please upgrade to WSLv2.
93
94 - If you encounter problems, please go to `Yocto Project
95 Bugzilla <http://bugzilla.yoctoproject.org>`__ and submit a bug. We are
96 interested in hearing about your experience. For information on
97 how to submit a bug, see the Yocto Project
98 :yocto_wiki:`Bugzilla wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`
99 and the ":ref:`dev-manual/dev-manual-common-tasks:submitting a defect against the yocto project`"
100 section in the Yocto Project Development Tasks Manual.
101
102
103Required Packages for the Build Host
104====================================
105
106The list of packages you need on the host development system can be
107large when covering all build scenarios using the Yocto Project. This
108section describes required packages according to Linux distribution and
109function.
110
111.. _ubuntu-packages:
112
113Ubuntu and Debian
114-----------------
115
116The following list shows the required packages by function given a
117supported Ubuntu or Debian Linux distribution:
118
119.. note::
120
121 - If your build system has the ``oss4-dev`` package installed, you
122 might experience QEMU build failures due to the package installing
123 its own custom ``/usr/include/linux/soundcard.h`` on the Debian
124 system. If you run into this situation, either of the following
125 solutions exist:
126 ::
127
128 $ sudo apt-get build-dep qemu
129 $ sudo apt-get remove oss4-dev
130
131 - For Debian-8, ``python3-git`` and ``pylint3`` are no longer
132 available via ``apt-get``.
133 ::
134
135 $ sudo pip3 install GitPython pylint==1.9.5
136
137- *Essentials:* Packages needed to build an image on a headless system:
138 ::
139
140 $ sudo apt-get install &UBUNTU_HOST_PACKAGES_ESSENTIAL;
141
142- *Documentation:* Packages needed if you are going to build out the
143 Yocto Project documentation manuals:
144 ::
145
146 $ sudo apt-get install make xsltproc docbook-utils fop dblatex xmlto
147
148Fedora Packages
149---------------
150
151The following list shows the required packages by function given a
152supported Fedora Linux distribution:
153
154- *Essentials:* Packages needed to build an image for a headless
155 system:
156 ::
157
158 $ sudo dnf install &FEDORA_HOST_PACKAGES_ESSENTIAL;
159
160- *Documentation:* Packages needed if you are going to build out the
161 Yocto Project documentation manuals:
162 ::
163
164 $ sudo dnf install docbook-style-dsssl docbook-style-xsl \
165 docbook-dtds docbook-utils fop libxslt dblatex xmlto
166
167openSUSE Packages
168-----------------
169
170The following list shows the required packages by function given a
171supported openSUSE Linux distribution:
172
173- *Essentials:* Packages needed to build an image for a headless
174 system:
175 ::
176
177 $ sudo zypper install &OPENSUSE_HOST_PACKAGES_ESSENTIAL;
178
179- *Documentation:* Packages needed if you are going to build out the
180 Yocto Project documentation manuals: $ sudo zypper install dblatex
181 xmlto
182
183CentOS-7 Packages
184-----------------
185
186The following list shows the required packages by function given a
187supported CentOS-7 Linux distribution:
188
189- *Essentials:* Packages needed to build an image for a headless
190 system:
191 ::
192
193 $ sudo yum install &CENTOS7_HOST_PACKAGES_ESSENTIAL;
194
195 .. note::
196
197 - Extra Packages for Enterprise Linux (i.e. ``epel-release``) is
198 a collection of packages from Fedora built on RHEL/CentOS for
199 easy installation of packages not included in enterprise Linux
200 by default. You need to install these packages separately.
201
202 - The ``makecache`` command consumes additional Metadata from
203 ``epel-release``.
204
205- *Documentation:* Packages needed if you are going to build out the
206 Yocto Project documentation manuals:
207 ::
208
209 $ sudo yum install docbook-style-dsssl docbook-style-xsl \
210 docbook-dtds docbook-utils fop libxslt dblatex xmlto
211
212CentOS-8 Packages
213-----------------
214
215The following list shows the required packages by function given a
216supported CentOS-8 Linux distribution:
217
218- *Essentials:* Packages needed to build an image for a headless
219 system:
220 ::
221
222 $ sudo dnf install &CENTOS8_HOST_PACKAGES_ESSENTIAL;
223
224 .. note::
225
226 - Extra Packages for Enterprise Linux (i.e. ``epel-release``) is
227 a collection of packages from Fedora built on RHEL/CentOS for
228 easy installation of packages not included in enterprise Linux
229 by default. You need to install these packages separately.
230
231 - The ``PowerTools`` repo provides additional packages such as
232 ``rpcgen`` and ``texinfo``.
233
234 - The ``makecache`` command consumes additional Metadata from
235 ``epel-release``.
236
237- *Documentation:* Packages needed if you are going to build out the
238 Yocto Project documentation manuals:
239 ::
240
241 $ sudo dnf install docbook-style-dsssl docbook-style-xsl \\
242 docbook-dtds docbook-utils fop libxslt dblatex xmlto
243
244Required Git, tar, Python and gcc Versions
245==========================================
246
247In order to use the build system, your host development system must meet
248the following version requirements for Git, tar, and Python:
249
250- Git 1.8.3.1 or greater
251
252- tar 1.28 or greater
253
254- Python 3.5.0 or greater
255
256If your host development system does not meet all these requirements,
257you can resolve this by installing a ``buildtools`` tarball that
258contains these tools. You can get the tarball one of two ways: download
259a pre-built tarball or use BitBake to build the tarball.
260
261In addition, your host development system must meet the following
262version requirement for gcc:
263
264- gcc 5.0 or greater
265
266If your host development system does not meet this requirement, you can
267resolve this by installing a ``buildtools-extended`` tarball that
268contains additional tools, the equivalent of ``buildtools-essential``.
269
270Installing a Pre-Built ``buildtools`` Tarball with ``install-buildtools`` script
271--------------------------------------------------------------------------------
272
273The ``install-buildtools`` script is the easiest of the three methods by
274which you can get these tools. It downloads a pre-built buildtools
275installer and automatically installs the tools for you:
276
2771. Execute the ``install-buildtools`` script. Here is an example:
278 ::
279
280 $ cd poky
281 $ scripts/install-buildtools --without-extended-buildtools \
282 --base-url https://downloads.yoctoproject.org/releases/yocto \
283 --release yocto-&DISTRO; \
284 --installer-version &DISTRO;
285
286 During execution, the buildtools tarball will be downloaded, the
287 checksum of the download will be verified, the installer will be run
288 for you, and some basic checks will be run to to make sure the
289 installation is functional.
290
291 To avoid the need of ``sudo`` privileges, the ``install-buildtools``
292 script will by default tell the installer to install in:
293 ::
294
295 /path/to/poky/buildtools
296
297 If your host development system needs the additional tools provided
298 in the ``buildtools-extended`` tarball, you can instead execute the
299 ``install-buildtools`` script with the default parameters:
300 ::
301
302 $ cd poky
303 $ scripts/install-buildtools
304
3052. Source the tools environment setup script by using a command like the
306 following:
307 ::
308
309 $ source /path/to/poky/buildtools/environment-setup-x86_64-pokysdk-linux
310
311 Of course, you need to supply your installation directory and be sure to
312 use the right file (i.e. i586 or x86_64).
313
314 After you have sourced the setup script, the tools are added to
315 ``PATH`` and any other environment variables required to run the
316 tools are initialized. The results are working versions versions of
317 Git, tar, Python and ``chrpath``. And in the case of the
318 ``buildtools-extended`` tarball, additional working versions of tools
319 including ``gcc``, ``make`` and the other tools included in
320 ``packagegroup-core-buildessential``.
321
322Downloading a Pre-Built ``buildtools`` Tarball
323----------------------------------------------
324
325Downloading and running a pre-built buildtools installer is the easiest
326of the two methods by which you can get these tools:
327
3281. Locate and download the ``*.sh`` at &YOCTO_RELEASE_DL_URL;/buildtools/
329
3302. Execute the installation script. Here is an example for the
331 traditional installer:
332 ::
333
334 $ sh ~/Downloads/x86_64-buildtools-nativesdk-standalone-DISTRO.sh
335
336 Here is an example for the extended installer:
337 ::
338
339 $ sh ~/Downloads/x86_64-buildtools-extended-nativesdk-standalone-DISTRO.sh
340
341 During execution, a prompt appears that allows you to choose the
342 installation directory. For example, you could choose the following:
343 /home/your-username/buildtools
344
3453. Source the tools environment setup script by using a command like the
346 following:
347 ::
348
349 $ source /home/your_username/buildtools/environment-setup-i586-poky-linux
350
351 Of
352 course, you need to supply your installation directory and be sure to
353 use the right file (i.e. i585 or x86-64).
354
355 After you have sourced the setup script, the tools are added to
356 ``PATH`` and any other environment variables required to run the
357 tools are initialized. The results are working versions versions of
358 Git, tar, Python and ``chrpath``. And in the case of the
359 ``buildtools-extended`` tarball, additional working versions of tools
360 including ``gcc``, ``make`` and the other tools included in
361 ``packagegroup-core-buildessential``.
362
363Building Your Own ``buildtools`` Tarball
364----------------------------------------
365
366Building and running your own buildtools installer applies only when you
367have a build host that can already run BitBake. In this case, you use
368that machine to build the ``.sh`` file and then take steps to transfer
369and run it on a machine that does not meet the minimal Git, tar, and
370Python (or gcc) requirements.
371
372Here are the steps to take to build and run your own buildtools
373installer:
374
3751. On the machine that is able to run BitBake, be sure you have set up
376 your build environment with the setup script
377 (:ref:`structure-core-script`).
378
3792. Run the BitBake command to build the tarball:
380 ::
381
382 $ bitbake buildtools-tarball
383
384 or run the BitBake command to build the extended tarball:
385 ::
386
387 $ bitbake buildtools-extended-tarball
388
389 .. note::
390
391 The
392 SDKMACHINE
393 variable in your
394 local.conf
395 file determines whether you build tools for a 32-bit or 64-bit
396 system.
397
398 Once the build completes, you can find the ``.sh`` file that installs
399 the tools in the ``tmp/deploy/sdk`` subdirectory of the
400 :term:`Build Directory`. The installer file has the string
401 "buildtools" (or "buildtools-extended") in the name.
402
4033. Transfer the ``.sh`` file from the build host to the machine that
404 does not meet the Git, tar, or Python (or gcc) requirements.
405
4064. On the machine that does not meet the requirements, run the ``.sh``
407 file to install the tools. Here is an example for the traditional
408 installer:
409 ::
410
411 $ sh ~/Downloads/x86_64-buildtools-nativesdk-standalone-&DISTRO;.sh
412
413 Here is an example for the extended installer:
414 ::
415
416 $ sh ~/Downloads/x86_64-buildtools-extended-nativesdk-standalone-&DISTRO;.sh
417
418 During execution, a prompt appears that allows you to choose the
419 installation directory. For example, you could choose the following:
420 /home/your_username/buildtools
421
4225. Source the tools environment setup script by using a command like the
423 following:
424 ::
425
426 $ source /home/your_username/buildtools/environment-setup-x86_64-poky-linux
427
428 Of course, you need to supply your installation directory and be sure to
429 use the right file (i.e. i586 or x86_64).
430
431 After you have sourced the setup script, the tools are added to
432 ``PATH`` and any other environment variables required to run the
433 tools are initialized. The results are working versions versions of
434 Git, tar, Python and ``chrpath``. And in the case of the
435 ``buildtools-extended`` tarball, additional working versions of tools
436 including ``gcc``, ``make`` and the other tools included in
437 ``packagegroup-core-buildessential``.
diff --git a/documentation/ref-manual/ref-system-requirements.xml b/documentation/ref-manual/ref-system-requirements.xml
index c6e1eb9716..ac8b0032db 100644
--- a/documentation/ref-manual/ref-system-requirements.xml
+++ b/documentation/ref-manual/ref-system-requirements.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-manual-system-requirements'> 6<chapter id='ref-manual-system-requirements'>
6<title>System Requirements</title> 7<title>System Requirements</title>
@@ -93,14 +94,12 @@
93 <itemizedlist> 94 <itemizedlist>
94 <listitem><para>Ubuntu 16.04 (LTS)</para></listitem> 95 <listitem><para>Ubuntu 16.04 (LTS)</para></listitem>
95 <listitem><para>Ubuntu 18.04 (LTS)</para></listitem> 96 <listitem><para>Ubuntu 18.04 (LTS)</para></listitem>
96 <listitem><para>Ubuntu 19.04</para></listitem>
97 <listitem><para>Ubuntu 20.04</para></listitem> 97 <listitem><para>Ubuntu 20.04</para></listitem>
98 <listitem><para>Fedora 28</para></listitem>
99 <listitem><para>Fedora 29</para></listitem>
100 <listitem><para>Fedora 30</para></listitem> 98 <listitem><para>Fedora 30</para></listitem>
101 <listitem><para>Fedora 31</para></listitem> 99 <listitem><para>Fedora 31</para></listitem>
102 <listitem><para>Fedora 32</para></listitem> 100 <listitem><para>Fedora 32</para></listitem>
103 <listitem><para>CentOS 7.x</para></listitem> 101 <listitem><para>CentOS 7.x</para></listitem>
102 <listitem><para>CentOS 8.x</para></listitem>
104 <listitem><para>Debian GNU/Linux 8.x (Jessie)</para></listitem> 103 <listitem><para>Debian GNU/Linux 8.x (Jessie)</para></listitem>
105 <listitem><para>Debian GNU/Linux 9.x (Stretch)</para></listitem> 104 <listitem><para>Debian GNU/Linux 9.x (Stretch)</para></listitem>
106 <listitem><para>Debian GNU/Linux 10.x (Buster)</para></listitem> 105 <listitem><para>Debian GNU/Linux 10.x (Buster)</para></listitem>
diff --git a/documentation/ref-manual/ref-tasks.rst b/documentation/ref-manual/ref-tasks.rst
new file mode 100644
index 0000000000..dcdff05dce
--- /dev/null
+++ b/documentation/ref-manual/ref-tasks.rst
@@ -0,0 +1,875 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****
4Tasks
5*****
6
7Tasks are units of execution for BitBake. Recipes (``.bb`` files) use
8tasks to complete configuring, compiling, and packaging software. This
9chapter provides a reference of the tasks defined in the OpenEmbedded
10build system.
11
12Normal Recipe Build Tasks
13=========================
14
15The following sections describe normal tasks associated with building a
16recipe. For more information on tasks and dependencies, see the
17":ref:`Tasks <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:tasks>`" and
18":ref:`Dependencies <bitbake:bitbake-user-manual/bitbake-user-manual-execution:dependencies>`" sections in the
19BitBake User Manual.
20
21.. _ref-tasks-build:
22
23``do_build``
24------------
25
26The default task for all recipes. This task depends on all other normal
27tasks required to build a recipe.
28
29.. _ref-tasks-compile:
30
31``do_compile``
32--------------
33
34Compiles the source code. This task runs with the current working
35directory set to ``${``\ :term:`B`\ ``}``.
36
37The default behavior of this task is to run the ``oe_runmake`` function
38if a makefile (``Makefile``, ``makefile``, or ``GNUmakefile``) is found.
39If no such file is found, the ``do_compile`` task does nothing.
40
41.. _ref-tasks-compile_ptest_base:
42
43``do_compile_ptest_base``
44-------------------------
45
46Compiles the runtime test suite included in the software being built.
47
48.. _ref-tasks-configure:
49
50``do_configure``
51----------------
52
53Configures the source by enabling and disabling any build-time and
54configuration options for the software being built. The task runs with
55the current working directory set to ``${``\ :term:`B`\ ``}``.
56
57The default behavior of this task is to run ``oe_runmake clean`` if a
58makefile (``Makefile``, ``makefile``, or ``GNUmakefile``) is found and
59:term:`CLEANBROKEN` is not set to "1". If no such
60file is found or the ``CLEANBROKEN`` variable is set to "1", the
61``do_configure`` task does nothing.
62
63.. _ref-tasks-configure_ptest_base:
64
65``do_configure_ptest_base``
66---------------------------
67
68Configures the runtime test suite included in the software being built.
69
70.. _ref-tasks-deploy:
71
72``do_deploy``
73-------------
74
75Writes output files that are to be deployed to
76``${``\ :term:`DEPLOY_DIR_IMAGE`\ ``}``. The
77task runs with the current working directory set to
78``${``\ :term:`B`\ ``}``.
79
80Recipes implementing this task should inherit the
81:ref:`deploy <ref-classes-deploy>` class and should write the output
82to ``${``\ :term:`DEPLOYDIR`\ ``}``, which is not to be
83confused with ``${DEPLOY_DIR}``. The ``deploy`` class sets up
84``do_deploy`` as a shared state (sstate) task that can be accelerated
85through sstate use. The sstate mechanism takes care of copying the
86output from ``${DEPLOYDIR}`` to ``${DEPLOY_DIR_IMAGE}``.
87
88.. note::
89
90 Do not write the output directly to
91 ${DEPLOY_DIR_IMAGE}
92 , as this causes the sstate mechanism to malfunction.
93
94The ``do_deploy`` task is not added as a task by default and
95consequently needs to be added manually. If you want the task to run
96after :ref:`ref-tasks-compile`, you can add it by doing
97the following: addtask deploy after do_compile Adding ``do_deploy``
98after other tasks works the same way.
99
100.. note::
101
102 You do not need to add
103 before do_build
104 to the
105 addtask
106 command (though it is harmless), because the
107 base
108 class contains the following:
109 ::
110
111 do_build[recrdeptask] += "do_deploy"
112
113
114 See the "
115 Dependencies
116 " section in the BitBake User Manual for more information.
117
118If the ``do_deploy`` task re-executes, any previous output is removed
119(i.e. "cleaned").
120
121.. _ref-tasks-fetch:
122
123``do_fetch``
124------------
125
126Fetches the source code. This task uses the
127:term:`SRC_URI` variable and the argument's prefix to
128determine the correct :ref:`fetcher <bitbake:bb-fetchers>`
129module.
130
131.. _ref-tasks-image:
132
133``do_image``
134------------
135
136Starts the image generation process. The ``do_image`` task runs after
137the OpenEmbedded build system has run the
138:ref:`ref-tasks-rootfs` task during which packages are
139identified for installation into the image and the root filesystem is
140created, complete with post-processing.
141
142The ``do_image`` task performs pre-processing on the image through the
143:term:`IMAGE_PREPROCESS_COMMAND` and
144dynamically generates supporting ``do_image_*`` tasks as needed.
145
146For more information on image creation, see the ":ref:`image-generation-dev-environment`"
147section in the Yocto Project Overview and Concepts Manual.
148
149.. _ref-tasks-image-complete:
150
151``do_image_complete``
152---------------------
153
154Completes the image generation process. The ``do_image_complete`` task
155runs after the OpenEmbedded build system has run the
156:ref:`ref-tasks-image` task during which image
157pre-processing occurs and through dynamically generated ``do_image_*``
158tasks the image is constructed.
159
160The ``do_image_complete`` task performs post-processing on the image
161through the
162:term:`IMAGE_POSTPROCESS_COMMAND`.
163
164For more information on image creation, see the
165":ref:`image-generation-dev-environment`"
166section in the Yocto Project Overview and Concepts Manual.
167
168.. _ref-tasks-install:
169
170``do_install``
171--------------
172
173Copies files that are to be packaged into the holding area
174``${``\ :term:`D`\ ``}``. This task runs with the current
175working directory set to ``${``\ :term:`B`\ ``}``, which is the
176compilation directory. The ``do_install`` task, as well as other tasks
177that either directly or indirectly depend on the installed files (e.g.
178:ref:`ref-tasks-package`, ``do_package_write_*``, and
179:ref:`ref-tasks-rootfs`), run under
180:ref:`fakeroot <overview-manual/overview-manual-concepts:fakeroot and pseudo>`.
181
182.. note::
183
184 When installing files, be careful not to set the owner and group IDs
185 of the installed files to unintended values. Some methods of copying
186 files, notably when using the recursive ``cp`` command, can preserve
187 the UID and/or GID of the original file, which is usually not what
188 you want. The ``host-user-contaminated`` QA check checks for files
189 that probably have the wrong ownership.
190
191 Safe methods for installing files include the following:
192
193 - The ``install`` utility. This utility is the preferred method.
194
195 - The ``cp`` command with the "--no-preserve=ownership" option.
196
197 - The ``tar`` command with the "--no-same-owner" option. See the
198 ``bin_package.bbclass`` file in the ``meta/classes`` directory of
199 the :term:`Source Directory` for an example.
200
201.. _ref-tasks-install_ptest_base:
202
203``do_install_ptest_base``
204-------------------------
205
206Copies the runtime test suite files from the compilation directory to a
207holding area.
208
209.. _ref-tasks-package:
210
211``do_package``
212--------------
213
214Analyzes the content of the holding area
215``${``\ :term:`D`\ ``}`` and splits the content into subsets
216based on available packages and files. This task makes use of the
217:term:`PACKAGES` and :term:`FILES`
218variables.
219
220The ``do_package`` task, in conjunction with the
221:ref:`ref-tasks-packagedata` task, also saves some
222important package metadata. For additional information, see the
223:term:`PKGDESTWORK` variable and the
224":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
225section in the Yocto Project Overview and Concepts Manual.
226
227.. _ref-tasks-package_qa:
228
229``do_package_qa``
230-----------------
231
232Runs QA checks on packaged files. For more information on these checks,
233see the :ref:`insane <ref-classes-insane>` class.
234
235.. _ref-tasks-package_write_deb:
236
237``do_package_write_deb``
238------------------------
239
240Creates Debian packages (i.e. ``*.deb`` files) and places them in the
241``${``\ :term:`DEPLOY_DIR_DEB`\ ``}`` directory in
242the package feeds area. For more information, see the
243":ref:`package-feeds-dev-environment`" section in
244the Yocto Project Overview and Concepts Manual.
245
246.. _ref-tasks-package_write_ipk:
247
248``do_package_write_ipk``
249------------------------
250
251Creates IPK packages (i.e. ``*.ipk`` files) and places them in the
252``${``\ :term:`DEPLOY_DIR_IPK`\ ``}`` directory in
253the package feeds area. For more information, see the
254":ref:`package-feeds-dev-environment`" section in
255the Yocto Project Overview and Concepts Manual.
256
257.. _ref-tasks-package_write_rpm:
258
259``do_package_write_rpm``
260------------------------
261
262Creates RPM packages (i.e. ``*.rpm`` files) and places them in the
263``${``\ :term:`DEPLOY_DIR_RPM`\ ``}`` directory in
264the package feeds area. For more information, see the
265":ref:`package-feeds-dev-environment`" section in
266the Yocto Project Overview and Concepts Manual.
267
268.. _ref-tasks-package_write_tar:
269
270``do_package_write_tar``
271------------------------
272
273Creates tarballs and places them in the
274``${``\ :term:`DEPLOY_DIR_TAR`\ ``}`` directory in
275the package feeds area. For more information, see the
276":ref:`package-feeds-dev-environment`" section in
277the Yocto Project Overview and Concepts Manual.
278
279.. _ref-tasks-packagedata:
280
281``do_packagedata``
282------------------
283
284Saves package metadata generated by the
285:ref:`ref-tasks-package` task in
286:term:`PKGDATA_DIR` to make it available globally.
287
288.. _ref-tasks-patch:
289
290``do_patch``
291------------
292
293Locates patch files and applies them to the source code.
294
295After fetching and unpacking source files, the build system uses the
296recipe's :term:`SRC_URI` statements
297to locate and apply patch files to the source code.
298
299.. note::
300
301 The build system uses the
302 FILESPATH
303 variable to determine the default set of directories when searching
304 for patches.
305
306Patch files, by default, are ``*.patch`` and ``*.diff`` files created
307and kept in a subdirectory of the directory holding the recipe file. For
308example, consider the
309:yocto_git:`bluez5 </cgit/cgit.cgi/poky/tree/meta/recipes-connectivity/bluez5>`
310recipe from the OE-Core layer (i.e. ``poky/meta``):
311::
312
313 poky/meta/recipes-connectivity/bluez5
314
315This recipe has two patch files located here:
316::
317
318 poky/meta/recipes-connectivity/bluez5/bluez5
319
320In the ``bluez5`` recipe, the ``SRC_URI`` statements point to the source
321and patch files needed to build the package.
322
323.. note::
324
325 In the case for the
326 bluez5_5.48.bb
327 recipe, the
328 SRC_URI
329 statements are from an include file
330 bluez5.inc
331 .
332
333As mentioned earlier, the build system treats files whose file types are
334``.patch`` and ``.diff`` as patch files. However, you can use the
335"apply=yes" parameter with the ``SRC_URI`` statement to indicate any
336file as a patch file:
337::
338
339 SRC_URI = " \\
340 git://path_to_repo/some_package \\
341 file://file;apply=yes \\
342 "
343
344Conversely, if you have a directory full of patch files and you want to
345exclude some so that the ``do_patch`` task does not apply them during
346the patch phase, you can use the "apply=no" parameter with the
347``SRC_URI`` statement:
348::
349
350 SRC_URI = " \
351 git://path_to_repo/some_package \
352 file://path_to_lots_of_patch_files \
353 file://path_to_lots_of_patch_files/patch_file5;apply=no \
354 "
355
356In the
357previous example, assuming all the files in the directory holding the
358patch files end with either ``.patch`` or ``.diff``, every file would be
359applied as a patch by default except for the patch_file5 patch.
360
361You can find out more about the patching process in the
362":ref:`patching-dev-environment`" section in
363the Yocto Project Overview and Concepts Manual and the
364":ref:`new-recipe-patching-code`" section in the
365Yocto Project Development Tasks Manual.
366
367.. _ref-tasks-populate_lic:
368
369``do_populate_lic``
370-------------------
371
372Writes license information for the recipe that is collected later when
373the image is constructed.
374
375.. _ref-tasks-populate_sdk:
376
377``do_populate_sdk``
378-------------------
379
380Creates the file and directory structure for an installable SDK. See the
381":ref:`sdk-generation-dev-environment`"
382section in the Yocto Project Overview and Concepts Manual for more
383information.
384
385.. _ref-tasks-populate_sdk_ext:
386
387``do_populate_sdk_ext``
388-----------------------
389
390Creates the file and directory structure for an installable extensible
391SDK (eSDK). See the ":ref:`sdk-generation-dev-environment`"
392section in the Yocto Project Overview and Concepts Manual for more
393information.
394
395
396.. _ref-tasks-populate_sysroot:
397
398``do_populate_sysroot``
399-----------------------
400
401Stages (copies) a subset of the files installed by the
402:ref:`ref-tasks-install` task into the appropriate
403sysroot. For information on how to access these files from other
404recipes, see the :term:`STAGING_DIR* <STAGING_DIR_HOST>` variables.
405Directories that would typically not be needed by other recipes at build
406time (e.g. ``/etc``) are not copied by default.
407
408For information on what directories are copied by default, see the
409:term:`SYSROOT_DIRS* <SYSROOT_DIRS>` variables. You can change
410these variables inside your recipe if you need to make additional (or
411fewer) directories available to other recipes at build time.
412
413The ``do_populate_sysroot`` task is a shared state (sstate) task, which
414means that the task can be accelerated through sstate use. Realize also
415that if the task is re-executed, any previous output is removed (i.e.
416"cleaned").
417
418.. _ref-tasks-prepare_recipe_sysroot:
419
420``do_prepare_recipe_sysroot``
421-----------------------------
422
423Installs the files into the individual recipe specific sysroots (i.e.
424``recipe-sysroot`` and ``recipe-sysroot-native`` under
425``${``\ :term:`WORKDIR`\ ``}`` based upon the
426dependencies specified by :term:`DEPENDS`). See the
427":ref:`staging <ref-classes-staging>`" class for more information.
428
429.. _ref-tasks-rm_work:
430
431``do_rm_work``
432--------------
433
434Removes work files after the OpenEmbedded build system has finished with
435them. You can learn more by looking at the
436":ref:`rm_work.bbclass <ref-classes-rm-work>`" section.
437
438.. _ref-tasks-unpack:
439
440``do_unpack``
441-------------
442
443Unpacks the source code into a working directory pointed to by
444``${``\ :term:`WORKDIR`\ ``}``. The :term:`S`
445variable also plays a role in where unpacked source files ultimately
446reside. For more information on how source files are unpacked, see the
447":ref:`source-fetching-dev-environment`"
448section in the Yocto Project Overview and Concepts Manual and also see
449the ``WORKDIR`` and ``S`` variable descriptions.
450
451Manually Called Tasks
452=====================
453
454These tasks are typically manually triggered (e.g. by using the
455``bitbake -c`` command-line option):
456
457.. _ref-tasks-checkpkg:
458
459``do_checkpkg``
460---------------
461
462Provides information about the recipe including its upstream version and
463status. The upstream version and status reveals whether or not a version
464of the recipe exists upstream and a status of not updated, updated, or
465unknown.
466
467To check the upstream version and status of a recipe, use the following
468devtool commands:
469::
470
471 $ devtool latest-version
472 $ devtool check-upgrade-status
473
474See the ":ref:`ref-manual/ref-devtool-reference:\`\`devtool\`\` quick reference`"
475chapter for more information on
476``devtool``. See the ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
477section for information on checking the upgrade status of a recipe.
478
479To build the ``checkpkg`` task, use the ``bitbake`` command with the
480"-c" option and task name:
481::
482
483 $ bitbake core-image-minimal -c checkpkg
484
485By default, the results are stored in :term:`$LOG_DIR <LOG_DIR>` (e.g.
486``$BUILD_DIR/tmp/log``).
487
488.. _ref-tasks-checkuri:
489
490``do_checkuri``
491---------------
492
493Validates the :term:`SRC_URI` value.
494
495.. _ref-tasks-clean:
496
497``do_clean``
498------------
499
500Removes all output files for a target from the
501:ref:`ref-tasks-unpack` task forward (i.e. ``do_unpack``,
502:ref:`ref-tasks-configure`,
503:ref:`ref-tasks-compile`,
504:ref:`ref-tasks-install`, and
505:ref:`ref-tasks-package`).
506
507You can run this task using BitBake as follows:
508::
509
510 $ bitbake -c clean recipe
511
512Running this task does not remove the
513:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>` cache files.
514Consequently, if no changes have been made and the recipe is rebuilt
515after cleaning, output files are simply restored from the sstate cache.
516If you want to remove the sstate cache files for the recipe, you need to
517use the :ref:`ref-tasks-cleansstate` task instead
518(i.e. ``bitbake -c cleansstate`` recipe).
519
520.. _ref-tasks-cleanall:
521
522``do_cleanall``
523---------------
524
525Removes all output files, shared state
526(:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>`) cache, and
527downloaded source files for a target (i.e. the contents of
528:term:`DL_DIR`). Essentially, the ``do_cleanall`` task is
529identical to the :ref:`ref-tasks-cleansstate` task
530with the added removal of downloaded source files.
531
532You can run this task using BitBake as follows:
533::
534
535 $ bitbake -c cleanall recipe
536
537Typically, you would not normally use the ``cleanall`` task. Do so only
538if you want to start fresh with the :ref:`ref-tasks-fetch`
539task.
540
541.. _ref-tasks-cleansstate:
542
543``do_cleansstate``
544------------------
545
546Removes all output files and shared state
547(:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>`) cache for a
548target. Essentially, the ``do_cleansstate`` task is identical to the
549:ref:`ref-tasks-clean` task with the added removal of
550shared state (`:ref:`sstate <overview-manual/overview-manual-concepts:shared state cache>`)
551cache.
552
553You can run this task using BitBake as follows:
554::
555
556 $ bitbake -c cleansstate recipe
557
558When you run the ``do_cleansstate`` task, the OpenEmbedded build system
559no longer uses any sstate. Consequently, building the recipe from
560scratch is guaranteed.
561
562.. note::
563
564 The
565 do_cleansstate
566 task cannot remove sstate from a remote sstate mirror. If you need to
567 build a target from scratch using remote mirrors, use the "-f" option
568 as follows:
569 ::
570
571 $ bitbake -f -c do_cleansstate target
572
573
574.. _ref-tasks-devpyshell:
575
576``do_devpyshell``
577-----------------
578
579Starts a shell in which an interactive Python interpreter allows you to
580interact with the BitBake build environment. From within this shell, you
581can directly examine and set bits from the data store and execute
582functions as if within the BitBake environment. See the ":ref:`platdev-appdev-devpyshell`" section in
583the Yocto Project Development Tasks Manual for more information about
584using ``devpyshell``.
585
586.. _ref-tasks-devshell:
587
588``do_devshell``
589---------------
590
591Starts a shell whose environment is set up for development, debugging,
592or both. See the ":ref:`platdev-appdev-devshell`" section in the
593Yocto Project Development Tasks Manual for more information about using
594``devshell``.
595
596.. _ref-tasks-listtasks:
597
598``do_listtasks``
599----------------
600
601Lists all defined tasks for a target.
602
603.. _ref-tasks-package_index:
604
605``do_package_index``
606--------------------
607
608Creates or updates the index in the `:ref:`package-feeds-dev-environment` area.
609
610.. note::
611
612 This task is not triggered with the
613 bitbake -c
614 command-line option as are the other tasks in this section. Because
615 this task is specifically for the
616 package-index
617 recipe, you run it using
618 bitbake package-index
619 .
620
621Image-Related Tasks
622===================
623
624The following tasks are applicable to image recipes.
625
626.. _ref-tasks-bootimg:
627
628``do_bootimg``
629--------------
630
631Creates a bootable live image. See the
632:term:`IMAGE_FSTYPES` variable for additional
633information on live image types.
634
635.. _ref-tasks-bundle_initramfs:
636
637``do_bundle_initramfs``
638-----------------------
639
640Combines an initial RAM disk (initramfs) image and kernel together to
641form a single image. The
642:term:`CONFIG_INITRAMFS_SOURCE` variable
643has some more information about these types of images.
644
645.. _ref-tasks-rootfs:
646
647``do_rootfs``
648-------------
649
650Creates the root filesystem (file and directory structure) for an image.
651See the ":ref:`image-generation-dev-environment`"
652section in the Yocto Project Overview and Concepts Manual for more
653information on how the root filesystem is created.
654
655.. _ref-tasks-testimage:
656
657``do_testimage``
658----------------
659
660Boots an image and performs runtime tests within the image. For
661information on automatically testing images, see the
662":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
663section in the Yocto Project Development Tasks Manual.
664
665.. _ref-tasks-testimage_auto:
666
667``do_testimage_auto``
668---------------------
669
670Boots an image and performs runtime tests within the image immediately
671after it has been built. This task is enabled when you set
672:term:`TESTIMAGE_AUTO` equal to "1".
673
674For information on automatically testing images, see the
675":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
676section in the Yocto Project Development Tasks Manual.
677
678Kernel-Related Tasks
679====================
680
681The following tasks are applicable to kernel recipes. Some of these
682tasks (e.g. the :ref:`ref-tasks-menuconfig` task) are
683also applicable to recipes that use Linux kernel style configuration
684such as the BusyBox recipe.
685
686.. _ref-tasks-compile_kernelmodules:
687
688``do_compile_kernelmodules``
689----------------------------
690
691Runs the step that builds the kernel modules (if needed). Building a
692kernel consists of two steps: 1) the kernel (``vmlinux``) is built, and
6932) the modules are built (i.e. ``make modules``).
694
695.. _ref-tasks-diffconfig:
696
697``do_diffconfig``
698-----------------
699
700When invoked by the user, this task creates a file containing the
701differences between the original config as produced by
702:ref:`ref-tasks-kernel_configme` task and the
703changes made by the user with other methods (i.e. using
704(:ref:`ref-tasks-kernel_menuconfig`). Once the
705file of differences is created, it can be used to create a config
706fragment that only contains the differences. You can invoke this task
707from the command line as follows:
708::
709
710 $ bitbake linux-yocto -c diffconfig
711
712For more information, see the
713":ref:`kernel-dev/kernel-dev-common:creating configuration fragments`"
714section in the Yocto Project Linux Kernel Development Manual.
715
716.. _ref-tasks-kernel_checkout:
717
718``do_kernel_checkout``
719----------------------
720
721Converts the newly unpacked kernel source into a form with which the
722OpenEmbedded build system can work. Because the kernel source can be
723fetched in several different ways, the ``do_kernel_checkout`` task makes
724sure that subsequent tasks are given a clean working tree copy of the
725kernel with the correct branches checked out.
726
727.. _ref-tasks-kernel_configcheck:
728
729``do_kernel_configcheck``
730-------------------------
731
732Validates the configuration produced by the
733:ref:`ref-tasks-kernel_menuconfig` task. The
734``do_kernel_configcheck`` task produces warnings when a requested
735configuration does not appear in the final ``.config`` file or when you
736override a policy configuration in a hardware configuration fragment.
737You can run this task explicitly and view the output by using the
738following command:
739::
740
741 $ bitbake linux-yocto -c kernel_configcheck -f
742
743For more information, see the
744":ref:`kernel-dev/kernel-dev-common:validating configuration`"
745section in the Yocto Project Linux Kernel Development Manual.
746
747.. _ref-tasks-kernel_configme:
748
749``do_kernel_configme``
750----------------------
751
752After the kernel is patched by the :ref:`ref-tasks-patch`
753task, the ``do_kernel_configme`` task assembles and merges all the
754kernel config fragments into a merged configuration that can then be
755passed to the kernel configuration phase proper. This is also the time
756during which user-specified defconfigs are applied if present, and where
757configuration modes such as ``--allnoconfig`` are applied.
758
759.. _ref-tasks-kernel_menuconfig:
760
761``do_kernel_menuconfig``
762------------------------
763
764Invoked by the user to manipulate the ``.config`` file used to build a
765linux-yocto recipe. This task starts the Linux kernel configuration
766tool, which you then use to modify the kernel configuration.
767
768.. note::
769
770 You can also invoke this tool from the command line as follows:
771 ::
772
773 $ bitbake linux-yocto -c menuconfig
774
775
776See the ":ref:`kernel-dev/kernel-dev-common:using \`\`menuconfig\`\``"
777section in the Yocto Project Linux Kernel Development Manual for more
778information on this configuration tool.
779
780.. _ref-tasks-kernel_metadata:
781
782``do_kernel_metadata``
783----------------------
784
785Collects all the features required for a given kernel build, whether the
786features come from :term:`SRC_URI` or from Git
787repositories. After collection, the ``do_kernel_metadata`` task
788processes the features into a series of config fragments and patches,
789which can then be applied by subsequent tasks such as
790:ref:`ref-tasks-patch` and
791:ref:`ref-tasks-kernel_configme`.
792
793.. _ref-tasks-menuconfig:
794
795``do_menuconfig``
796-----------------
797
798Runs ``make menuconfig`` for the kernel. For information on
799``menuconfig``, see the
800":ref:`kernel-dev/kernel-dev-common:using \`\`menuconfig\`\``"
801section in the Yocto Project Linux Kernel Development Manual.
802
803.. _ref-tasks-savedefconfig:
804
805``do_savedefconfig``
806--------------------
807
808When invoked by the user, creates a defconfig file that can be used
809instead of the default defconfig. The saved defconfig contains the
810differences between the default defconfig and the changes made by the
811user using other methods (i.e. the
812:ref:`ref-tasks-kernel_menuconfig` task. You
813can invoke the task using the following command:
814::
815
816 $ bitbake linux-yocto -c savedefconfig
817
818.. _ref-tasks-shared_workdir:
819
820``do_shared_workdir``
821---------------------
822
823After the kernel has been compiled but before the kernel modules have
824been compiled, this task copies files required for module builds and
825which are generated from the kernel build into the shared work
826directory. With these copies successfully copied, the
827:ref:`ref-tasks-compile_kernelmodules` task
828can successfully build the kernel modules in the next step of the build.
829
830.. _ref-tasks-sizecheck:
831
832``do_sizecheck``
833----------------
834
835After the kernel has been built, this task checks the size of the
836stripped kernel image against
837:term:`KERNEL_IMAGE_MAXSIZE`. If that
838variable was set and the size of the stripped kernel exceeds that size,
839the kernel build produces a warning to that effect.
840
841.. _ref-tasks-strip:
842
843``do_strip``
844------------
845
846If ``KERNEL_IMAGE_STRIP_EXTRA_SECTIONS`` is defined, this task strips
847the sections named in that variable from ``vmlinux``. This stripping is
848typically used to remove nonessential sections such as ``.comment``
849sections from a size-sensitive configuration.
850
851.. _ref-tasks-validate_branches:
852
853``do_validate_branches``
854------------------------
855
856After the kernel is unpacked but before it is patched, this task makes
857sure that the machine and metadata branches as specified by the
858:term:`SRCREV` variables actually exist on the specified
859branches. If these branches do not exist and
860:term:`AUTOREV` is not being used, the
861``do_validate_branches`` task fails during the build.
862
863Miscellaneous Tasks
864===================
865
866The following sections describe miscellaneous tasks.
867
868.. _ref-tasks-spdx:
869
870``do_spdx``
871-----------
872
873A build stage that takes the source code and scans it on a remote
874FOSSOLOGY server in order to produce an SPDX document. This task applies
875only to the :ref:`spdx <ref-classes-spdx>` class.
diff --git a/documentation/ref-manual/ref-tasks.xml b/documentation/ref-manual/ref-tasks.xml
index 011e0d7496..5b09b3f2e8 100644
--- a/documentation/ref-manual/ref-tasks.xml
+++ b/documentation/ref-manual/ref-tasks.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-tasks'> 6<chapter id='ref-tasks'>
6<title>Tasks</title> 7<title>Tasks</title>
diff --git a/documentation/ref-manual/ref-terms.rst b/documentation/ref-manual/ref-terms.rst
new file mode 100644
index 0000000000..600cc23c3e
--- /dev/null
+++ b/documentation/ref-manual/ref-terms.rst
@@ -0,0 +1,397 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************
4Yocto Project Terms
5*******************
6
7Following is a list of terms and definitions users new to the Yocto Project
8development environment might find helpful. While some of these terms are
9universal, the list includes them just in case:
10
11.. glossary::
12
13 Append Files
14 Files that append build information to a recipe file. Append files are
15 known as BitBake append files and ``.bbappend`` files. The OpenEmbedded
16 build system expects every append file to have a corresponding recipe
17 (``.bb``) file. Furthermore, the append file and corresponding recipe file
18 must use the same root filename. The filenames can differ only in the
19 file type suffix used (e.g. ``formfactor_0.0.bb`` and
20 ``formfactor_0.0.bbappend``).
21
22 Information in append files extends or overrides the information in the
23 similarly-named recipe file. For an example of an append file in use, see
24 the ":ref:`dev-manual/dev-manual-common-tasks:Using .bbappend Files in
25 Your Layer`" section in the Yocto Project Development Tasks Manual.
26
27 When you name an append file, you can use the "``%``" wildcard character
28 to allow for matching recipe names. For example, suppose you have an
29 append file named as follows:
30 ::
31
32 busybox_1.21.%.bbappend
33
34 That append file
35 would match any ``busybox_1.21.``\ x\ ``.bb`` version of the recipe. So,
36 the append file would match any of the following recipe names:
37
38 .. code-block:: shell
39
40 busybox_1.21.1.bb
41 busybox_1.21.2.bb
42 busybox_1.21.3.bb
43 busybox_1.21.10.bb
44 busybox_1.21.25.bb
45
46 .. note::
47
48 The use of the " % " character is limited in that it only works
49 directly in front of the .bbappend portion of the append file's
50 name. You cannot use the wildcard character in any other location of
51 the name.
52
53 BitBake
54 The task executor and scheduler used by the OpenEmbedded build system to
55 build images. For more information on BitBake, see the :doc:`BitBake User
56 Manual <bitbake:index>`.
57
58 Board Support Package (BSP)
59 A group of drivers, definitions, and other components that provide support
60 for a specific hardware configuration. For more information on BSPs, see
61 the :ref:`bsp-guide/bsp-guide:Yocto Project Board Support Package
62 Developer's Guide`.
63
64 Build Directory
65 This term refers to the area used by the OpenEmbedded build system for
66 builds. The area is created when you ``source`` the setup environment
67 script that is found in the Source Directory
68 (i.e. :ref:`ref-manual/ref-structure:\`\`oe-init-build-env\`\``). The
69 :term:`TOPDIR` variable points to the Build Directory.
70
71 You have a lot of flexibility when creating the Build Directory.
72 Following are some examples that show how to create the directory. The
73 examples assume your :term:`Source Directory` is named ``poky``:
74
75 - Create the Build Directory inside your Source Directory and let
76 the name of the Build Directory default to ``build``:
77
78 .. code-block:: shell
79
80 $ cd $HOME/poky
81 $ source oe-init-build-env
82
83 - Create the Build Directory inside your home directory and
84 specifically name it ``test-builds``:
85
86 .. code-block:: shell
87
88 $ cd $HOME
89 $ source poky/oe-init-build-env test-builds
90
91 - Provide a directory path and specifically name the Build
92 Directory. Any intermediate folders in the pathname must exist.
93 This next example creates a Build Directory named
94 ``YP-POKYVERSION`` in your home directory within the existing
95 directory ``mybuilds``:
96
97 .. code-block:: shell
98
99 $ cd $HOME
100 $ source $HOME/poky/oe-init-build-env $HOME/mybuilds/YP-POKYVERSION
101
102 .. note::
103
104 By default, the Build Directory contains :term:`TMPDIR` , which is a
105 temporary directory the build system uses for its work. TMPDIR cannot
106 be under NFS. Thus, by default, the Build Directory cannot be under
107 NFS. However, if you need the Build Directory to be under NFS, you can
108 set this up by setting TMPDIR in your local.conf file to use a local
109 drive. Doing so effectively separates TMPDIR from TOPDIR , which is the
110 Build Directory.
111
112 Build Host
113 The system used to build images in a Yocto Project Development
114 environment. The build system is sometimes referred to as the development
115 host.
116
117 Classes
118 Files that provide for logic encapsulation and inheritance so that
119 commonly used patterns can be defined once and then easily used in
120 multiple recipes. For reference information on the Yocto Project classes,
121 see the ":ref:`ref-manual/ref-classes:Classes`" chapter. Class files end with the
122 ``.bbclass`` filename extension.
123
124 Configuration File
125 Files that hold global definitions of variables, user-defined variables,
126 and hardware configuration information. These files tell the OpenEmbedded
127 build system what to build and what to put into the image to support a
128 particular platform.
129
130 Configuration files end with a ``.conf`` filename extension. The
131 :file:`conf/local.conf` configuration file in the :term:`Build Directory`
132 contains user-defined variables that affect every build. The
133 :file:`meta-poky/conf/distro/poky.conf` configuration file defines Yocto
134 "distro" configuration variables used only when building with this
135 policy. Machine configuration files, which are located throughout the
136 :term:`Source Directory`, define variables for specific hardware and are
137 only used when building for that target (e.g. the
138 :file:`machine/beaglebone.conf` configuration file defines variables for
139 the Texas Instruments ARM Cortex-A8 development board).
140
141 Container Layer
142 Layers that hold other layers. An example of a container layer is
143 OpenEmbedded's `meta-openembedded
144 <https://github.com/openembedded/meta-openembedded>`_ layer. The
145 ``meta-openembedded`` layer contains many ``meta-*`` layers.
146
147 Cross-Development Toolchain
148 In general, a cross-development toolchain is a collection of software
149 development tools and utilities that run on one architecture and allow you
150 to develop software for a different, or targeted, architecture. These
151 toolchains contain cross-compilers, linkers, and debuggers that are
152 specific to the target architecture.
153
154 The Yocto Project supports two different cross-development toolchains:
155
156 - A toolchain only used by and within BitBake when building an image for a
157 target architecture.
158
159 - A relocatable toolchain used outside of BitBake by developers when
160 developing applications that will run on a targeted device.
161
162 Creation of these toolchains is simple and automated. For information on
163 toolchain concepts as they apply to the Yocto Project, see the
164 ":ref:`overview-manual/overview-manual-concepts:Cross-Development
165 Toolchain Generation`" section in the Yocto Project Overview and Concepts
166 Manual. You can also find more information on using the relocatable
167 toolchain in the :ref:`sdk-manual/sdk-manual:Yocto Project Application
168 Development and the Extensible Software Development Kit (eSDK)` manual.
169
170 Extensible Software Development Kit (eSDK)
171 A custom SDK for application developers. This eSDK allows developers to
172 incorporate their library and programming changes back into the image to
173 make their code available to other application developers.
174
175 For information on the eSDK, see the :ref:`sdk-manual/sdk-manual:Yocto
176 Project Application Development and the Extensible Software Development
177 Kit (eSDK)` manual.
178
179 Image
180 An image is an artifact of the BitBake build process given a collection of
181 recipes and related Metadata. Images are the binary output that run on
182 specific hardware or QEMU and are used for specific use-cases. For a list
183 of the supported image types that the Yocto Project provides, see the
184 ":ref:`ref-manual/ref-images:Images`" chapter.
185
186 Layer
187 A collection of related recipes. Layers allow you to consolidate related
188 metadata to customize your build. Layers also isolate information used
189 when building for multiple architectures. Layers are hierarchical in
190 their ability to override previous specifications. You can include any
191 number of available layers from the Yocto Project and customize the build
192 by adding your layers after them. You can search the Layer Index for
193 layers used within Yocto Project.
194
195 For introductory information on layers, see the
196 ":ref:`overview-manual/overview-manual-yp-intro:The Yocto Project Layer
197 Model`" section in the Yocto Project Overview and Concepts Manual. For
198 more detailed information on layers, see the
199 ":ref:`dev-manual/dev-manual-common-tasks:Understanding and Creating
200 Layers`" section in the Yocto Project Development Tasks Manual. For a
201 discussion specifically on BSP Layers, see the ":ref:`bsp-guide/bsp:BSP
202 Layers`" section in the Yocto Project Board Support Packages (BSP)
203 Developer's Guide.
204
205 Metadata
206 A key element of the Yocto Project is the Metadata that
207 is used to construct a Linux distribution and is contained in the
208 files that the :term:`OpenEmbedded Build System`
209 parses when building an image. In general, Metadata includes recipes,
210 configuration files, and other information that refers to the build
211 instructions themselves, as well as the data used to control what
212 things get built and the effects of the build. Metadata also includes
213 commands and data used to indicate what versions of software are
214 used, from where they are obtained, and changes or additions to the
215 software itself (patches or auxiliary files) that are used to fix
216 bugs or customize the software for use in a particular situation.
217 OpenEmbedded-Core is an important set of validated metadata.
218
219 In the context of the kernel ("kernel Metadata"), the term refers to
220 the kernel config fragments and features contained in the
221 :yocto_git:`yocto-kernel-cache </cgit/cgit.cgi/yocto-kernel-cache>`
222 Git repository.
223
224 OpenEmbedded-Core (OE-Core)
225 OE-Core is metadata comprised of
226 foundational recipes, classes, and associated files that are meant to
227 be common among many different OpenEmbedded-derived systems,
228 including the Yocto Project. OE-Core is a curated subset of an
229 original repository developed by the OpenEmbedded community that has
230 been pared down into a smaller, core set of continuously validated
231 recipes. The result is a tightly controlled and an quality-assured
232 core set of recipes.
233
234 You can see the Metadata in the ``meta`` directory of the Yocto
235 Project :yocto_git:`Source Repositories <>`.
236
237 OpenEmbedded Build System
238 The build system specific to the Yocto
239 Project. The OpenEmbedded build system is based on another project
240 known as "Poky", which uses :term:`BitBake` as the task
241 executor. Throughout the Yocto Project documentation set, the
242 OpenEmbedded build system is sometimes referred to simply as "the
243 build system". If other build systems, such as a host or target build
244 system are referenced, the documentation clearly states the
245 difference.
246
247 .. note::
248
249 For some historical information about Poky, see the
250 Poky
251 term.
252
253 Package
254 In the context of the Yocto Project, this term refers to a
255 recipe's packaged output produced by BitBake (i.e. a "baked recipe").
256 A package is generally the compiled binaries produced from the
257 recipe's sources. You "bake" something by running it through BitBake.
258
259 It is worth noting that the term "package" can, in general, have
260 subtle meanings. For example, the packages referred to in the
261 "`Required Packages for the Build
262 Host <#required-packages-for-the-build-host>`__" section are compiled
263 binaries that, when installed, add functionality to your Linux
264 distribution.
265
266 Another point worth noting is that historically within the Yocto
267 Project, recipes were referred to as packages - thus, the existence
268 of several BitBake variables that are seemingly mis-named, (e.g.
269 :term:`PR`, :term:`PV`, and
270 :term:`PE`).
271
272 Package Groups
273 Arbitrary groups of software Recipes. You use
274 package groups to hold recipes that, when built, usually accomplish a
275 single task. For example, a package group could contain the recipes
276 for a company's proprietary or value-add software. Or, the package
277 group could contain the recipes that enable graphics. A package group
278 is really just another recipe. Because package group files are
279 recipes, they end with the ``.bb`` filename extension.
280
281 Poky
282 Poky, which is pronounced *Pock*-ee, is a reference embedded
283 distribution and a reference test configuration. Poky provides the
284 following:
285
286 - A base-level functional distro used to illustrate how to customize
287 a distribution.
288
289 - A means by which to test the Yocto Project components (i.e. Poky
290 is used to validate the Yocto Project).
291
292 - A vehicle through which you can download the Yocto Project.
293
294 Poky is not a product level distro. Rather, it is a good starting
295 point for customization.
296
297 .. note::
298
299 Poky began as an open-source project initially developed by
300 OpenedHand. OpenedHand developed Poky from the existing
301 OpenEmbedded build system to create a commercially supportable
302 build system for embedded Linux. After Intel Corporation acquired
303 OpenedHand, the poky project became the basis for the Yocto
304 Project's build system.
305
306 Recipe
307 A set of instructions for building packages. A recipe
308 describes where you get source code, which patches to apply, how to
309 configure the source, how to compile it and so on. Recipes also
310 describe dependencies for libraries or for other recipes. Recipes
311 represent the logical unit of execution, the software to build, the
312 images to build, and use the ``.bb`` file extension.
313
314 Reference Kit
315 A working example of a system, which includes a
316 :term:`BSP<Board Support Package (BSP)>` as well as a
317 :term:`build host<Build Host>` and other components, that can
318 work on specific hardware.
319
320 Source Directory
321 This term refers to the directory structure
322 created as a result of creating a local copy of the ``poky`` Git
323 repository ``git://git.yoctoproject.org/poky`` or expanding a
324 released ``poky`` tarball.
325
326 .. note::
327
328 Creating a local copy of the
329 poky
330 Git repository is the recommended method for setting up your
331 Source Directory.
332
333 Sometimes you might hear the term "poky directory" used to refer to
334 this directory structure.
335
336 .. note::
337
338 The OpenEmbedded build system does not support file or directory
339 names that contain spaces. Be sure that the Source Directory you
340 use does not contain these types of names.
341
342 The Source Directory contains BitBake, Documentation, Metadata and
343 other files that all support the Yocto Project. Consequently, you
344 must have the Source Directory in place on your development system in
345 order to do any development using the Yocto Project.
346
347 When you create a local copy of the Git repository, you can name the
348 repository anything you like. Throughout much of the documentation,
349 "poky" is used as the name of the top-level folder of the local copy
350 of the poky Git repository. So, for example, cloning the ``poky`` Git
351 repository results in a local Git repository whose top-level folder
352 is also named "poky".
353
354 While it is not recommended that you use tarball expansion to set up
355 the Source Directory, if you do, the top-level directory name of the
356 Source Directory is derived from the Yocto Project release tarball.
357 For example, downloading and unpacking
358 :yocto_dl:`/releases/yocto/&DISTRO_REL_TAG;/&YOCTO_POKY;.tar.bz2`
359 results in a Source Directory whose root folder is named ``poky``.
360
361 It is important to understand the differences between the Source
362 Directory created by unpacking a released tarball as compared to
363 cloning ``git://git.yoctoproject.org/poky``. When you unpack a
364 tarball, you have an exact copy of the files based on the time of
365 release - a fixed release point. Any changes you make to your local
366 files in the Source Directory are on top of the release and will
367 remain local only. On the other hand, when you clone the ``poky`` Git
368 repository, you have an active development repository with access to
369 the upstream repository's branches and tags. In this case, any local
370 changes you make to the local Source Directory can be later applied
371 to active development branches of the upstream ``poky`` Git
372 repository.
373
374 For more information on concepts related to Git repositories,
375 branches, and tags, see the
376 ":ref:`overview-manual/overview-manual-development-environment:repositories, tags, and branches`"
377 section in the Yocto Project Overview and Concepts Manual.
378
379 Task
380 A unit of execution for BitBake (e.g.
381 :ref:`ref-tasks-compile`,
382 :ref:`ref-tasks-fetch`,
383 :ref:`ref-tasks-patch`, and so forth).
384
385 Toaster
386 A web interface to the Yocto Project's :term:`OpenEmbedded Build System`.
387 The interface enables you to
388 configure and run your builds. Information about builds is collected
389 and stored in a database. For information on Toaster, see the
390 :doc:`../toaster-manual/toaster-manual`.
391
392 Upstream
393 A reference to source code or repositories that are not
394 local to the development system but located in a master area that is
395 controlled by the maintainer of the source code. For example, in
396 order for a developer to work on a particular piece of code, they
397 need to first get a copy of it from an "upstream" source.
diff --git a/documentation/ref-manual/ref-terms.xml b/documentation/ref-manual/ref-terms.xml
index 722fa7ee27..2a0452bd78 100644
--- a/documentation/ref-manual/ref-terms.xml
+++ b/documentation/ref-manual/ref-terms.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-terms'> 6<chapter id='ref-terms'>
6<title>Yocto Project Terms</title> 7<title>Yocto Project Terms</title>
@@ -364,7 +365,7 @@
364 You use package groups to hold recipes that, when built, 365 You use package groups to hold recipes that, when built,
365 usually accomplish a single task. 366 usually accomplish a single task.
366 For example, a package group could contain the recipes for a 367 For example, a package group could contain the recipes for a
367 companys proprietary or value-add software. 368 company's proprietary or value-add software.
368 Or, the package group could contain the recipes that enable 369 Or, the package group could contain the recipes that enable
369 graphics. 370 graphics.
370 A package group is really just another recipe. 371 A package group is really just another recipe.
diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
new file mode 100644
index 0000000000..c49c208bc0
--- /dev/null
+++ b/documentation/ref-manual/ref-variables.rst
@@ -0,0 +1,8899 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************
4Variables Glossary
5******************
6
7This chapter lists common variables used in the OpenEmbedded build
8system and gives an overview of their function and contents.
9
10`A <#var-ABIEXTENSION>`__ :term:`B` `C <#var-CACHE>`__
11:term:`D` `E <#var-EFI_PROVIDER>`__ `F <#var-FEATURE_PACKAGES>`__
12`G <#var-GCCPIE>`__ `H <#var-HOMEPAGE>`__ `I <#var-ICECC_DISABLED>`__
13`K <#var-KARCH>`__ `L <#var-LABELS>`__ `M <#var-MACHINE>`__
14`N <#var-NATIVELSBSTRING>`__ `O <#var-OBJCOPY>`__ :term:`P`
15`R <#var-RANLIB>`__ :term:`S` :term:`T`
16`U <#var-UBOOT_CONFIG>`__ `V <#var-VOLATILE_LOG_DIR>`__
17`W <#var-WARN_QA>`__ `X <#var-XSERVER>`__
18
19.. glossary::
20
21 ABIEXTENSION
22 Extension to the Application Binary Interface (ABI) field of the GNU
23 canonical architecture name (e.g. "eabi").
24
25 ABI extensions are set in the machine include files. For example, the
26 ``meta/conf/machine/include/arm/arch-arm.inc`` file sets the
27 following extension:
28 ::
29
30 ABIEXTENSION = "eabi"
31
32 ALLOW_EMPTY
33 Specifies whether to produce an output package even if it is empty.
34 By default, BitBake does not produce empty packages. This default
35 behavior can cause issues when there is an
36 :term:`RDEPENDS` or some other hard runtime
37 requirement on the existence of the package.
38
39 Like all package-controlling variables, you must always use them in
40 conjunction with a package name override, as in:
41 ::
42
43 ALLOW_EMPTY_${PN} = "1"
44 ALLOW_EMPTY_${PN}-dev = "1"
45 ALLOW_EMPTY_${PN}-staticdev = "1"
46
47 ALTERNATIVE
48 Lists commands in a package that need an alternative binary naming
49 scheme. Sometimes the same command is provided in multiple packages.
50 When this occurs, the OpenEmbedded build system needs to use the
51 alternatives system to create a different binary naming scheme so the
52 commands can co-exist.
53
54 To use the variable, list out the package's commands that also exist
55 as part of another package. For example, if the ``busybox`` package
56 has four commands that also exist as part of another package, you
57 identify them as follows:
58 ::
59
60 ALTERNATIVE_busybox = "sh sed test bracket"
61
62 For more information on the alternatives system, see the
63 ":ref:`update-alternatives.bbclass <ref-classes-update-alternatives>`"
64 section.
65
66 ALTERNATIVE_LINK_NAME
67 Used by the alternatives system to map duplicated commands to actual
68 locations. For example, if the ``bracket`` command provided by the
69 ``busybox`` package is duplicated through another package, you must
70 use the ``ALTERNATIVE_LINK_NAME`` variable to specify the actual
71 location:
72 ::
73
74 ALTERNATIVE_LINK_NAME[bracket] = "/usr/bin/["
75
76 In this example, the binary for the ``bracket`` command (i.e. ``[``)
77 from the ``busybox`` package resides in ``/usr/bin/``.
78
79 .. note::
80
81 If ALTERNATIVE_LINK_NAME is not defined, it defaults to ${bindir}/ name.
82
83 For more information on the alternatives system, see the
84 ":ref:`update-alternatives.bbclass <ref-classes-update-alternatives>`"
85 section.
86
87 ALTERNATIVE_PRIORITY
88 Used by the alternatives system to create default priorities for
89 duplicated commands. You can use the variable to create a single
90 default regardless of the command name or package, a default for
91 specific duplicated commands regardless of the package, or a default
92 for specific commands tied to particular packages. Here are the
93 available syntax forms:
94 ::
95
96 ALTERNATIVE_PRIORITY = "priority"
97 ALTERNATIVE_PRIORITY[name] = "priority"
98 ALTERNATIVE_PRIORITY_pkg[name] = "priority"
99
100 For more information on the alternatives system, see the
101 ":ref:`update-alternatives.bbclass <ref-classes-update-alternatives>`"
102 section.
103
104 ALTERNATIVE_TARGET
105 Used by the alternatives system to create default link locations for
106 duplicated commands. You can use the variable to create a single
107 default location for all duplicated commands regardless of the
108 command name or package, a default for specific duplicated commands
109 regardless of the package, or a default for specific commands tied to
110 particular packages. Here are the available syntax forms:
111 ::
112
113 ALTERNATIVE_TARGET = "target"
114 ALTERNATIVE_TARGET[name] = "target"
115 ALTERNATIVE_TARGET_pkg[name] = "target"
116
117 .. note::
118
119 If ``ALTERNATIVE_TARGET`` is not defined, it inherits the value
120 from the :term:`ALTERNATIVE_LINK_NAME` variable.
121
122 If ``ALTERNATIVE_LINK_NAME`` and ``ALTERNATIVE_TARGET`` are the
123 same, the target for ``ALTERNATIVE_TARGET`` has "``.{BPN}``"
124 appended to it.
125
126 Finally, if the file referenced has not been renamed, the
127 alternatives system will rename it to avoid the need to rename
128 alternative files in the :ref:`ref-tasks-install`
129 task while retaining support for the command if necessary.
130
131 For more information on the alternatives system, see the
132 ":ref:`update-alternatives.bbclass <ref-classes-update-alternatives>`"
133 section.
134
135 APPEND
136 An override list of append strings for each target specified with
137 :term:`LABELS`.
138
139 See the :ref:`grub-efi <ref-classes-grub-efi>` class for more
140 information on how this variable is used.
141
142 AR
143 The minimal command and arguments used to run ``ar``.
144
145 ARCHIVER_MODE
146 When used with the :ref:`archiver <ref-classes-archiver>` class,
147 determines the type of information used to create a released archive.
148 You can use this variable to create archives of patched source,
149 original source, configured source, and so forth by employing the
150 following variable flags (varflags):
151 ::
152
153 ARCHIVER_MODE[src] = "original" # Uses original (unpacked) source files.
154 ARCHIVER_MODE[src] = "patched" # Uses patched source files. This is the default.
155 ARCHIVER_MODE[src] = "configured" # Uses configured source files.
156 ARCHIVER_MODE[diff] = "1" # Uses patches between do_unpack and do_patch.
157 ARCHIVER_MODE[diff-exclude] ?= "file file ..." # Lists files and directories to exclude from diff.
158 ARCHIVER_MODE[dumpdata] = "1" # Uses environment data.
159 ARCHIVER_MODE[recipe] = "1" # Uses recipe and include files.
160 ARCHIVER_MODE[srpm] = "1" # Uses RPM package files.
161
162 For information on how the variable works, see the
163 ``meta/classes/archiver.bbclass`` file in the :term:`Source Directory`.
164
165 AS
166 Minimal command and arguments needed to run the assembler.
167
168 ASSUME_PROVIDED
169 Lists recipe names (:term:`PN` values) BitBake does not
170 attempt to build. Instead, BitBake assumes these recipes have already
171 been built.
172
173 In OpenEmbedded-Core, ``ASSUME_PROVIDED`` mostly specifies native
174 tools that should not be built. An example is ``git-native``, which
175 when specified, allows for the Git binary from the host to be used
176 rather than building ``git-native``.
177
178 ASSUME_SHLIBS
179 Provides additional ``shlibs`` provider mapping information, which
180 adds to or overwrites the information provided automatically by the
181 system. Separate multiple entries using spaces.
182
183 As an example, use the following form to add an ``shlib`` provider of
184 shlibname in packagename with the optional version:
185 ::
186
187 shlibname:packagename[_version]
188
189 Here is an example that adds a shared library named ``libEGL.so.1``
190 as being provided by the ``libegl-implementation`` package:
191 ::
192
193 ASSUME_SHLIBS = "libEGL.so.1:libegl-implementation"
194
195 AUTHOR
196 The email address used to contact the original author or authors in
197 order to send patches and forward bugs.
198
199 AUTO_LIBNAME_PKGS
200 When the :ref:`debian <ref-classes-debian>` class is inherited,
201 which is the default behavior, ``AUTO_LIBNAME_PKGS`` specifies which
202 packages should be checked for libraries and renamed according to
203 Debian library package naming.
204
205 The default value is "${PACKAGES}", which causes the debian class to
206 act on all packages that are explicitly generated by the recipe.
207
208 AUTO_SYSLINUXMENU
209 Enables creating an automatic menu for the syslinux bootloader. You
210 must set this variable in your recipe. The
211 :ref:`syslinux <ref-classes-syslinux>` class checks this variable.
212
213 AUTOREV
214 When ``SRCREV`` is set to the value of this variable, it specifies to
215 use the latest source revision in the repository. Here is an example:
216 ::
217
218 SRCREV = "${AUTOREV}"
219
220 If you use the previous statement to retrieve the latest version of
221 software, you need to be sure :term:`PV` contains
222 ``${``\ :term:`SRCPV`\ ``}``. For example, suppose you
223 have a kernel recipe that inherits the
224 :ref:`kernel <ref-classes-kernel>` class and you use the previous
225 statement. In this example, ``${SRCPV}`` does not automatically get
226 into ``PV``. Consequently, you need to change ``PV`` in your recipe
227 so that it does contain ``${SRCPV}``.
228
229 For more information see the
230 ":ref:`dev-manual/dev-manual-common-tasks:automatically incrementing a package version number`"
231 section in the Yocto Project Development Tasks Manual.
232
233 AVAILABLE_LICENSES
234 List of licenses found in the directories specified by
235 :term:`COMMON_LICENSE_DIR` and
236 :term:`LICENSE_PATH`.
237
238 .. note::
239
240 It is assumed that all changes to
241 COMMON_LICENSE_DIR
242 and
243 LICENSE_PATH
244 have been done before
245 AVAILABLE_LICENSES
246 is defined (in
247 license.bbclass
248 ).
249
250 AVAILTUNES
251 The list of defined CPU and Application Binary Interface (ABI)
252 tunings (i.e. "tunes") available for use by the OpenEmbedded build
253 system.
254
255 The list simply presents the tunes that are available. Not all tunes
256 may be compatible with a particular machine configuration, or with
257 each other in a
258 :ref:`Multilib <dev-manual/dev-manual-common-tasks:combining multiple versions of library files into one image>`
259 configuration.
260
261 To add a tune to the list, be sure to append it with spaces using the
262 "+=" BitBake operator. Do not simply replace the list by using the
263 "=" operator. See the
264 ":ref:`Basic Syntax <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:basic syntax>`" section in the BitBake
265 User Manual for more information.
266
267 B
268 The directory within the :term:`Build Directory` in
269 which the OpenEmbedded build system places generated objects during a
270 recipe's build process. By default, this directory is the same as the
271 :term:`S` directory, which is defined as:
272 ::
273
274 S = "${WORKDIR}/${BP}"
275
276 You can separate the (``S``) directory and the directory pointed to
277 by the ``B`` variable. Most Autotools-based recipes support
278 separating these directories. The build system defaults to using
279 separate directories for ``gcc`` and some kernel recipes.
280
281 BAD_RECOMMENDATIONS
282 Lists "recommended-only" packages to not install. Recommended-only
283 packages are packages installed only through the
284 :term:`RRECOMMENDS` variable. You can prevent any
285 of these "recommended" packages from being installed by listing them
286 with the ``BAD_RECOMMENDATIONS`` variable:
287 ::
288
289 BAD_RECOMMENDATIONS = "package_name package_name package_name ..."
290
291 You can set this variable globally in your ``local.conf`` file or you
292 can attach it to a specific image recipe by using the recipe name
293 override:
294 ::
295
296 BAD_RECOMMENDATIONS_pn-target_image = "package_name"
297
298 It is important to realize that if you choose to not install packages
299 using this variable and some other packages are dependent on them
300 (i.e. listed in a recipe's :term:`RDEPENDS`
301 variable), the OpenEmbedded build system ignores your request and
302 will install the packages to avoid dependency errors.
303
304 Support for this variable exists only when using the IPK and RPM
305 packaging backend. Support does not exist for DEB.
306
307 See the :term:`NO_RECOMMENDATIONS` and the
308 :term:`PACKAGE_EXCLUDE` variables for related
309 information.
310
311 BASE_LIB
312 The library directory name for the CPU or Application Binary
313 Interface (ABI) tune. The ``BASE_LIB`` applies only in the Multilib
314 context. See the ":ref:`dev-manual/dev-manual-common-tasks:combining multiple versions of library files into one image`"
315 section in the Yocto Project Development Tasks Manual for information
316 on Multilib.
317
318 The ``BASE_LIB`` variable is defined in the machine include files in
319 the :term:`Source Directory`. If Multilib is not
320 being used, the value defaults to "lib".
321
322 BASE_WORKDIR
323 Points to the base of the work directory for all recipes. The default
324 value is "${TMPDIR}/work".
325
326 BB_ALLOWED_NETWORKS
327 Specifies a space-delimited list of hosts that the fetcher is allowed
328 to use to obtain the required source code. Following are
329 considerations surrounding this variable:
330
331 - This host list is only used if ``BB_NO_NETWORK`` is either not set
332 or set to "0".
333
334 - Limited support for wildcard matching against the beginning of
335 host names exists. For example, the following setting matches
336 ``git.gnu.org``, ``ftp.gnu.org``, and ``foo.git.gnu.org``.
337 ::
338
339 BB_ALLOWED_NETWORKS = "*.gnu.org"
340
341 .. note::
342
343 The use of the "``*``" character only works at the beginning of
344 a host name and it must be isolated from the remainder of the
345 host name. You cannot use the wildcard character in any other
346 location of the name or combined with the front part of the
347 name.
348
349 For example, ``*.foo.bar`` is supported, while ``*aa.foo.bar``
350 is not.
351
352 - Mirrors not in the host list are skipped and logged in debug.
353
354 - Attempts to access networks not in the host list cause a failure.
355
356 Using ``BB_ALLOWED_NETWORKS`` in conjunction with
357 :term:`PREMIRRORS` is very useful. Adding the host
358 you want to use to ``PREMIRRORS`` results in the source code being
359 fetched from an allowed location and avoids raising an error when a
360 host that is not allowed is in a :term:`SRC_URI`
361 statement. This is because the fetcher does not attempt to use the
362 host listed in ``SRC_URI`` after a successful fetch from the
363 ``PREMIRRORS`` occurs.
364
365 BB_DANGLINGAPPENDS_WARNONLY
366 Defines how BitBake handles situations where an append file
367 (``.bbappend``) has no corresponding recipe file (``.bb``). This
368 condition often occurs when layers get out of sync (e.g. ``oe-core``
369 bumps a recipe version and the old recipe no longer exists and the
370 other layer has not been updated to the new version of the recipe
371 yet).
372
373 The default fatal behavior is safest because it is the sane reaction
374 given something is out of sync. It is important to realize when your
375 changes are no longer being applied.
376
377 You can change the default behavior by setting this variable to "1",
378 "yes", or "true" in your ``local.conf`` file, which is located in the
379 :term:`Build Directory`: Here is an example:
380 ::
381
382 BB_DANGLINGAPPENDS_WARNONLY = "1"
383
384 BB_DISKMON_DIRS
385 Monitors disk space and available inodes during the build and allows
386 you to control the build based on these parameters.
387
388 Disk space monitoring is disabled by default. To enable monitoring,
389 add the ``BB_DISKMON_DIRS`` variable to your ``conf/local.conf`` file
390 found in the :term:`Build Directory`. Use the
391 following form:
392 ::
393
394 BB_DISKMON_DIRS = "action,dir,threshold [...]"
395
396 where:
397
398 action is:
399 ABORT: Immediately abort the build when
400 a threshold is broken.
401 STOPTASKS: Stop the build after the currently
402 executing tasks have finished when
403 a threshold is broken.
404 WARN: Issue a warning but continue the
405 build when a threshold is broken.
406 Subsequent warnings are issued as
407 defined by the BB_DISKMON_WARNINTERVAL
408 variable, which must be defined in
409 the conf/local.conf file.
410
411 dir is:
412 Any directory you choose. You can specify one or
413 more directories to monitor by separating the
414 groupings with a space. If two directories are
415 on the same device, only the first directory
416 is monitored.
417
418 threshold is:
419 Either the minimum available disk space,
420 the minimum number of free inodes, or
421 both. You must specify at least one. To
422 omit one or the other, simply omit the value.
423 Specify the threshold using G, M, K for Gbytes,
424 Mbytes, and Kbytes, respectively. If you do
425 not specify G, M, or K, Kbytes is assumed by
426 default. Do not use GB, MB, or KB.
427
428 Here are some examples:
429 ::
430
431 BB_DISKMON_DIRS = "ABORT,${TMPDIR},1G,100K WARN,${SSTATE_DIR},1G,100K"
432 BB_DISKMON_DIRS = "STOPTASKS,${TMPDIR},1G"
433 BB_DISKMON_DIRS = "ABORT,${TMPDIR},,100K"
434
435 The first example works only if you also provide the
436 :term:`BB_DISKMON_WARNINTERVAL`
437 variable in the ``conf/local.conf``. This example causes the build
438 system to immediately abort when either the disk space in
439 ``${TMPDIR}`` drops below 1 Gbyte or the available free inodes drops
440 below 100 Kbytes. Because two directories are provided with the
441 variable, the build system also issue a warning when the disk space
442 in the ``${SSTATE_DIR}`` directory drops below 1 Gbyte or the number
443 of free inodes drops below 100 Kbytes. Subsequent warnings are issued
444 during intervals as defined by the ``BB_DISKMON_WARNINTERVAL``
445 variable.
446
447 The second example stops the build after all currently executing
448 tasks complete when the minimum disk space in the ``${TMPDIR}``
449 directory drops below 1 Gbyte. No disk monitoring occurs for the free
450 inodes in this case.
451
452 The final example immediately aborts the build when the number of
453 free inodes in the ``${TMPDIR}`` directory drops below 100 Kbytes. No
454 disk space monitoring for the directory itself occurs in this case.
455
456 BB_DISKMON_WARNINTERVAL
457 Defines the disk space and free inode warning intervals. To set these
458 intervals, define the variable in your ``conf/local.conf`` file in
459 the :term:`Build Directory`.
460
461 If you are going to use the ``BB_DISKMON_WARNINTERVAL`` variable, you
462 must also use the :term:`BB_DISKMON_DIRS`
463 variable and define its action as "WARN". During the build,
464 subsequent warnings are issued each time disk space or number of free
465 inodes further reduces by the respective interval.
466
467 If you do not provide a ``BB_DISKMON_WARNINTERVAL`` variable and you
468 do use ``BB_DISKMON_DIRS`` with the "WARN" action, the disk
469 monitoring interval defaults to the following:
470 ::
471
472 BB_DISKMON_WARNINTERVAL = "50M,5K"
473
474 When specifying the variable in your configuration file, use the
475 following form:
476 ::
477
478 BB_DISKMON_WARNINTERVAL = "disk_space_interval,disk_inode_interval"
479
480 where:
481
482 disk_space_interval is:
483 An interval of memory expressed in either
484 G, M, or K for Gbytes, Mbytes, or Kbytes,
485 respectively. You cannot use GB, MB, or KB.
486
487 disk_inode_interval is:
488 An interval of free inodes expressed in either
489 G, M, or K for Gbytes, Mbytes, or Kbytes,
490 respectively. You cannot use GB, MB, or KB.
491
492 Here is an example:
493 ::
494
495 BB_DISKMON_DIRS = "WARN,${SSTATE_DIR},1G,100K"
496 BB_DISKMON_WARNINTERVAL = "50M,5K"
497
498 These variables cause the
499 OpenEmbedded build system to issue subsequent warnings each time the
500 available disk space further reduces by 50 Mbytes or the number of
501 free inodes further reduces by 5 Kbytes in the ``${SSTATE_DIR}``
502 directory. Subsequent warnings based on the interval occur each time
503 a respective interval is reached beyond the initial warning (i.e. 1
504 Gbytes and 100 Kbytes).
505
506 BB_GENERATE_MIRROR_TARBALLS
507 Causes tarballs of the source control repositories (e.g. Git
508 repositories), including metadata, to be placed in the
509 :term:`DL_DIR` directory.
510
511 For performance reasons, creating and placing tarballs of these
512 repositories is not the default action by the OpenEmbedded build
513 system.
514 ::
515
516 BB_GENERATE_MIRROR_TARBALLS = "1"
517
518 Set this variable in your
519 ``local.conf`` file in the :term:`Build Directory`.
520
521 Once you have the tarballs containing your source files, you can
522 clean up your ``DL_DIR`` directory by deleting any Git or other
523 source control work directories.
524
525 BB_NUMBER_THREADS
526 The maximum number of tasks BitBake should run in parallel at any one
527 time. The OpenEmbedded build system automatically configures this
528 variable to be equal to the number of cores on the build system. For
529 example, a system with a dual core processor that also uses
530 hyper-threading causes the ``BB_NUMBER_THREADS`` variable to default
531 to "4".
532
533 For single socket systems (i.e. one CPU), you should not have to
534 override this variable to gain optimal parallelism during builds.
535 However, if you have very large systems that employ multiple physical
536 CPUs, you might want to make sure the ``BB_NUMBER_THREADS`` variable
537 is not set higher than "20".
538
539 For more information on speeding up builds, see the
540 ":ref:`dev-manual/dev-manual-common-tasks:speeding up a build`"
541 section in the Yocto Project Development Tasks Manual.
542
543 BB_SERVER_TIMEOUT
544 Specifies the time (in seconds) after which to unload the BitBake
545 server due to inactivity. Set ``BB_SERVER_TIMEOUT`` to determine how
546 long the BitBake server stays resident between invocations.
547
548 For example, the following statement in your ``local.conf`` file
549 instructs the server to be unloaded after 20 seconds of inactivity:
550 ::
551
552 BB_SERVER_TIMEOUT = "20"
553
554 If you want the server to never be unloaded,
555 set ``BB_SERVER_TIMEOUT`` to "-1".
556
557 BBCLASSEXTEND
558 Allows you to extend a recipe so that it builds variants of the
559 software. Common variants for recipes exist such as "natives" like
560 ``quilt-native``, which is a copy of Quilt built to run on the build
561 system; "crosses" such as ``gcc-cross``, which is a compiler built to
562 run on the build machine but produces binaries that run on the target
563 :term:`MACHINE`; "nativesdk", which targets the SDK
564 machine instead of ``MACHINE``; and "mulitlibs" in the form
565 "``multilib:``\ multilib_name".
566
567 To build a different variant of the recipe with a minimal amount of
568 code, it usually is as simple as adding the following to your recipe:
569 ::
570
571 BBCLASSEXTEND =+ "native nativesdk"
572 BBCLASSEXTEND =+ "multilib:multilib_name"
573
574 .. note::
575
576 Internally, the ``BBCLASSEXTEND`` mechanism generates recipe
577 variants by rewriting variable values and applying overrides such
578 as ``_class-native``. For example, to generate a native version of
579 a recipe, a :term:`DEPENDS` on "foo" is rewritten
580 to a ``DEPENDS`` on "foo-native".
581
582 Even when using ``BBCLASSEXTEND``, the recipe is only parsed once.
583 Parsing once adds some limitations. For example, it is not
584 possible to include a different file depending on the variant,
585 since ``include`` statements are processed when the recipe is
586 parsed.
587
588 BBFILE_COLLECTIONS
589 Lists the names of configured layers. These names are used to find
590 the other ``BBFILE_*`` variables. Typically, each layer will append
591 its name to this variable in its ``conf/layer.conf`` file.
592
593 BBFILE_PATTERN
594 Variable that expands to match files from
595 :term:`BBFILES` in a particular layer. This variable
596 is used in the ``conf/layer.conf`` file and must be suffixed with the
597 name of the specific layer (e.g. ``BBFILE_PATTERN_emenlow``).
598
599 BBFILE_PRIORITY
600 Assigns the priority for recipe files in each layer.
601
602 This variable is useful in situations where the same recipe appears
603 in more than one layer. Setting this variable allows you to
604 prioritize a layer against other layers that contain the same recipe
605 - effectively letting you control the precedence for the multiple
606 layers. The precedence established through this variable stands
607 regardless of a recipe's version (:term:`PV` variable). For
608 example, a layer that has a recipe with a higher ``PV`` value but for
609 which the ``BBFILE_PRIORITY`` is set to have a lower precedence still
610 has a lower precedence.
611
612 A larger value for the ``BBFILE_PRIORITY`` variable results in a
613 higher precedence. For example, the value 6 has a higher precedence
614 than the value 5. If not specified, the ``BBFILE_PRIORITY`` variable
615 is set based on layer dependencies (see the ``LAYERDEPENDS`` variable
616 for more information. The default priority, if unspecified for a
617 layer with no dependencies, is the lowest defined priority + 1 (or 1
618 if no priorities are defined).
619
620 .. tip::
621
622 You can use the command
623 bitbake-layers show-layers
624 to list all configured layers along with their priorities.
625
626 BBFILES
627 A space-separated list of recipe files BitBake uses to build
628 software.
629
630 When specifying recipe files, you can pattern match using Python's
631 `glob <https://docs.python.org/3/library/glob.html>`_ syntax.
632 For details on the syntax, see the documentation by following the
633 previous link.
634
635 BBFILES_DYNAMIC
636 Activates content when identified layers are present. You identify
637 the layers by the collections that the layers define.
638
639 Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files
640 whose corresponding ``.bb`` file is in a layer that attempts to
641 modify other layers through ``.bbappend`` but does not want to
642 introduce a hard dependency on those other layers.
643
644 Use the following form for ``BBFILES_DYNAMIC``:
645 collection_name:filename_pattern The following example identifies two
646 collection names and two filename patterns:
647 ::
648
649 BBFILES_DYNAMIC += " \
650 clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \
651 core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \
652 "
653
654 This next example shows an error message that occurs because invalid
655 entries are found, which cause parsing to abort:
656 ::
657
658 ERROR: BBFILES_DYNAMIC entries must be of the form <collection name>:<filename pattern>, not:
659 /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend
660 /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend
661
662 BBINCLUDELOGS
663 Variable that controls how BitBake displays logs on build failure.
664
665 BBINCLUDELOGS_LINES
666 If :term:`BBINCLUDELOGS` is set, specifies the
667 maximum number of lines from the task log file to print when
668 reporting a failed task. If you do not set ``BBINCLUDELOGS_LINES``,
669 the entire log is printed.
670
671 BBLAYERS
672 Lists the layers to enable during the build. This variable is defined
673 in the ``bblayers.conf`` configuration file in the :term:`Build Directory`.
674 Here is an example:
675 ::
676
677 BBLAYERS = " \
678 /home/scottrif/poky/meta \ /home/scottrif/poky/meta-poky \
679 /home/scottrif/poky/meta-yocto-bsp \
680 /home/scottrif/poky/meta-mykernel \
681 "
682
683 This example enables four layers, one of which is a custom,
684 user-defined layer named ``meta-mykernel``.
685
686 BBMASK
687 Prevents BitBake from processing recipes and recipe append files.
688
689 You can use the ``BBMASK`` variable to "hide" these ``.bb`` and
690 ``.bbappend`` files. BitBake ignores any recipe or recipe append
691 files that match any of the expressions. It is as if BitBake does not
692 see them at all. Consequently, matching files are not parsed or
693 otherwise used by BitBake.
694
695 The values you provide are passed to Python's regular expression
696 compiler. Consequently, the syntax follows Python's Regular
697 Expression (re) syntax. The expressions are compared against the full
698 paths to the files. For complete syntax information, see Python's
699 documentation at https://docs.python.org/3/library/re.html#regular-expression-syntax.
700
701 The following example uses a complete regular expression to tell
702 BitBake to ignore all recipe and recipe append files in the
703 ``meta-ti/recipes-misc/`` directory:
704 ::
705
706 BBMASK = "meta-ti/recipes-misc/"
707
708 If you want to mask out multiple directories or recipes, you can
709 specify multiple regular expression fragments. This next example
710 masks out multiple directories and individual recipes: ::
711
712 BBMASK += "/meta-ti/recipes-misc/ meta-ti/recipes-ti/packagegroup/"
713 BBMASK += "/meta-oe/recipes-support/"
714 BBMASK += "/meta-foo/.*/openldap"
715 BBMASK += "opencv.*\.bbappend"
716 BBMASK += "lzma"
717
718 .. note::
719
720 When specifying a directory name, use the trailing slash character
721 to ensure you match just that directory name.
722
723 BBMULTICONFIG
724 Specifies each additional separate configuration when you are
725 building targets with multiple configurations. Use this variable in
726 your ``conf/local.conf`` configuration file. Specify a
727 multiconfigname for each configuration file you are using. For
728 example, the following line specifies three configuration files:
729 ::
730
731 BBMULTICONFIG = "configA configB configC"
732
733 Each configuration file you
734 use must reside in the :term:`Build Directory`
735 ``conf/multiconfig`` directory (e.g.
736 build_directory\ ``/conf/multiconfig/configA.conf``).
737
738 For information on how to use ``BBMULTICONFIG`` in an environment
739 that supports building targets with multiple configurations, see the
740 ":ref:`dev-building-images-for-multiple-targets-using-multiple-configurations`"
741 section in the Yocto Project Development Tasks Manual.
742
743 BBPATH
744 Used by BitBake to locate ``.bbclass`` and configuration files. This
745 variable is analogous to the ``PATH`` variable.
746
747 .. note::
748
749 If you run BitBake from a directory outside of the
750 Build Directory
751 , you must be sure to set
752 BBPATH
753 to point to the Build Directory. Set the variable as you would any
754 environment variable and then run BitBake:
755 ::
756
757 $ BBPATH = "build_directory"
758 $ export BBPATH
759 $ bitbake target
760
761
762 BBSERVER
763 If defined in the BitBake environment, ``BBSERVER`` points to the
764 BitBake remote server.
765
766 Use the following format to export the variable to the BitBake
767 environment:
768 ::
769
770 export BBSERVER=localhost:$port
771
772 By default, ``BBSERVER`` also appears in
773 :term:`bitbake:BB_HASHBASE_WHITELIST`.
774 Consequently, ``BBSERVER`` is excluded from checksum and dependency
775 data.
776
777 BINCONFIG
778 When inheriting the
779 :ref:`binconfig-disabled <ref-classes-binconfig-disabled>` class,
780 this variable specifies binary configuration scripts to disable in
781 favor of using ``pkg-config`` to query the information. The
782 ``binconfig-disabled`` class will modify the specified scripts to
783 return an error so that calls to them can be easily found and
784 replaced.
785
786 To add multiple scripts, separate them by spaces. Here is an example
787 from the ``libpng`` recipe:
788 ::
789
790 BINCONFIG = "${bindir}/libpng-config ${bindir}/libpng16-config"
791
792 BINCONFIG_GLOB
793 When inheriting the :ref:`binconfig <ref-classes-binconfig>` class,
794 this variable specifies a wildcard for configuration scripts that
795 need editing. The scripts are edited to correct any paths that have
796 been set up during compilation so that they are correct for use when
797 installed into the sysroot and called by the build processes of other
798 recipes.
799
800 .. note::
801
802 The
803 BINCONFIG_GLOB
804 variable uses
805 shell globbing
806 , which is recognition and expansion of wildcards during pattern
807 matching. Shell globbing is very similar to
808 fnmatch
809 and
810 glob
811 .
812
813 For more information on how this variable works, see
814 ``meta/classes/binconfig.bbclass`` in the :term:`Source Directory`.
815 You can also find general
816 information on the class in the
817 ":ref:`binconfig.bbclass <ref-classes-binconfig>`" section.
818
819 BP
820 The base recipe name and version but without any special recipe name
821 suffix (i.e. ``-native``, ``lib64-``, and so forth). ``BP`` is
822 comprised of the following:
823 ::
824
825 ${BPN}-${PV}
826
827 BPN
828 This variable is a version of the :term:`PN` variable with
829 common prefixes and suffixes removed, such as ``nativesdk-``,
830 ``-cross``, ``-native``, and multilib's ``lib64-`` and ``lib32-``.
831 The exact lists of prefixes and suffixes removed are specified by the
832 :term:`MLPREFIX` and
833 :term:`SPECIAL_PKGSUFFIX` variables,
834 respectively.
835
836 BUGTRACKER
837 Specifies a URL for an upstream bug tracking website for a recipe.
838 The OpenEmbedded build system does not use this variable. Rather, the
839 variable is a useful pointer in case a bug in the software being
840 built needs to be manually reported.
841
842 BUILD_ARCH
843 Specifies the architecture of the build host (e.g. ``i686``). The
844 OpenEmbedded build system sets the value of ``BUILD_ARCH`` from the
845 machine name reported by the ``uname`` command.
846
847 BUILD_AS_ARCH
848 Specifies the architecture-specific assembler flags for the build
849 host. By default, the value of ``BUILD_AS_ARCH`` is empty.
850
851 BUILD_CC_ARCH
852 Specifies the architecture-specific C compiler flags for the build
853 host. By default, the value of ``BUILD_CC_ARCH`` is empty.
854
855 BUILD_CCLD
856 Specifies the linker command to be used for the build host when the C
857 compiler is being used as the linker. By default, ``BUILD_CCLD``
858 points to GCC and passes as arguments the value of
859 :term:`BUILD_CC_ARCH`, assuming
860 ``BUILD_CC_ARCH`` is set.
861
862 BUILD_CFLAGS
863 Specifies the flags to pass to the C compiler when building for the
864 build host. When building in the ``-native`` context,
865 :term:`CFLAGS` is set to the value of this variable by
866 default.
867
868 BUILD_CPPFLAGS
869 Specifies the flags to pass to the C preprocessor (i.e. to both the C
870 and the C++ compilers) when building for the build host. When
871 building in the ``-native`` context, :term:`CPPFLAGS`
872 is set to the value of this variable by default.
873
874 BUILD_CXXFLAGS
875 Specifies the flags to pass to the C++ compiler when building for the
876 build host. When building in the ``-native`` context,
877 :term:`CXXFLAGS` is set to the value of this variable
878 by default.
879
880 BUILD_FC
881 Specifies the Fortran compiler command for the build host. By
882 default, ``BUILD_FC`` points to Gfortran and passes as arguments the
883 value of :term:`BUILD_CC_ARCH`, assuming
884 ``BUILD_CC_ARCH`` is set.
885
886 BUILD_LD
887 Specifies the linker command for the build host. By default,
888 ``BUILD_LD`` points to the GNU linker (ld) and passes as arguments
889 the value of :term:`BUILD_LD_ARCH`, assuming
890 ``BUILD_LD_ARCH`` is set.
891
892 BUILD_LD_ARCH
893 Specifies architecture-specific linker flags for the build host. By
894 default, the value of ``BUILD_LD_ARCH`` is empty.
895
896 BUILD_LDFLAGS
897 Specifies the flags to pass to the linker when building for the build
898 host. When building in the ``-native`` context,
899 :term:`LDFLAGS` is set to the value of this variable
900 by default.
901
902 BUILD_OPTIMIZATION
903 Specifies the optimization flags passed to the C compiler when
904 building for the build host or the SDK. The flags are passed through
905 the :term:`BUILD_CFLAGS` and
906 :term:`BUILDSDK_CFLAGS` default values.
907
908 The default value of the ``BUILD_OPTIMIZATION`` variable is "-O2
909 -pipe".
910
911 BUILD_OS
912 Specifies the operating system in use on the build host (e.g.
913 "linux"). The OpenEmbedded build system sets the value of
914 ``BUILD_OS`` from the OS reported by the ``uname`` command - the
915 first word, converted to lower-case characters.
916
917 BUILD_PREFIX
918 The toolchain binary prefix used for native recipes. The OpenEmbedded
919 build system uses the ``BUILD_PREFIX`` value to set the
920 :term:`TARGET_PREFIX` when building for
921 ``native`` recipes.
922
923 BUILD_STRIP
924 Specifies the command to be used to strip debugging symbols from
925 binaries produced for the build host. By default, ``BUILD_STRIP``
926 points to
927 ``${``\ :term:`BUILD_PREFIX`\ ``}strip``.
928
929 BUILD_SYS
930 Specifies the system, including the architecture and the operating
931 system, to use when building for the build host (i.e. when building
932 ``native`` recipes).
933
934 The OpenEmbedded build system automatically sets this variable based
935 on :term:`BUILD_ARCH`,
936 :term:`BUILD_VENDOR`, and
937 :term:`BUILD_OS`. You do not need to set the
938 ``BUILD_SYS`` variable yourself.
939
940 BUILD_VENDOR
941 Specifies the vendor name to use when building for the build host.
942 The default value is an empty string ("").
943
944 BUILDDIR
945 Points to the location of the :term:`Build Directory`.
946 You can define this directory indirectly through the
947 ````` <#structure-core-script>`__ script by passing in a Build
948 Directory path when you run the script. If you run the script and do
949 not provide a Build Directory path, the ``BUILDDIR`` defaults to
950 ``build`` in the current directory.
951
952 BUILDHISTORY_COMMIT
953 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
954 class, this variable specifies whether or not to commit the build
955 history output in a local Git repository. If set to "1", this local
956 repository will be maintained automatically by the ``buildhistory``
957 class and a commit will be created on every build for changes to each
958 top-level subdirectory of the build history output (images, packages,
959 and sdk). If you want to track changes to build history over time,
960 you should set this value to "1".
961
962 By default, the ``buildhistory`` class does not commit the build
963 history output in a local Git repository:
964 ::
965
966 BUILDHISTORY_COMMIT ?= "0"
967
968 BUILDHISTORY_COMMIT_AUTHOR
969 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
970 class, this variable specifies the author to use for each Git commit.
971 In order for the ``BUILDHISTORY_COMMIT_AUTHOR`` variable to work, the
972 :term:`BUILDHISTORY_COMMIT` variable must
973 be set to "1".
974
975 Git requires that the value you provide for the
976 ``BUILDHISTORY_COMMIT_AUTHOR`` variable takes the form of "name
977 email@host". Providing an email address or host that is not valid
978 does not produce an error.
979
980 By default, the ``buildhistory`` class sets the variable as follows:
981 ::
982
983 BUILDHISTORY_COMMIT_AUTHOR ?= "buildhistory <buildhistory@${DISTRO}>"
984
985 BUILDHISTORY_DIR
986 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
987 class, this variable specifies the directory in which build history
988 information is kept. For more information on how the variable works,
989 see the ``buildhistory.class``.
990
991 By default, the ``buildhistory`` class sets the directory as follows:
992 ::
993
994 BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
995
996 BUILDHISTORY_FEATURES
997 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
998 class, this variable specifies the build history features to be
999 enabled. For more information on how build history works, see the
1000 ":ref:`dev-manual/dev-manual-common-tasks:maintaining build output quality`"
1001 section in the Yocto Project Development Tasks Manual.
1002
1003 You can specify these features in the form of a space-separated list:
1004
1005 - *image:* Analysis of the contents of images, which includes the
1006 list of installed packages among other things.
1007
1008 - *package:* Analysis of the contents of individual packages.
1009
1010 - *sdk:* Analysis of the contents of the software development kit
1011 (SDK).
1012
1013 - *task:* Save output file signatures for
1014 :ref:`shared state <overview-manual/overview-manual-concepts:shared state cache>`
1015 (sstate) tasks.
1016 This saves one file per task and lists the SHA-256 checksums for
1017 each file staged (i.e. the output of the task).
1018
1019 By default, the ``buildhistory`` class enables the following
1020 features:
1021 ::
1022
1023 BUILDHISTORY_FEATURES ?= "image package sdk"
1024
1025 BUILDHISTORY_IMAGE_FILES
1026 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
1027 class, this variable specifies a list of paths to files copied from
1028 the image contents into the build history directory under an
1029 "image-files" directory in the directory for the image, so that you
1030 can track the contents of each file. The default is to copy
1031 ``/etc/passwd`` and ``/etc/group``, which allows you to monitor for
1032 changes in user and group entries. You can modify the list to include
1033 any file. Specifying an invalid path does not produce an error.
1034 Consequently, you can include files that might not always be present.
1035
1036 By default, the ``buildhistory`` class provides paths to the
1037 following files:
1038 ::
1039
1040 BUILDHISTORY_IMAGE_FILES ?= "/etc/passwd /etc/group"
1041
1042 BUILDHISTORY_PUSH_REPO
1043 When inheriting the :ref:`buildhistory <ref-classes-buildhistory>`
1044 class, this variable optionally specifies a remote repository to
1045 which build history pushes Git changes. In order for
1046 ``BUILDHISTORY_PUSH_REPO`` to work,
1047 :term:`BUILDHISTORY_COMMIT` must be set to
1048 "1".
1049
1050 The repository should correspond to a remote address that specifies a
1051 repository as understood by Git, or alternatively to a remote name
1052 that you have set up manually using ``git remote`` within the local
1053 repository.
1054
1055 By default, the ``buildhistory`` class sets the variable as follows:
1056 ::
1057
1058 BUILDHISTORY_PUSH_REPO ?= ""
1059
1060 BUILDSDK_CFLAGS
1061 Specifies the flags to pass to the C compiler when building for the
1062 SDK. When building in the ``nativesdk-`` context,
1063 :term:`CFLAGS` is set to the value of this variable by
1064 default.
1065
1066 BUILDSDK_CPPFLAGS
1067 Specifies the flags to pass to the C pre-processor (i.e. to both the
1068 C and the C++ compilers) when building for the SDK. When building in
1069 the ``nativesdk-`` context, :term:`CPPFLAGS` is set
1070 to the value of this variable by default.
1071
1072 BUILDSDK_CXXFLAGS
1073 Specifies the flags to pass to the C++ compiler when building for the
1074 SDK. When building in the ``nativesdk-`` context,
1075 :term:`CXXFLAGS` is set to the value of this variable
1076 by default.
1077
1078 BUILDSDK_LDFLAGS
1079 Specifies the flags to pass to the linker when building for the SDK.
1080 When building in the ``nativesdk-`` context,
1081 :term:`LDFLAGS` is set to the value of this variable
1082 by default.
1083
1084 BUILDSTATS_BASE
1085 Points to the location of the directory that holds build statistics
1086 when you use and enable the
1087 :ref:`buildstats <ref-classes-buildstats>` class. The
1088 ``BUILDSTATS_BASE`` directory defaults to
1089 ``${``\ :term:`TMPDIR`\ ``}/buildstats/``.
1090
1091 BUSYBOX_SPLIT_SUID
1092 For the BusyBox recipe, specifies whether to split the output
1093 executable file into two parts: one for features that require
1094 ``setuid root``, and one for the remaining features (i.e. those that
1095 do not require ``setuid root``).
1096
1097 The ``BUSYBOX_SPLIT_SUID`` variable defaults to "1", which results in
1098 splitting the output executable file. Set the variable to "0" to get
1099 a single output executable file.
1100
1101 CACHE
1102 Specifies the directory BitBake uses to store a cache of the
1103 :term:`Metadata` so it does not need to be parsed every time
1104 BitBake is started.
1105
1106 CC
1107 The minimal command and arguments used to run the C compiler.
1108
1109 CFLAGS
1110 Specifies the flags to pass to the C compiler. This variable is
1111 exported to an environment variable and thus made visible to the
1112 software being built during the compilation step.
1113
1114 Default initialization for ``CFLAGS`` varies depending on what is
1115 being built:
1116
1117 - :term:`TARGET_CFLAGS` when building for the
1118 target
1119
1120 - :term:`BUILD_CFLAGS` when building for the
1121 build host (i.e. ``-native``)
1122
1123 - :term:`BUILDSDK_CFLAGS` when building for
1124 an SDK (i.e. ``nativesdk-``)
1125
1126 CLASSOVERRIDE
1127 An internal variable specifying the special class override that
1128 should currently apply (e.g. "class-target", "class-native", and so
1129 forth). The classes that use this variable (e.g.
1130 :ref:`native <ref-classes-native>`,
1131 :ref:`nativesdk <ref-classes-nativesdk>`, and so forth) set the
1132 variable to appropriate values.
1133
1134 .. note::
1135
1136 CLASSOVERRIDE
1137 gets its default "class-target" value from the
1138 bitbake.conf
1139 file.
1140
1141 As an example, the following override allows you to install extra
1142 files, but only when building for the target:
1143 ::
1144
1145 do_install_append_class-target() {
1146 install my-extra-file ${D}${sysconfdir}
1147 }
1148
1149 Here is an example where ``FOO`` is set to
1150 "native" when building for the build host, and to "other" when not
1151 building for the build host:
1152 ::
1153
1154 FOO_class-native = "native"
1155 FOO = "other"
1156
1157 The underlying mechanism behind ``CLASSOVERRIDE`` is simply
1158 that it is included in the default value of
1159 :term:`OVERRIDES`.
1160
1161 CLEANBROKEN
1162 If set to "1" within a recipe, ``CLEANBROKEN`` specifies that the
1163 ``make clean`` command does not work for the software being built.
1164 Consequently, the OpenEmbedded build system will not try to run
1165 ``make clean`` during the :ref:`ref-tasks-configure`
1166 task, which is the default behavior.
1167
1168 COMBINED_FEATURES
1169 Provides a list of hardware features that are enabled in both
1170 :term:`MACHINE_FEATURES` and
1171 :term:`DISTRO_FEATURES`. This select list of
1172 features contains features that make sense to be controlled both at
1173 the machine and distribution configuration level. For example, the
1174 "bluetooth" feature requires hardware support but should also be
1175 optional at the distribution level, in case the hardware supports
1176 Bluetooth but you do not ever intend to use it.
1177
1178 COMMON_LICENSE_DIR
1179 Points to ``meta/files/common-licenses`` in the
1180 :term:`Source Directory`, which is where generic license
1181 files reside.
1182
1183 COMPATIBLE_HOST
1184 A regular expression that resolves to one or more hosts (when the
1185 recipe is native) or one or more targets (when the recipe is
1186 non-native) with which a recipe is compatible. The regular expression
1187 is matched against :term:`HOST_SYS`. You can use the
1188 variable to stop recipes from being built for classes of systems with
1189 which the recipes are not compatible. Stopping these builds is
1190 particularly useful with kernels. The variable also helps to increase
1191 parsing speed since the build system skips parsing recipes not
1192 compatible with the current system.
1193
1194 COMPATIBLE_MACHINE
1195 A regular expression that resolves to one or more target machines
1196 with which a recipe is compatible. The regular expression is matched
1197 against :term:`MACHINEOVERRIDES`. You can use
1198 the variable to stop recipes from being built for machines with which
1199 the recipes are not compatible. Stopping these builds is particularly
1200 useful with kernels. The variable also helps to increase parsing
1201 speed since the build system skips parsing recipes not compatible
1202 with the current machine.
1203
1204 COMPLEMENTARY_GLOB
1205 Defines wildcards to match when installing a list of complementary
1206 packages for all the packages explicitly (or implicitly) installed in
1207 an image.
1208
1209 .. note::
1210
1211 The
1212 COMPLEMENTARY_GLOB
1213 variable uses Unix filename pattern matching (
1214 fnmatch
1215 ), which is similar to the Unix style pathname pattern expansion (
1216 glob
1217 ).
1218
1219 The resulting list of complementary packages is associated with an
1220 item that can be added to
1221 :term:`IMAGE_FEATURES`. An example usage of
1222 this is the "dev-pkgs" item that when added to ``IMAGE_FEATURES``
1223 will install -dev packages (containing headers and other development
1224 files) for every package in the image.
1225
1226 To add a new feature item pointing to a wildcard, use a variable flag
1227 to specify the feature item name and use the value to specify the
1228 wildcard. Here is an example:
1229 ::
1230
1231 COMPLEMENTARY_GLOB[dev-pkgs] = '*-dev'
1232
1233 COMPONENTS_DIR
1234 Stores sysroot components for each recipe. The OpenEmbedded build
1235 system uses ``COMPONENTS_DIR`` when constructing recipe-specific
1236 sysroots for other recipes.
1237
1238 The default is
1239 "``${``\ :term:`STAGING_DIR`\ ``}-components``."
1240 (i.e.
1241 "``${``\ :term:`TMPDIR`\ ``}/sysroots-components``").
1242
1243 CONF_VERSION
1244 Tracks the version of the local configuration file (i.e.
1245 ``local.conf``). The value for ``CONF_VERSION`` increments each time
1246 ``build/conf/`` compatibility changes.
1247
1248 CONFFILES
1249 Identifies editable or configurable files that are part of a package.
1250 If the Package Management System (PMS) is being used to update
1251 packages on the target system, it is possible that configuration
1252 files you have changed after the original installation and that you
1253 now want to remain unchanged are overwritten. In other words,
1254 editable files might exist in the package that you do not want reset
1255 as part of the package update process. You can use the ``CONFFILES``
1256 variable to list the files in the package that you wish to prevent
1257 the PMS from overwriting during this update process.
1258
1259 To use the ``CONFFILES`` variable, provide a package name override
1260 that identifies the resulting package. Then, provide a
1261 space-separated list of files. Here is an example:
1262 ::
1263
1264 CONFFILES_${PN} += "${sysconfdir}/file1 \
1265 ${sysconfdir}/file2 ${sysconfdir}/file3"
1266
1267 A relationship exists between the ``CONFFILES`` and ``FILES``
1268 variables. The files listed within ``CONFFILES`` must be a subset of
1269 the files listed within ``FILES``. Because the configuration files
1270 you provide with ``CONFFILES`` are simply being identified so that
1271 the PMS will not overwrite them, it makes sense that the files must
1272 already be included as part of the package through the ``FILES``
1273 variable.
1274
1275 .. note::
1276
1277 When specifying paths as part of the
1278 CONFFILES
1279 variable, it is good practice to use appropriate path variables.
1280 For example,
1281 ${sysconfdir}
1282 rather than
1283 /etc
1284 or
1285 ${bindir}
1286 rather than
1287 /usr/bin
1288 . You can find a list of these variables at the top of the
1289 meta/conf/bitbake.conf
1290 file in the
1291 Source Directory
1292 .
1293
1294 CONFIG_INITRAMFS_SOURCE
1295 Identifies the initial RAM filesystem (initramfs) source files. The
1296 OpenEmbedded build system receives and uses this kernel Kconfig
1297 variable as an environment variable. By default, the variable is set
1298 to null ("").
1299
1300 The ``CONFIG_INITRAMFS_SOURCE`` can be either a single cpio archive
1301 with a ``.cpio`` suffix or a space-separated list of directories and
1302 files for building the initramfs image. A cpio archive should contain
1303 a filesystem archive to be used as an initramfs image. Directories
1304 should contain a filesystem layout to be included in the initramfs
1305 image. Files should contain entries according to the format described
1306 by the ``usr/gen_init_cpio`` program in the kernel tree.
1307
1308 If you specify multiple directories and files, the initramfs image
1309 will be the aggregate of all of them.
1310
1311 For information on creating an initramfs, see the
1312 ":ref:`building-an-initramfs-image`" section
1313 in the Yocto Project Development Tasks Manual.
1314
1315 CONFIG_SITE
1316 A list of files that contains ``autoconf`` test results relevant to
1317 the current build. This variable is used by the Autotools utilities
1318 when running ``configure``.
1319
1320 CONFIGURE_FLAGS
1321 The minimal arguments for GNU configure.
1322
1323 CONFLICT_DISTRO_FEATURES
1324 When inheriting the
1325 :ref:`distro_features_check <ref-classes-distro_features_check>`
1326 class, this variable identifies distribution features that would be
1327 in conflict should the recipe be built. In other words, if the
1328 ``CONFLICT_DISTRO_FEATURES`` variable lists a feature that also
1329 appears in ``DISTRO_FEATURES`` within the current configuration, an
1330 error occurs and the build stops.
1331
1332 COPYLEFT_LICENSE_EXCLUDE
1333 A space-separated list of licenses to exclude from the source
1334 archived by the :ref:`archiver <ref-classes-archiver>` class. In
1335 other words, if a license in a recipe's
1336 :term:`LICENSE` value is in the value of
1337 ``COPYLEFT_LICENSE_EXCLUDE``, then its source is not archived by the
1338 class.
1339
1340 .. note::
1341
1342 The
1343 COPYLEFT_LICENSE_EXCLUDE
1344 variable takes precedence over the
1345 COPYLEFT_LICENSE_INCLUDE
1346 variable.
1347
1348 The default value, which is "CLOSED Proprietary", for
1349 ``COPYLEFT_LICENSE_EXCLUDE`` is set by the
1350 :ref:`copyleft_filter <ref-classes-copyleft_filter>` class, which
1351 is inherited by the ``archiver`` class.
1352
1353 COPYLEFT_LICENSE_INCLUDE
1354 A space-separated list of licenses to include in the source archived
1355 by the :ref:`archiver <ref-classes-archiver>` class. In other
1356 words, if a license in a recipe's :term:`LICENSE`
1357 value is in the value of ``COPYLEFT_LICENSE_INCLUDE``, then its
1358 source is archived by the class.
1359
1360 The default value is set by the
1361 :ref:`copyleft_filter <ref-classes-copyleft_filter>` class, which
1362 is inherited by the ``archiver`` class. The default value includes
1363 "GPL*", "LGPL*", and "AGPL*".
1364
1365 COPYLEFT_PN_EXCLUDE
1366 A list of recipes to exclude in the source archived by the
1367 :ref:`archiver <ref-classes-archiver>` class. The
1368 ``COPYLEFT_PN_EXCLUDE`` variable overrides the license inclusion and
1369 exclusion caused through the
1370 :term:`COPYLEFT_LICENSE_INCLUDE` and
1371 :term:`COPYLEFT_LICENSE_EXCLUDE`
1372 variables, respectively.
1373
1374 The default value, which is "" indicating to not explicitly exclude
1375 any recipes by name, for ``COPYLEFT_PN_EXCLUDE`` is set by the
1376 :ref:`copyleft_filter <ref-classes-copyleft_filter>` class, which
1377 is inherited by the ``archiver`` class.
1378
1379 COPYLEFT_PN_INCLUDE
1380 A list of recipes to include in the source archived by the
1381 :ref:`archiver <ref-classes-archiver>` class. The
1382 ``COPYLEFT_PN_INCLUDE`` variable overrides the license inclusion and
1383 exclusion caused through the
1384 :term:`COPYLEFT_LICENSE_INCLUDE` and
1385 :term:`COPYLEFT_LICENSE_EXCLUDE`
1386 variables, respectively.
1387
1388 The default value, which is "" indicating to not explicitly include
1389 any recipes by name, for ``COPYLEFT_PN_INCLUDE`` is set by the
1390 :ref:`copyleft_filter <ref-classes-copyleft_filter>` class, which
1391 is inherited by the ``archiver`` class.
1392
1393 COPYLEFT_RECIPE_TYPES
1394 A space-separated list of recipe types to include in the source
1395 archived by the :ref:`archiver <ref-classes-archiver>` class.
1396 Recipe types are ``target``, ``native``, ``nativesdk``, ``cross``,
1397 ``crosssdk``, and ``cross-canadian``.
1398
1399 The default value, which is "target*", for ``COPYLEFT_RECIPE_TYPES``
1400 is set by the :ref:`copyleft_filter <ref-classes-copyleft_filter>`
1401 class, which is inherited by the ``archiver`` class.
1402
1403 COPY_LIC_DIRS
1404 If set to "1" along with the
1405 :term:`COPY_LIC_MANIFEST` variable, the
1406 OpenEmbedded build system copies into the image the license files,
1407 which are located in ``/usr/share/common-licenses``, for each
1408 package. The license files are placed in directories within the image
1409 itself during build time.
1410
1411 .. note::
1412
1413 The
1414 COPY_LIC_DIRS
1415 does not offer a path for adding licenses for newly installed
1416 packages to an image, which might be most suitable for read-only
1417 filesystems that cannot be upgraded. See the
1418 LICENSE_CREATE_PACKAGE
1419 variable for additional information. You can also reference the "
1420 Providing License Text
1421 " section in the Yocto Project Development Tasks Manual for
1422 information on providing license text.
1423
1424 COPY_LIC_MANIFEST
1425 If set to "1", the OpenEmbedded build system copies the license
1426 manifest for the image to
1427 ``/usr/share/common-licenses/license.manifest`` within the image
1428 itself during build time.
1429
1430 .. note::
1431
1432 The
1433 COPY_LIC_MANIFEST
1434 does not offer a path for adding licenses for newly installed
1435 packages to an image, which might be most suitable for read-only
1436 filesystems that cannot be upgraded. See the
1437 LICENSE_CREATE_PACKAGE
1438 variable for additional information. You can also reference the "
1439 Providing License Text
1440 " section in the Yocto Project Development Tasks Manual for
1441 information on providing license text.
1442
1443 CORE_IMAGE_EXTRA_INSTALL
1444 Specifies the list of packages to be added to the image. You should
1445 only set this variable in the ``local.conf`` configuration file found
1446 in the :term:`Build Directory`.
1447
1448 This variable replaces ``POKY_EXTRA_INSTALL``, which is no longer
1449 supported.
1450
1451 COREBASE
1452 Specifies the parent directory of the OpenEmbedded-Core Metadata
1453 layer (i.e. ``meta``).
1454
1455 It is an important distinction that ``COREBASE`` points to the parent
1456 of this layer and not the layer itself. Consider an example where you
1457 have cloned the Poky Git repository and retained the ``poky`` name
1458 for your local copy of the repository. In this case, ``COREBASE``
1459 points to the ``poky`` folder because it is the parent directory of
1460 the ``poky/meta`` layer.
1461
1462 COREBASE_FILES
1463 Lists files from the :term:`COREBASE` directory that
1464 should be copied other than the layers listed in the
1465 ``bblayers.conf`` file. The ``COREBASE_FILES`` variable exists for
1466 the purpose of copying metadata from the OpenEmbedded build system
1467 into the extensible SDK.
1468
1469 Explicitly listing files in ``COREBASE`` is needed because it
1470 typically contains build directories and other files that should not
1471 normally be copied into the extensible SDK. Consequently, the value
1472 of ``COREBASE_FILES`` is used in order to only copy the files that
1473 are actually needed.
1474
1475 CPP
1476 The minimal command and arguments used to run the C preprocessor.
1477
1478 CPPFLAGS
1479 Specifies the flags to pass to the C pre-processor (i.e. to both the
1480 C and the C++ compilers). This variable is exported to an environment
1481 variable and thus made visible to the software being built during the
1482 compilation step.
1483
1484 Default initialization for ``CPPFLAGS`` varies depending on what is
1485 being built:
1486
1487 - :term:`TARGET_CPPFLAGS` when building for
1488 the target
1489
1490 - :term:`BUILD_CPPFLAGS` when building for the
1491 build host (i.e. ``-native``)
1492
1493 - :term:`BUILDSDK_CPPFLAGS` when building
1494 for an SDK (i.e. ``nativesdk-``)
1495
1496 CROSS_COMPILE
1497 The toolchain binary prefix for the target tools. The
1498 ``CROSS_COMPILE`` variable is the same as the
1499 :term:`TARGET_PREFIX` variable.
1500
1501 .. note::
1502
1503 The OpenEmbedded build system sets the
1504 CROSS_COMPILE
1505 variable only in certain contexts (e.g. when building for kernel
1506 and kernel module recipes).
1507
1508 CVSDIR
1509 The directory in which files checked out under the CVS system are
1510 stored.
1511
1512 CXX
1513 The minimal command and arguments used to run the C++ compiler.
1514
1515 CXXFLAGS
1516 Specifies the flags to pass to the C++ compiler. This variable is
1517 exported to an environment variable and thus made visible to the
1518 software being built during the compilation step.
1519
1520 Default initialization for ``CXXFLAGS`` varies depending on what is
1521 being built:
1522
1523 - :term:`TARGET_CXXFLAGS` when building for
1524 the target
1525
1526 - :term:`BUILD_CXXFLAGS` when building for the
1527 build host (i.e. ``-native``)
1528
1529 - :term:`BUILDSDK_CXXFLAGS` when building
1530 for an SDK (i.e. ``nativesdk-``)
1531
1532 D
1533 The destination directory. The location in the :term:`Build Directory`
1534 where components are installed by the
1535 :ref:`ref-tasks-install` task. This location defaults
1536 to:
1537 ::
1538
1539 ${WORKDIR}/image
1540
1541 .. note::
1542
1543 Tasks that read from or write to this directory should run under
1544 fakeroot
1545 .
1546
1547 DATE
1548 The date the build was started. Dates appear using the year, month,
1549 and day (YMD) format (e.g. "20150209" for February 9th, 2015).
1550
1551 DATETIME
1552 The date and time on which the current build started. The format is
1553 suitable for timestamps.
1554
1555 DEBIAN_NOAUTONAME
1556 When the :ref:`debian <ref-classes-debian>` class is inherited,
1557 which is the default behavior, ``DEBIAN_NOAUTONAME`` specifies a
1558 particular package should not be renamed according to Debian library
1559 package naming. You must use the package name as an override when you
1560 set this variable. Here is an example from the ``fontconfig`` recipe:
1561 ::
1562
1563 DEBIAN_NOAUTONAME_fontconfig-utils = "1"
1564
1565 DEBIANNAME
1566 When the :ref:`debian <ref-classes-debian>` class is inherited,
1567 which is the default behavior, ``DEBIANNAME`` allows you to override
1568 the library name for an individual package. Overriding the library
1569 name in these cases is rare. You must use the package name as an
1570 override when you set this variable. Here is an example from the
1571 ``dbus`` recipe:
1572 ::
1573
1574 DEBIANNAME_${PN} = "dbus-1"
1575
1576 DEBUG_BUILD
1577 Specifies to build packages with debugging information. This
1578 influences the value of the ``SELECTED_OPTIMIZATION`` variable.
1579
1580 DEBUG_OPTIMIZATION
1581 The options to pass in ``TARGET_CFLAGS`` and ``CFLAGS`` when
1582 compiling a system for debugging. This variable defaults to "-O
1583 -fno-omit-frame-pointer ${DEBUG_FLAGS} -pipe".
1584
1585 DEFAULT_PREFERENCE
1586 Specifies a weak bias for recipe selection priority.
1587
1588 The most common usage of this is variable is to set it to "-1" within
1589 a recipe for a development version of a piece of software. Using the
1590 variable in this way causes the stable version of the recipe to build
1591 by default in the absence of ``PREFERRED_VERSION`` being used to
1592 build the development version.
1593
1594 .. note::
1595
1596 The bias provided by
1597 DEFAULT_PREFERENCE
1598 is weak and is overridden by
1599 BBFILE_PRIORITY
1600 if that variable is different between two layers that contain
1601 different versions of the same recipe.
1602
1603 DEFAULTTUNE
1604 The default CPU and Application Binary Interface (ABI) tunings (i.e.
1605 the "tune") used by the OpenEmbedded build system. The
1606 ``DEFAULTTUNE`` helps define
1607 :term:`TUNE_FEATURES`.
1608
1609 The default tune is either implicitly or explicitly set by the
1610 machine (:term:`MACHINE`). However, you can override
1611 the setting using available tunes as defined with
1612 :term:`AVAILTUNES`.
1613
1614 DEPENDS
1615 Lists a recipe's build-time dependencies. These are dependencies on
1616 other recipes whose contents (e.g. headers and shared libraries) are
1617 needed by the recipe at build time.
1618
1619 As an example, consider a recipe ``foo`` that contains the following
1620 assignment:
1621 ::
1622
1623 DEPENDS = "bar"
1624
1625 The practical effect of the previous
1626 assignment is that all files installed by bar will be available in
1627 the appropriate staging sysroot, given by the
1628 :term:`STAGING_DIR* <STAGING_DIR>` variables, by the time the
1629 :ref:`ref-tasks-configure` task for ``foo`` runs.
1630 This mechanism is implemented by having ``do_configure`` depend on
1631 the :ref:`ref-tasks-populate_sysroot` task of
1632 each recipe listed in ``DEPENDS``, through a
1633 ``[``\ :ref:`deptask <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]``
1634 declaration in the :ref:`base <ref-classes-base>` class.
1635
1636 .. note::
1637
1638 It seldom is necessary to reference, for example,
1639 STAGING_DIR_HOST
1640 explicitly. The standard classes and build-related variables are
1641 configured to automatically use the appropriate staging sysroots.
1642
1643 As another example, ``DEPENDS`` can also be used to add utilities
1644 that run on the build machine during the build. For example, a recipe
1645 that makes use of a code generator built by the recipe ``codegen``
1646 might have the following:
1647 ::
1648
1649 DEPENDS = "codegen-native"
1650
1651 For more
1652 information, see the :ref:`native <ref-classes-native>` class and
1653 the :term:`EXTRANATIVEPATH` variable.
1654
1655 .. note::
1656
1657 - ``DEPENDS`` is a list of recipe names. Or, to be more precise,
1658 it is a list of :term:`PROVIDES` names, which
1659 usually match recipe names. Putting a package name such as
1660 "foo-dev" in ``DEPENDS`` does not make sense. Use "foo"
1661 instead, as this will put files from all the packages that make
1662 up ``foo``, which includes those from ``foo-dev``, into the
1663 sysroot.
1664
1665 - One recipe having another recipe in ``DEPENDS`` does not by
1666 itself add any runtime dependencies between the packages
1667 produced by the two recipes. However, as explained in the
1668 ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
1669 section in the Yocto Project Overview and Concepts Manual,
1670 runtime dependencies will often be added automatically, meaning
1671 ``DEPENDS`` alone is sufficient for most recipes.
1672
1673 - Counterintuitively, ``DEPENDS`` is often necessary even for
1674 recipes that install precompiled components. For example, if
1675 ``libfoo`` is a precompiled library that links against
1676 ``libbar``, then linking against ``libfoo`` requires both
1677 ``libfoo`` and ``libbar`` to be available in the sysroot.
1678 Without a ``DEPENDS`` from the recipe that installs ``libfoo``
1679 to the recipe that installs ``libbar``, other recipes might
1680 fail to link against ``libfoo``.
1681
1682 For information on runtime dependencies, see the
1683 :term:`RDEPENDS` variable. You can also see the
1684 ":ref:`Tasks <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:tasks>`" and
1685 ":ref:`Dependencies <bitbake:bitbake-user-manual/bitbake-user-manual-execution:dependencies>`" sections in the
1686 BitBake User Manual for additional information on tasks and
1687 dependencies.
1688
1689 DEPLOY_DIR
1690 Points to the general area that the OpenEmbedded build system uses to
1691 place images, packages, SDKs, and other output files that are ready
1692 to be used outside of the build system. By default, this directory
1693 resides within the :term:`Build Directory` as
1694 ``${TMPDIR}/deploy``.
1695
1696 For more information on the structure of the Build Directory, see
1697 ":ref:`ref-manual/ref-structure:the build directory - \`\`build/\`\``" section.
1698 For more detail on the contents of the ``deploy`` directory, see the
1699 ":ref:`Images <images-dev-environment>`", ":ref:`Package
1700 Feeds <package-feeds-dev-environment>`", and
1701 ":ref:`sdk-dev-environment`" sections all in the
1702 Yocto Project Overview and Concepts Manual.
1703
1704 DEPLOY_DIR_DEB
1705 Points to the area that the OpenEmbedded build system uses to place
1706 Debian packages that are ready to be used outside of the build
1707 system. This variable applies only when
1708 :term:`PACKAGE_CLASSES` contains
1709 "package_deb".
1710
1711 The BitBake configuration file initially defines the
1712 ``DEPLOY_DIR_DEB`` variable as a sub-folder of
1713 :term:`DEPLOY_DIR`:
1714 ::
1715
1716 DEPLOY_DIR_DEB = "${DEPLOY_DIR}/deb"
1717
1718 The :ref:`package_deb <ref-classes-package_deb>` class uses the
1719 ``DEPLOY_DIR_DEB`` variable to make sure the
1720 :ref:`ref-tasks-package_write_deb` task
1721 writes Debian packages into the appropriate folder. For more
1722 information on how packaging works, see the ":ref:`Package
1723 Feeds <package-feeds-dev-environment>`" section
1724 in the Yocto Project Overview and Concepts Manual.
1725
1726 DEPLOY_DIR_IMAGE
1727 Points to the area that the OpenEmbedded build system uses to place
1728 images and other associated output files that are ready to be
1729 deployed onto the target machine. The directory is machine-specific
1730 as it contains the ``${MACHINE}`` name. By default, this directory
1731 resides within the :term:`Build Directory` as
1732 ``${DEPLOY_DIR}/images/${MACHINE}/``.
1733
1734 For more information on the structure of the Build Directory, see
1735 ":ref:`ref-manual/ref-structure:the build directory - \`\`build/\`\``" section.
1736 For more detail on the contents of the ``deploy`` directory, see the
1737 ":ref:`Images <images-dev-environment>`" and
1738 ":ref:`sdk-dev-environment`" sections both in
1739 the Yocto Project Overview and Concepts Manual.
1740
1741 DEPLOY_DIR_IPK
1742 Points to the area that the OpenEmbedded build system uses to place
1743 IPK packages that are ready to be used outside of the build system.
1744 This variable applies only when
1745 :term:`PACKAGE_CLASSES` contains
1746 "package_ipk".
1747
1748 The BitBake configuration file initially defines this variable as a
1749 sub-folder of :term:`DEPLOY_DIR`:
1750 ::
1751
1752 DEPLOY_DIR_IPK = "${DEPLOY_DIR}/ipk"
1753
1754 The :ref:`package_ipk <ref-classes-package_ipk>` class uses the
1755 ``DEPLOY_DIR_IPK`` variable to make sure the
1756 :ref:`ref-tasks-package_write_ipk` task
1757 writes IPK packages into the appropriate folder. For more information
1758 on how packaging works, see the ":ref:`Package
1759 Feeds <package-feeds-dev-environment>`" section
1760 in the Yocto Project Overview and Concepts Manual.
1761
1762 DEPLOY_DIR_RPM
1763 Points to the area that the OpenEmbedded build system uses to place
1764 RPM packages that are ready to be used outside of the build system.
1765 This variable applies only when
1766 :term:`PACKAGE_CLASSES` contains
1767 "package_rpm".
1768
1769 The BitBake configuration file initially defines this variable as a
1770 sub-folder of :term:`DEPLOY_DIR`:
1771 ::
1772
1773 DEPLOY_DIR_RPM = "${DEPLOY_DIR}/rpm"
1774
1775 The :ref:`package_rpm <ref-classes-package_rpm>` class uses the
1776 ``DEPLOY_DIR_RPM`` variable to make sure the
1777 :ref:`ref-tasks-package_write_rpm` task
1778 writes RPM packages into the appropriate folder. For more information
1779 on how packaging works, see the ":ref:`Package
1780 Feeds <package-feeds-dev-environment>`" section
1781 in the Yocto Project Overview and Concepts Manual.
1782
1783 DEPLOY_DIR_TAR
1784 Points to the area that the OpenEmbedded build system uses to place
1785 tarballs that are ready to be used outside of the build system. This
1786 variable applies only when
1787 :term:`PACKAGE_CLASSES` contains
1788 "package_tar".
1789
1790 The BitBake configuration file initially defines this variable as a
1791 sub-folder of :term:`DEPLOY_DIR`:
1792 ::
1793
1794 DEPLOY_DIR_TAR = "${DEPLOY_DIR}/tar"
1795
1796 The :ref:`package_tar <ref-classes-package_tar>` class uses the
1797 ``DEPLOY_DIR_TAR`` variable to make sure the
1798 :ref:`ref-tasks-package_write_tar` task
1799 writes TAR packages into the appropriate folder. For more information
1800 on how packaging works, see the ":ref:`Package
1801 Feeds <package-feeds-dev-environment>`" section
1802 in the Yocto Project Overview and Concepts Manual.
1803
1804 DEPLOYDIR
1805 When inheriting the :ref:`deploy <ref-classes-deploy>` class, the
1806 ``DEPLOYDIR`` points to a temporary work area for deployed files that
1807 is set in the ``deploy`` class as follows:
1808 ::
1809
1810 DEPLOYDIR = "${WORKDIR}/deploy-${:term:`PN`}"
1811
1812 Recipes inheriting the ``deploy`` class should copy files to be
1813 deployed into ``DEPLOYDIR``, and the class will take care of copying
1814 them into :term:`DEPLOY_DIR_IMAGE`
1815 afterwards.
1816
1817 DESCRIPTION
1818 The package description used by package managers. If not set,
1819 ``DESCRIPTION`` takes the value of the :term:`SUMMARY`
1820 variable.
1821
1822 DISTRO
1823 The short name of the distribution. For information on the long name
1824 of the distribution, see the :term:`DISTRO_NAME`
1825 variable.
1826
1827 The ``DISTRO`` variable corresponds to a distribution configuration
1828 file whose root name is the same as the variable's argument and whose
1829 filename extension is ``.conf``. For example, the distribution
1830 configuration file for the Poky distribution is named ``poky.conf``
1831 and resides in the ``meta-poky/conf/distro`` directory of the
1832 :term:`Source Directory`.
1833
1834 Within that ``poky.conf`` file, the ``DISTRO`` variable is set as
1835 follows:
1836 ::
1837
1838 DISTRO = "poky"
1839
1840 Distribution configuration files are located in a ``conf/distro``
1841 directory within the :term:`Metadata` that contains the
1842 distribution configuration. The value for ``DISTRO`` must not contain
1843 spaces, and is typically all lower-case.
1844
1845 .. note::
1846
1847 If the
1848 DISTRO
1849 variable is blank, a set of default configurations are used, which
1850 are specified within
1851 meta/conf/distro/defaultsetup.conf
1852 also in the Source Directory.
1853
1854 DISTRO_CODENAME
1855 Specifies a codename for the distribution being built.
1856
1857 DISTRO_EXTRA_RDEPENDS
1858 Specifies a list of distro-specific packages to add to all images.
1859 This variable takes affect through ``packagegroup-base`` so the
1860 variable only really applies to the more full-featured images that
1861 include ``packagegroup-base``. You can use this variable to keep
1862 distro policy out of generic images. As with all other distro
1863 variables, you set this variable in the distro ``.conf`` file.
1864
1865 DISTRO_EXTRA_RRECOMMENDS
1866 Specifies a list of distro-specific packages to add to all images if
1867 the packages exist. The packages might not exist or be empty (e.g.
1868 kernel modules). The list of packages are automatically installed but
1869 you can remove them.
1870
1871 DISTRO_FEATURES
1872 The software support you want in your distribution for various
1873 features. You define your distribution features in the distribution
1874 configuration file.
1875
1876 In most cases, the presence or absence of a feature in
1877 ``DISTRO_FEATURES`` is translated to the appropriate option supplied
1878 to the configure script during the
1879 :ref:`ref-tasks-configure` task for recipes that
1880 optionally support the feature. For example, specifying "x11" in
1881 ``DISTRO_FEATURES``, causes every piece of software built for the
1882 target that can optionally support X11 to have its X11 support
1883 enabled.
1884
1885 Two more examples are Bluetooth and NFS support. For a more complete
1886 list of features that ships with the Yocto Project and that you can
1887 provide with this variable, see the "`Distro
1888 Features <#ref-features-distro>`__" section.
1889
1890 DISTRO_FEATURES_BACKFILL
1891 Features to be added to ``DISTRO_FEATURES`` if not also present in
1892 ``DISTRO_FEATURES_BACKFILL_CONSIDERED``.
1893
1894 This variable is set in the ``meta/conf/bitbake.conf`` file. It is
1895 not intended to be user-configurable. It is best to just reference
1896 the variable to see which distro features are being backfilled for
1897 all distro configurations. See the "`Feature
1898 Backfilling <#ref-features-backfill>`__" section for more
1899 information.
1900
1901 DISTRO_FEATURES_BACKFILL_CONSIDERED
1902 Features from ``DISTRO_FEATURES_BACKFILL`` that should not be
1903 backfilled (i.e. added to ``DISTRO_FEATURES``) during the build. See
1904 the "`Feature Backfilling <#ref-features-backfill>`__" section for
1905 more information.
1906
1907 DISTRO_FEATURES_DEFAULT
1908 A convenience variable that gives you the default list of distro
1909 features with the exception of any features specific to the C library
1910 (``libc``).
1911
1912 When creating a custom distribution, you might find it useful to be
1913 able to reuse the default
1914 :term:`DISTRO_FEATURES` options without the
1915 need to write out the full set. Here is an example that uses
1916 ``DISTRO_FEATURES_DEFAULT`` from a custom distro configuration file:
1917 ::
1918
1919 DISTRO_FEATURES ?= "${DISTRO_FEATURES_DEFAULT} myfeature"
1920
1921 DISTRO_FEATURES_FILTER_NATIVE
1922 Specifies a list of features that if present in the target
1923 :term:`DISTRO_FEATURES` value should be
1924 included in ``DISTRO_FEATURES`` when building native recipes. This
1925 variable is used in addition to the features filtered using the
1926 :term:`DISTRO_FEATURES_NATIVE`
1927 variable.
1928
1929 DISTRO_FEATURES_FILTER_NATIVESDK
1930 Specifies a list of features that if present in the target
1931 :term:`DISTRO_FEATURES` value should be
1932 included in ``DISTRO_FEATURES`` when building nativesdk recipes. This
1933 variable is used in addition to the features filtered using the
1934 :term:`DISTRO_FEATURES_NATIVESDK`
1935 variable.
1936
1937 DISTRO_FEATURES_NATIVE
1938 Specifies a list of features that should be included in
1939 :term:`DISTRO_FEATURES` when building native
1940 recipes. This variable is used in addition to the features filtered
1941 using the
1942 :term:`DISTRO_FEATURES_FILTER_NATIVE`
1943 variable.
1944
1945 DISTRO_FEATURES_NATIVESDK
1946 Specifies a list of features that should be included in
1947 :term:`DISTRO_FEATURES` when building
1948 nativesdk recipes. This variable is used in addition to the features
1949 filtered using the
1950 :term:`DISTRO_FEATURES_FILTER_NATIVESDK`
1951 variable.
1952
1953 DISTRO_NAME
1954 The long name of the distribution. For information on the short name
1955 of the distribution, see the :term:`DISTRO` variable.
1956
1957 The ``DISTRO_NAME`` variable corresponds to a distribution
1958 configuration file whose root name is the same as the variable's
1959 argument and whose filename extension is ``.conf``. For example, the
1960 distribution configuration file for the Poky distribution is named
1961 ``poky.conf`` and resides in the ``meta-poky/conf/distro`` directory
1962 of the :term:`Source Directory`.
1963
1964 Within that ``poky.conf`` file, the ``DISTRO_NAME`` variable is set
1965 as follows:
1966 ::
1967
1968 DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
1969
1970 Distribution configuration files are located in a ``conf/distro``
1971 directory within the :term:`Metadata` that contains the
1972 distribution configuration.
1973
1974 .. note::
1975
1976 If the
1977 DISTRO_NAME
1978 variable is blank, a set of default configurations are used, which
1979 are specified within
1980 meta/conf/distro/defaultsetup.conf
1981 also in the Source Directory.
1982
1983 DISTRO_VERSION
1984 The version of the distribution.
1985
1986 DISTROOVERRIDES
1987 A colon-separated list of overrides specific to the current
1988 distribution. By default, this list includes the value of
1989 :term:`DISTRO`.
1990
1991 You can extend ``DISTROOVERRIDES`` to add extra overrides that should
1992 apply to the distribution.
1993
1994 The underlying mechanism behind ``DISTROOVERRIDES`` is simply that it
1995 is included in the default value of
1996 :term:`OVERRIDES`.
1997
1998 DL_DIR
1999 The central download directory used by the build process to store
2000 downloads. By default, ``DL_DIR`` gets files suitable for mirroring
2001 for everything except Git repositories. If you want tarballs of Git
2002 repositories, use the
2003 :term:`BB_GENERATE_MIRROR_TARBALLS`
2004 variable.
2005
2006 You can set this directory by defining the ``DL_DIR`` variable in the
2007 ``conf/local.conf`` file. This directory is self-maintaining and you
2008 should not have to touch it. By default, the directory is
2009 ``downloads`` in the :term:`Build Directory`.
2010 ::
2011
2012 #DL_DIR ?= "${TOPDIR}/downloads"
2013
2014 To specify a different download directory,
2015 simply remove the comment from the line and provide your directory.
2016
2017 During a first build, the system downloads many different source code
2018 tarballs from various upstream projects. Downloading can take a
2019 while, particularly if your network connection is slow. Tarballs are
2020 all stored in the directory defined by ``DL_DIR`` and the build
2021 system looks there first to find source tarballs.
2022
2023 .. note::
2024
2025 When wiping and rebuilding, you can preserve this directory to
2026 speed up this part of subsequent builds.
2027
2028 You can safely share this directory between multiple builds on the
2029 same development machine. For additional information on how the build
2030 process gets source files when working behind a firewall or proxy
2031 server, see this specific question in the
2032 "`FAQ <#how-does-the-yocto-project-obtain-source-code-and-will-it-work-behind-my-firewall-or-proxy-server>`__"
2033 chapter. You can also refer to the
2034 ":yocto_wiki:`Working Behind a Network Proxy </wiki/Working_Behind_a_Network_Proxy>`"
2035 Wiki page.
2036
2037 DOC_COMPRESS
2038 When inheriting the :ref:`compress_doc <ref-classes-compress_doc>`
2039 class, this variable sets the compression policy used when the
2040 OpenEmbedded build system compresses man pages and info pages. By
2041 default, the compression method used is gz (gzip). Other policies
2042 available are xz and bz2.
2043
2044 For information on policies and on how to use this variable, see the
2045 comments in the ``meta/classes/compress_doc.bbclass`` file.
2046
2047 EFI_PROVIDER
2048 When building bootable images (i.e. where ``hddimg``, ``iso``, or
2049 ``wic.vmdk`` is in :term:`IMAGE_FSTYPES`), the
2050 ``EFI_PROVIDER`` variable specifies the EFI bootloader to use. The
2051 default is "grub-efi", but "systemd-boot" can be used instead.
2052
2053 See the :ref:`systemd-boot <ref-classes-systemd-boot>` and
2054 :ref:`image-live <ref-classes-image-live>` classes for more
2055 information.
2056
2057 ENABLE_BINARY_LOCALE_GENERATION
2058 Variable that controls which locales for ``glibc`` are generated
2059 during the build (useful if the target device has 64Mbytes of RAM or
2060 less).
2061
2062 ERR_REPORT_DIR
2063 When used with the :ref:`report-error <ref-classes-report-error>`
2064 class, specifies the path used for storing the debug files created by
2065 the :ref:`error reporting
2066 tool <dev-manual/dev-manual-common-tasks:using the error reporting tool>`, which
2067 allows you to submit build errors you encounter to a central
2068 database. By default, the value of this variable is
2069 ``${``\ :term:`LOG_DIR`\ ``}/error-report``.
2070
2071 You can set ``ERR_REPORT_DIR`` to the path you want the error
2072 reporting tool to store the debug files as follows in your
2073 ``local.conf`` file:
2074 ::
2075
2076 ERR_REPORT_DIR = "path"
2077
2078 ERROR_QA
2079 Specifies the quality assurance checks whose failures are reported as
2080 errors by the OpenEmbedded build system. You set this variable in
2081 your distribution configuration file. For a list of the checks you
2082 can control with this variable, see the
2083 ":ref:`insane.bbclass <ref-classes-insane>`" section.
2084
2085 EXCLUDE_FROM_SHLIBS
2086 Triggers the OpenEmbedded build system's shared libraries resolver to
2087 exclude an entire package when scanning for shared libraries.
2088
2089 .. note::
2090
2091 The shared libraries resolver's functionality results in part from
2092 the internal function
2093 package_do_shlibs
2094 , which is part of the
2095 do_package
2096 task. You should be aware that the shared libraries resolver might
2097 implicitly define some dependencies between packages.
2098
2099 The ``EXCLUDE_FROM_SHLIBS`` variable is similar to the
2100 :term:`PRIVATE_LIBS` variable, which excludes a
2101 package's particular libraries only and not the whole package.
2102
2103 Use the ``EXCLUDE_FROM_SHLIBS`` variable by setting it to "1" for a
2104 particular package:
2105 ::
2106
2107 EXCLUDE_FROM_SHLIBS = "1"
2108
2109 EXCLUDE_FROM_WORLD
2110 Directs BitBake to exclude a recipe from world builds (i.e.
2111 ``bitbake world``). During world builds, BitBake locates, parses and
2112 builds all recipes found in every layer exposed in the
2113 ``bblayers.conf`` configuration file.
2114
2115 To exclude a recipe from a world build using this variable, set the
2116 variable to "1" in the recipe.
2117
2118 .. note::
2119
2120 Recipes added to
2121 EXCLUDE_FROM_WORLD
2122 may still be built during a world build in order to satisfy
2123 dependencies of other recipes. Adding a recipe to
2124 EXCLUDE_FROM_WORLD
2125 only ensures that the recipe is not explicitly added to the list
2126 of build targets in a world build.
2127
2128 EXTENDPE
2129 Used with file and pathnames to create a prefix for a recipe's
2130 version based on the recipe's :term:`PE` value. If ``PE``
2131 is set and greater than zero for a recipe, ``EXTENDPE`` becomes that
2132 value (e.g if ``PE`` is equal to "1" then ``EXTENDPE`` becomes "1").
2133 If a recipe's ``PE`` is not set (the default) or is equal to zero,
2134 ``EXTENDPE`` becomes "".
2135
2136 See the :term:`STAMP` variable for an example.
2137
2138 EXTENDPKGV
2139 The full package version specification as it appears on the final
2140 packages produced by a recipe. The variable's value is normally used
2141 to fix a runtime dependency to the exact same version of another
2142 package in the same recipe:
2143 ::
2144
2145 RDEPENDS_${PN}-additional-module = "${PN} (= ${EXTENDPKGV})"
2146
2147 The dependency relationships are intended to force the package
2148 manager to upgrade these types of packages in lock-step.
2149
2150 EXTERNAL_KERNEL_TOOLS
2151 When set, the ``EXTERNAL_KERNEL_TOOLS`` variable indicates that these
2152 tools are not in the source tree.
2153
2154 When kernel tools are available in the tree, they are preferred over
2155 any externally installed tools. Setting the ``EXTERNAL_KERNEL_TOOLS``
2156 variable tells the OpenEmbedded build system to prefer the installed
2157 external tools. See the
2158 :ref:`kernel-yocto <ref-classes-kernel-yocto>` class in
2159 ``meta/classes`` to see how the variable is used.
2160
2161 EXTERNALSRC
2162 When inheriting the :ref:`externalsrc <ref-classes-externalsrc>`
2163 class, this variable points to the source tree, which is outside of
2164 the OpenEmbedded build system. When set, this variable sets the
2165 :term:`S` variable, which is what the OpenEmbedded build
2166 system uses to locate unpacked recipe source code.
2167
2168 For more information on ``externalsrc.bbclass``, see the
2169 ":ref:`externalsrc.bbclass <ref-classes-externalsrc>`" section. You
2170 can also find information on how to use this variable in the
2171 ":ref:`dev-manual/dev-manual-common-tasks:building software from an external source`"
2172 section in the Yocto Project Development Tasks Manual.
2173
2174 EXTERNALSRC_BUILD
2175 When inheriting the :ref:`externalsrc <ref-classes-externalsrc>`
2176 class, this variable points to the directory in which the recipe's
2177 source code is built, which is outside of the OpenEmbedded build
2178 system. When set, this variable sets the :term:`B` variable,
2179 which is what the OpenEmbedded build system uses to locate the Build
2180 Directory.
2181
2182 For more information on ``externalsrc.bbclass``, see the
2183 ":ref:`externalsrc.bbclass <ref-classes-externalsrc>`" section. You
2184 can also find information on how to use this variable in the
2185 ":ref:`dev-manual/dev-manual-common-tasks:building software from an external source`"
2186 section in the Yocto Project Development Tasks Manual.
2187
2188 EXTRA_AUTORECONF
2189 For recipes inheriting the :ref:`autotools <ref-classes-autotools>`
2190 class, you can use ``EXTRA_AUTORECONF`` to specify extra options to
2191 pass to the ``autoreconf`` command that is executed during the
2192 :ref:`ref-tasks-configure` task.
2193
2194 The default value is "--exclude=autopoint".
2195
2196 EXTRA_IMAGE_FEATURES
2197 A list of additional features to include in an image. When listing
2198 more than one feature, separate them with a space.
2199
2200 Typically, you configure this variable in your ``local.conf`` file,
2201 which is found in the :term:`Build Directory`.
2202 Although you can use this variable from within a recipe, best
2203 practices dictate that you do not.
2204
2205 .. note::
2206
2207 To enable primary features from within the image recipe, use the
2208 IMAGE_FEATURES
2209 variable.
2210
2211 Here are some examples of features you can add:
2212
2213 - "dbg-pkgs" - Adds -dbg packages for all installed packages including
2214 symbol information for debugging and profiling.
2215
2216 - "debug-tweaks" - Makes an image suitable for debugging. For example, allows root logins without passwords and
2217 enables post-installation logging. See the 'allow-empty-password' and
2218 'post-install-logging' features in the "`Image
2219 Features <#ref-features-image>`__" section for more information.
2220 - "dev-pkgs" - Adds -dev packages for all installed packages. This is
2221 useful if you want to develop against the libraries in the image.
2222 - "read-only-rootfs" - Creates an image whose root filesystem is
2223 read-only. See the
2224 ":ref:`dev-manual/dev-manual-common-tasks:creating a read-only root filesystem`"
2225 section in the Yocto Project Development Tasks Manual for more
2226 information
2227 - "tools-debug" - Adds debugging tools such as gdb and strace.
2228 - "tools-sdk" - Adds development tools such as gcc, make,
2229 pkgconfig and so forth.
2230 - "tools-testapps" - Adds useful testing tools
2231 such as ts_print, aplay, arecord and so forth.
2232
2233 For a complete list of image features that ships with the Yocto
2234 Project, see the "`Image Features <#ref-features-image>`__" section.
2235
2236 For an example that shows how to customize your image by using this
2237 variable, see the ":ref:`usingpoky-extend-customimage-imagefeatures`"
2238 section in the Yocto Project Development Tasks Manual.
2239
2240 EXTRA_IMAGECMD
2241 Specifies additional options for the image creation command that has
2242 been specified in :term:`IMAGE_CMD`. When setting
2243 this variable, use an override for the associated image type. Here is
2244 an example:
2245 ::
2246
2247 EXTRA_IMAGECMD_ext3 ?= "-i 4096"
2248
2249 EXTRA_IMAGEDEPENDS
2250 A list of recipes to build that do not provide packages for
2251 installing into the root filesystem.
2252
2253 Sometimes a recipe is required to build the final image but is not
2254 needed in the root filesystem. You can use the ``EXTRA_IMAGEDEPENDS``
2255 variable to list these recipes and thus specify the dependencies. A
2256 typical example is a required bootloader in a machine configuration.
2257
2258 .. note::
2259
2260 To add packages to the root filesystem, see the various
2261 \*RDEPENDS and \*RRECOMMENDS
2262 variables.
2263
2264 EXTRANATIVEPATH
2265 A list of subdirectories of
2266 ``${``\ :term:`STAGING_BINDIR_NATIVE`\ ``}``
2267 added to the beginning of the environment variable ``PATH``. As an
2268 example, the following prepends
2269 "${STAGING_BINDIR_NATIVE}/foo:${STAGING_BINDIR_NATIVE}/bar:" to
2270 ``PATH``:
2271 ::
2272
2273 EXTRANATIVEPATH = "foo bar"
2274
2275 EXTRA_OECMAKE
2276 Additional `CMake <https://cmake.org/overview/>`__ options. See the
2277 :ref:`cmake <ref-classes-cmake>` class for additional information.
2278
2279 EXTRA_OECONF
2280 Additional ``configure`` script options. See
2281 :term:`PACKAGECONFIG_CONFARGS` for
2282 additional information on passing configure script options.
2283
2284 EXTRA_OEMAKE
2285 Additional GNU ``make`` options.
2286
2287 Because the ``EXTRA_OEMAKE`` defaults to "", you need to set the
2288 variable to specify any required GNU options.
2289
2290 :term:`PARALLEL_MAKE` and
2291 :term:`PARALLEL_MAKEINST` also make use of
2292 ``EXTRA_OEMAKE`` to pass the required flags.
2293
2294 EXTRA_OESCONS
2295 When inheriting the :ref:`scons <ref-classes-scons>` class, this
2296 variable specifies additional configuration options you want to pass
2297 to the ``scons`` command line.
2298
2299 EXTRA_USERS_PARAMS
2300 When inheriting the :ref:`extrausers <ref-classes-extrausers>`
2301 class, this variable provides image level user and group operations.
2302 This is a more global method of providing user and group
2303 configuration as compared to using the
2304 :ref:`useradd <ref-classes-useradd>` class, which ties user and
2305 group configurations to a specific recipe.
2306
2307 The set list of commands you can configure using the
2308 ``EXTRA_USERS_PARAMS`` is shown in the ``extrausers`` class. These
2309 commands map to the normal Unix commands of the same names:
2310 ::
2311
2312 # EXTRA_USERS_PARAMS = "\
2313 # useradd -p '' tester; \
2314 # groupadd developers; \
2315 # userdel nobody; \
2316 # groupdel -g video; \
2317 # groupmod -g 1020 developers; \
2318 # usermod -s /bin/sh tester; \
2319 # "
2320
2321 FEATURE_PACKAGES
2322 Defines one or more packages to include in an image when a specific
2323 item is included in :term:`IMAGE_FEATURES`.
2324 When setting the value, ``FEATURE_PACKAGES`` should have the name of
2325 the feature item as an override. Here is an example:
2326 ::
2327
2328 FEATURE_PACKAGES_widget = "package1 package2"
2329
2330 In this example, if "widget" were added to ``IMAGE_FEATURES``,
2331 package1 and package2 would be included in the image.
2332
2333 .. note::
2334
2335 Packages installed by features defined through
2336 FEATURE_PACKAGES
2337 are often package groups. While similarly named, you should not
2338 confuse the
2339 FEATURE_PACKAGES
2340 variable with package groups, which are discussed elsewhere in the
2341 documentation.
2342
2343 FEED_DEPLOYDIR_BASE_URI
2344 Points to the base URL of the server and location within the
2345 document-root that provides the metadata and packages required by
2346 OPKG to support runtime package management of IPK packages. You set
2347 this variable in your ``local.conf`` file.
2348
2349 Consider the following example:
2350 ::
2351
2352 FEED_DEPLOYDIR_BASE_URI = "http://192.168.7.1/BOARD-dir"
2353
2354 This example assumes you are serving
2355 your packages over HTTP and your databases are located in a directory
2356 named ``BOARD-dir``, which is underneath your HTTP server's
2357 document-root. In this case, the OpenEmbedded build system generates
2358 a set of configuration files for you in your target that work with
2359 the feed.
2360
2361 FILES
2362 The list of files and directories that are placed in a package. The
2363 :term:`PACKAGES` variable lists the packages
2364 generated by a recipe.
2365
2366 To use the ``FILES`` variable, provide a package name override that
2367 identifies the resulting package. Then, provide a space-separated
2368 list of files or paths that identify the files you want included as
2369 part of the resulting package. Here is an example:
2370 ::
2371
2372 FILES_${PN} += "${bindir}/mydir1 ${bindir}/mydir2/myfile"
2373
2374 .. note::
2375
2376 - When specifying files or paths, you can pattern match using
2377 Python's
2378 `glob <https://docs.python.org/3/library/glob.html>`_
2379 syntax. For details on the syntax, see the documentation by
2380 following the previous link.
2381
2382 - When specifying paths as part of the ``FILES`` variable, it is
2383 good practice to use appropriate path variables. For example,
2384 use ``${sysconfdir}`` rather than ``/etc``, or ``${bindir}``
2385 rather than ``/usr/bin``. You can find a list of these
2386 variables at the top of the ``meta/conf/bitbake.conf`` file in
2387 the :term:`Source Directory`. You will also
2388 find the default values of the various ``FILES_*`` variables in
2389 this file.
2390
2391 If some of the files you provide with the ``FILES`` variable are
2392 editable and you know they should not be overwritten during the
2393 package update process by the Package Management System (PMS), you
2394 can identify these files so that the PMS will not overwrite them. See
2395 the :term:`CONFFILES` variable for information on
2396 how to identify these files to the PMS.
2397
2398 FILES_SOLIBSDEV
2399 Defines the file specification to match
2400 :term:`SOLIBSDEV`. In other words,
2401 ``FILES_SOLIBSDEV`` defines the full path name of the development
2402 symbolic link (symlink) for shared libraries on the target platform.
2403
2404 The following statement from the ``bitbake.conf`` shows how it is
2405 set:
2406 ::
2407
2408 FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
2409
2410 FILESEXTRAPATHS
2411 Extends the search path the OpenEmbedded build system uses when
2412 looking for files and patches as it processes recipes and append
2413 files. The default directories BitBake uses when it processes recipes
2414 are initially defined by the :term:`FILESPATH`
2415 variable. You can extend ``FILESPATH`` variable by using
2416 ``FILESEXTRAPATHS``.
2417
2418 Best practices dictate that you accomplish this by using
2419 ``FILESEXTRAPATHS`` from within a ``.bbappend`` file and that you
2420 prepend paths as follows:
2421 ::
2422
2423 FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
2424
2425 In the above example, the build system first
2426 looks for files in a directory that has the same name as the
2427 corresponding append file.
2428
2429 .. note::
2430
2431 When extending ``FILESEXTRAPATHS``, be sure to use the immediate
2432 expansion (``:=``) operator. Immediate expansion makes sure that
2433 BitBake evaluates :term:`THISDIR` at the time the
2434 directive is encountered rather than at some later time when
2435 expansion might result in a directory that does not contain the
2436 files you need.
2437
2438 Also, include the trailing separating colon character if you are
2439 prepending. The trailing colon character is necessary because you
2440 are directing BitBake to extend the path by prepending directories
2441 to the search path.
2442
2443 Here is another common use:
2444 ::
2445
2446 FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
2447
2448 In this example, the build system extends the
2449 ``FILESPATH`` variable to include a directory named ``files`` that is
2450 in the same directory as the corresponding append file.
2451
2452 This next example specifically adds three paths:
2453 ::
2454
2455 FILESEXTRAPATHS_prepend := "path_1:path_2:path_3:"
2456
2457 A final example shows how you can extend the search path and include
2458 a :term:`MACHINE`-specific override, which is useful
2459 in a BSP layer:
2460 ::
2461
2462 FILESEXTRAPATHS_prepend_intel-x86-common := "${THISDIR}/${PN}:"
2463
2464 The previous statement appears in the
2465 ``linux-yocto-dev.bbappend`` file, which is found in the
2466 :ref:`overview-manual/overview-manual-development-environment:yocto project source repositories` in
2467 ``meta-intel/common/recipes-kernel/linux``. Here, the machine
2468 override is a special :term:`PACKAGE_ARCH`
2469 definition for multiple ``meta-intel`` machines.
2470
2471 .. note::
2472
2473 For a layer that supports a single BSP, the override could just be
2474 the value of
2475 MACHINE
2476 .
2477
2478 By prepending paths in ``.bbappend`` files, you allow multiple append
2479 files that reside in different layers but are used for the same
2480 recipe to correctly extend the path.
2481
2482 FILESOVERRIDES
2483 A subset of :term:`OVERRIDES` used by the
2484 OpenEmbedded build system for creating
2485 :term:`FILESPATH`. The ``FILESOVERRIDES`` variable
2486 uses overrides to automatically extend the
2487 :term:`FILESPATH` variable. For an example of how
2488 that works, see the :term:`FILESPATH` variable
2489 description. Additionally, you find more information on how overrides
2490 are handled in the
2491 ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`"
2492 section of the BitBake User Manual.
2493
2494 By default, the ``FILESOVERRIDES`` variable is defined as:
2495 ::
2496
2497 FILESOVERRIDES = "${TRANSLATED_TARGET_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}"
2498
2499 .. note::
2500
2501 Do not hand-edit the
2502 FILESOVERRIDES
2503 variable. The values match up with expected overrides and are used
2504 in an expected manner by the build system.
2505
2506 FILESPATH
2507 The default set of directories the OpenEmbedded build system uses
2508 when searching for patches and files.
2509
2510 During the build process, BitBake searches each directory in
2511 ``FILESPATH`` in the specified order when looking for files and
2512 patches specified by each ``file://`` URI in a recipe's
2513 :term:`SRC_URI` statements.
2514
2515 The default value for the ``FILESPATH`` variable is defined in the
2516 ``base.bbclass`` class found in ``meta/classes`` in the
2517 :term:`Source Directory`:
2518 ::
2519
2520 FILESPATH = "${@base_set_filespath(["${FILE_DIRNAME}/${BP}", \
2521 "${FILE_DIRNAME}/${BPN}", "${FILE_DIRNAME}/files"], d)}"
2522
2523 The
2524 ``FILESPATH`` variable is automatically extended using the overrides
2525 from the :term:`FILESOVERRIDES` variable.
2526
2527 .. note::
2528
2529 - Do not hand-edit the ``FILESPATH`` variable. If you want the
2530 build system to look in directories other than the defaults,
2531 extend the ``FILESPATH`` variable by using the
2532 :term:`FILESEXTRAPATHS` variable.
2533
2534 - Be aware that the default ``FILESPATH`` directories do not map
2535 to directories in custom layers where append files
2536 (``.bbappend``) are used. If you want the build system to find
2537 patches or files that reside with your append files, you need
2538 to extend the ``FILESPATH`` variable by using the
2539 ``FILESEXTRAPATHS`` variable.
2540
2541 You can take advantage of this searching behavior in useful ways. For
2542 example, consider a case where the following directory structure
2543 exists for general and machine-specific configurations:
2544 ::
2545
2546 files/defconfig
2547 files/MACHINEA/defconfig
2548 files/MACHINEB/defconfig
2549
2550 Also in the example, the ``SRC_URI`` statement contains
2551 "file://defconfig". Given this scenario, you can set
2552 :term:`MACHINE` to "MACHINEA" and cause the build
2553 system to use files from ``files/MACHINEA``. Set ``MACHINE`` to
2554 "MACHINEB" and the build system uses files from ``files/MACHINEB``.
2555 Finally, for any machine other than "MACHINEA" and "MACHINEB", the
2556 build system uses files from ``files/defconfig``.
2557
2558 You can find out more about the patching process in the
2559 ":ref:`patching-dev-environment`" section
2560 in the Yocto Project Overview and Concepts Manual and the
2561 ":ref:`new-recipe-patching-code`" section in
2562 the Yocto Project Development Tasks Manual. See the
2563 :ref:`ref-tasks-patch` task as well.
2564
2565 FILESYSTEM_PERMS_TABLES
2566 Allows you to define your own file permissions settings table as part
2567 of your configuration for the packaging process. For example, suppose
2568 you need a consistent set of custom permissions for a set of groups
2569 and users across an entire work project. It is best to do this in the
2570 packages themselves but this is not always possible.
2571
2572 By default, the OpenEmbedded build system uses the ``fs-perms.txt``,
2573 which is located in the ``meta/files`` folder in the :term:`Source Directory`.
2574 If you create your own file
2575 permissions setting table, you should place it in your layer or the
2576 distro's layer.
2577
2578 You define the ``FILESYSTEM_PERMS_TABLES`` variable in the
2579 ``conf/local.conf`` file, which is found in the :term:`Build Directory`,
2580 to point to your custom
2581 ``fs-perms.txt``. You can specify more than a single file permissions
2582 setting table. The paths you specify to these files must be defined
2583 within the :term:`BBPATH` variable.
2584
2585 For guidance on how to create your own file permissions settings
2586 table file, examine the existing ``fs-perms.txt``.
2587
2588 FIT_HASH_ALG
2589 Specifies the hash algorithm used in creating the FIT Image. For e.g. sha256.
2590
2591 FIT_SIGN_ALG
2592 Specifies the signature algorithm used in creating the FIT Image.
2593 For e.g. rsa2048.
2594
2595 FONT_EXTRA_RDEPENDS
2596 When inheriting the :ref:`fontcache <ref-classes-fontcache>` class,
2597 this variable specifies the runtime dependencies for font packages.
2598 By default, the ``FONT_EXTRA_RDEPENDS`` is set to "fontconfig-utils".
2599
2600 FONT_PACKAGES
2601 When inheriting the :ref:`fontcache <ref-classes-fontcache>` class,
2602 this variable identifies packages containing font files that need to
2603 be cached by Fontconfig. By default, the ``fontcache`` class assumes
2604 that fonts are in the recipe's main package (i.e.
2605 ``${``\ :term:`PN`\ ``}``). Use this variable if fonts you
2606 need are in a package other than that main package.
2607
2608 FORCE_RO_REMOVE
2609 Forces the removal of the packages listed in ``ROOTFS_RO_UNNEEDED``
2610 during the generation of the root filesystem.
2611
2612 Set the variable to "1" to force the removal of these packages.
2613
2614 FULL_OPTIMIZATION
2615 The options to pass in ``TARGET_CFLAGS`` and ``CFLAGS`` when
2616 compiling an optimized system. This variable defaults to "-O2 -pipe
2617 ${DEBUG_FLAGS}".
2618
2619 GCCPIE
2620 Enables Position Independent Executables (PIE) within the GNU C
2621 Compiler (GCC). Enabling PIE in the GCC makes Return Oriented
2622 Programming (ROP) attacks much more difficult to execute.
2623
2624 By default the ``security_flags.inc`` file enables PIE by setting the
2625 variable as follows:
2626 ::
2627
2628 GCCPIE ?= "--enable-default-pie"
2629
2630 GCCVERSION
2631 Specifies the default version of the GNU C Compiler (GCC) used for
2632 compilation. By default, ``GCCVERSION`` is set to "8.x" in the
2633 ``meta/conf/distro/include/tcmode-default.inc`` include file:
2634 ::
2635
2636 GCCVERSION ?= "8.%"
2637
2638 You can override this value by setting it in a
2639 configuration file such as the ``local.conf``.
2640
2641 GDB
2642 The minimal command and arguments to run the GNU Debugger.
2643
2644 GITDIR
2645 The directory in which a local copy of a Git repository is stored
2646 when it is cloned.
2647
2648 GLIBC_GENERATE_LOCALES
2649 Specifies the list of GLIBC locales to generate should you not wish
2650 to generate all LIBC locals, which can be time consuming.
2651
2652 .. note::
2653
2654 If you specifically remove the locale
2655 en_US.UTF-8
2656 , you must set
2657 IMAGE_LINGUAS
2658 appropriately.
2659
2660 You can set ``GLIBC_GENERATE_LOCALES`` in your ``local.conf`` file.
2661 By default, all locales are generated.
2662 ::
2663
2664 GLIBC_GENERATE_LOCALES = "en_GB.UTF-8 en_US.UTF-8"
2665
2666 GROUPADD_PARAM
2667 When inheriting the :ref:`useradd <ref-classes-useradd>` class,
2668 this variable specifies for a package what parameters should be
2669 passed to the ``groupadd`` command if you wish to add a group to the
2670 system when the package is installed.
2671
2672 Here is an example from the ``dbus`` recipe:
2673 ::
2674
2675 GROUPADD_PARAM_${PN} = "-r netdev"
2676
2677 For information on the standard Linux shell command
2678 ``groupadd``, see http://linux.die.net/man/8/groupadd.
2679
2680 GROUPMEMS_PARAM
2681 When inheriting the :ref:`useradd <ref-classes-useradd>` class,
2682 this variable specifies for a package what parameters should be
2683 passed to the ``groupmems`` command if you wish to modify the members
2684 of a group when the package is installed.
2685
2686 For information on the standard Linux shell command ``groupmems``,
2687 see http://linux.die.net/man/8/groupmems.
2688
2689 GRUB_GFXSERIAL
2690 Configures the GNU GRand Unified Bootloader (GRUB) to have graphics
2691 and serial in the boot menu. Set this variable to "1" in your
2692 ``local.conf`` or distribution configuration file to enable graphics
2693 and serial in the menu.
2694
2695 See the :ref:`grub-efi <ref-classes-grub-efi>` class for more
2696 information on how this variable is used.
2697
2698 GRUB_OPTS
2699 Additional options to add to the GNU GRand Unified Bootloader (GRUB)
2700 configuration. Use a semi-colon character (``;``) to separate
2701 multiple options.
2702
2703 The ``GRUB_OPTS`` variable is optional. See the
2704 :ref:`grub-efi <ref-classes-grub-efi>` class for more information
2705 on how this variable is used.
2706
2707 GRUB_TIMEOUT
2708 Specifies the timeout before executing the default ``LABEL`` in the
2709 GNU GRand Unified Bootloader (GRUB).
2710
2711 The ``GRUB_TIMEOUT`` variable is optional. See the
2712 :ref:`grub-efi <ref-classes-grub-efi>` class for more information
2713 on how this variable is used.
2714
2715 GTKIMMODULES_PACKAGES
2716 When inheriting the
2717 :ref:`gtk-immodules-cache <ref-classes-gtk-immodules-cache>` class,
2718 this variable specifies the packages that contain the GTK+ input
2719 method modules being installed when the modules are in packages other
2720 than the main package.
2721
2722 HOMEPAGE
2723 Website where more information about the software the recipe is
2724 building can be found.
2725
2726 HOST_ARCH
2727 The name of the target architecture, which is normally the same as
2728 :term:`TARGET_ARCH`. The OpenEmbedded build system
2729 supports many architectures. Here is an example list of architectures
2730 supported. This list is by no means complete as the architecture is
2731 configurable:
2732
2733 - arm
2734 - i586
2735 - x86_64
2736 - powerpc
2737 - powerpc64
2738 - mips
2739 - mipsel
2740
2741 HOST_CC_ARCH
2742 Specifies architecture-specific compiler flags that are passed to the
2743 C compiler.
2744
2745 Default initialization for ``HOST_CC_ARCH`` varies depending on what
2746 is being built:
2747
2748 - :term:`TARGET_CC_ARCH` when building for the
2749 target
2750
2751 - ``BUILD_CC_ARCH`` when building for the build host (i.e.
2752 ``-native``)
2753
2754 - ``BUILDSDK_CC_ARCH`` when building for an SDK (i.e.
2755 ``nativesdk-``)
2756
2757 HOST_OS
2758 Specifies the name of the target operating system, which is normally
2759 the same as the :term:`TARGET_OS`. The variable can
2760 be set to "linux" for ``glibc``-based systems and to "linux-musl" for
2761 ``musl``. For ARM/EABI targets, there are also "linux-gnueabi" and
2762 "linux-musleabi" values possible.
2763
2764 HOST_PREFIX
2765 Specifies the prefix for the cross-compile toolchain. ``HOST_PREFIX``
2766 is normally the same as :term:`TARGET_PREFIX`.
2767
2768 HOST_SYS
2769 Specifies the system, including the architecture and the operating
2770 system, for which the build is occurring in the context of the
2771 current recipe.
2772
2773 The OpenEmbedded build system automatically sets this variable based
2774 on :term:`HOST_ARCH`,
2775 :term:`HOST_VENDOR`, and
2776 :term:`HOST_OS` variables.
2777
2778 .. note::
2779
2780 You do not need to set the variable yourself.
2781
2782 Consider these two examples:
2783
2784 - Given a native recipe on a 32-bit x86 machine running Linux, the
2785 value is "i686-linux".
2786
2787 - Given a recipe being built for a little-endian MIPS target running
2788 Linux, the value might be "mipsel-linux".
2789
2790 HOSTTOOLS
2791 A space-separated list (filter) of tools on the build host that
2792 should be allowed to be called from within build tasks. Using this
2793 filter helps reduce the possibility of host contamination. If a tool
2794 specified in the value of ``HOSTTOOLS`` is not found on the build
2795 host, the OpenEmbedded build system produces an error and the build
2796 is not started.
2797
2798 For additional information, see
2799 :term:`HOSTTOOLS_NONFATAL`.
2800
2801 HOSTTOOLS_NONFATAL
2802 A space-separated list (filter) of tools on the build host that
2803 should be allowed to be called from within build tasks. Using this
2804 filter helps reduce the possibility of host contamination. Unlike
2805 :term:`HOSTTOOLS`, the OpenEmbedded build system
2806 does not produce an error if a tool specified in the value of
2807 ``HOSTTOOLS_NONFATAL`` is not found on the build host. Thus, you can
2808 use ``HOSTTOOLS_NONFATAL`` to filter optional host tools.
2809
2810 HOST_VENDOR
2811 Specifies the name of the vendor. ``HOST_VENDOR`` is normally the
2812 same as :term:`TARGET_VENDOR`.
2813
2814 ICECC_DISABLED
2815 Disables or enables the ``icecc`` (Icecream) function. For more
2816 information on this function and best practices for using this
2817 variable, see the ":ref:`icecc.bbclass <ref-classes-icecc>`"
2818 section.
2819
2820 Setting this variable to "1" in your ``local.conf`` disables the
2821 function:
2822 ::
2823
2824 ICECC_DISABLED ??= "1"
2825
2826 To enable the function, set the variable as follows:
2827 ::
2828
2829 ICECC_DISABLED = ""
2830
2831 ICECC_ENV_EXEC
2832 Points to the ``icecc-create-env`` script that you provide. This
2833 variable is used by the :ref:`icecc <ref-classes-icecc>` class. You
2834 set this variable in your ``local.conf`` file.
2835
2836 If you do not point to a script that you provide, the OpenEmbedded
2837 build system uses the default script provided by the
2838 ``icecc-create-env.bb`` recipe, which is a modified version and not
2839 the one that comes with ``icecc``.
2840
2841 ICECC_PARALLEL_MAKE
2842 Extra options passed to the ``make`` command during the
2843 :ref:`ref-tasks-compile` task that specify parallel
2844 compilation. This variable usually takes the form of "-j x", where x
2845 represents the maximum number of parallel threads ``make`` can run.
2846
2847 .. note::
2848
2849 The options passed affect builds on all enabled machines on the
2850 network, which are machines running the
2851 iceccd
2852 daemon.
2853
2854 If your enabled machines support multiple cores, coming up with the
2855 maximum number of parallel threads that gives you the best
2856 performance could take some experimentation since machine speed,
2857 network lag, available memory, and existing machine loads can all
2858 affect build time. Consequently, unlike the
2859 :term:`PARALLEL_MAKE` variable, there is no
2860 rule-of-thumb for setting ``ICECC_PARALLEL_MAKE`` to achieve optimal
2861 performance.
2862
2863 If you do not set ``ICECC_PARALLEL_MAKE``, the build system does not
2864 use it (i.e. the system does not detect and assign the number of
2865 cores as is done with ``PARALLEL_MAKE``).
2866
2867 ICECC_PATH
2868 The location of the ``icecc`` binary. You can set this variable in
2869 your ``local.conf`` file. If your ``local.conf`` file does not define
2870 this variable, the :ref:`icecc <ref-classes-icecc>` class attempts
2871 to define it by locating ``icecc`` using ``which``.
2872
2873 ICECC_USER_CLASS_BL
2874 Identifies user classes that you do not want the Icecream distributed
2875 compile support to consider. This variable is used by the
2876 :ref:`icecc <ref-classes-icecc>` class. You set this variable in
2877 your ``local.conf`` file.
2878
2879 When you list classes using this variable, you are "blacklisting"
2880 them from distributed compilation across remote hosts. Any classes
2881 you list will be distributed and compiled locally.
2882
2883 ICECC_USER_PACKAGE_BL
2884 Identifies user recipes that you do not want the Icecream distributed
2885 compile support to consider. This variable is used by the
2886 :ref:`icecc <ref-classes-icecc>` class. You set this variable in
2887 your ``local.conf`` file.
2888
2889 When you list packages using this variable, you are "blacklisting"
2890 them from distributed compilation across remote hosts. Any packages
2891 you list will be distributed and compiled locally.
2892
2893 ICECC_USER_PACKAGE_WL
2894 Identifies user recipes that use an empty
2895 :term:`PARALLEL_MAKE` variable that you want to
2896 force remote distributed compilation on using the Icecream
2897 distributed compile support. This variable is used by the
2898 :ref:`icecc <ref-classes-icecc>` class. You set this variable in
2899 your ``local.conf`` file.
2900
2901 IMAGE_BASENAME
2902 The base name of image output files. This variable defaults to the
2903 recipe name (``${``\ :term:`PN`\ ``}``).
2904
2905 IMAGE_BOOT_FILES
2906 A space-separated list of files installed into the boot partition
2907 when preparing an image using the Wic tool with the
2908 ``bootimg-partition`` or ``bootimg-efi`` source plugin. By default,
2909 the files are
2910 installed under the same name as the source files. To change the
2911 installed name, separate it from the original name with a semi-colon
2912 (;). Source files need to be located in
2913 :term:`DEPLOY_DIR_IMAGE`. Here are two
2914 examples:
2915 ::
2916
2917 IMAGE_BOOT_FILES = "u-boot.img uImage;kernel"
2918 IMAGE_BOOT_FILES = "u-boot.${UBOOT_SUFFIX} ${KERNEL_IMAGETYPE}"
2919
2920 Alternatively, source files can be picked up using a glob pattern. In
2921 this case, the destination file must have the same name as the base
2922 name of the source file path. To install files into a directory
2923 within the target location, pass its name after a semi-colon (;).
2924 Here are two examples:
2925 ::
2926
2927 IMAGE_BOOT_FILES = "bcm2835-bootfiles/*"
2928 IMAGE_BOOT_FILES = "bcm2835-bootfiles/*;boot/"
2929
2930 The first example
2931 installs all files from ``${DEPLOY_DIR_IMAGE}/bcm2835-bootfiles``
2932 into the root of the target partition. The second example installs
2933 the same files into a ``boot`` directory within the target partition.
2934
2935 You can find information on how to use the Wic tool in the
2936 ":ref:`dev-manual/dev-manual-common-tasks:creating partitioned images using wic`"
2937 section of the Yocto Project Development Tasks Manual. Reference
2938 material for Wic is located in the
2939 ":doc:`../ref-manual/ref-kickstart`" chapter.
2940
2941 IMAGE_CLASSES
2942 A list of classes that all images should inherit. You typically use
2943 this variable to specify the list of classes that register the
2944 different types of images the OpenEmbedded build system creates.
2945
2946 The default value for ``IMAGE_CLASSES`` is ``image_types``. You can
2947 set this variable in your ``local.conf`` or in a distribution
2948 configuration file.
2949
2950 For more information, see ``meta/classes/image_types.bbclass`` in the
2951 :term:`Source Directory`.
2952
2953 IMAGE_CMD
2954 Specifies the command to create the image file for a specific image
2955 type, which corresponds to the value set set in
2956 :term:`IMAGE_FSTYPES`, (e.g. ``ext3``,
2957 ``btrfs``, and so forth). When setting this variable, you should use
2958 an override for the associated type. Here is an example:
2959 ::
2960
2961 IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} \
2962 --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 \
2963 ${EXTRA_IMAGECMD}"
2964
2965 You typically do not need to set this variable unless you are adding
2966 support for a new image type. For more examples on how to set this
2967 variable, see the :ref:`image_types <ref-classes-image_types>`
2968 class file, which is ``meta/classes/image_types.bbclass``.
2969
2970 IMAGE_DEVICE_TABLES
2971 Specifies one or more files that contain custom device tables that
2972 are passed to the ``makedevs`` command as part of creating an image.
2973 These files list basic device nodes that should be created under
2974 ``/dev`` within the image. If ``IMAGE_DEVICE_TABLES`` is not set,
2975 ``files/device_table-minimal.txt`` is used, which is located by
2976 :term:`BBPATH`. For details on how you should write
2977 device table files, see ``meta/files/device_table-minimal.txt`` as an
2978 example.
2979
2980 IMAGE_FEATURES
2981 The primary list of features to include in an image. Typically, you
2982 configure this variable in an image recipe. Although you can use this
2983 variable from your ``local.conf`` file, which is found in the
2984 :term:`Build Directory`, best practices dictate that you do
2985 not.
2986
2987 .. note::
2988
2989 To enable extra features from outside the image recipe, use the
2990 EXTRA_IMAGE_FEATURES
2991 variable.
2992
2993 For a list of image features that ships with the Yocto Project, see
2994 the "`Image Features <#ref-features-image>`__" section.
2995
2996 For an example that shows how to customize your image by using this
2997 variable, see the ":ref:`usingpoky-extend-customimage-imagefeatures`"
2998 section in the Yocto Project Development Tasks Manual.
2999
3000 IMAGE_FSTYPES
3001 Specifies the formats the OpenEmbedded build system uses during the
3002 build when creating the root filesystem. For example, setting
3003 ``IMAGE_FSTYPES`` as follows causes the build system to create root
3004 filesystems using two formats: ``.ext3`` and ``.tar.bz2``:
3005 ::
3006
3007 IMAGE_FSTYPES = "ext3 tar.bz2"
3008
3009 For the complete list of supported image formats from which you can
3010 choose, see :term:`IMAGE_TYPES`.
3011
3012 .. note::
3013
3014 - If an image recipe uses the "inherit image" line and you are
3015 setting ``IMAGE_FSTYPES`` inside the recipe, you must set
3016 ``IMAGE_FSTYPES`` prior to using the "inherit image" line.
3017
3018 - Due to the way the OpenEmbedded build system processes this
3019 variable, you cannot update its contents by using ``_append``
3020 or ``_prepend``. You must use the ``+=`` operator to add one or
3021 more options to the ``IMAGE_FSTYPES`` variable.
3022
3023 IMAGE_INSTALL
3024 Used by recipes to specify the packages to install into an image
3025 through the :ref:`image <ref-classes-image>` class. Use the
3026 ``IMAGE_INSTALL`` variable with care to avoid ordering issues.
3027
3028 Image recipes set ``IMAGE_INSTALL`` to specify the packages to
3029 install into an image through ``image.bbclass``. Additionally,
3030 "helper" classes such as the
3031 :ref:`core-image <ref-classes-core-image>` class exist that can
3032 take lists used with ``IMAGE_FEATURES`` and turn them into
3033 auto-generated entries in ``IMAGE_INSTALL`` in addition to its
3034 default contents.
3035
3036 When you use this variable, it is best to use it as follows:
3037 ::
3038
3039 IMAGE_INSTALL_append = " package-name"
3040
3041 Be sure to include the space
3042 between the quotation character and the start of the package name or
3043 names.
3044
3045 .. note::
3046
3047 - When working with a
3048 ```core-image-minimal-initramfs`` <#images-core-image-minimal-initramfs>`__
3049 image, do not use the ``IMAGE_INSTALL`` variable to specify
3050 packages for installation. Instead, use the
3051 :term:`PACKAGE_INSTALL` variable, which
3052 allows the initial RAM filesystem (initramfs) recipe to use a
3053 fixed set of packages and not be affected by ``IMAGE_INSTALL``.
3054 For information on creating an initramfs, see the
3055 ":ref:`building-an-initramfs-image`"
3056 section in the Yocto Project Development Tasks Manual.
3057
3058 - Using ``IMAGE_INSTALL`` with the
3059 :ref:`+= <bitbake:appending-and-prepending>`
3060 BitBake operator within the ``/conf/local.conf`` file or from
3061 within an image recipe is not recommended. Use of this operator
3062 in these ways can cause ordering issues. Since
3063 ``core-image.bbclass`` sets ``IMAGE_INSTALL`` to a default
3064 value using the
3065 :ref:`?= <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:setting a default value (?=)>`
3066 operator, using a ``+=`` operation against ``IMAGE_INSTALL``
3067 results in unexpected behavior when used within
3068 ``conf/local.conf``. Furthermore, the same operation from
3069 within an image recipe may or may not succeed depending on the
3070 specific situation. In both these cases, the behavior is
3071 contrary to how most users expect the ``+=`` operator to work.
3072
3073 IMAGE_LINGUAS
3074 Specifies the list of locales to install into the image during the
3075 root filesystem construction process. The OpenEmbedded build system
3076 automatically splits locale files, which are used for localization,
3077 into separate packages. Setting the ``IMAGE_LINGUAS`` variable
3078 ensures that any locale packages that correspond to packages already
3079 selected for installation into the image are also installed. Here is
3080 an example:
3081 ::
3082
3083 IMAGE_LINGUAS = "pt-br de-de"
3084
3085 In this example, the build system ensures any Brazilian Portuguese
3086 and German locale files that correspond to packages in the image are
3087 installed (i.e. ``*-locale-pt-br`` and ``*-locale-de-de`` as well as
3088 ``*-locale-pt`` and ``*-locale-de``, since some software packages
3089 only provide locale files by language and not by country-specific
3090 language).
3091
3092 See the :term:`GLIBC_GENERATE_LOCALES`
3093 variable for information on generating GLIBC locales.
3094
3095 IMAGE_MANIFEST
3096 The manifest file for the image. This file lists all the installed
3097 packages that make up the image. The file contains package
3098 information on a line-per-package basis as follows:
3099 ::
3100
3101 packagename packagearch version
3102
3103 The :ref:`image <ref-classes-image>` class defines the manifest
3104 file as follows:
3105 ::
3106
3107 IMAGE_MANIFEST ="${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.manifest"
3108
3109 The location is
3110 derived using the :term:`DEPLOY_DIR_IMAGE`
3111 and :term:`IMAGE_NAME` variables. You can find
3112 information on how the image is created in the ":ref:`image-generation-dev-environment`"
3113 section in the Yocto Project Overview and Concepts Manual.
3114
3115 IMAGE_NAME
3116 The name of the output image files minus the extension. This variable
3117 is derived using the :term:`IMAGE_BASENAME`,
3118 :term:`MACHINE`, and :term:`DATETIME`
3119 variables:
3120 ::
3121
3122 IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
3123
3124 IMAGE_OVERHEAD_FACTOR
3125 Defines a multiplier that the build system applies to the initial
3126 image size for cases when the multiplier times the returned disk
3127 usage value for the image is greater than the sum of
3128 ``IMAGE_ROOTFS_SIZE`` and ``IMAGE_ROOTFS_EXTRA_SPACE``. The result of
3129 the multiplier applied to the initial image size creates free disk
3130 space in the image as overhead. By default, the build process uses a
3131 multiplier of 1.3 for this variable. This default value results in
3132 30% free disk space added to the image when this method is used to
3133 determine the final generated image size. You should be aware that
3134 post install scripts and the package management system uses disk
3135 space inside this overhead area. Consequently, the multiplier does
3136 not produce an image with all the theoretical free disk space. See
3137 ``IMAGE_ROOTFS_SIZE`` for information on how the build system
3138 determines the overall image size.
3139
3140 The default 30% free disk space typically gives the image enough room
3141 to boot and allows for basic post installs while still leaving a
3142 small amount of free disk space. If 30% free space is inadequate, you
3143 can increase the default value. For example, the following setting
3144 gives you 50% free space added to the image:
3145 ::
3146
3147 IMAGE_OVERHEAD_FACTOR = "1.5"
3148
3149 Alternatively, you can ensure a specific amount of free disk space is
3150 added to the image by using the ``IMAGE_ROOTFS_EXTRA_SPACE``
3151 variable.
3152
3153 IMAGE_PKGTYPE
3154 Defines the package type (i.e. DEB, RPM, IPK, or TAR) used by the
3155 OpenEmbedded build system. The variable is defined appropriately by
3156 the :ref:`package_deb <ref-classes-package_deb>`,
3157 :ref:`package_rpm <ref-classes-package_rpm>`,
3158 :ref:`package_ipk <ref-classes-package_ipk>`, or
3159 :ref:`package_tar <ref-classes-package_tar>` class.
3160
3161 .. note::
3162
3163 The
3164 package_tar
3165 class is broken and is not supported. It is recommended that you
3166 do not use it.
3167
3168 The :ref:`populate_sdk_* <ref-classes-populate-sdk-*>` and
3169 :ref:`image <ref-classes-image>` classes use the ``IMAGE_PKGTYPE``
3170 for packaging up images and SDKs.
3171
3172 You should not set the ``IMAGE_PKGTYPE`` manually. Rather, the
3173 variable is set indirectly through the appropriate
3174 :ref:`package_* <ref-classes-package>` class using the
3175 :term:`PACKAGE_CLASSES` variable. The
3176 OpenEmbedded build system uses the first package type (e.g. DEB, RPM,
3177 or IPK) that appears with the variable
3178
3179 .. note::
3180
3181 Files using the
3182 .tar
3183 format are never used as a substitute packaging format for DEB,
3184 RPM, and IPK formatted files for your image or SDK.
3185
3186 IMAGE_POSTPROCESS_COMMAND
3187 Specifies a list of functions to call once the OpenEmbedded build
3188 system creates the final image output files. You can specify
3189 functions separated by semicolons:
3190 ::
3191
3192 IMAGE_POSTPROCESS_COMMAND += "function; ... "
3193
3194 If you need to pass the root filesystem path to a command within the
3195 function, you can use ``${IMAGE_ROOTFS}``, which points to the
3196 directory that becomes the root filesystem image. See the
3197 :term:`IMAGE_ROOTFS` variable for more
3198 information.
3199
3200 IMAGE_PREPROCESS_COMMAND
3201 Specifies a list of functions to call before the OpenEmbedded build
3202 system creates the final image output files. You can specify
3203 functions separated by semicolons:
3204 ::
3205
3206 IMAGE_PREPROCESS_COMMAND += "function; ... "
3207
3208 If you need to pass the root filesystem path to a command within the
3209 function, you can use ``${IMAGE_ROOTFS}``, which points to the
3210 directory that becomes the root filesystem image. See the
3211 :term:`IMAGE_ROOTFS` variable for more
3212 information.
3213
3214 IMAGE_ROOTFS
3215 The location of the root filesystem while it is under construction
3216 (i.e. during the :ref:`ref-tasks-rootfs` task). This
3217 variable is not configurable. Do not change it.
3218
3219 IMAGE_ROOTFS_ALIGNMENT
3220 Specifies the alignment for the output image file in Kbytes. If the
3221 size of the image is not a multiple of this value, then the size is
3222 rounded up to the nearest multiple of the value. The default value is
3223 "1". See :term:`IMAGE_ROOTFS_SIZE` for
3224 additional information.
3225
3226 IMAGE_ROOTFS_EXTRA_SPACE
3227 Defines additional free disk space created in the image in Kbytes. By
3228 default, this variable is set to "0". This free disk space is added
3229 to the image after the build system determines the image size as
3230 described in ``IMAGE_ROOTFS_SIZE``.
3231
3232 This variable is particularly useful when you want to ensure that a
3233 specific amount of free disk space is available on a device after an
3234 image is installed and running. For example, to be sure 5 Gbytes of
3235 free disk space is available, set the variable as follows:
3236 ::
3237
3238 IMAGE_ROOTFS_EXTRA_SPACE = "5242880"
3239
3240 For example, the Yocto Project Build Appliance specifically requests
3241 40 Gbytes of extra space with the line:
3242 ::
3243
3244 IMAGE_ROOTFS_EXTRA_SPACE = "41943040"
3245
3246 IMAGE_ROOTFS_SIZE
3247 Defines the size in Kbytes for the generated image. The OpenEmbedded
3248 build system determines the final size for the generated image using
3249 an algorithm that takes into account the initial disk space used for
3250 the generated image, a requested size for the image, and requested
3251 additional free disk space to be added to the image. Programatically,
3252 the build system determines the final size of the generated image as
3253 follows:
3254 ::
3255
3256 if (image-du * overhead) < rootfs-size:
3257 internal-rootfs-size = rootfs-size + xspace
3258 else:
3259 internal-rootfs-size = (image-du * overhead) + xspace
3260 where:
3261 image-du = Returned value of the du command on the image.
3262 overhead = IMAGE_OVERHEAD_FACTOR
3263 rootfs-size = IMAGE_ROOTFS_SIZE
3264 internal-rootfs-size = Initial root filesystem size before any modifications.
3265 xspace = IMAGE_ROOTFS_EXTRA_SPACE
3266
3267 See the :term:`IMAGE_OVERHEAD_FACTOR`
3268 and :term:`IMAGE_ROOTFS_EXTRA_SPACE`
3269 variables for related information.
3270
3271 IMAGE_TYPEDEP
3272 Specifies a dependency from one image type on another. Here is an
3273 example from the :ref:`image-live <ref-classes-image-live>` class:
3274 ::
3275
3276 IMAGE_TYPEDEP_live = "ext3"
3277
3278 In the previous example, the variable ensures that when "live" is
3279 listed with the :term:`IMAGE_FSTYPES` variable,
3280 the OpenEmbedded build system produces an ``ext3`` image first since
3281 one of the components of the live image is an ``ext3`` formatted
3282 partition containing the root filesystem.
3283
3284 IMAGE_TYPES
3285 Specifies the complete list of supported image types by default:
3286
3287 - btrfs
3288 - container
3289 - cpio
3290 - cpio.gz
3291 - cpio.lz4
3292 - cpio.lzma
3293 - cpio.xz
3294 - cramfs
3295 - ext2
3296 - ext2.bz2
3297 - ext2.gz
3298 - ext2.lzma
3299 - ext3
3300 - ext3.gz
3301 - ext4
3302 - ext4.gz
3303 - f2fs
3304 - hddimg
3305 - iso
3306 - jffs2
3307 - jffs2.sum
3308 - multiubi
3309 - squashfs
3310 - squashfs-lz4
3311 - squashfs-lzo
3312 - squashfs-xz
3313 - tar
3314 - tar.bz2
3315 - tar.gz
3316 - tar.lz4
3317 - tar.xz
3318 - tar.zst
3319 - ubi
3320 - ubifs
3321 - wic
3322 - wic.bz2
3323 - wic.gz
3324 - wic.lzma
3325
3326 For more information about these types of images, see
3327 ``meta/classes/image_types*.bbclass`` in the :term:`Source Directory`.
3328
3329 INC_PR
3330 Helps define the recipe revision for recipes that share a common
3331 ``include`` file. You can think of this variable as part of the
3332 recipe revision as set from within an include file.
3333
3334 Suppose, for example, you have a set of recipes that are used across
3335 several projects. And, within each of those recipes the revision (its
3336 :term:`PR` value) is set accordingly. In this case, when
3337 the revision of those recipes changes, the burden is on you to find
3338 all those recipes and be sure that they get changed to reflect the
3339 updated version of the recipe. In this scenario, it can get
3340 complicated when recipes that are used in many places and provide
3341 common functionality are upgraded to a new revision.
3342
3343 A more efficient way of dealing with this situation is to set the
3344 ``INC_PR`` variable inside the ``include`` files that the recipes
3345 share and then expand the ``INC_PR`` variable within the recipes to
3346 help define the recipe revision.
3347
3348 The following provides an example that shows how to use the
3349 ``INC_PR`` variable given a common ``include`` file that defines the
3350 variable. Once the variable is defined in the ``include`` file, you
3351 can use the variable to set the ``PR`` values in each recipe. You
3352 will notice that when you set a recipe's ``PR`` you can provide more
3353 granular revisioning by appending values to the ``INC_PR`` variable:
3354 ::
3355
3356 recipes-graphics/xorg-font/xorg-font-common.inc:INC_PR = "r2"
3357 recipes-graphics/xorg-font/encodings_1.0.4.bb:PR = "${INC_PR}.1"
3358 recipes-graphics/xorg-font/font-util_1.3.0.bb:PR = "${INC_PR}.0"
3359 recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
3360
3361 The
3362 first line of the example establishes the baseline revision to be
3363 used for all recipes that use the ``include`` file. The remaining
3364 lines in the example are from individual recipes and show how the
3365 ``PR`` value is set.
3366
3367 INCOMPATIBLE_LICENSE
3368 Specifies a space-separated list of license names (as they would
3369 appear in :term:`LICENSE`) that should be excluded
3370 from the build. Recipes that provide no alternatives to listed
3371 incompatible licenses are not built. Packages that are individually
3372 licensed with the specified incompatible licenses will be deleted.
3373
3374 .. note::
3375
3376 This functionality is only regularly tested using the following
3377 setting:
3378 ::
3379
3380 INCOMPATIBLE_LICENSE = "GPL-3.0 LGPL-3.0 AGPL-3.0"
3381
3382
3383 Although you can use other settings, you might be required to
3384 remove dependencies on or provide alternatives to components that
3385 are required to produce a functional system image.
3386
3387 .. note::
3388
3389 It is possible to define a list of licenses that are allowed to be
3390 used instead of the licenses that are excluded. To do this, define
3391 a variable
3392 COMPATIBLE_LICENSES
3393 with the names of the licences that are allowed. Then define
3394 INCOMPATIBLE_LICENSE
3395 as:
3396 ::
3397
3398 INCOMPATIBLE_LICENSE = "${@' '.join(sorted(set(d.getVar('AVAILABLE_LICENSES').split()) - set(d.getVar('COMPATIBLE_LICENSES').split())))}"
3399
3400
3401 This will result in
3402 INCOMPATIBLE_LICENSE
3403 containing the names of all licences from
3404 AVAILABLE_LICENSES
3405 except the ones specified in
3406 COMPATIBLE_LICENSES
3407 , thus only allowing the latter licences to be used.
3408
3409 INHERIT
3410 Causes the named class or classes to be inherited globally. Anonymous
3411 functions in the class or classes are not executed for the base
3412 configuration and in each individual recipe. The OpenEmbedded build
3413 system ignores changes to ``INHERIT`` in individual recipes.
3414
3415 For more information on ``INHERIT``, see the
3416 :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` configuration directive`"
3417 section in the Bitbake User Manual.
3418
3419 INHERIT_DISTRO
3420 Lists classes that will be inherited at the distribution level. It is
3421 unlikely that you want to edit this variable.
3422
3423 The default value of the variable is set as follows in the
3424 ``meta/conf/distro/defaultsetup.conf`` file:
3425 ::
3426
3427 INHERIT_DISTRO ?= "debian devshell sstate license"
3428
3429 INHIBIT_DEFAULT_DEPS
3430 Prevents the default dependencies, namely the C compiler and standard
3431 C library (libc), from being added to :term:`DEPENDS`.
3432 This variable is usually used within recipes that do not require any
3433 compilation using the C compiler.
3434
3435 Set the variable to "1" to prevent the default dependencies from
3436 being added.
3437
3438 INHIBIT_PACKAGE_DEBUG_SPLIT
3439 Prevents the OpenEmbedded build system from splitting out debug
3440 information during packaging. By default, the build system splits out
3441 debugging information during the
3442 :ref:`ref-tasks-package` task. For more information on
3443 how debug information is split out, see the
3444 :term:`PACKAGE_DEBUG_SPLIT_STYLE`
3445 variable.
3446
3447 To prevent the build system from splitting out debug information
3448 during packaging, set the ``INHIBIT_PACKAGE_DEBUG_SPLIT`` variable as
3449 follows:
3450 ::
3451
3452 INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
3453
3454 INHIBIT_PACKAGE_STRIP
3455 If set to "1", causes the build to not strip binaries in resulting
3456 packages and prevents the ``-dbg`` package from containing the source
3457 files.
3458
3459 By default, the OpenEmbedded build system strips binaries and puts
3460 the debugging symbols into ``${``\ :term:`PN`\ ``}-dbg``.
3461 Consequently, you should not set ``INHIBIT_PACKAGE_STRIP`` when you
3462 plan to debug in general.
3463
3464 INHIBIT_SYSROOT_STRIP
3465 If set to "1", causes the build to not strip binaries in the
3466 resulting sysroot.
3467
3468 By default, the OpenEmbedded build system strips binaries in the
3469 resulting sysroot. When you specifically set the
3470 ``INHIBIT_SYSROOT_STRIP`` variable to "1" in your recipe, you inhibit
3471 this stripping.
3472
3473 If you want to use this variable, include the
3474 :ref:`staging <ref-classes-staging>` class. This class uses a
3475 ``sys_strip()`` function to test for the variable and acts
3476 accordingly.
3477
3478 .. note::
3479
3480 Use of the
3481 INHIBIT_SYSROOT_STRIP
3482 variable occurs in rare and special circumstances. For example,
3483 suppose you are building bare-metal firmware by using an external
3484 GCC toolchain. Furthermore, even if the toolchain's binaries are
3485 strippable, other files exist that are needed for the build that
3486 are not strippable.
3487
3488 INITRAMFS_FSTYPES
3489 Defines the format for the output image of an initial RAM filesystem
3490 (initramfs), which is used during boot. Supported formats are the
3491 same as those supported by the
3492 :term:`IMAGE_FSTYPES` variable.
3493
3494 The default value of this variable, which is set in the
3495 ``meta/conf/bitbake.conf`` configuration file in the
3496 :term:`Source Directory`, is "cpio.gz". The Linux kernel's
3497 initramfs mechanism, as opposed to the initial RAM filesystem
3498 `initrd <https://en.wikipedia.org/wiki/Initrd>`__ mechanism, expects
3499 an optionally compressed cpio archive.
3500
3501 INITRAMFS_IMAGE
3502 Specifies the :term:`PROVIDES` name of an image
3503 recipe that is used to build an initial RAM filesystem (initramfs)
3504 image. In other words, the ``INITRAMFS_IMAGE`` variable causes an
3505 additional recipe to be built as a dependency to whatever root
3506 filesystem recipe you might be using (e.g. ``core-image-sato``). The
3507 initramfs image recipe you provide should set
3508 :term:`IMAGE_FSTYPES` to
3509 :term:`INITRAMFS_FSTYPES`.
3510
3511 An initramfs image provides a temporary root filesystem used for
3512 early system initialization (e.g. loading of modules needed to locate
3513 and mount the "real" root filesystem).
3514
3515 .. note::
3516
3517 See the
3518 meta/recipes-core/images/core-image-minimal-initramfs.bb
3519 recipe in the
3520 Source Directory
3521 for an example initramfs recipe. To select this sample recipe as
3522 the one built to provide the initramfs image, set
3523 INITRAMFS_IMAGE
3524 to "core-image-minimal-initramfs".
3525
3526 You can also find more information by referencing the
3527 ``meta-poky/conf/local.conf.sample.extended`` configuration file in
3528 the Source Directory, the :ref:`image <ref-classes-image>` class,
3529 and the :ref:`kernel <ref-classes-kernel>` class to see how to use
3530 the ``INITRAMFS_IMAGE`` variable.
3531
3532 If ``INITRAMFS_IMAGE`` is empty, which is the default, then no
3533 initramfs image is built.
3534
3535 For more information, you can also see the
3536 :term:`INITRAMFS_IMAGE_BUNDLE`
3537 variable, which allows the generated image to be bundled inside the
3538 kernel image. Additionally, for information on creating an initramfs
3539 image, see the ":ref:`building-an-initramfs-image`" section
3540 in the Yocto Project Development Tasks Manual.
3541
3542 INITRAMFS_IMAGE_BUNDLE
3543 Controls whether or not the image recipe specified by
3544 :term:`INITRAMFS_IMAGE` is run through an
3545 extra pass
3546 (:ref:`ref-tasks-bundle_initramfs`) during
3547 kernel compilation in order to build a single binary that contains
3548 both the kernel image and the initial RAM filesystem (initramfs)
3549 image. This makes use of the
3550 :term:`CONFIG_INITRAMFS_SOURCE` kernel
3551 feature.
3552
3553 .. note::
3554
3555 Using an extra compilation pass to bundle the initramfs avoids a
3556 circular dependency between the kernel recipe and the initramfs
3557 recipe should the initramfs include kernel modules. Should that be
3558 the case, the initramfs recipe depends on the kernel for the
3559 kernel modules, and the kernel depends on the initramfs recipe
3560 since the initramfs is bundled inside the kernel image.
3561
3562 The combined binary is deposited into the ``tmp/deploy`` directory,
3563 which is part of the :term:`Build Directory`.
3564
3565 Setting the variable to "1" in a configuration file causes the
3566 OpenEmbedded build system to generate a kernel image with the
3567 initramfs specified in ``INITRAMFS_IMAGE`` bundled within:
3568 ::
3569
3570 INITRAMFS_IMAGE_BUNDLE = "1"
3571
3572 By default, the
3573 :ref:`kernel <ref-classes-kernel>` class sets this variable to a
3574 null string as follows:
3575 ::
3576
3577 INITRAMFS_IMAGE_BUNDLE ?= ""
3578
3579 .. note::
3580
3581 You must set the
3582 INITRAMFS_IMAGE_BUNDLE
3583 variable in a configuration file. You cannot set the variable in a
3584 recipe file.
3585
3586 See the
3587 :yocto_git:`local.conf.sample.extended </cgit/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample.extended>`
3588 file for additional information. Also, for information on creating an
3589 initramfs, see the ":ref:`building-an-initramfs-image`" section
3590 in the Yocto Project Development Tasks Manual.
3591
3592 INITRAMFS_LINK_NAME
3593 The link name of the initial RAM filesystem image. This variable is
3594 set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3595 follows:
3596 ::
3597
3598 INITRAMFS_LINK_NAME ?= "initramfs-${KERNEL_ARTIFACT_LINK_NAME}"
3599
3600 The value of the
3601 ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3602 file, has the following value:
3603 ::
3604
3605 KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
3606
3607 See the :term:`MACHINE` variable for additional
3608 information.
3609
3610 INITRAMFS_NAME
3611 The base name of the initial RAM filesystem image. This variable is
3612 set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3613 follows:
3614 ::
3615
3616 INITRAMFS_NAME ?= "initramfs-${KERNEL_ARTIFACT_NAME}"
3617
3618 The value of the :term:`KERNEL_ARTIFACT_NAME`
3619 variable, which is set in the same file, has the following value:
3620 ::
3621
3622 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3623
3624 INITRD
3625 Indicates list of filesystem images to concatenate and use as an
3626 initial RAM disk (``initrd``).
3627
3628 The ``INITRD`` variable is an optional variable used with the
3629 :ref:`image-live <ref-classes-image-live>` class.
3630
3631 INITRD_IMAGE
3632 When building a "live" bootable image (i.e. when
3633 :term:`IMAGE_FSTYPES` contains "live"),
3634 ``INITRD_IMAGE`` specifies the image recipe that should be built to
3635 provide the initial RAM disk image. The default value is
3636 "core-image-minimal-initramfs".
3637
3638 See the :ref:`image-live <ref-classes-image-live>` class for more
3639 information.
3640
3641 INITSCRIPT_NAME
3642 The filename of the initialization script as installed to
3643 ``${sysconfdir}/init.d``.
3644
3645 This variable is used in recipes when using ``update-rc.d.bbclass``.
3646 The variable is mandatory.
3647
3648 INITSCRIPT_PACKAGES
3649 A list of the packages that contain initscripts. If multiple packages
3650 are specified, you need to append the package name to the other
3651 ``INITSCRIPT_*`` as an override.
3652
3653 This variable is used in recipes when using ``update-rc.d.bbclass``.
3654 The variable is optional and defaults to the :term:`PN`
3655 variable.
3656
3657 INITSCRIPT_PARAMS
3658 Specifies the options to pass to ``update-rc.d``. Here is an example:
3659 ::
3660
3661 INITSCRIPT_PARAMS = "start 99 5 2 . stop 20 0 1 6 ."
3662
3663 In this example, the script has a runlevel of 99, starts the script
3664 in initlevels 2 and 5, and stops the script in levels 0, 1 and 6.
3665
3666 The variable's default value is "defaults", which is set in the
3667 :ref:`update-rc.d <ref-classes-update-rc.d>` class.
3668
3669 The value in ``INITSCRIPT_PARAMS`` is passed through to the
3670 ``update-rc.d`` command. For more information on valid parameters,
3671 please see the ``update-rc.d`` manual page at
3672 https://manpages.debian.org/buster/init-system-helpers/update-rc.d.8.en.html
3673
3674 INSANE_SKIP
3675 Specifies the QA checks to skip for a specific package within a
3676 recipe. For example, to skip the check for symbolic link ``.so``
3677 files in the main package of a recipe, add the following to the
3678 recipe. The package name override must be used, which in this example
3679 is ``${PN}``:
3680 ::
3681
3682 INSANE_SKIP_${PN} += "dev-so"
3683
3684 See the ":ref:`insane.bbclass <ref-classes-insane>`" section for a
3685 list of the valid QA checks you can specify using this variable.
3686
3687 INSTALL_TIMEZONE_FILE
3688 By default, the ``tzdata`` recipe packages an ``/etc/timezone`` file.
3689 Set the ``INSTALL_TIMEZONE_FILE`` variable to "0" at the
3690 configuration level to disable this behavior.
3691
3692 IPK_FEED_URIS
3693 When the IPK backend is in use and package management is enabled on
3694 the target, you can use this variable to set up ``opkg`` in the
3695 target image to point to package feeds on a nominated server. Once
3696 the feed is established, you can perform installations or upgrades
3697 using the package manager at runtime.
3698
3699 KARCH
3700 Defines the kernel architecture used when assembling the
3701 configuration. Architectures supported for this release are:
3702
3703 - powerpc
3704 - i386
3705 - x86_64
3706 - arm
3707 - qemu
3708 - mips
3709
3710 You define the ``KARCH`` variable in the :ref:`kernel-dev/kernel-dev-advanced:bsp descriptions`.
3711
3712 KBRANCH
3713 A regular expression used by the build process to explicitly identify
3714 the kernel branch that is validated, patched, and configured during a
3715 build. You must set this variable to ensure the exact kernel branch
3716 you want is being used by the build process.
3717
3718 Values for this variable are set in the kernel's recipe file and the
3719 kernel's append file. For example, if you are using the
3720 ``linux-yocto_4.12`` kernel, the kernel recipe file is the
3721 ``meta/recipes-kernel/linux/linux-yocto_4.12.bb`` file. ``KBRANCH``
3722 is set as follows in that kernel recipe file:
3723 ::
3724
3725 KBRANCH ?= "standard/base"
3726
3727 This variable is also used from the kernel's append file to identify
3728 the kernel branch specific to a particular machine or target
3729 hardware. Continuing with the previous kernel example, the kernel's
3730 append file (i.e. ``linux-yocto_4.12.bbappend``) is located in the
3731 BSP layer for a given machine. For example, the append file for the
3732 Beaglebone, EdgeRouter, and generic versions of both 32 and 64-bit IA
3733 machines (``meta-yocto-bsp``) is named
3734 ``meta-yocto-bsp/recipes-kernel/linux/linux-yocto_4.12.bbappend``.
3735 Here are the related statements from that append file:
3736 ::
3737
3738 KBRANCH_genericx86 = "standard/base"
3739 KBRANCH_genericx86-64 = "standard/base"
3740 KBRANCH_edgerouter = "standard/edgerouter"
3741 KBRANCH_beaglebone = "standard/beaglebone"
3742
3743 The ``KBRANCH`` statements
3744 identify the kernel branch to use when building for each supported
3745 BSP.
3746
3747 KBUILD_DEFCONFIG
3748 When used with the :ref:`kernel-yocto <ref-classes-kernel-yocto>`
3749 class, specifies an "in-tree" kernel configuration file for use
3750 during a kernel build.
3751
3752 Typically, when using a ``defconfig`` to configure a kernel during a
3753 build, you place the file in your layer in the same manner as you
3754 would place patch files and configuration fragment files (i.e.
3755 "out-of-tree"). However, if you want to use a ``defconfig`` file that
3756 is part of the kernel tree (i.e. "in-tree"), you can use the
3757 ``KBUILD_DEFCONFIG`` variable and append the
3758 :term:`KMACHINE` variable to point to the
3759 ``defconfig`` file.
3760
3761 To use the variable, set it in the append file for your kernel recipe
3762 using the following form:
3763 ::
3764
3765 KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file
3766
3767 Here is an example from a "raspberrypi2" ``KMACHINE`` build that uses
3768 a ``defconfig`` file named "bcm2709_defconfig":
3769 ::
3770
3771 KBUILD_DEFCONFIG_raspberrypi2 = "bcm2709_defconfig"
3772
3773 As an alternative, you can use the following within your append file:
3774 ::
3775
3776 KBUILD_DEFCONFIG_pn-linux-yocto ?= defconfig_file
3777
3778 For more
3779 information on how to use the ``KBUILD_DEFCONFIG`` variable, see the
3780 ":ref:`kernel-dev/kernel-dev-common:using an "in-tree" \`\`defconfig\`\` file`"
3781 section in the Yocto Project Linux Kernel Development Manual.
3782
3783 KERNEL_ALT_IMAGETYPE
3784 Specifies an alternate kernel image type for creation in addition to
3785 the kernel image type specified using the
3786 :term:`KERNEL_IMAGETYPE` variable.
3787
3788 KERNEL_ARTIFACT_NAME
3789 Specifies the name of all of the build artifacts. You can change the
3790 name of the artifacts by changing the ``KERNEL_ARTIFACT_NAME``
3791 variable.
3792
3793 The value of ``KERNEL_ARTIFACT_NAME``, which is set in the
3794 ``meta/classes/kernel-artifact-names.bbclass`` file, has the
3795 following default value:
3796 ::
3797
3798 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3799
3800 See the :term:`PKGE`, :term:`PKGV`, :term:`PKGR`, and :term:`MACHINE`
3801 variables for additional information.
3802
3803 .. note::
3804
3805 The IMAGE_VERSION_SUFFIX variable is set to DATETIME.
3806
3807 KERNEL_CLASSES
3808 A list of classes defining kernel image types that the
3809 :ref:`kernel <ref-classes-kernel>` class should inherit. You
3810 typically append this variable to enable extended image types. An
3811 example is the "kernel-fitimage", which enables fitImage support and
3812 resides in ``meta/classes/kernel-fitimage.bbclass``. You can register
3813 custom kernel image types with the ``kernel`` class using this
3814 variable.
3815
3816 KERNEL_DEVICETREE
3817 Specifies the name of the generated Linux kernel device tree (i.e.
3818 the ``.dtb``) file.
3819
3820 .. note::
3821
3822 Legacy support exists for specifying the full path to the device
3823 tree. However, providing just the .dtb file is preferred.
3824
3825 In order to use this variable, the
3826 :ref:`kernel-devicetree <ref-classes-kernel-devicetree>` class must
3827 be inherited.
3828
3829 KERNEL_DTB_LINK_NAME
3830 The link name of the kernel device tree binary (DTB). This variable
3831 is set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3832 follows:
3833 ::
3834
3835 KERNEL_DTB_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
3836
3837 The
3838 value of the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in
3839 the same file, has the following value:
3840 ::
3841
3842 KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
3843
3844 See the :term:`MACHINE` variable for additional
3845 information.
3846
3847 KERNEL_DTB_NAME
3848 The base name of the kernel device tree binary (DTB). This variable
3849 is set in the ``meta/classes/kernel-artifact-names.bbclass`` file as
3850 follows:
3851 ::
3852
3853 KERNEL_DTB_NAME ?= "${KERNEL_ARTIFACT_NAME}"
3854
3855 The value of the :term:`KERNEL_ARTIFACT_NAME`
3856 variable, which is set in the same file, has the following value:
3857 ::
3858
3859 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3860
3861 KERNEL_EXTRA_ARGS
3862 Specifies additional ``make`` command-line arguments the OpenEmbedded
3863 build system passes on when compiling the kernel.
3864
3865 KERNEL_FEATURES
3866 Includes additional kernel metadata. In the OpenEmbedded build
3867 system, the default Board Support Packages (BSPs)
3868 :term:`Metadata` is provided through the
3869 :term:`KMACHINE` and :term:`KBRANCH`
3870 variables. You can use the ``KERNEL_FEATURES`` variable from within
3871 the kernel recipe or kernel append file to further add metadata for
3872 all BSPs or specific BSPs.
3873
3874 The metadata you add through this variable includes config fragments
3875 and features descriptions, which usually includes patches as well as
3876 config fragments. You typically override the ``KERNEL_FEATURES``
3877 variable for a specific machine. In this way, you can provide
3878 validated, but optional, sets of kernel configurations and features.
3879
3880 For example, the following example from the ``linux-yocto-rt_4.12``
3881 kernel recipe adds "netfilter" and "taskstats" features to all BSPs
3882 as well as "virtio" configurations to all QEMU machines. The last two
3883 statements add specific configurations to targeted machine types:
3884 ::
3885
3886 KERNEL_EXTRA_FEATURES ?= "features/netfilter/netfilter.scc features/taskstats/taskstats.scc"
3887 KERNEL_FEATURES_append = "${KERNEL_EXTRA_FEATURES}"
3888 KERNEL_FEATURES_append_qemuall = "cfg/virtio.scc"
3889 KERNEL_FEATURES_append_qemux86 = " cfg/sound.scc cfg/paravirt_kvm.scc"
3890 KERNEL_FEATURES_append_qemux86-64 = "cfg/sound.scc"
3891
3892 KERNEL_FIT_LINK_NAME
3893 The link name of the kernel flattened image tree (FIT) image. This
3894 variable is set in the ``meta/classes/kernel-artifact-names.bbclass``
3895 file as follows:
3896 ::
3897
3898 KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
3899
3900 The value of the
3901 ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3902 file, has the following value:
3903 ::
3904
3905 KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
3906
3907 See the :term:`MACHINE` variable for additional
3908 information.
3909
3910 KERNEL_FIT_NAME
3911 The base name of the kernel flattened image tree (FIT) image. This
3912 variable is set in the ``meta/classes/kernel-artifact-names.bbclass``
3913 file as follows:
3914 ::
3915
3916 KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}"
3917
3918 The value of the :term:`KERNEL_ARTIFACT_NAME`
3919 variable, which is set in the same file, has the following value:
3920 ::
3921
3922 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3923
3924 KERNEL_IMAGE_LINK_NAME
3925 The link name for the kernel image. This variable is set in the
3926 ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
3927 ::
3928
3929 KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
3930
3931 The value of
3932 the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the same
3933 file, has the following value:
3934 ::
3935
3936 KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
3937
3938 See the :term:`MACHINE` variable for additional
3939 information.
3940
3941 KERNEL_IMAGE_MAXSIZE
3942 Specifies the maximum size of the kernel image file in kilobytes. If
3943 ``KERNEL_IMAGE_MAXSIZE`` is set, the size of the kernel image file is
3944 checked against the set value during the
3945 :ref:`ref-tasks-sizecheck` task. The task fails if
3946 the kernel image file is larger than the setting.
3947
3948 ``KERNEL_IMAGE_MAXSIZE`` is useful for target devices that have a
3949 limited amount of space in which the kernel image must be stored.
3950
3951 By default, this variable is not set, which means the size of the
3952 kernel image is not checked.
3953
3954 KERNEL_IMAGE_NAME
3955 The base name of the kernel image. This variable is set in the
3956 ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
3957 ::
3958
3959 KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
3960
3961 The value of the
3962 :term:`KERNEL_ARTIFACT_NAME` variable,
3963 which is set in the same file, has the following value:
3964 ::
3965
3966 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
3967
3968 KERNEL_IMAGETYPE
3969 The type of kernel to build for a device, usually set by the machine
3970 configuration files and defaults to "zImage". This variable is used
3971 when building the kernel and is passed to ``make`` as the target to
3972 build.
3973
3974 If you want to build an alternate kernel image type, use the
3975 :term:`KERNEL_ALT_IMAGETYPE` variable.
3976
3977 KERNEL_MODULE_AUTOLOAD
3978 Lists kernel modules that need to be auto-loaded during boot.
3979
3980 .. note::
3981
3982 This variable replaces the deprecated
3983 module_autoload
3984 variable.
3985
3986 You can use the ``KERNEL_MODULE_AUTOLOAD`` variable anywhere that it
3987 can be recognized by the kernel recipe or by an out-of-tree kernel
3988 module recipe (e.g. a machine configuration file, a distribution
3989 configuration file, an append file for the recipe, or the recipe
3990 itself).
3991
3992 Specify it as follows:
3993 ::
3994
3995 KERNEL_MODULE_AUTOLOAD += "module_name1 module_name2 module_name3"
3996
3997 Including ``KERNEL_MODULE_AUTOLOAD`` causes the OpenEmbedded build
3998 system to populate the ``/etc/modules-load.d/modname.conf`` file with
3999 the list of modules to be auto-loaded on boot. The modules appear
4000 one-per-line in the file. Here is an example of the most common use
4001 case:
4002 ::
4003
4004 KERNEL_MODULE_AUTOLOAD += "module_name"
4005
4006 For information on how to populate the ``modname.conf`` file with
4007 ``modprobe.d`` syntax lines, see the :term:`KERNEL_MODULE_PROBECONF` variable.
4008
4009 KERNEL_MODULE_PROBECONF
4010 Provides a list of modules for which the OpenEmbedded build system
4011 expects to find ``module_conf_``\ modname values that specify
4012 configuration for each of the modules. For information on how to
4013 provide those module configurations, see the
4014 :term:`module_conf_* <module_conf>` variable.
4015
4016 KERNEL_PATH
4017 The location of the kernel sources. This variable is set to the value
4018 of the :term:`STAGING_KERNEL_DIR` within
4019 the :ref:`module <ref-classes-module>` class. For information on
4020 how this variable is used, see the
4021 ":ref:`kernel-dev/kernel-dev-common:incorporating out-of-tree modules`"
4022 section in the Yocto Project Linux Kernel Development Manual.
4023
4024 To help maximize compatibility with out-of-tree drivers used to build
4025 modules, the OpenEmbedded build system also recognizes and uses the
4026 :term:`KERNEL_SRC` variable, which is identical to
4027 the ``KERNEL_PATH`` variable. Both variables are common variables
4028 used by external Makefiles to point to the kernel source directory.
4029
4030 KERNEL_SRC
4031 The location of the kernel sources. This variable is set to the value
4032 of the :term:`STAGING_KERNEL_DIR` within
4033 the :ref:`module <ref-classes-module>` class. For information on
4034 how this variable is used, see the
4035 ":ref:`kernel-dev/kernel-dev-common:incorporating out-of-tree modules`"
4036 section in the Yocto Project Linux Kernel Development Manual.
4037
4038 To help maximize compatibility with out-of-tree drivers used to build
4039 modules, the OpenEmbedded build system also recognizes and uses the
4040 :term:`KERNEL_PATH` variable, which is identical
4041 to the ``KERNEL_SRC`` variable. Both variables are common variables
4042 used by external Makefiles to point to the kernel source directory.
4043
4044 KERNEL_VERSION
4045 Specifies the version of the kernel as extracted from ``version.h``
4046 or ``utsrelease.h`` within the kernel sources. Effects of setting
4047 this variable do not take affect until the kernel has been
4048 configured. Consequently, attempting to refer to this variable in
4049 contexts prior to configuration will not work.
4050
4051 KERNELDEPMODDEPEND
4052 Specifies whether the data referenced through
4053 :term:`PKGDATA_DIR` is needed or not. The
4054 ``KERNELDEPMODDEPEND`` does not control whether or not that data
4055 exists, but simply whether or not it is used. If you do not need to
4056 use the data, set the ``KERNELDEPMODDEPEND`` variable in your
4057 ``initramfs`` recipe. Setting the variable there when the data is not
4058 needed avoids a potential dependency loop.
4059
4060 KFEATURE_DESCRIPTION
4061 Provides a short description of a configuration fragment. You use
4062 this variable in the ``.scc`` file that describes a configuration
4063 fragment file. Here is the variable used in a file named ``smp.scc``
4064 to describe SMP being enabled:
4065 ::
4066
4067 define KFEATURE_DESCRIPTION "Enable SMP"
4068
4069 KMACHINE
4070 The machine as known by the kernel. Sometimes the machine name used
4071 by the kernel does not match the machine name used by the
4072 OpenEmbedded build system. For example, the machine name that the
4073 OpenEmbedded build system understands as ``core2-32-intel-common``
4074 goes by a different name in the Linux Yocto kernel. The kernel
4075 understands that machine as ``intel-core2-32``. For cases like these,
4076 the ``KMACHINE`` variable maps the kernel machine name to the
4077 OpenEmbedded build system machine name.
4078
4079 These mappings between different names occur in the Yocto Linux
4080 Kernel's ``meta`` branch. As an example take a look in the
4081 ``common/recipes-kernel/linux/linux-yocto_3.19.bbappend`` file:
4082 ::
4083
4084 LINUX_VERSION_core2-32-intel-common = "3.19.0"
4085 COMPATIBLE_MACHINE_core2-32-intel-common = "${MACHINE}"
4086 SRCREV_meta_core2-32-intel-common = "8897ef68b30e7426bc1d39895e71fb155d694974"
4087 SRCREV_machine_core2-32-intel-common = "43b9eced9ba8a57add36af07736344dcc383f711"
4088 KMACHINE_core2-32-intel-common = "intel-core2-32"
4089 KBRANCH_core2-32-intel-common = "standard/base"
4090 KERNEL_FEATURES_append_core2-32-intel-common = "${KERNEL_FEATURES_INTEL_COMMON}"
4091
4092 The ``KMACHINE`` statement says
4093 that the kernel understands the machine name as "intel-core2-32".
4094 However, the OpenEmbedded build system understands the machine as
4095 "core2-32-intel-common".
4096
4097 KTYPE
4098 Defines the kernel type to be used in assembling the configuration.
4099 The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
4100 kernel types. See the ":ref:`kernel-dev/kernel-dev-advanced:kernel types`"
4101 section in the
4102 Yocto Project Linux Kernel Development Manual for more information on
4103 kernel types.
4104
4105 You define the ``KTYPE`` variable in the
4106 :ref:`kernel-dev/kernel-dev-advanced:bsp descriptions`. The
4107 value you use must match the value used for the
4108 :term:`LINUX_KERNEL_TYPE` value used by the
4109 kernel recipe.
4110
4111 LABELS
4112 Provides a list of targets for automatic configuration.
4113
4114 See the :ref:`grub-efi <ref-classes-grub-efi>` class for more
4115 information on how this variable is used.
4116
4117 LAYERDEPENDS
4118 Lists the layers, separated by spaces, on which this recipe depends.
4119 Optionally, you can specify a specific layer version for a dependency
4120 by adding it to the end of the layer name. Here is an example:
4121 ::
4122
4123 LAYERDEPENDS_mylayer = "anotherlayer (=3)"
4124
4125 In this previous example,
4126 version 3 of "anotherlayer" is compared against
4127 :term:`LAYERVERSION`\ ``_anotherlayer``.
4128
4129 An error is produced if any dependency is missing or the version
4130 numbers (if specified) do not match exactly. This variable is used in
4131 the ``conf/layer.conf`` file and must be suffixed with the name of
4132 the specific layer (e.g. ``LAYERDEPENDS_mylayer``).
4133
4134 LAYERDIR
4135 When used inside the ``layer.conf`` configuration file, this variable
4136 provides the path of the current layer. This variable is not
4137 available outside of ``layer.conf`` and references are expanded
4138 immediately when parsing of the file completes.
4139
4140 LAYERRECOMMENDS
4141 Lists the layers, separated by spaces, recommended for use with this
4142 layer.
4143
4144 Optionally, you can specify a specific layer version for a
4145 recommendation by adding the version to the end of the layer name.
4146 Here is an example:
4147 ::
4148
4149 LAYERRECOMMENDS_mylayer = "anotherlayer (=3)"
4150
4151 In this previous example, version 3 of "anotherlayer" is compared
4152 against ``LAYERVERSION_anotherlayer``.
4153
4154 This variable is used in the ``conf/layer.conf`` file and must be
4155 suffixed with the name of the specific layer (e.g.
4156 ``LAYERRECOMMENDS_mylayer``).
4157
4158 LAYERSERIES_COMPAT
4159 Lists the versions of the :term:`OpenEmbedded-Core (OE-Core)` for which
4160 a layer is compatible. Using the ``LAYERSERIES_COMPAT`` variable
4161 allows the layer maintainer to indicate which combinations of the
4162 layer and OE-Core can be expected to work. The variable gives the
4163 system a way to detect when a layer has not been tested with new
4164 releases of OE-Core (e.g. the layer is not maintained).
4165
4166 To specify the OE-Core versions for which a layer is compatible, use
4167 this variable in your layer's ``conf/layer.conf`` configuration file.
4168 For the list, use the Yocto Project
4169 :yocto_wiki:`Release Name </wiki/Releases>` (e.g.
4170 DISTRO_NAME_NO_CAP). To specify multiple OE-Core versions for the
4171 layer, use a space-separated list:
4172 ::
4173
4174 LAYERSERIES_COMPAT_layer_root_name = "DISTRO_NAME_NO_CAP DISTRO_NAME_NO_CAP_MINUS_ONE"
4175
4176 .. note::
4177
4178 Setting
4179 LAYERSERIES_COMPAT
4180 is required by the Yocto Project Compatible version 2 standard.
4181 The OpenEmbedded build system produces a warning if the variable
4182 is not set for any given layer.
4183
4184 See the ":ref:`dev-manual/dev-manual-common-tasks:creating your own layer`"
4185 section in the Yocto Project Development Tasks Manual.
4186
4187 LAYERVERSION
4188 Optionally specifies the version of a layer as a single number. You
4189 can use this within :term:`LAYERDEPENDS` for
4190 another layer in order to depend on a specific version of the layer.
4191 This variable is used in the ``conf/layer.conf`` file and must be
4192 suffixed with the name of the specific layer (e.g.
4193 ``LAYERVERSION_mylayer``).
4194
4195 LD
4196 The minimal command and arguments used to run the linker.
4197
4198 LDFLAGS
4199 Specifies the flags to pass to the linker. This variable is exported
4200 to an environment variable and thus made visible to the software
4201 being built during the compilation step.
4202
4203 Default initialization for ``LDFLAGS`` varies depending on what is
4204 being built:
4205
4206 - :term:`TARGET_LDFLAGS` when building for the
4207 target
4208
4209 - :term:`BUILD_LDFLAGS` when building for the
4210 build host (i.e. ``-native``)
4211
4212 - :term:`BUILDSDK_LDFLAGS` when building for
4213 an SDK (i.e. ``nativesdk-``)
4214
4215 LEAD_SONAME
4216 Specifies the lead (or primary) compiled library file (i.e. ``.so``)
4217 that the :ref:`debian <ref-classes-debian>` class applies its
4218 naming policy to given a recipe that packages multiple libraries.
4219
4220 This variable works in conjunction with the ``debian`` class.
4221
4222 LIC_FILES_CHKSUM
4223 Checksums of the license text in the recipe source code.
4224
4225 This variable tracks changes in license text of the source code
4226 files. If the license text is changed, it will trigger a build
4227 failure, which gives the developer an opportunity to review any
4228 license change.
4229
4230 This variable must be defined for all recipes (unless
4231 :term:`LICENSE` is set to "CLOSED").
4232
4233 For more information, see the ":ref:`usingpoky-configuring-lic_files_chksum`"
4234 section in the Yocto Project Development Tasks Manual.
4235
4236 LICENSE
4237 The list of source licenses for the recipe. Follow these rules:
4238
4239 - Do not use spaces within individual license names.
4240
4241 - Separate license names using \| (pipe) when there is a choice
4242 between licenses.
4243
4244 - Separate license names using & (ampersand) when multiple licenses
4245 exist that cover different parts of the source.
4246
4247 - You can use spaces between license names.
4248
4249 - For standard licenses, use the names of the files in
4250 ``meta/files/common-licenses/`` or the
4251 :term:`SPDXLICENSEMAP` flag names defined in
4252 ``meta/conf/licenses.conf``.
4253
4254 Here are some examples:
4255 ::
4256
4257 LICENSE = "LGPLv2.1 | GPLv3"
4258 LICENSE = "MPL-1 & LGPLv2.1"
4259 LICENSE = "GPLv2+"
4260
4261 The first example is from the
4262 recipes for Qt, which the user may choose to distribute under either
4263 the LGPL version 2.1 or GPL version 3. The second example is from
4264 Cairo where two licenses cover different parts of the source code.
4265 The final example is from ``sysstat``, which presents a single
4266 license.
4267
4268 You can also specify licenses on a per-package basis to handle
4269 situations where components of the output have different licenses.
4270 For example, a piece of software whose code is licensed under GPLv2
4271 but has accompanying documentation licensed under the GNU Free
4272 Documentation License 1.2 could be specified as follows:
4273 ::
4274
4275 LICENSE = "GFDL-1.2 & GPLv2"
4276 LICENSE_${PN} = "GPLv2"
4277 LICENSE_${PN}-doc = "GFDL-1.2"
4278
4279 LICENSE_CREATE_PACKAGE
4280 Setting ``LICENSE_CREATE_PACKAGE`` to "1" causes the OpenEmbedded
4281 build system to create an extra package (i.e.
4282 ``${``\ :term:`PN`\ ``}-lic``) for each recipe and to add
4283 those packages to the
4284 :term:`RRECOMMENDS`\ ``_${PN}``.
4285
4286 The ``${PN}-lic`` package installs a directory in
4287 ``/usr/share/licenses`` named ``${PN}``, which is the recipe's base
4288 name, and installs files in that directory that contain license and
4289 copyright information (i.e. copies of the appropriate license files
4290 from ``meta/common-licenses`` that match the licenses specified in
4291 the :term:`LICENSE` variable of the recipe metadata
4292 and copies of files marked in
4293 :term:`LIC_FILES_CHKSUM` as containing
4294 license text).
4295
4296 For related information on providing license text, see the
4297 :term:`COPY_LIC_DIRS` variable, the
4298 :term:`COPY_LIC_MANIFEST` variable, and the
4299 ":ref:`dev-manual/dev-manual-common-tasks:providing license text`"
4300 section in the Yocto Project Development Tasks Manual.
4301
4302 LICENSE_FLAGS
4303 Specifies additional flags for a recipe you must whitelist through
4304 :term:`LICENSE_FLAGS_WHITELIST` in
4305 order to allow the recipe to be built. When providing multiple flags,
4306 separate them with spaces.
4307
4308 This value is independent of :term:`LICENSE` and is
4309 typically used to mark recipes that might require additional licenses
4310 in order to be used in a commercial product. For more information,
4311 see the
4312 ":ref:`dev-manual/dev-manual-common-tasks:enabling commercially licensed recipes`"
4313 section in the Yocto Project Development Tasks Manual.
4314
4315 LICENSE_FLAGS_WHITELIST
4316 Lists license flags that when specified in
4317 :term:`LICENSE_FLAGS` within a recipe should not
4318 prevent that recipe from being built. This practice is otherwise
4319 known as "whitelisting" license flags. For more information, see the
4320 ":ref:`dev-manual/dev-manual-common-tasks:enabling commercially licensed recipes`"
4321 section in the Yocto Project Development Tasks Manual.
4322
4323 LICENSE_PATH
4324 Path to additional licenses used during the build. By default, the
4325 OpenEmbedded build system uses ``COMMON_LICENSE_DIR`` to define the
4326 directory that holds common license text used during the build. The
4327 ``LICENSE_PATH`` variable allows you to extend that location to other
4328 areas that have additional licenses:
4329 ::
4330
4331 LICENSE_PATH += "path-to-additional-common-licenses"
4332
4333 LINUX_KERNEL_TYPE
4334 Defines the kernel type to be used in assembling the configuration.
4335 The linux-yocto recipes define "standard", "tiny", and "preempt-rt"
4336 kernel types. See the ":ref:`kernel-dev/kernel-dev-advanced:kernel types`"
4337 section in the
4338 Yocto Project Linux Kernel Development Manual for more information on
4339 kernel types.
4340
4341 If you do not specify a ``LINUX_KERNEL_TYPE``, it defaults to
4342 "standard". Together with :term:`KMACHINE`, the
4343 ``LINUX_KERNEL_TYPE`` variable defines the search arguments used by
4344 the kernel tools to find the appropriate description within the
4345 kernel :term:`Metadata` with which to build out the sources
4346 and configuration.
4347
4348 LINUX_VERSION
4349 The Linux version from ``kernel.org`` on which the Linux kernel image
4350 being built using the OpenEmbedded build system is based. You define
4351 this variable in the kernel recipe. For example, the
4352 ``linux-yocto-3.4.bb`` kernel recipe found in
4353 ``meta/recipes-kernel/linux`` defines the variables as follows:
4354 ::
4355
4356 LINUX_VERSION ?= "3.4.24"
4357
4358 The ``LINUX_VERSION`` variable is used to define :term:`PV`
4359 for the recipe:
4360 ::
4361
4362 PV = "${LINUX_VERSION}+git${SRCPV}"
4363
4364 LINUX_VERSION_EXTENSION
4365 A string extension compiled into the version string of the Linux
4366 kernel built with the OpenEmbedded build system. You define this
4367 variable in the kernel recipe. For example, the linux-yocto kernel
4368 recipes all define the variable as follows:
4369 ::
4370
4371 LINUX_VERSION_EXTENSION ?= "-yocto-${LINUX_KERNEL_TYPE}"
4372
4373 Defining this variable essentially sets the Linux kernel
4374 configuration item ``CONFIG_LOCALVERSION``, which is visible through
4375 the ``uname`` command. Here is an example that shows the extension
4376 assuming it was set as previously shown:
4377 ::
4378
4379 $ uname -r
4380 3.7.0-rc8-custom
4381
4382 LOG_DIR
4383 Specifies the directory to which the OpenEmbedded build system writes
4384 overall log files. The default directory is ``${TMPDIR}/log``.
4385
4386 For the directory containing logs specific to each task, see the
4387 :term:`T` variable.
4388
4389 MACHINE
4390 Specifies the target device for which the image is built. You define
4391 ``MACHINE`` in the ``local.conf`` file found in the
4392 :term:`Build Directory`. By default, ``MACHINE`` is set to
4393 "qemux86", which is an x86-based architecture machine to be emulated
4394 using QEMU:
4395 ::
4396
4397 MACHINE ?= "qemux86"
4398
4399 The variable corresponds to a machine configuration file of the same
4400 name, through which machine-specific configurations are set. Thus,
4401 when ``MACHINE`` is set to "qemux86" there exists the corresponding
4402 ``qemux86.conf`` machine configuration file, which can be found in
4403 the :term:`Source Directory` in
4404 ``meta/conf/machine``.
4405
4406 The list of machines supported by the Yocto Project as shipped
4407 include the following:
4408 ::
4409
4410 MACHINE ?= "qemuarm"
4411 MACHINE ?= "qemuarm64"
4412 MACHINE ?= "qemumips"
4413 MACHINE ?= "qemumips64"
4414 MACHINE ?= "qemuppc"
4415 MACHINE ?= "qemux86"
4416 MACHINE ?= "qemux86-64"
4417 MACHINE ?= "genericx86"
4418 MACHINE ?= "genericx86-64"
4419 MACHINE ?= "beaglebone"
4420 MACHINE ?= "edgerouter"
4421
4422 The last five are Yocto Project reference hardware
4423 boards, which are provided in the ``meta-yocto-bsp`` layer.
4424
4425 .. note::
4426
4427 Adding additional Board Support Package (BSP) layers to your
4428 configuration adds new possible settings for
4429 MACHINE
4430 .
4431
4432 MACHINE_ARCH
4433 Specifies the name of the machine-specific architecture. This
4434 variable is set automatically from :term:`MACHINE` or
4435 :term:`TUNE_PKGARCH`. You should not hand-edit
4436 the ``MACHINE_ARCH`` variable.
4437
4438 MACHINE_ESSENTIAL_EXTRA_RDEPENDS
4439 A list of required machine-specific packages to install as part of
4440 the image being built. The build process depends on these packages
4441 being present. Furthermore, because this is a "machine-essential"
4442 variable, the list of packages are essential for the machine to boot.
4443 The impact of this variable affects images based on
4444 ``packagegroup-core-boot``, including the ``core-image-minimal``
4445 image.
4446
4447 This variable is similar to the
4448 ``MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`` variable with the exception
4449 that the image being built has a build dependency on the variable's
4450 list of packages. In other words, the image will not build if a file
4451 in this list is not found.
4452
4453 As an example, suppose the machine for which you are building
4454 requires ``example-init`` to be run during boot to initialize the
4455 hardware. In this case, you would use the following in the machine's
4456 ``.conf`` configuration file:
4457 ::
4458
4459 MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "example-init"
4460
4461 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
4462 A list of recommended machine-specific packages to install as part of
4463 the image being built. The build process does not depend on these
4464 packages being present. However, because this is a
4465 "machine-essential" variable, the list of packages are essential for
4466 the machine to boot. The impact of this variable affects images based
4467 on ``packagegroup-core-boot``, including the ``core-image-minimal``
4468 image.
4469
4470 This variable is similar to the ``MACHINE_ESSENTIAL_EXTRA_RDEPENDS``
4471 variable with the exception that the image being built does not have
4472 a build dependency on the variable's list of packages. In other
4473 words, the image will still build if a package in this list is not
4474 found. Typically, this variable is used to handle essential kernel
4475 modules, whose functionality may be selected to be built into the
4476 kernel rather than as a module, in which case a package will not be
4477 produced.
4478
4479 Consider an example where you have a custom kernel where a specific
4480 touchscreen driver is required for the machine to be usable. However,
4481 the driver can be built as a module or into the kernel depending on
4482 the kernel configuration. If the driver is built as a module, you
4483 want it to be installed. But, when the driver is built into the
4484 kernel, you still want the build to succeed. This variable sets up a
4485 "recommends" relationship so that in the latter case, the build will
4486 not fail due to the missing package. To accomplish this, assuming the
4487 package for the module was called ``kernel-module-ab123``, you would
4488 use the following in the machine's ``.conf`` configuration file:
4489 ::
4490
4491 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "kernel-module-ab123"
4492
4493 .. note::
4494
4495 In this example, the
4496 kernel-module-ab123
4497 recipe needs to explicitly set its
4498 PACKAGES
4499 variable to ensure that BitBake does not use the kernel recipe's
4500 PACKAGES_DYNAMIC
4501 variable to satisfy the dependency.
4502
4503 Some examples of these machine essentials are flash, screen,
4504 keyboard, mouse, or touchscreen drivers (depending on the machine).
4505
4506 MACHINE_EXTRA_RDEPENDS
4507 A list of machine-specific packages to install as part of the image
4508 being built that are not essential for the machine to boot. However,
4509 the build process for more fully-featured images depends on the
4510 packages being present.
4511
4512 This variable affects all images based on ``packagegroup-base``,
4513 which does not include the ``core-image-minimal`` or
4514 ``core-image-full-cmdline`` images.
4515
4516 The variable is similar to the ``MACHINE_EXTRA_RRECOMMENDS`` variable
4517 with the exception that the image being built has a build dependency
4518 on the variable's list of packages. In other words, the image will
4519 not build if a file in this list is not found.
4520
4521 An example is a machine that has WiFi capability but is not essential
4522 for the machine to boot the image. However, if you are building a
4523 more fully-featured image, you want to enable the WiFi. The package
4524 containing the firmware for the WiFi hardware is always expected to
4525 exist, so it is acceptable for the build process to depend upon
4526 finding the package. In this case, assuming the package for the
4527 firmware was called ``wifidriver-firmware``, you would use the
4528 following in the ``.conf`` file for the machine:
4529 ::
4530
4531 MACHINE_EXTRA_RDEPENDS += "wifidriver-firmware"
4532
4533 MACHINE_EXTRA_RRECOMMENDS
4534 A list of machine-specific packages to install as part of the image
4535 being built that are not essential for booting the machine. The image
4536 being built has no build dependency on this list of packages.
4537
4538 This variable affects only images based on ``packagegroup-base``,
4539 which does not include the ``core-image-minimal`` or
4540 ``core-image-full-cmdline`` images.
4541
4542 This variable is similar to the ``MACHINE_EXTRA_RDEPENDS`` variable
4543 with the exception that the image being built does not have a build
4544 dependency on the variable's list of packages. In other words, the
4545 image will build if a file in this list is not found.
4546
4547 An example is a machine that has WiFi capability but is not essential
4548 For the machine to boot the image. However, if you are building a
4549 more fully-featured image, you want to enable WiFi. In this case, the
4550 package containing the WiFi kernel module will not be produced if the
4551 WiFi driver is built into the kernel, in which case you still want
4552 the build to succeed instead of failing as a result of the package
4553 not being found. To accomplish this, assuming the package for the
4554 module was called ``kernel-module-examplewifi``, you would use the
4555 following in the ``.conf`` file for the machine:
4556 ::
4557
4558 MACHINE_EXTRA_RRECOMMENDS += "kernel-module-examplewifi"
4559
4560 MACHINE_FEATURES
4561 Specifies the list of hardware features the
4562 :term:`MACHINE` is capable of supporting. For related
4563 information on enabling features, see the
4564 :term:`DISTRO_FEATURES`,
4565 :term:`COMBINED_FEATURES`, and
4566 :term:`IMAGE_FEATURES` variables.
4567
4568 For a list of hardware features supported by the Yocto Project as
4569 shipped, see the "`Machine Features <#ref-features-machine>`__"
4570 section.
4571
4572 MACHINE_FEATURES_BACKFILL
4573 Features to be added to ``MACHINE_FEATURES`` if not also present in
4574 ``MACHINE_FEATURES_BACKFILL_CONSIDERED``.
4575
4576 This variable is set in the ``meta/conf/bitbake.conf`` file. It is
4577 not intended to be user-configurable. It is best to just reference
4578 the variable to see which machine features are being backfilled for
4579 all machine configurations. See the "`Feature
4580 Backfilling <#ref-features-backfill>`__" section for more
4581 information.
4582
4583 MACHINE_FEATURES_BACKFILL_CONSIDERED
4584 Features from ``MACHINE_FEATURES_BACKFILL`` that should not be
4585 backfilled (i.e. added to ``MACHINE_FEATURES``) during the build. See
4586 the "`Feature Backfilling <#ref-features-backfill>`__" section for
4587 more information.
4588
4589 MACHINEOVERRIDES
4590 A colon-separated list of overrides that apply to the current
4591 machine. By default, this list includes the value of
4592 :term:`MACHINE`.
4593
4594 You can extend ``MACHINEOVERRIDES`` to add extra overrides that
4595 should apply to a machine. For example, all machines emulated in QEMU
4596 (e.g. ``qemuarm``, ``qemux86``, and so forth) include a file named
4597 ``meta/conf/machine/include/qemu.inc`` that prepends the following
4598 override to ``MACHINEOVERRIDES``:
4599 ::
4600
4601 MACHINEOVERRIDES =. "qemuall:"
4602
4603 This
4604 override allows variables to be overriden for all machines emulated
4605 in QEMU, like in the following example from the ``connman-conf``
4606 recipe:
4607 ::
4608
4609 SRC_URI_append_qemuall = "file://wired.config \
4610 file://wired-setup \
4611 "
4612
4613 The underlying mechanism behind
4614 ``MACHINEOVERRIDES`` is simply that it is included in the default
4615 value of :term:`OVERRIDES`.
4616
4617 MAINTAINER
4618 The email address of the distribution maintainer.
4619
4620 MIRRORS
4621 Specifies additional paths from which the OpenEmbedded build system
4622 gets source code. When the build system searches for source code, it
4623 first tries the local download directory. If that location fails, the
4624 build system tries locations defined by
4625 :term:`PREMIRRORS`, the upstream source, and then
4626 locations specified by ``MIRRORS`` in that order.
4627
4628 Assuming your distribution (:term:`DISTRO`) is "poky",
4629 the default value for ``MIRRORS`` is defined in the
4630 ``conf/distro/poky.conf`` file in the ``meta-poky`` Git repository.
4631
4632 MLPREFIX
4633 Specifies a prefix has been added to :term:`PN` to create a
4634 special version of a recipe or package (i.e. a Multilib version). The
4635 variable is used in places where the prefix needs to be added to or
4636 removed from a the name (e.g. the :term:`BPN` variable).
4637 ``MLPREFIX`` gets set when a prefix has been added to ``PN``.
4638
4639 .. note::
4640
4641 The "ML" in
4642 MLPREFIX
4643 stands for "MultiLib". This representation is historical and comes
4644 from a time when
4645 nativesdk
4646 was a suffix rather than a prefix on the recipe name. When
4647 nativesdk
4648 was turned into a prefix, it made sense to set
4649 MLPREFIX
4650 for it as well.
4651
4652 To help understand when ``MLPREFIX`` might be needed, consider when
4653 :term:`BBCLASSEXTEND` is used to provide a
4654 ``nativesdk`` version of a recipe in addition to the target version.
4655 If that recipe declares build-time dependencies on tasks in other
4656 recipes by using :term:`DEPENDS`, then a dependency on
4657 "foo" will automatically get rewritten to a dependency on
4658 "nativesdk-foo". However, dependencies like the following will not
4659 get rewritten automatically:
4660 ::
4661
4662 do_foo[depends] += "recipe:do_foo"
4663
4664 If you want such a dependency to also get transformed, you can do the
4665 following:
4666 ::
4667
4668 do_foo[depends] += "${MLPREFIX}recipe:do_foo"
4669
4670 module_autoload
4671 This variable has been replaced by the ``KERNEL_MODULE_AUTOLOAD``
4672 variable. You should replace all occurrences of ``module_autoload``
4673 with additions to ``KERNEL_MODULE_AUTOLOAD``, for example:
4674 ::
4675
4676 module_autoload_rfcomm = "rfcomm"
4677
4678 should now be replaced with:
4679 ::
4680
4681 KERNEL_MODULE_AUTOLOAD += "rfcomm"
4682
4683 See the :term:`KERNEL_MODULE_AUTOLOAD` variable for more information.
4684
4685 module_conf
4686 Specifies `modprobe.d <http://linux.die.net/man/5/modprobe.d>`_
4687 syntax lines for inclusion in the ``/etc/modprobe.d/modname.conf``
4688 file.
4689
4690 You can use this variable anywhere that it can be recognized by the
4691 kernel recipe or out-of-tree kernel module recipe (e.g. a machine
4692 configuration file, a distribution configuration file, an append file
4693 for the recipe, or the recipe itself). If you use this variable, you
4694 must also be sure to list the module name in the
4695 :term:`KERNEL_MODULE_AUTOLOAD`
4696 variable.
4697
4698 Here is the general syntax:
4699 ::
4700
4701 module_conf_module_name = "modprobe.d-syntax"
4702
4703 You must use the kernel module name override.
4704
4705 Run ``man modprobe.d`` in the shell to find out more information on
4706 the exact syntax you want to provide with ``module_conf``.
4707
4708 Including ``module_conf`` causes the OpenEmbedded build system to
4709 populate the ``/etc/modprobe.d/modname.conf`` file with
4710 ``modprobe.d`` syntax lines. Here is an example that adds the options
4711 ``arg1`` and ``arg2`` to a module named ``mymodule``:
4712 ::
4713
4714 module_conf_mymodule = "options mymodule arg1=val1 arg2=val2"
4715
4716 For information on how to specify kernel modules to auto-load on
4717 boot, see the :term:`KERNEL_MODULE_AUTOLOAD` variable.
4718
4719 MODULE_TARBALL_DEPLOY
4720 Controls creation of the ``modules-*.tgz`` file. Set this variable to
4721 "0" to disable creation of this file, which contains all of the
4722 kernel modules resulting from a kernel build.
4723
4724 MODULE_TARBALL_LINK_NAME
4725 The link name of the kernel module tarball. This variable is set in
4726 the ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
4727 ::
4728
4729 MODULE_TARBALL_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
4730
4731 The value
4732 of the ``KERNEL_ARTIFACT_LINK_NAME`` variable, which is set in the
4733 same file, has the following value:
4734 ::
4735
4736 KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
4737
4738 See the :term:`MACHINE` variable for additional information.
4739
4740 MODULE_TARBALL_NAME
4741 The base name of the kernel module tarball. This variable is set in
4742 the ``meta/classes/kernel-artifact-names.bbclass`` file as follows:
4743 ::
4744
4745 MODULE_TARBALL_NAME ?= "${KERNEL_ARTIFACT_NAME}"
4746
4747 The value of the :term:`KERNEL_ARTIFACT_NAME` variable,
4748 which is set in the same file, has the following value:
4749 ::
4750
4751 KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
4752
4753 MULTIMACH_TARGET_SYS
4754 Uniquely identifies the type of the target system for which packages
4755 are being built. This variable allows output for different types of
4756 target systems to be put into different subdirectories of the same
4757 output directory.
4758
4759 The default value of this variable is:
4760 ::
4761
4762 ${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}
4763
4764 Some classes (e.g.
4765 :ref:`cross-canadian <ref-classes-cross-canadian>`) modify the
4766 ``MULTIMACH_TARGET_SYS`` value.
4767
4768 See the :term:`STAMP` variable for an example. See the
4769 :term:`STAGING_DIR_TARGET` variable for more information.
4770
4771 NATIVELSBSTRING
4772 A string identifying the host distribution. Strings consist of the
4773 host distributor ID followed by the release, as reported by the
4774 ``lsb_release`` tool or as read from ``/etc/lsb-release``. For
4775 example, when running a build on Ubuntu 12.10, the value is
4776 "Ubuntu-12.10". If this information is unable to be determined, the
4777 value resolves to "Unknown".
4778
4779 This variable is used by default to isolate native shared state
4780 packages for different distributions (e.g. to avoid problems with
4781 ``glibc`` version incompatibilities). Additionally, the variable is
4782 checked against
4783 :term:`SANITY_TESTED_DISTROS` if that
4784 variable is set.
4785
4786 NM
4787 The minimal command and arguments to run ``nm``.
4788
4789 NO_GENERIC_LICENSE
4790 Avoids QA errors when you use a non-common, non-CLOSED license in a
4791 recipe. Packages exist, such as the linux-firmware package, with many
4792 licenses that are not in any way common. Also, new licenses are added
4793 occasionally to avoid introducing a lot of common license files,
4794 which are only applicable to a specific package.
4795 ``NO_GENERIC_LICENSE`` is used to allow copying a license that does
4796 not exist in common licenses.
4797
4798 The following example shows how to add ``NO_GENERIC_LICENSE`` to a
4799 recipe:
4800 ::
4801
4802 NO_GENERIC_LICENSE[license_name] = "license_file_in_fetched_source"
4803
4804 The following is an example that
4805 uses the ``LICENSE.Abilis.txt`` file as the license from the fetched
4806 source:
4807 ::
4808
4809 NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
4810
4811 NO_RECOMMENDATIONS
4812 Prevents installation of all "recommended-only" packages.
4813 Recommended-only packages are packages installed only through the
4814 :term:`RRECOMMENDS` variable). Setting the
4815 ``NO_RECOMMENDATIONS`` variable to "1" turns this feature on: ::
4816
4817 NO_RECOMMENDATIONS = "1"
4818
4819 You can set this variable globally in your ``local.conf`` file or you
4820 can attach it to a specific image recipe by using the recipe name
4821 override: ::
4822
4823 NO_RECOMMENDATIONS_pn-target_image = "1"
4824
4825 It is important to realize that if you choose to not install packages
4826 using this variable and some other packages are dependent on them
4827 (i.e. listed in a recipe's :term:`RDEPENDS`
4828 variable), the OpenEmbedded build system ignores your request and
4829 will install the packages to avoid dependency errors.
4830
4831 .. note::
4832
4833 Some recommended packages might be required for certain system
4834 functionality, such as kernel modules. It is up to you to add
4835 packages with the IMAGE_INSTALL variable.
4836
4837 Support for this variable exists only when using the IPK and RPM
4838 packaging backend. Support does not exist for DEB.
4839
4840 See the :term:`BAD_RECOMMENDATIONS` and
4841 the :term:`PACKAGE_EXCLUDE` variables for
4842 related information.
4843
4844 NOAUTOPACKAGEDEBUG
4845 Disables auto package from splitting ``.debug`` files. If a recipe
4846 requires ``FILES_${PN}-dbg`` to be set manually, the
4847 ``NOAUTOPACKAGEDEBUG`` can be defined allowing you to define the
4848 content of the debug package. For example:
4849 ::
4850
4851 NOAUTOPACKAGEDEBUG = "1"
4852 FILES_${PN}-dev = "${includedir}/${QT_DIR_NAME}/Qt/*"
4853 FILES_${PN}-dbg = "/usr/src/debug/"
4854 FILES_${QT_BASE_NAME}-demos-doc = "${docdir}/${QT_DIR_NAME}/qch/qt.qch"
4855
4856 OBJCOPY
4857 The minimal command and arguments to run ``objcopy``.
4858
4859 OBJDUMP
4860 The minimal command and arguments to run ``objdump``.
4861
4862 OE_BINCONFIG_EXTRA_MANGLE
4863 When inheriting the :ref:`binconfig <ref-classes-binconfig>` class,
4864 this variable specifies additional arguments passed to the "sed"
4865 command. The sed command alters any paths in configuration scripts
4866 that have been set up during compilation. Inheriting this class
4867 results in all paths in these scripts being changed to point into the
4868 ``sysroots/`` directory so that all builds that use the script will
4869 use the correct directories for the cross compiling layout.
4870
4871 See the ``meta/classes/binconfig.bbclass`` in the
4872 :term:`Source Directory` for details on how this class
4873 applies these additional sed command arguments. For general
4874 information on the ``binconfig`` class, see the
4875 ":ref:`binconfig.bbclass <ref-classes-binconfig>`" section.
4876
4877 OE_IMPORTS
4878 An internal variable used to tell the OpenEmbedded build system what
4879 Python modules to import for every Python function run by the system.
4880
4881 .. note::
4882
4883 Do not set this variable. It is for internal use only.
4884
4885 OE_INIT_ENV_SCRIPT
4886 The name of the build environment setup script for the purposes of
4887 setting up the environment within the extensible SDK. The default
4888 value is "oe-init-build-env".
4889
4890 If you use a custom script to set up your build environment, set the
4891 ``OE_INIT_ENV_SCRIPT`` variable to its name.
4892
4893 OE_TERMINAL
4894 Controls how the OpenEmbedded build system spawns interactive
4895 terminals on the host development system (e.g. using the BitBake
4896 command with the ``-c devshell`` command-line option). For more
4897 information, see the ":ref:`platdev-appdev-devshell`" section in
4898 the Yocto Project Development Tasks Manual.
4899
4900 You can use the following values for the ``OE_TERMINAL`` variable:
4901
4902 - auto
4903 - gnome
4904 - xfce
4905 - rxvt
4906 - screen
4907 - konsole
4908 - none
4909
4910 OEROOT
4911 The directory from which the top-level build environment setup script
4912 is sourced. The Yocto Project provides a top-level build environment
4913 setup script: ````` <#structure-core-script>`__. When you run this
4914 script, the ``OEROOT`` variable resolves to the directory that
4915 contains the script.
4916
4917 For additional information on how this variable is used, see the
4918 initialization script.
4919
4920 OLDEST_KERNEL
4921 Declares the oldest version of the Linux kernel that the produced
4922 binaries must support. This variable is passed into the build of the
4923 Embedded GNU C Library (``glibc``).
4924
4925 The default for this variable comes from the
4926 ``meta/conf/bitbake.conf`` configuration file. You can override this
4927 default by setting the variable in a custom distribution
4928 configuration file.
4929
4930 OVERRIDES
4931 A colon-separated list of overrides that currently apply. Overrides
4932 are a BitBake mechanism that allows variables to be selectively
4933 overridden at the end of parsing. The set of overrides in
4934 ``OVERRIDES`` represents the "state" during building, which includes
4935 the current recipe being built, the machine for which it is being
4936 built, and so forth.
4937
4938 As an example, if the string "an-override" appears as an element in
4939 the colon-separated list in ``OVERRIDES``, then the following
4940 assignment will override ``FOO`` with the value "overridden" at the
4941 end of parsing:
4942 ::
4943
4944 FOO_an-override = "overridden"
4945
4946 See the
4947 ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`"
4948 section in the BitBake User Manual for more information on the
4949 overrides mechanism.
4950
4951 The default value of ``OVERRIDES`` includes the values of the
4952 :term:`CLASSOVERRIDE`,
4953 :term:`MACHINEOVERRIDES`, and
4954 :term:`DISTROOVERRIDES` variables. Another
4955 important override included by default is ``pn-${PN}``. This override
4956 allows variables to be set for a single recipe within configuration
4957 (``.conf``) files. Here is an example:
4958 ::
4959
4960 FOO_pn-myrecipe = "myrecipe-specific value"
4961
4962 .. note::
4963
4964 An easy way to see what overrides apply is to search for
4965 OVERRIDES
4966 in the output of the
4967 bitbake -e
4968 command. See the "
4969 Viewing Variable Values
4970 " section in the Yocto Project Development Tasks Manual for more
4971 information.
4972
4973 P
4974 The recipe name and version. ``P`` is comprised of the following:
4975 ::
4976
4977 ${PN}-${PV}
4978
4979 PACKAGE_ADD_METADATA
4980 This variable defines additional metdata to add to packages.
4981
4982 You may find you need to inject additional metadata into packages.
4983 This variable allows you to do that by setting the injected data as
4984 the value. Multiple fields can be added by splitting the content with
4985 the literal separator "\n".
4986
4987 The suffixes '_IPK', '_DEB', or '_RPM' can be applied to the variable
4988 to do package type specific settings. It can also be made package
4989 specific by using the package name as a suffix.
4990
4991 You can find out more about applying this variable in the
4992 ":ref:`dev-manual/dev-manual-common-tasks:adding custom metadata to packages`"
4993 section in the Yocto Project Development Tasks Manual.
4994
4995 PACKAGE_ARCH
4996 The architecture of the resulting package or packages.
4997
4998 By default, the value of this variable is set to
4999 :term:`TUNE_PKGARCH` when building for the
5000 target, :term:`BUILD_ARCH` when building for the
5001 build host, and "${SDK_ARCH}-${SDKPKGSUFFIX}" when building for the
5002 SDK.
5003
5004 .. note::
5005
5006 See
5007 SDK_ARCH
5008 for more information.
5009
5010 However, if your recipe's output packages are built specific to the
5011 target machine rather than generally for the architecture of the
5012 machine, you should set ``PACKAGE_ARCH`` to the value of
5013 :term:`MACHINE_ARCH` in the recipe as follows:
5014 ::
5015
5016 PACKAGE_ARCH = "${MACHINE_ARCH}"
5017
5018 PACKAGE_ARCHS
5019 Specifies a list of architectures compatible with the target machine.
5020 This variable is set automatically and should not normally be
5021 hand-edited. Entries are separated using spaces and listed in order
5022 of priority. The default value for ``PACKAGE_ARCHS`` is "all any
5023 noarch ${PACKAGE_EXTRA_ARCHS} ${MACHINE_ARCH}".
5024
5025 PACKAGE_BEFORE_PN
5026 Enables easily adding packages to ``PACKAGES`` before ``${PN}`` so
5027 that those added packages can pick up files that would normally be
5028 included in the default package.
5029
5030 PACKAGE_CLASSES
5031 This variable, which is set in the ``local.conf`` configuration file
5032 found in the ``conf`` folder of the
5033 :term:`Build Directory`, specifies the package manager the
5034 OpenEmbedded build system uses when packaging data.
5035
5036 You can provide one or more of the following arguments for the
5037 variable: PACKAGE_CLASSES ?= "package_rpm package_deb package_ipk
5038 package_tar"
5039
5040 .. note::
5041
5042 While it is a legal option, the
5043 package_tar
5044 class has limited functionality due to no support for package
5045 dependencies by that backend. Therefore, it is recommended that
5046 you do not use it.
5047
5048 The build system uses only the first argument in the list as the
5049 package manager when creating your image or SDK. However, packages
5050 will be created using any additional packaging classes you specify.
5051 For example, if you use the following in your ``local.conf`` file:
5052 ::
5053
5054 PACKAGE_CLASSES ?= "package_ipk"
5055
5056 The OpenEmbedded build system uses
5057 the IPK package manager to create your image or SDK.
5058
5059 For information on packaging and build performance effects as a
5060 result of the package manager in use, see the
5061 ":ref:`package.bbclass <ref-classes-package>`" section.
5062
5063 PACKAGE_DEBUG_SPLIT_STYLE
5064 Determines how to split up the binary and debug information when
5065 creating ``*-dbg`` packages to be used with the GNU Project Debugger
5066 (GDB).
5067
5068 With the ``PACKAGE_DEBUG_SPLIT_STYLE`` variable, you can control
5069 where debug information, which can include or exclude source files,
5070 is stored:
5071
5072 - ".debug": Debug symbol files are placed next to the binary in a
5073 ``.debug`` directory on the target. For example, if a binary is
5074 installed into ``/bin``, the corresponding debug symbol files are
5075 installed in ``/bin/.debug``. Source files are placed in
5076 ``/usr/src/debug``.
5077
5078 - "debug-file-directory": Debug symbol files are placed under
5079 ``/usr/lib/debug`` on the target, and separated by the path from
5080 where the binary is installed. For example, if a binary is
5081 installed in ``/bin``, the corresponding debug symbols are
5082 installed in ``/usr/lib/debug/bin``. Source files are placed in
5083 ``/usr/src/debug``.
5084
5085 - "debug-without-src": The same behavior as ".debug" previously
5086 described with the exception that no source files are installed.
5087
5088 - "debug-with-srcpkg": The same behavior as ".debug" previously
5089 described with the exception that all source files are placed in a
5090 separate ``*-src`` pkg. This is the default behavior.
5091
5092 You can find out more about debugging using GDB by reading the
5093 ":ref:`platdev-gdb-remotedebug`" section
5094 in the Yocto Project Development Tasks Manual.
5095
5096 PACKAGE_EXCLUDE_COMPLEMENTARY
5097 Prevents specific packages from being installed when you are
5098 installing complementary packages.
5099
5100 You might find that you want to prevent installing certain packages
5101 when you are installing complementary packages. For example, if you
5102 are using :term:`IMAGE_FEATURES` to install
5103 ``dev-pkgs``, you might not want to install all packages from a
5104 particular multilib. If you find yourself in this situation, you can
5105 use the ``PACKAGE_EXCLUDE_COMPLEMENTARY`` variable to specify regular
5106 expressions to match the packages you want to exclude.
5107
5108 PACKAGE_EXCLUDE
5109 Lists packages that should not be installed into an image. For
5110 example:
5111 ::
5112
5113 PACKAGE_EXCLUDE = "package_name package_name package_name ..."
5114
5115 You can set this variable globally in your ``local.conf`` file or you
5116 can attach it to a specific image recipe by using the recipe name
5117 override:
5118 ::
5119
5120 PACKAGE_EXCLUDE_pn-target_image = "package_name"
5121
5122 If you choose to not install a package using this variable and some
5123 other package is dependent on it (i.e. listed in a recipe's
5124 :term:`RDEPENDS` variable), the OpenEmbedded build
5125 system generates a fatal installation error. Because the build system
5126 halts the process with a fatal error, you can use the variable with
5127 an iterative development process to remove specific components from a
5128 system.
5129
5130 Support for this variable exists only when using the IPK and RPM
5131 packaging backend. Support does not exist for DEB.
5132
5133 See the :term:`NO_RECOMMENDATIONS` and the
5134 :term:`BAD_RECOMMENDATIONS` variables for
5135 related information.
5136
5137 PACKAGE_EXTRA_ARCHS
5138 Specifies the list of architectures compatible with the device CPU.
5139 This variable is useful when you build for several different devices
5140 that use miscellaneous processors such as XScale and ARM926-EJS.
5141
5142 PACKAGE_FEED_ARCHS
5143 Optionally specifies the package architectures used as part of the
5144 package feed URIs during the build. When used, the
5145 ``PACKAGE_FEED_ARCHS`` variable is appended to the final package feed
5146 URI, which is constructed using the
5147 :term:`PACKAGE_FEED_URIS` and
5148 :term:`PACKAGE_FEED_BASE_PATHS`
5149 variables.
5150
5151 .. note::
5152
5153 You can use the
5154 PACKAGE_FEEDS_ARCHS
5155 variable to whitelist specific package architectures. If you do
5156 not need to whitelist specific architectures, which is a common
5157 case, you can omit this variable. Omitting the variable results in
5158 all available architectures for the current machine being included
5159 into remote package feeds.
5160
5161 Consider the following example where the ``PACKAGE_FEED_URIS``,
5162 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
5163 defined in your ``local.conf`` file:
5164 ::
5165
5166 PACKAGE_FEED_URIS = "https://example.com/packagerepos/release \
5167 https://example.com/packagerepos/updates"
5168 PACKAGE_FEED_BASE_PATHS = "rpm rpm-dev"
5169 PACKAGE_FEED_ARCHS = "all core2-64"
5170
5171 Given these settings, the resulting package feeds are as follows:
5172 ::
5173
5174 https://example.com/packagerepos/release/rpm/all
5175 https://example.com/packagerepos/release/rpm/core2-64
5176 https://example.com/packagerepos/release/rpm-dev/all
5177 https://example.com/packagerepos/release/rpm-dev/core2-64
5178 https://example.com/packagerepos/updates/rpm/all
5179 https://example.com/packagerepos/updates/rpm/core2-64
5180 https://example.com/packagerepos/updates/rpm-dev/all
5181 https://example.com/packagerepos/updates/rpm-dev/core2-64
5182
5183 PACKAGE_FEED_BASE_PATHS
5184 Specifies the base path used when constructing package feed URIs. The
5185 ``PACKAGE_FEED_BASE_PATHS`` variable makes up the middle portion of a
5186 package feed URI used by the OpenEmbedded build system. The base path
5187 lies between the :term:`PACKAGE_FEED_URIS`
5188 and :term:`PACKAGE_FEED_ARCHS` variables.
5189
5190 Consider the following example where the ``PACKAGE_FEED_URIS``,
5191 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
5192 defined in your ``local.conf`` file:
5193 ::
5194
5195 PACKAGE_FEED_URIS = "https://example.com/packagerepos/release \
5196 https://example.com/packagerepos/updates"
5197 PACKAGE_FEED_BASE_PATHS = "rpm rpm-dev"
5198 PACKAGE_FEED_ARCHS = "all core2-64"
5199
5200 Given these settings, the resulting package feeds are as follows:
5201 ::
5202
5203 https://example.com/packagerepos/release/rpm/all
5204 https://example.com/packagerepos/release/rpm/core2-64
5205 https://example.com/packagerepos/release/rpm-dev/all
5206 https://example.com/packagerepos/release/rpm-dev/core2-64
5207 https://example.com/packagerepos/updates/rpm/all
5208 https://example.com/packagerepos/updates/rpm/core2-64
5209 https://example.com/packagerepos/updates/rpm-dev/all
5210 https://example.com/packagerepos/updates/rpm-dev/core2-64
5211
5212 PACKAGE_FEED_URIS
5213 Specifies the front portion of the package feed URI used by the
5214 OpenEmbedded build system. Each final package feed URI is comprised
5215 of ``PACKAGE_FEED_URIS``,
5216 :term:`PACKAGE_FEED_BASE_PATHS`, and
5217 :term:`PACKAGE_FEED_ARCHS` variables.
5218
5219 Consider the following example where the ``PACKAGE_FEED_URIS``,
5220 ``PACKAGE_FEED_BASE_PATHS``, and ``PACKAGE_FEED_ARCHS`` variables are
5221 defined in your ``local.conf`` file:
5222 ::
5223
5224 PACKAGE_FEED_URIS = "https://example.com/packagerepos/release \
5225 https://example.com/packagerepos/updates"
5226 PACKAGE_FEED_BASE_PATHS = "rpm rpm-dev"
5227 PACKAGE_FEED_ARCHS = "all core2-64"
5228
5229 Given these settings, the resulting package feeds are as follows:
5230 ::
5231
5232 https://example.com/packagerepos/release/rpm/all
5233 https://example.com/packagerepos/release/rpm/core2-64
5234 https://example.com/packagerepos/release/rpm-dev/all
5235 https://example.com/packagerepos/release/rpm-dev/core2-64
5236 https://example.com/packagerepos/updates/rpm/all
5237 https://example.com/packagerepos/updates/rpm/core2-64
5238 https://example.com/packagerepos/updates/rpm-dev/all
5239 https://example.com/packagerepos/updates/rpm-dev/core2-64
5240
5241 PACKAGE_INSTALL
5242 The final list of packages passed to the package manager for
5243 installation into the image.
5244
5245 Because the package manager controls actual installation of all
5246 packages, the list of packages passed using ``PACKAGE_INSTALL`` is
5247 not the final list of packages that are actually installed. This
5248 variable is internal to the image construction code. Consequently, in
5249 general, you should use the
5250 :term:`IMAGE_INSTALL` variable to specify
5251 packages for installation. The exception to this is when working with
5252 the
5253 ```core-image-minimal-initramfs`` <#images-core-image-minimal-initramfs>`__
5254 image. When working with an initial RAM filesystem (initramfs) image,
5255 use the ``PACKAGE_INSTALL`` variable. For information on creating an
5256 initramfs, see the ":ref:`building-an-initramfs-image`" section
5257 in the Yocto Project Development Tasks Manual.
5258
5259 PACKAGE_INSTALL_ATTEMPTONLY
5260 Specifies a list of packages the OpenEmbedded build system attempts
5261 to install when creating an image. If a listed package fails to
5262 install, the build system does not generate an error. This variable
5263 is generally not user-defined.
5264
5265 PACKAGE_PREPROCESS_FUNCS
5266 Specifies a list of functions run to pre-process the
5267 :term:`PKGD` directory prior to splitting the files out
5268 to individual packages.
5269
5270 PACKAGE_WRITE_DEPS
5271 Specifies a list of dependencies for post-installation and
5272 pre-installation scripts on native/cross tools. If your
5273 post-installation or pre-installation script can execute at rootfs
5274 creation time rather than on the target but depends on a native tool
5275 in order to execute, you need to list the tools in
5276 ``PACKAGE_WRITE_DEPS``.
5277
5278 For information on running post-installation scripts, see the
5279 ":ref:`dev-manual/dev-manual-common-tasks:post-installation scripts`"
5280 section in the Yocto Project Development Tasks Manual.
5281
5282 PACKAGECONFIG
5283 This variable provides a means of enabling or disabling features of a
5284 recipe on a per-recipe basis. ``PACKAGECONFIG`` blocks are defined in
5285 recipes when you specify features and then arguments that define
5286 feature behaviors. Here is the basic block structure (broken over
5287 multiple lines for readability):
5288 ::
5289
5290 PACKAGECONFIG ??= "f1 f2 f3 ..."
5291 PACKAGECONFIG[f1] = "\
5292 --with-f1, \
5293 --without-f1, \
5294 build-deps-for-f1, \
5295 runtime-deps-for-f1, \
5296 runtime-recommends-for-f1, \
5297 packageconfig-conflicts-for-f1"
5298 PACKAGECONFIG[f2] = "\
5299 ... and so on and so on ...
5300
5301 The ``PACKAGECONFIG`` variable itself specifies a space-separated
5302 list of the features to enable. Following the features, you can
5303 determine the behavior of each feature by providing up to six
5304 order-dependent arguments, which are separated by commas. You can
5305 omit any argument you like but must retain the separating commas. The
5306 order is important and specifies the following:
5307
5308 1. Extra arguments that should be added to the configure script
5309 argument list (:term:`EXTRA_OECONF` or
5310 :term:`PACKAGECONFIG_CONFARGS`) if
5311 the feature is enabled.
5312
5313 2. Extra arguments that should be added to ``EXTRA_OECONF`` or
5314 ``PACKAGECONFIG_CONFARGS`` if the feature is disabled.
5315
5316 3. Additional build dependencies (:term:`DEPENDS`)
5317 that should be added if the feature is enabled.
5318
5319 4. Additional runtime dependencies (:term:`RDEPENDS`)
5320 that should be added if the feature is enabled.
5321
5322 5. Additional runtime recommendations
5323 (:term:`RRECOMMENDS`) that should be added if
5324 the feature is enabled.
5325
5326 6. Any conflicting (that is, mutually exclusive) ``PACKAGECONFIG``
5327 settings for this feature.
5328
5329 Consider the following ``PACKAGECONFIG`` block taken from the
5330 ``librsvg`` recipe. In this example the feature is ``gtk``, which has
5331 three arguments that determine the feature's behavior.
5332 ::
5333
5334 PACKAGECONFIG[gtk] = "--with-gtk3,--without-gtk3,gtk+3"
5335
5336 The
5337 ``--with-gtk3`` and ``gtk+3`` arguments apply only if the feature is
5338 enabled. In this case, ``--with-gtk3`` is added to the configure
5339 script argument list and ``gtk+3`` is added to ``DEPENDS``. On the
5340 other hand, if the feature is disabled say through a ``.bbappend``
5341 file in another layer, then the second argument ``--without-gtk3`` is
5342 added to the configure script instead.
5343
5344 The basic ``PACKAGECONFIG`` structure previously described holds true
5345 regardless of whether you are creating a block or changing a block.
5346 When creating a block, use the structure inside your recipe.
5347
5348 If you want to change an existing ``PACKAGECONFIG`` block, you can do
5349 so one of two ways:
5350
5351 - *Append file:* Create an append file named
5352 recipename\ ``.bbappend`` in your layer and override the value of
5353 ``PACKAGECONFIG``. You can either completely override the
5354 variable:
5355 ::
5356
5357 PACKAGECONFIG = "f4 f5"
5358
5359 Or, you can just append the variable:
5360 ::
5361
5362 PACKAGECONFIG_append = " f4"
5363
5364 - *Configuration file:* This method is identical to changing the
5365 block through an append file except you edit your ``local.conf``
5366 or ``mydistro.conf`` file. As with append files previously
5367 described, you can either completely override the variable:
5368 PACKAGECONFIG_pn-recipename = "f4 f5" Or, you can just amend the
5369 variable:
5370 ::
5371
5372 PACKAGECONFIG_append_pn-recipename = " f4"
5373
5374 PACKAGECONFIG_CONFARGS
5375 A space-separated list of configuration options generated from the
5376 :term:`PACKAGECONFIG` setting.
5377
5378 Classes such as :ref:`autotools <ref-classes-autotools>` and
5379 :ref:`cmake <ref-classes-cmake>` use ``PACKAGECONFIG_CONFARGS`` to
5380 pass ``PACKAGECONFIG`` options to ``configure`` and ``cmake``,
5381 respectively. If you are using ``PACKAGECONFIG`` but not a class that
5382 handles the ``do_configure`` task, then you need to use
5383 ``PACKAGECONFIG_CONFARGS`` appropriately.
5384
5385 PACKAGEGROUP_DISABLE_COMPLEMENTARY
5386 For recipes inheriting the
5387 :ref:`packagegroup <ref-classes-packagegroup>` class, setting
5388 ``PACKAGEGROUP_DISABLE_COMPLEMENTARY`` to "1" specifies that the
5389 normal complementary packages (i.e. ``-dev``, ``-dbg``, and so forth)
5390 should not be automatically created by the ``packagegroup`` recipe,
5391 which is the default behavior.
5392
5393 PACKAGES
5394 The list of packages the recipe creates. The default value is the
5395 following:
5396 ::
5397
5398 ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}
5399
5400 During packaging, the :ref:`ref-tasks-package` task
5401 goes through ``PACKAGES`` and uses the :term:`FILES`
5402 variable corresponding to each package to assign files to the
5403 package. If a file matches the ``FILES`` variable for more than one
5404 package in ``PACKAGES``, it will be assigned to the earliest
5405 (leftmost) package.
5406
5407 Packages in the variable's list that are empty (i.e. where none of
5408 the patterns in ``FILES_``\ pkg match any files installed by the
5409 :ref:`ref-tasks-install` task) are not generated,
5410 unless generation is forced through the
5411 :term:`ALLOW_EMPTY` variable.
5412
5413 PACKAGES_DYNAMIC
5414 A promise that your recipe satisfies runtime dependencies for
5415 optional modules that are found in other recipes.
5416 ``PACKAGES_DYNAMIC`` does not actually satisfy the dependencies, it
5417 only states that they should be satisfied. For example, if a hard,
5418 runtime dependency (:term:`RDEPENDS`) of another
5419 package is satisfied at build time through the ``PACKAGES_DYNAMIC``
5420 variable, but a package with the module name is never actually
5421 produced, then the other package will be broken. Thus, if you attempt
5422 to include that package in an image, you will get a dependency
5423 failure from the packaging system during the
5424 :ref:`ref-tasks-rootfs` task.
5425
5426 Typically, if there is a chance that such a situation can occur and
5427 the package that is not created is valid without the dependency being
5428 satisfied, then you should use :term:`RRECOMMENDS`
5429 (a soft runtime dependency) instead of ``RDEPENDS``.
5430
5431 For an example of how to use the ``PACKAGES_DYNAMIC`` variable when
5432 you are splitting packages, see the
5433 ":ref:`dev-manual/dev-manual-common-tasks:handling optional module packaging`"
5434 section in the Yocto Project Development Tasks Manual.
5435
5436 PACKAGESPLITFUNCS
5437 Specifies a list of functions run to perform additional splitting of
5438 files into individual packages. Recipes can either prepend to this
5439 variable or prepend to the ``populate_packages`` function in order to
5440 perform additional package splitting. In either case, the function
5441 should set :term:`PACKAGES`,
5442 :term:`FILES`, :term:`RDEPENDS` and
5443 other packaging variables appropriately in order to perform the
5444 desired splitting.
5445
5446 PARALLEL_MAKE
5447 Extra options passed to the ``make`` command during the
5448 :ref:`ref-tasks-compile` task in order to specify
5449 parallel compilation on the local build host. This variable is
5450 usually in the form "-j x", where x represents the maximum number of
5451 parallel threads ``make`` can run.
5452
5453 .. note::
5454
5455 In order for
5456 PARALLEL_MAKE
5457 to be effective,
5458 make
5459 must be called with
5460 ${
5461 EXTRA_OEMAKE
5462 }
5463 . An easy way to ensure this is to use the
5464 oe_runmake
5465 function.
5466
5467 By default, the OpenEmbedded build system automatically sets this
5468 variable to be equal to the number of cores the build system uses.
5469
5470 .. note::
5471
5472 If the software being built experiences dependency issues during
5473 the
5474 do_compile
5475 task that result in race conditions, you can clear the
5476 PARALLEL_MAKE
5477 variable within the recipe as a workaround. For information on
5478 addressing race conditions, see the "
5479 Debugging Parallel Make Races
5480 " section in the Yocto Project Development Tasks Manual.
5481
5482 For single socket systems (i.e. one CPU), you should not have to
5483 override this variable to gain optimal parallelism during builds.
5484 However, if you have very large systems that employ multiple physical
5485 CPUs, you might want to make sure the ``PARALLEL_MAKE`` variable is
5486 not set higher than "-j 20".
5487
5488 For more information on speeding up builds, see the
5489 ":ref:`dev-manual/dev-manual-common-tasks:speeding up a build`"
5490 section in the Yocto Project Development Tasks Manual.
5491
5492 PARALLEL_MAKEINST
5493 Extra options passed to the ``make install`` command during the
5494 :ref:`ref-tasks-install` task in order to specify
5495 parallel installation. This variable defaults to the value of
5496 :term:`PARALLEL_MAKE`.
5497
5498 .. note::
5499
5500 In order for ``PARALLEL_MAKEINST`` to be effective, ``make`` must
5501 be called with
5502 ``${``\ :term:`EXTRA_OEMAKE`\ ``}``. An easy
5503 way to ensure this is to use the ``oe_runmake`` function.
5504
5505 If the software being built experiences dependency issues during
5506 the ``do_install`` task that result in race conditions, you can
5507 clear the ``PARALLEL_MAKEINST`` variable within the recipe as a
5508 workaround. For information on addressing race conditions, see the
5509 ":ref:`dev-manual/dev-manual-common-tasks:debugging parallel make races`"
5510 section in the Yocto Project Development Tasks Manual.
5511
5512 PATCHRESOLVE
5513 Determines the action to take when a patch fails. You can set this
5514 variable to one of two values: "noop" and "user".
5515
5516 The default value of "noop" causes the build to simply fail when the
5517 OpenEmbedded build system cannot successfully apply a patch. Setting
5518 the value to "user" causes the build system to launch a shell and
5519 places you in the right location so that you can manually resolve the
5520 conflicts.
5521
5522 Set this variable in your ``local.conf`` file.
5523
5524 PATCHTOOL
5525 Specifies the utility used to apply patches for a recipe during the
5526 :ref:`ref-tasks-patch` task. You can specify one of
5527 three utilities: "patch", "quilt", or "git". The default utility used
5528 is "quilt" except for the quilt-native recipe itself. Because the
5529 quilt tool is not available at the time quilt-native is being
5530 patched, it uses "patch".
5531
5532 If you wish to use an alternative patching tool, set the variable in
5533 the recipe using one of the following:
5534 ::
5535
5536 PATCHTOOL = "patch"
5537 PATCHTOOL = "quilt"
5538 PATCHTOOL = "git"
5539
5540 PE
5541 The epoch of the recipe. By default, this variable is unset. The
5542 variable is used to make upgrades possible when the versioning scheme
5543 changes in some backwards incompatible way.
5544
5545 ``PE`` is the default value of the :term:`PKGE` variable.
5546
5547 PF
5548 Specifies the recipe or package name and includes all version and
5549 revision numbers (i.e. ``glibc-2.13-r20+svnr15508/`` and
5550 ``bash-4.2-r1/``). This variable is comprised of the following:
5551 ${:term:`PN`}-${:term:`EXTENDPE`}${:term:`PV`}-${:term:`PR`}
5552
5553 PIXBUF_PACKAGES
5554 When inheriting the :ref:`pixbufcache <ref-classes-pixbufcache>`
5555 class, this variable identifies packages that contain the pixbuf
5556 loaders used with ``gdk-pixbuf``. By default, the ``pixbufcache``
5557 class assumes that the loaders are in the recipe's main package (i.e.
5558 ``${``\ :term:`PN`\ ``}``). Use this variable if the
5559 loaders you need are in a package other than that main package.
5560
5561 PKG
5562 The name of the resulting package created by the OpenEmbedded build
5563 system.
5564
5565 .. note::
5566
5567 When using the
5568 PKG
5569 variable, you must use a package name override.
5570
5571 For example, when the :ref:`debian <ref-classes-debian>` class
5572 renames the output package, it does so by setting
5573 ``PKG_packagename``.
5574
5575 PKG_CONFIG_PATH
5576 The path to ``pkg-config`` files for the current build context.
5577 ``pkg-config`` reads this variable from the environment.
5578
5579 PKGD
5580 Points to the destination directory for files to be packaged before
5581 they are split into individual packages. This directory defaults to
5582 the following:
5583 ::
5584
5585 ${WORKDIR}/package
5586
5587 Do not change this default.
5588
5589 PKGDATA_DIR
5590 Points to a shared, global-state directory that holds data generated
5591 during the packaging process. During the packaging process, the
5592 :ref:`ref-tasks-packagedata` task packages data
5593 for each recipe and installs it into this temporary, shared area.
5594 This directory defaults to the following, which you should not
5595 change:
5596 ::
5597
5598 ${STAGING_DIR_HOST}/pkgdata
5599
5600 For examples of how this data is used, see the
5601 ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
5602 section in the Yocto Project Overview and Concepts Manual and the
5603 ":ref:`dev-manual/dev-manual-common-tasks:viewing package information with \`\`oe-pkgdata-util\`\``"
5604 section in the Yocto Project Development Tasks Manual. For more
5605 information on the shared, global-state directory, see
5606 :term:`STAGING_DIR_HOST`.
5607
5608 PKGDEST
5609 Points to the parent directory for files to be packaged after they
5610 have been split into individual packages. This directory defaults to
5611 the following:
5612 ::
5613
5614 ${WORKDIR}/packages-split
5615
5616 Under this directory, the build system creates directories for each
5617 package specified in :term:`PACKAGES`. Do not change
5618 this default.
5619
5620 PKGDESTWORK
5621 Points to a temporary work area where the
5622 :ref:`ref-tasks-package` task saves package metadata.
5623 The ``PKGDESTWORK`` location defaults to the following:
5624 ::
5625
5626 ${WORKDIR}/pkgdata
5627
5628 Do not change this default.
5629
5630 The :ref:`ref-tasks-packagedata` task copies the
5631 package metadata from ``PKGDESTWORK`` to
5632 :term:`PKGDATA_DIR` to make it available globally.
5633
5634 PKGE
5635 The epoch of the package(s) built by the recipe. By default, ``PKGE``
5636 is set to :term:`PE`.
5637
5638 PKGR
5639 The revision of the package(s) built by the recipe. By default,
5640 ``PKGR`` is set to :term:`PR`.
5641
5642 PKGV
5643 The version of the package(s) built by the recipe. By default,
5644 ``PKGV`` is set to :term:`PV`.
5645
5646 PN
5647 This variable can have two separate functions depending on the
5648 context: a recipe name or a resulting package name.
5649
5650 ``PN`` refers to a recipe name in the context of a file used by the
5651 OpenEmbedded build system as input to create a package. The name is
5652 normally extracted from the recipe file name. For example, if the
5653 recipe is named ``expat_2.0.1.bb``, then the default value of ``PN``
5654 will be "expat".
5655
5656 The variable refers to a package name in the context of a file
5657 created or produced by the OpenEmbedded build system.
5658
5659 If applicable, the ``PN`` variable also contains any special suffix
5660 or prefix. For example, using ``bash`` to build packages for the
5661 native machine, ``PN`` is ``bash-native``. Using ``bash`` to build
5662 packages for the target and for Multilib, ``PN`` would be ``bash``
5663 and ``lib64-bash``, respectively.
5664
5665 PNBLACKLIST
5666 Lists recipes you do not want the OpenEmbedded build system to build.
5667 This variable works in conjunction with the
5668 :ref:`blacklist <ref-classes-blacklist>` class, which is inherited
5669 globally.
5670
5671 To prevent a recipe from being built, use the ``PNBLACKLIST``
5672 variable in your ``local.conf`` file. Here is an example that
5673 prevents ``myrecipe`` from being built:
5674 ::
5675
5676 PNBLACKLIST[myrecipe] = "Not supported by our organization."
5677
5678 POPULATE_SDK_POST_HOST_COMMAND
5679 Specifies a list of functions to call once the OpenEmbedded build
5680 system has created the host part of the SDK. You can specify
5681 functions separated by semicolons:
5682 ::
5683
5684 POPULATE_SDK_POST_HOST_COMMAND += "function; ... "
5685
5686 If you need to pass the SDK path to a command within a function, you
5687 can use ``${SDK_DIR}``, which points to the parent directory used by
5688 the OpenEmbedded build system when creating SDK output. See the
5689 :term:`SDK_DIR` variable for more information.
5690
5691 POPULATE_SDK_POST_TARGET_COMMAND
5692 Specifies a list of functions to call once the OpenEmbedded build
5693 system has created the target part of the SDK. You can specify
5694 functions separated by semicolons:
5695 ::
5696
5697 POPULATE_SDK_POST_TARGET_COMMAND += "function; ... "
5698
5699 If you need to pass the SDK path to a command within a function, you
5700 can use ``${SDK_DIR}``, which points to the parent directory used by
5701 the OpenEmbedded build system when creating SDK output. See the
5702 :term:`SDK_DIR` variable for more information.
5703
5704 PR
5705 The revision of the recipe. The default value for this variable is
5706 "r0". Subsequent revisions of the recipe conventionally have the
5707 values "r1", "r2", and so forth. When :term:`PV` increases,
5708 ``PR`` is conventionally reset to "r0".
5709
5710 .. note::
5711
5712 The OpenEmbedded build system does not need the aid of
5713 PR
5714 to know when to rebuild a recipe. The build system uses the task
5715 input checksums
5716 along with the
5717 stamp
5718 and
5719 shared state cache
5720 mechanisms.
5721
5722 The ``PR`` variable primarily becomes significant when a package
5723 manager dynamically installs packages on an already built image. In
5724 this case, ``PR``, which is the default value of
5725 :term:`PKGR`, helps the package manager distinguish which
5726 package is the most recent one in cases where many packages have the
5727 same ``PV`` (i.e. ``PKGV``). A component having many packages with
5728 the same ``PV`` usually means that the packages all install the same
5729 upstream version, but with later (``PR``) version packages including
5730 packaging fixes.
5731
5732 .. note::
5733
5734 PR
5735 does not need to be increased for changes that do not change the
5736 package contents or metadata.
5737
5738 Because manually managing ``PR`` can be cumbersome and error-prone,
5739 an automated solution exists. See the
5740 ":ref:`dev-manual/dev-manual-common-tasks:working with a pr service`" section
5741 in the Yocto Project Development Tasks Manual for more information.
5742
5743 PREFERRED_PROVIDER
5744 If multiple recipes provide the same item, this variable determines
5745 which recipe is preferred and thus provides the item (i.e. the
5746 preferred provider). You should always suffix this variable with the
5747 name of the provided item. And, you should define the variable using
5748 the preferred recipe's name (:term:`PN`). Here is a common
5749 example:
5750 ::
5751
5752 PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
5753
5754 In the previous example, multiple recipes are providing "virtual/kernel".
5755 The ``PREFERRED_PROVIDER`` variable is set with the name (``PN``) of
5756 the recipe you prefer to provide "virtual/kernel".
5757
5758 Following are more examples:
5759 ::
5760
5761 PREFERRED_PROVIDER_virtual/xserver = "xserver-xf86"
5762 PREFERRED_PROVIDER_virtual/libgl ?= "mesa"
5763
5764 For more
5765 information, see the ":ref:`metadata-virtual-providers`"
5766 section in the Yocto Project Development Tasks Manual.
5767
5768 .. note::
5769
5770 If you use a
5771 virtual/\*
5772 item with
5773 PREFERRED_PROVIDER
5774 , then any recipe that
5775 PROVIDES
5776 that item but is not selected (defined) by
5777 PREFERRED_PROVIDER
5778 is prevented from building, which is usually desirable since this
5779 mechanism is designed to select between mutually exclusive
5780 alternative providers.
5781
5782 PREFERRED_VERSION
5783 If multiple versions of recipes exist, this variable determines which
5784 version is given preference. You must always suffix the variable with
5785 the :term:`PN` you want to select, and you should set the
5786 :term:`PV` accordingly for precedence.
5787
5788 The ``PREFERRED_VERSION`` variable supports limited wildcard use
5789 through the "``%``" character. You can use the character to match any
5790 number of characters, which can be useful when specifying versions
5791 that contain long revision numbers that potentially change. Here are
5792 two examples:
5793 ::
5794
5795 PREFERRED_VERSION_python = "3.4.0"
5796 PREFERRED_VERSION_linux-yocto = "5.0%"
5797
5798 .. note::
5799
5800 The use of the "%" character is limited in that it only works at the end of the
5801 string. You cannot use the wildcard character in any other
5802 location of the string.
5803
5804 The specified version is matched against :term:`PV`, which
5805 does not necessarily match the version part of the recipe's filename.
5806 For example, consider two recipes ``foo_1.2.bb`` and ``foo_git.bb``
5807 where ``foo_git.bb`` contains the following assignment:
5808 ::
5809
5810 PV = "1.1+git${SRCPV}"
5811
5812 In this case, the correct way to select
5813 ``foo_git.bb`` is by using an assignment such as the following:
5814 ::
5815
5816 PREFERRED_VERSION_foo = "1.1+git%"
5817
5818 Compare that previous example
5819 against the following incorrect example, which does not work:
5820 ::
5821
5822 PREFERRED_VERSION_foo = "git"
5823
5824 Sometimes the ``PREFERRED_VERSION`` variable can be set by
5825 configuration files in a way that is hard to change. You can use
5826 :term:`OVERRIDES` to set a machine-specific
5827 override. Here is an example:
5828 ::
5829
5830 PREFERRED_VERSION_linux-yocto_qemux86 = "5.0%"
5831
5832 Although not recommended, worst case, you can also use the
5833 "forcevariable" override, which is the strongest override possible.
5834 Here is an example:
5835 ::
5836
5837 PREFERRED_VERSION_linux-yocto_forcevariable = "5.0%"
5838
5839 .. note::
5840
5841 The \_forcevariable override is not handled specially. This override
5842 only works because the default value of OVERRIDES includes "forcevariable".
5843
5844 PREMIRRORS
5845 Specifies additional paths from which the OpenEmbedded build system
5846 gets source code. When the build system searches for source code, it
5847 first tries the local download directory. If that location fails, the
5848 build system tries locations defined by ``PREMIRRORS``, the upstream
5849 source, and then locations specified by
5850 :term:`MIRRORS` in that order.
5851
5852 Assuming your distribution (:term:`DISTRO`) is "poky",
5853 the default value for ``PREMIRRORS`` is defined in the
5854 ``conf/distro/poky.conf`` file in the ``meta-poky`` Git repository.
5855
5856 Typically, you could add a specific server for the build system to
5857 attempt before any others by adding something like the following to
5858 the ``local.conf`` configuration file in the
5859 :term:`Build Directory`:
5860 ::
5861
5862 PREMIRRORS_prepend = "\
5863 git://.*/.* http://www.yoctoproject.org/sources/ \n \
5864 ftp://.*/.* http://www.yoctoproject.org/sources/ \n \
5865 http://.*/.* http://www.yoctoproject.org/sources/ \n \
5866 https://.*/.* http://www.yoctoproject.org/sources/ \n"
5867
5868 These changes cause the
5869 build system to intercept Git, FTP, HTTP, and HTTPS requests and
5870 direct them to the ``http://`` sources mirror. You can use
5871 ``file://`` URLs to point to local directories or network shares as
5872 well.
5873
5874 PRIORITY
5875 Indicates the importance of a package.
5876
5877 ``PRIORITY`` is considered to be part of the distribution policy
5878 because the importance of any given recipe depends on the purpose for
5879 which the distribution is being produced. Thus, ``PRIORITY`` is not
5880 normally set within recipes.
5881
5882 You can set ``PRIORITY`` to "required", "standard", "extra", and
5883 "optional", which is the default.
5884
5885 PRIVATE_LIBS
5886 Specifies libraries installed within a recipe that should be ignored
5887 by the OpenEmbedded build system's shared library resolver. This
5888 variable is typically used when software being built by a recipe has
5889 its own private versions of a library normally provided by another
5890 recipe. In this case, you would not want the package containing the
5891 private libraries to be set as a dependency on other unrelated
5892 packages that should instead depend on the package providing the
5893 standard version of the library.
5894
5895 Libraries specified in this variable should be specified by their
5896 file name. For example, from the Firefox recipe in meta-browser:
5897 ::
5898
5899 PRIVATE_LIBS = "libmozjs.so \
5900 libxpcom.so \
5901 libnspr4.so \
5902 libxul.so \
5903 libmozalloc.so \
5904 libplc4.so \
5905 libplds4.so"
5906
5907 For more information, see the
5908 ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
5909 section in the Yocto Project Overview and Concepts Manual.
5910
5911 PROVIDES
5912 A list of aliases by which a particular recipe can be known. By
5913 default, a recipe's own ``PN`` is implicitly already in its
5914 ``PROVIDES`` list and therefore does not need to mention that it
5915 provides itself. If a recipe uses ``PROVIDES``, the additional
5916 aliases are synonyms for the recipe and can be useful for satisfying
5917 dependencies of other recipes during the build as specified by
5918 ``DEPENDS``.
5919
5920 Consider the following example ``PROVIDES`` statement from the recipe
5921 file ``eudev_3.2.9.bb``:
5922 ::
5923
5924 PROVIDES = "udev"
5925
5926 The ``PROVIDES`` statement
5927 results in the "eudev" recipe also being available as simply "udev".
5928
5929 .. note::
5930
5931 Given that a recipe's own recipe name is already implicitly in its
5932 own
5933 PROVIDES
5934 list, it is unnecessary to add aliases with the "+=" operator;
5935 using a simple assignment will be sufficient. In other words,
5936 while you could write:
5937 ::
5938
5939 PROVIDES += "udev"
5940
5941
5942 in the above, the "+=" is overkill and unnecessary.
5943
5944 In addition to providing recipes under alternate names, the
5945 ``PROVIDES`` mechanism is also used to implement virtual targets. A
5946 virtual target is a name that corresponds to some particular
5947 functionality (e.g. a Linux kernel). Recipes that provide the
5948 functionality in question list the virtual target in ``PROVIDES``.
5949 Recipes that depend on the functionality in question can include the
5950 virtual target in ``DEPENDS`` to leave the choice of provider open.
5951
5952 Conventionally, virtual targets have names on the form
5953 "virtual/function" (e.g. "virtual/kernel"). The slash is simply part
5954 of the name and has no syntactical significance.
5955
5956 The :term:`PREFERRED_PROVIDER` variable is
5957 used to select which particular recipe provides a virtual target.
5958
5959 .. note::
5960
5961 A corresponding mechanism for virtual runtime dependencies
5962 (packages) exists. However, the mechanism does not depend on any
5963 special functionality beyond ordinary variable assignments. For
5964 example, ``VIRTUAL-RUNTIME_dev_manager`` refers to the package of
5965 the component that manages the ``/dev`` directory.
5966
5967 Setting the "preferred provider" for runtime dependencies is as
5968 simple as using the following assignment in a configuration file:
5969 ::
5970
5971 VIRTUAL-RUNTIME_dev_manager = "udev"
5972
5973
5974 PRSERV_HOST
5975 The network based :term:`PR` service host and port.
5976
5977 The ``conf/local.conf.sample.extended`` configuration file in the
5978 :term:`Source Directory` shows how the
5979 ``PRSERV_HOST`` variable is set:
5980 ::
5981
5982 PRSERV_HOST = "localhost:0"
5983
5984 You must
5985 set the variable if you want to automatically start a local :ref:`PR
5986 service <dev-manual/dev-manual-common-tasks:working with a pr service>`. You can
5987 set ``PRSERV_HOST`` to other values to use a remote PR service.
5988
5989 PTEST_ENABLED
5990 Specifies whether or not :ref:`Package
5991 Test <dev-manual/dev-manual-common-tasks:testing packages with ptest>` (ptest)
5992 functionality is enabled when building a recipe. You should not set
5993 this variable directly. Enabling and disabling building Package Tests
5994 at build time should be done by adding "ptest" to (or removing it
5995 from) :term:`DISTRO_FEATURES`.
5996
5997 PV
5998 The version of the recipe. The version is normally extracted from the
5999 recipe filename. For example, if the recipe is named
6000 ``expat_2.0.1.bb``, then the default value of ``PV`` will be "2.0.1".
6001 ``PV`` is generally not overridden within a recipe unless it is
6002 building an unstable (i.e. development) version from a source code
6003 repository (e.g. Git or Subversion).
6004
6005 ``PV`` is the default value of the :term:`PKGV` variable.
6006
6007 PYTHON_ABI
6008 When used by recipes that inherit the
6009 :ref:`distutils3 <ref-classes-distutils3>`,
6010 :ref:`setuptools3 <ref-classes-setuptools3>`,
6011 :ref:`distutils <ref-classes-distutils>`, or
6012 :ref:`setuptools <ref-classes-setuptools>` classes, denotes the
6013 Application Binary Interface (ABI) currently in use for Python. By
6014 default, the ABI is "m". You do not have to set this variable as the
6015 OpenEmbedded build system sets it for you.
6016
6017 The OpenEmbedded build system uses the ABI to construct directory
6018 names used when installing the Python headers and libraries in
6019 sysroot (e.g. ``.../python3.3m/...``).
6020
6021 Recipes that inherit the ``distutils`` class during cross-builds also
6022 use this variable to locate the headers and libraries of the
6023 appropriate Python that the extension is targeting.
6024
6025 PYTHON_PN
6026 When used by recipes that inherit the
6027 `distutils3 <ref-classes-distutils3>`,
6028 :ref:`setuptools3 <ref-classes-setuptools3>`,
6029 :ref:`distutils <ref-classes-distutils>`, or
6030 :ref:`setuptools <ref-classes-setuptools>` classes, specifies the
6031 major Python version being built. For Python 3.x, ``PYTHON_PN`` would
6032 be "python3". You do not have to set this variable as the
6033 OpenEmbedded build system automatically sets it for you.
6034
6035 The variable allows recipes to use common infrastructure such as the
6036 following:
6037 ::
6038
6039 DEPENDS += "${PYTHON_PN}-native"
6040
6041 In the previous example,
6042 the version of the dependency is ``PYTHON_PN``.
6043
6044 RANLIB
6045 The minimal command and arguments to run ``ranlib``.
6046
6047 RCONFLICTS
6048 The list of packages that conflict with packages. Note that packages
6049 will not be installed if conflicting packages are not first removed.
6050
6051 Like all package-controlling variables, you must always use them in
6052 conjunction with a package name override. Here is an example:
6053 ::
6054
6055 RCONFLICTS_${PN} = "another_conflicting_package_name"
6056
6057 BitBake, which the OpenEmbedded build system uses, supports
6058 specifying versioned dependencies. Although the syntax varies
6059 depending on the packaging format, BitBake hides these differences
6060 from you. Here is the general syntax to specify versions with the
6061 ``RCONFLICTS`` variable:
6062 ::
6063
6064 RCONFLICTS_${PN} = "package (operator version)"
6065
6066 For ``operator``, you can specify the following: = < > <=
6067 >= For example, the following sets up a dependency on version 1.2 or
6068 greater of the package ``foo``:
6069 ::
6070
6071 RCONFLICTS_${PN} = "foo (>= 1.2)"
6072
6073 RDEPENDS
6074 Lists runtime dependencies of a package. These dependencies are other
6075 packages that must be installed in order for the package to function
6076 correctly. As an example, the following assignment declares that the
6077 package ``foo`` needs the packages ``bar`` and ``baz`` to be
6078 installed:
6079 ::
6080
6081 RDEPENDS_foo = "bar baz"
6082
6083 The most common types of package
6084 runtime dependencies are automatically detected and added. Therefore,
6085 most recipes do not need to set ``RDEPENDS``. For more information,
6086 see the
6087 ":ref:`overview-manual/overview-manual-concepts:automatically added runtime dependencies`"
6088 section in the Yocto Project Overview and Concepts Manual.
6089
6090 The practical effect of the above ``RDEPENDS`` assignment is that
6091 ``bar`` and ``baz`` will be declared as dependencies inside the
6092 package ``foo`` when it is written out by one of the
6093 ```do_package_write_*`` <#ref-tasks-package_write_deb>`__ tasks.
6094 Exactly how this is done depends on which package format is used,
6095 which is determined by
6096 :term:`PACKAGE_CLASSES`. When the
6097 corresponding package manager installs the package, it will know to
6098 also install the packages on which it depends.
6099
6100 To ensure that the packages ``bar`` and ``baz`` get built, the
6101 previous ``RDEPENDS`` assignment also causes a task dependency to be
6102 added. This dependency is from the recipe's
6103 :ref:`ref-tasks-build` (not to be confused with
6104 :ref:`ref-tasks-compile`) task to the
6105 ``do_package_write_*`` task of the recipes that build ``bar`` and
6106 ``baz``.
6107
6108 The names of the packages you list within ``RDEPENDS`` must be the
6109 names of other packages - they cannot be recipe names. Although
6110 package names and recipe names usually match, the important point
6111 here is that you are providing package names within the ``RDEPENDS``
6112 variable. For an example of the default list of packages created from
6113 a recipe, see the :term:`PACKAGES` variable.
6114
6115 Because the ``RDEPENDS`` variable applies to packages being built,
6116 you should always use the variable in a form with an attached package
6117 name (remember that a single recipe can build multiple packages). For
6118 example, suppose you are building a development package that depends
6119 on the ``perl`` package. In this case, you would use the following
6120 ``RDEPENDS`` statement:
6121 ::
6122
6123 RDEPENDS_${PN}-dev += "perl"
6124
6125 In the example,
6126 the development package depends on the ``perl`` package. Thus, the
6127 ``RDEPENDS`` variable has the ``${PN}-dev`` package name as part of
6128 the variable.
6129
6130 .. note::
6131
6132 RDEPENDS_${PN}-dev
6133 includes
6134 ${
6135 PN
6136 }
6137 by default. This default is set in the BitBake configuration file
6138 (
6139 meta/conf/bitbake.conf
6140 ). Be careful not to accidentally remove
6141 ${PN}
6142 when modifying
6143 RDEPENDS_${PN}-dev
6144 . Use the "+=" operator rather than the "=" operator.
6145
6146 The package names you use with ``RDEPENDS`` must appear as they would
6147 in the ``PACKAGES`` variable. The :term:`PKG` variable
6148 allows a different name to be used for the final package (e.g. the
6149 :ref:`debian <ref-classes-debian>` class uses this to rename
6150 packages), but this final package name cannot be used with
6151 ``RDEPENDS``, which makes sense as ``RDEPENDS`` is meant to be
6152 independent of the package format used.
6153
6154 BitBake, which the OpenEmbedded build system uses, supports
6155 specifying versioned dependencies. Although the syntax varies
6156 depending on the packaging format, BitBake hides these differences
6157 from you. Here is the general syntax to specify versions with the
6158 ``RDEPENDS`` variable:
6159 ::
6160
6161 RDEPENDS_${PN} = "package (operator version)"
6162
6163 For operator, you can specify the following: = < > <= >= For version,
6164 provide the version number.
6165
6166 .. note::
6167
6168 You can use
6169 EXTENDPKGV
6170 to provide a full package version specification.
6171
6172 For example, the following sets up a dependency on version 1.2 or
6173 greater of the package ``foo``:
6174 ::
6175
6176 RDEPENDS_${PN} = "foo (>= 1.2)"
6177
6178 For information on build-time dependencies, see the
6179 :term:`DEPENDS` variable. You can also see the
6180 ":ref:`Tasks <bitbake:bitbake-user-manual/bitbake-user-manual-metadata:tasks>`" and
6181 ":ref:`Dependencies <bitbake:bitbake-user-manual/bitbake-user-manual-execution:dependencies>`" sections in the
6182 BitBake User Manual for additional information on tasks and
6183 dependencies.
6184
6185 REQUIRED_DISTRO_FEATURES
6186 When inheriting the
6187 :ref:`distro_features_check <ref-classes-distro_features_check>`
6188 class, this variable identifies distribution features that must exist
6189 in the current configuration in order for the OpenEmbedded build
6190 system to build the recipe. In other words, if the
6191 ``REQUIRED_DISTRO_FEATURES`` variable lists a feature that does not
6192 appear in ``DISTRO_FEATURES`` within the current configuration, an
6193 error occurs and the build stops.
6194
6195 RM_WORK_EXCLUDE
6196 With ``rm_work`` enabled, this variable specifies a list of recipes
6197 whose work directories should not be removed. See the
6198 ":ref:`rm_work.bbclass <ref-classes-rm-work>`" section for more
6199 details.
6200
6201 ROOT_HOME
6202 Defines the root home directory. By default, this directory is set as
6203 follows in the BitBake configuration file:
6204 ::
6205
6206 ROOT_HOME ??= "/home/root"
6207
6208 .. note::
6209
6210 This default value is likely used because some embedded solutions
6211 prefer to have a read-only root filesystem and prefer to keep
6212 writeable data in one place.
6213
6214 You can override the default by setting the variable in any layer or
6215 in the ``local.conf`` file. Because the default is set using a "weak"
6216 assignment (i.e. "??="), you can use either of the following forms to
6217 define your override:
6218 ::
6219
6220 ROOT_HOME = "/root"
6221 ROOT_HOME ?= "/root"
6222
6223 These
6224 override examples use ``/root``, which is probably the most commonly
6225 used override.
6226
6227 ROOTFS
6228 Indicates a filesystem image to include as the root filesystem.
6229
6230 The ``ROOTFS`` variable is an optional variable used with the
6231 :ref:`image-live <ref-classes-image-live>` class.
6232
6233 ROOTFS_POSTINSTALL_COMMAND
6234 Specifies a list of functions to call after the OpenEmbedded build
6235 system has installed packages. You can specify functions separated by
6236 semicolons:
6237 ::
6238
6239 ROOTFS_POSTINSTALL_COMMAND += "function; ... "
6240
6241 If you need to pass the root filesystem path to a command within a
6242 function, you can use ``${IMAGE_ROOTFS}``, which points to the
6243 directory that becomes the root filesystem image. See the
6244 :term:`IMAGE_ROOTFS` variable for more
6245 information.
6246
6247 ROOTFS_POSTPROCESS_COMMAND
6248 Specifies a list of functions to call once the OpenEmbedded build
6249 system has created the root filesystem. You can specify functions
6250 separated by semicolons:
6251 ::
6252
6253 ROOTFS_POSTPROCESS_COMMAND += "function; ... "
6254
6255 If you need to pass the root filesystem path to a command within a
6256 function, you can use ``${IMAGE_ROOTFS}``, which points to the
6257 directory that becomes the root filesystem image. See the
6258 :term:`IMAGE_ROOTFS` variable for more
6259 information.
6260
6261 ROOTFS_POSTUNINSTALL_COMMAND
6262 Specifies a list of functions to call after the OpenEmbedded build
6263 system has removed unnecessary packages. When runtime package
6264 management is disabled in the image, several packages are removed
6265 including ``base-passwd``, ``shadow``, and ``update-alternatives``.
6266 You can specify functions separated by semicolons:
6267 ::
6268
6269 ROOTFS_POSTUNINSTALL_COMMAND += "function; ... "
6270
6271 If you need to pass the root filesystem path to a command within a
6272 function, you can use ``${IMAGE_ROOTFS}``, which points to the
6273 directory that becomes the root filesystem image. See the
6274 :term:`IMAGE_ROOTFS` variable for more
6275 information.
6276
6277 ROOTFS_PREPROCESS_COMMAND
6278 Specifies a list of functions to call before the OpenEmbedded build
6279 system has created the root filesystem. You can specify functions
6280 separated by semicolons:
6281 ::
6282
6283 ROOTFS_PREPROCESS_COMMAND += "function; ... "
6284
6285 If you need to pass the root filesystem path to a command within a
6286 function, you can use ``${IMAGE_ROOTFS}``, which points to the
6287 directory that becomes the root filesystem image. See the
6288 :term:`IMAGE_ROOTFS` variable for more
6289 information.
6290
6291 RPROVIDES
6292 A list of package name aliases that a package also provides. These
6293 aliases are useful for satisfying runtime dependencies of other
6294 packages both during the build and on the target (as specified by
6295 ``RDEPENDS``).
6296
6297 .. note::
6298
6299 A package's own name is implicitly already in its
6300 RPROVIDES
6301 list.
6302
6303 As with all package-controlling variables, you must always use the
6304 variable in conjunction with a package name override. Here is an
6305 example:
6306 ::
6307
6308 RPROVIDES_${PN} = "widget-abi-2"
6309
6310 RRECOMMENDS
6311 A list of packages that extends the usability of a package being
6312 built. The package being built does not depend on this list of
6313 packages in order to successfully build, but rather uses them for
6314 extended usability. To specify runtime dependencies for packages, see
6315 the ``RDEPENDS`` variable.
6316
6317 The package manager will automatically install the ``RRECOMMENDS``
6318 list of packages when installing the built package. However, you can
6319 prevent listed packages from being installed by using the
6320 :term:`BAD_RECOMMENDATIONS`,
6321 :term:`NO_RECOMMENDATIONS`, and
6322 :term:`PACKAGE_EXCLUDE` variables.
6323
6324 Packages specified in ``RRECOMMENDS`` need not actually be produced.
6325 However, a recipe must exist that provides each package, either
6326 through the :term:`PACKAGES` or
6327 :term:`PACKAGES_DYNAMIC` variables or the
6328 :term:`RPROVIDES` variable, or an error will occur
6329 during the build. If such a recipe does exist and the package is not
6330 produced, the build continues without error.
6331
6332 Because the ``RRECOMMENDS`` variable applies to packages being built,
6333 you should always attach an override to the variable to specify the
6334 particular package whose usability is being extended. For example,
6335 suppose you are building a development package that is extended to
6336 support wireless functionality. In this case, you would use the
6337 following:
6338 ::
6339
6340 RRECOMMENDS_${PN}-dev += "wireless_package_name"
6341
6342 In the
6343 example, the package name (``${PN}-dev``) must appear as it would in
6344 the ``PACKAGES`` namespace before any renaming of the output package
6345 by classes such as ``debian.bbclass``.
6346
6347 BitBake, which the OpenEmbedded build system uses, supports
6348 specifying versioned recommends. Although the syntax varies depending
6349 on the packaging format, BitBake hides these differences from you.
6350 Here is the general syntax to specify versions with the
6351 ``RRECOMMENDS`` variable:
6352 ::
6353
6354 RRECOMMENDS_${PN} = "package (operator version)"
6355
6356 For ``operator``, you can specify the following:
6357
6358 - =
6359 - <
6360 - >
6361 - <=
6362 - >=
6363
6364 For example, the following sets up a recommend on version 1.2 or
6365 greater of the package ``foo``:
6366 ::
6367
6368 RRECOMMENDS_${PN} = "foo (>= 1.2)"
6369
6370 RREPLACES
6371 A list of packages replaced by a package. The package manager uses
6372 this variable to determine which package should be installed to
6373 replace other package(s) during an upgrade. In order to also have the
6374 other package(s) removed at the same time, you must add the name of
6375 the other package to the ``RCONFLICTS`` variable.
6376
6377 As with all package-controlling variables, you must use this variable
6378 in conjunction with a package name override. Here is an example:
6379 ::
6380
6381 RREPLACES_${PN} = "other_package_being_replaced"
6382
6383 BitBake, which the OpenEmbedded build system uses, supports
6384 specifying versioned replacements. Although the syntax varies
6385 depending on the packaging format, BitBake hides these differences
6386 from you. Here is the general syntax to specify versions with the
6387 ``RREPLACES`` variable:
6388 ::
6389
6390 RREPLACES_${PN} = "package (operator version)"
6391
6392 For ``operator``, you can specify the following:
6393
6394 - =
6395 - <
6396 - >
6397 - <=
6398 - >=
6399
6400 For example, the following sets up a replacement using version 1.2
6401 or greater of the package ``foo``:
6402 ::
6403
6404 RREPLACES_${PN} = "foo (>= 1.2)"
6405
6406 RSUGGESTS
6407 A list of additional packages that you can suggest for installation
6408 by the package manager at the time a package is installed. Not all
6409 package managers support this functionality.
6410
6411 As with all package-controlling variables, you must always use this
6412 variable in conjunction with a package name override. Here is an
6413 example:
6414 ::
6415
6416 RSUGGESTS_${PN} = "useful_package another_package"
6417
6418 S
6419 The location in the :term:`Build Directory` where
6420 unpacked recipe source code resides. By default, this directory is
6421 ``${``\ :term:`WORKDIR`\ ``}/${``\ :term:`BPN`\ ``}-${``\ :term:`PV`\ ``}``,
6422 where ``${BPN}`` is the base recipe name and ``${PV}`` is the recipe
6423 version. If the source tarball extracts the code to a directory named
6424 anything other than ``${BPN}-${PV}``, or if the source code is
6425 fetched from an SCM such as Git or Subversion, then you must set
6426 ``S`` in the recipe so that the OpenEmbedded build system knows where
6427 to find the unpacked source.
6428
6429 As an example, assume a :term:`Source Directory`
6430 top-level folder named ``poky`` and a default Build Directory at
6431 ``poky/build``. In this case, the work directory the build system
6432 uses to keep the unpacked recipe for ``db`` is the following:
6433 ::
6434
6435 poky/build/tmp/work/qemux86-poky-linux/db/5.1.19-r3/db-5.1.19
6436
6437 The unpacked source code resides in the ``db-5.1.19`` folder.
6438
6439 This next example assumes a Git repository. By default, Git
6440 repositories are cloned to ``${WORKDIR}/git`` during
6441 :ref:`ref-tasks-fetch`. Since this path is different
6442 from the default value of ``S``, you must set it specifically so the
6443 source can be located:
6444 ::
6445
6446 SRC_URI = "git://path/to/repo.git"
6447 S = "${WORKDIR}/git"
6448
6449 SANITY_REQUIRED_UTILITIES
6450 Specifies a list of command-line utilities that should be checked for
6451 during the initial sanity checking process when running BitBake. If
6452 any of the utilities are not installed on the build host, then
6453 BitBake immediately exits with an error.
6454
6455 SANITY_TESTED_DISTROS
6456 A list of the host distribution identifiers that the build system has
6457 been tested against. Identifiers consist of the host distributor ID
6458 followed by the release, as reported by the ``lsb_release`` tool or
6459 as read from ``/etc/lsb-release``. Separate the list items with
6460 explicit newline characters (``\n``). If ``SANITY_TESTED_DISTROS`` is
6461 not empty and the current value of
6462 :term:`NATIVELSBSTRING` does not appear in the
6463 list, then the build system reports a warning that indicates the
6464 current host distribution has not been tested as a build host.
6465
6466 SDK_ARCH
6467 The target architecture for the SDK. Typically, you do not directly
6468 set this variable. Instead, use :term:`SDKMACHINE`.
6469
6470 SDK_DEPLOY
6471 The directory set up and used by the
6472 :ref:`populate_sdk_base <ref-classes-populate-sdk>` class to which
6473 the SDK is deployed. The ``populate_sdk_base`` class defines
6474 ``SDK_DEPLOY`` as follows:
6475 ::
6476
6477 SDK_DEPLOY = "${TMPDIR}/deploy/sdk"
6478
6479 SDK_DIR
6480 The parent directory used by the OpenEmbedded build system when
6481 creating SDK output. The
6482 :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class defines
6483 the variable as follows:
6484 ::
6485
6486 SDK_DIR = "${WORKDIR}/sdk"
6487
6488 .. note::
6489
6490 The
6491 SDK_DIR
6492 directory is a temporary directory as it is part of
6493 WORKDIR
6494 . The final output directory is
6495 SDK_DEPLOY
6496 .
6497
6498 SDK_EXT_TYPE
6499 Controls whether or not shared state artifacts are copied into the
6500 extensible SDK. The default value of "full" copies all of the
6501 required shared state artifacts into the extensible SDK. The value
6502 "minimal" leaves these artifacts out of the SDK.
6503
6504 .. note::
6505
6506 If you set the variable to "minimal", you need to ensure
6507 SSTATE_MIRRORS
6508 is set in the SDK's configuration to enable the artifacts to be
6509 fetched as needed.
6510
6511 SDK_HOST_MANIFEST
6512 The manifest file for the host part of the SDK. This file lists all
6513 the installed packages that make up the host part of the SDK. The
6514 file contains package information on a line-per-package basis as
6515 follows:
6516 ::
6517
6518 packagename packagearch version
6519
6520 The :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class
6521 defines the manifest file as follows:
6522 ::
6523
6524 SDK_HOST_MANIFEST = "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.host.manifest"
6525
6526 The location is derived using the :term:`SDK_DEPLOY` and
6527 :term:`TOOLCHAIN_OUTPUTNAME` variables.
6528
6529 SDK_INCLUDE_PKGDATA
6530 When set to "1", specifies to include the packagedata for all recipes
6531 in the "world" target in the extensible SDK. Including this data
6532 allows the ``devtool search`` command to find these recipes in search
6533 results, as well as allows the ``devtool add`` command to map
6534 dependencies more effectively.
6535
6536 .. note::
6537
6538 Enabling the
6539 SDK_INCLUDE_PKGDATA
6540 variable significantly increases build time because all of world
6541 needs to be built. Enabling the variable also slightly increases
6542 the size of the extensible SDK.
6543
6544 SDK_INCLUDE_TOOLCHAIN
6545 When set to "1", specifies to include the toolchain in the extensible
6546 SDK. Including the toolchain is useful particularly when
6547 :term:`SDK_EXT_TYPE` is set to "minimal" to keep
6548 the SDK reasonably small but you still want to provide a usable
6549 toolchain. For example, suppose you want to use the toolchain from an
6550 IDE or from other tools and you do not want to perform additional
6551 steps to install the toolchain.
6552
6553 The ``SDK_INCLUDE_TOOLCHAIN`` variable defaults to "0" if
6554 ``SDK_EXT_TYPE`` is set to "minimal", and defaults to "1" if
6555 ``SDK_EXT_TYPE`` is set to "full".
6556
6557 SDK_INHERIT_BLACKLIST
6558 A list of classes to remove from the :term:`INHERIT`
6559 value globally within the extensible SDK configuration. The
6560 :ref:`populate-sdk-ext <ref-classes-populate-sdk-*>` class sets the
6561 default value:
6562 ::
6563
6564 SDK_INHERIT_BLACKLIST ?= "buildhistory icecc"
6565
6566 Some classes are not generally applicable within the extensible SDK
6567 context. You can use this variable to disable those classes.
6568
6569 For additional information on how to customize the extensible SDK's
6570 configuration, see the
6571 ":ref:`sdk-manual/sdk-appendix-customizing:configuring the extensible sdk`"
6572 section in the Yocto Project Application Development and the
6573 Extensible Software Development Kit (eSDK) manual.
6574
6575 SDK_LOCAL_CONF_BLACKLIST
6576 A list of variables not allowed through from the OpenEmbedded build
6577 system configuration into the extensible SDK configuration. Usually,
6578 these are variables that are specific to the machine on which the
6579 build system is running and thus would be potentially problematic
6580 within the extensible SDK.
6581
6582 By default, ``SDK_LOCAL_CONF_BLACKLIST`` is set in the
6583 :ref:`populate-sdk-ext <ref-classes-populate-sdk-*>` class and
6584 excludes the following variables:
6585
6586 - :term:`CONF_VERSION`
6587 - :term:`BB_NUMBER_THREADS`
6588 - :term:`bitbake:BB_NUMBER_PARSE_THREADS`
6589 - :term:`PARALLEL_MAKE`
6590 - :term:`PRSERV_HOST`
6591 - :term:`SSTATE_MIRRORS` :term:`DL_DIR`
6592 - :term:`SSTATE_DIR` :term:`TMPDIR`
6593 - :term:`BB_SERVER_TIMEOUT`
6594
6595 For additional information on how to customize the extensible SDK's
6596 configuration, see the
6597 ":ref:`sdk-manual/sdk-appendix-customizing:configuring the extensible sdk`"
6598 section in the Yocto Project Application Development and the
6599 Extensible Software Development Kit (eSDK) manual.
6600
6601 SDK_LOCAL_CONF_WHITELIST
6602 A list of variables allowed through from the OpenEmbedded build
6603 system configuration into the extensible SDK configuration. By
6604 default, the list of variables is empty and is set in the
6605 :ref:`populate-sdk-ext <ref-classes-populate-sdk-*>` class.
6606
6607 This list overrides the variables specified using the
6608 :term:`SDK_LOCAL_CONF_BLACKLIST`
6609 variable as well as any variables identified by automatic
6610 blacklisting due to the "/" character being found at the start of the
6611 value, which is usually indicative of being a path and thus might not
6612 be valid on the system where the SDK is installed.
6613
6614 For additional information on how to customize the extensible SDK's
6615 configuration, see the
6616 ":ref:`sdk-manual/sdk-appendix-customizing:configuring the extensible sdk`"
6617 section in the Yocto Project Application Development and the
6618 Extensible Software Development Kit (eSDK) manual.
6619
6620 SDK_NAME
6621 The base name for SDK output files. The name is derived from the
6622 :term:`DISTRO`, :term:`TCLIBC`,
6623 :term:`SDK_ARCH`,
6624 :term:`IMAGE_BASENAME`, and
6625 :term:`TUNE_PKGARCH` variables:
6626 ::
6627
6628 SDK_NAME = "${DISTRO}-${TCLIBC}-${SDK_ARCH}-${IMAGE_BASENAME}-${TUNE_PKGARCH}"
6629
6630 SDK_OS
6631 Specifies the operating system for which the SDK will be built. The
6632 default value is the value of :term:`BUILD_OS`.
6633
6634 SDK_OUTPUT
6635 The location used by the OpenEmbedded build system when creating SDK
6636 output. The :ref:`populate_sdk_base <ref-classes-populate-sdk-*>`
6637 class defines the variable as follows:
6638 ::
6639
6640 SDK_DIR = "${WORKDIR}/sdk"
6641 SDK_OUTPUT = "${SDK_DIR}/image"
6642 SDK_DEPLOY = "${DEPLOY_DIR}/sdk"
6643
6644 .. note::
6645
6646 The SDK_OUTPUT directory is a temporary directory as it is part of
6647 WORKDIR by way of SDK_DIR. The final output directory is
6648 SDK_DEPLOY.
6649
6650 SDK_PACKAGE_ARCHS
6651 Specifies a list of architectures compatible with the SDK machine.
6652 This variable is set automatically and should not normally be
6653 hand-edited. Entries are separated using spaces and listed in order
6654 of priority. The default value for ``SDK_PACKAGE_ARCHS`` is "all any
6655 noarch ${SDK_ARCH}-${SDKPKGSUFFIX}".
6656
6657 SDK_POSTPROCESS_COMMAND
6658 Specifies a list of functions to call once the OpenEmbedded build
6659 system creates the SDK. You can specify functions separated by
6660 semicolons: SDK_POSTPROCESS_COMMAND += "function; ... "
6661
6662 If you need to pass an SDK path to a command within a function, you
6663 can use ``${SDK_DIR}``, which points to the parent directory used by
6664 the OpenEmbedded build system when creating SDK output. See the
6665 :term:`SDK_DIR` variable for more information.
6666
6667 SDK_PREFIX
6668 The toolchain binary prefix used for ``nativesdk`` recipes. The
6669 OpenEmbedded build system uses the ``SDK_PREFIX`` value to set the
6670 :term:`TARGET_PREFIX` when building
6671 ``nativesdk`` recipes. The default value is "${SDK_SYS}-".
6672
6673 SDK_RECRDEP_TASKS
6674 A list of shared state tasks added to the extensible SDK. By default,
6675 the following tasks are added:
6676
6677 - do_populate_lic
6678 - do_package_qa
6679 - do_populate_sysroot
6680 - do_deploy
6681
6682 Despite the default value of "" for the
6683 ``SDK_RECRDEP_TASKS`` variable, the above four tasks are always added
6684 to the SDK. To specify tasks beyond these four, you need to use the
6685 ``SDK_RECRDEP_TASKS`` variable (e.g. you are defining additional
6686 tasks that are needed in order to build
6687 :term:`SDK_TARGETS`).
6688
6689 SDK_SYS
6690 Specifies the system, including the architecture and the operating
6691 system, for which the SDK will be built.
6692
6693 The OpenEmbedded build system automatically sets this variable based
6694 on :term:`SDK_ARCH`,
6695 :term:`SDK_VENDOR`, and
6696 :term:`SDK_OS`. You do not need to set the ``SDK_SYS``
6697 variable yourself.
6698
6699 SDK_TARGET_MANIFEST
6700 The manifest file for the target part of the SDK. This file lists all
6701 the installed packages that make up the target part of the SDK. The
6702 file contains package information on a line-per-package basis as
6703 follows:
6704 ::
6705
6706 packagename packagearch version
6707
6708 The :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class
6709 defines the manifest file as follows:
6710 ::
6711
6712 SDK_TARGET_MANIFEST = "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.target.manifest"
6713
6714 The location is derived using the :term:`SDK_DEPLOY` and
6715 :term:`TOOLCHAIN_OUTPUTNAME` variables.
6716
6717 SDK_TARGETS
6718 A list of targets to install from shared state as part of the
6719 standard or extensible SDK installation. The default value is "${PN}"
6720 (i.e. the image from which the SDK is built).
6721
6722 The ``SDK_TARGETS`` variable is an internal variable and typically
6723 would not be changed.
6724
6725 SDK_TITLE
6726 The title to be printed when running the SDK installer. By default,
6727 this title is based on the :term:`DISTRO_NAME` or
6728 :term:`DISTRO` variable and is set in the
6729 :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class as
6730 follows:
6731 ::
6732
6733 SDK_TITLE ??= "${@d.getVar('DISTRO_NAME') or d.getVar('DISTRO')} SDK"
6734
6735 For the default distribution "poky",
6736 ``SDK_TITLE`` is set to "Poky (Yocto Project Reference Distro)".
6737
6738 For information on how to change this default title, see the
6739 ":ref:`sdk-manual/sdk-appendix-customizing:changing the extensible sdk installer title`"
6740 section in the Yocto Project Application Development and the
6741 Extensible Software Development Kit (eSDK) manual.
6742
6743 SDK_UPDATE_URL
6744 An optional URL for an update server for the extensible SDK. If set,
6745 the value is used as the default update server when running
6746 ``devtool sdk-update`` within the extensible SDK.
6747
6748 SDK_VENDOR
6749 Specifies the name of the SDK vendor.
6750
6751 SDK_VERSION
6752 Specifies the version of the SDK. The distribution configuration file
6753 (e.g. ``/meta-poky/conf/distro/poky.conf``) defines the
6754 ``SDK_VERSION`` as follows:
6755 ::
6756
6757 SDK_VERSION = "${@d.getVar('DISTRO_VERSION').replace('snapshot-${DATE}','snapshot')}"
6758
6759 For additional information, see the
6760 :term:`DISTRO_VERSION` and
6761 :term:`DATE` variables.
6762
6763 SDKEXTPATH
6764 The default installation directory for the Extensible SDK. By
6765 default, this directory is based on the :term:`DISTRO`
6766 variable and is set in the
6767 :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class as
6768 follows:
6769 ::
6770
6771 SDKEXTPATH ??= "~/${@d.getVar('DISTRO')}_sdk"
6772
6773 For the
6774 default distribution "poky", the ``SDKEXTPATH`` is set to "poky_sdk".
6775
6776 For information on how to change this default directory, see the
6777 ":ref:`sdk-manual/sdk-appendix-customizing:changing the default sdk installation directory`"
6778 section in the Yocto Project Application Development and the
6779 Extensible Software Development Kit (eSDK) manual.
6780
6781 SDKIMAGE_FEATURES
6782 Equivalent to ``IMAGE_FEATURES``. However, this variable applies to
6783 the SDK generated from an image using the following command:
6784 ::
6785
6786 $ bitbake -c populate_sdk imagename
6787
6788 SDKMACHINE
6789 The machine for which the SDK is built. In other words, the SDK is
6790 built such that it runs on the target you specify with the
6791 ``SDKMACHINE`` value. The value points to a corresponding ``.conf``
6792 file under ``conf/machine-sdk/``.
6793
6794 You can use "i686" and "x86_64" as possible values for this variable.
6795 The variable defaults to "i686" and is set in the local.conf file in
6796 the Build Directory.
6797 ::
6798
6799 SDKMACHINE ?= "i686"
6800
6801 .. note::
6802
6803 You cannot set the
6804 SDKMACHINE
6805 variable in your distribution configuration file. If you do, the
6806 configuration will not take affect.
6807
6808 SDKPATH
6809 Defines the path offered to the user for installation of the SDK that
6810 is generated by the OpenEmbedded build system. The path appears as
6811 the default location for installing the SDK when you run the SDK's
6812 installation script. You can override the offered path when you run
6813 the script.
6814
6815 SDKTARGETSYSROOT
6816 The full path to the sysroot used for cross-compilation within an SDK
6817 as it will be when installed into the default
6818 :term:`SDKPATH`.
6819
6820 SECTION
6821 The section in which packages should be categorized. Package
6822 management utilities can make use of this variable.
6823
6824 SELECTED_OPTIMIZATION
6825 Specifies the optimization flags passed to the C compiler when
6826 building for the target. The flags are passed through the default
6827 value of the :term:`TARGET_CFLAGS` variable.
6828
6829 The ``SELECTED_OPTIMIZATION`` variable takes the value of
6830 ``FULL_OPTIMIZATION`` unless ``DEBUG_BUILD`` = "1". If that is the
6831 case, the value of ``DEBUG_OPTIMIZATION`` is used.
6832
6833 SERIAL_CONSOLE
6834 Defines a serial console (TTY) to enable using
6835 `getty <https://en.wikipedia.org/wiki/Getty_(Unix)>`__. Provide a
6836 value that specifies the baud rate followed by the TTY device name
6837 separated by a space. You cannot specify more than one TTY device:
6838 ::
6839
6840 SERIAL_CONSOLE = "115200 ttyS0"
6841
6842 .. note::
6843
6844 The
6845 SERIAL_CONSOLE
6846 variable is deprecated. Please use the
6847 SERIAL_CONSOLES
6848 variable.
6849
6850 SERIAL_CONSOLES
6851 Defines a serial console (TTY) to enable using
6852 `getty <https://en.wikipedia.org/wiki/Getty_(Unix)>`__. Provide a
6853 value that specifies the baud rate followed by the TTY device name
6854 separated by a semicolon. Use spaces to separate multiple devices:
6855 ::
6856
6857 SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1"
6858
6859 SERIAL_CONSOLES_CHECK
6860 Specifies serial consoles, which must be listed in
6861 :term:`SERIAL_CONSOLES`, to check against
6862 ``/proc/console`` before enabling them using getty. This variable
6863 allows aliasing in the format: <device>:<alias>. If a device was
6864 listed as "sclp_line0" in ``/dev/`` and "ttyS0" was listed in
6865 ``/proc/console``, you would do the following: ::
6866
6867 SERIAL_CONSOLES_CHECK = "slcp_line0:ttyS0"
6868
6869 This variable is currently only supported with SysVinit (i.e. not
6870 with systemd).
6871
6872 SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS
6873 A list of recipe dependencies that should not be used to determine
6874 signatures of tasks from one recipe when they depend on tasks from
6875 another recipe. For example: ::
6876
6877 SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += "intone->mplayer2"
6878
6879 In the previous example, ``intone`` depends on ``mplayer2``.
6880
6881 You can use the special token ``"*"`` on the left-hand side of the
6882 dependency to match all recipes except the one on the right-hand
6883 side. Here is an example: ::
6884
6885 SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += "*->quilt-native"
6886
6887 In the previous example, all recipes except ``quilt-native`` ignore
6888 task signatures from the ``quilt-native`` recipe when determining
6889 their task signatures.
6890
6891 Use of this variable is one mechanism to remove dependencies that
6892 affect task signatures and thus force rebuilds when a recipe changes.
6893
6894 .. note::
6895
6896 If you add an inappropriate dependency for a recipe relationship,
6897 the software might break during runtime if the interface of the
6898 second recipe was changed after the first recipe had been built.
6899
6900 SIGGEN_EXCLUDERECIPES_ABISAFE
6901 A list of recipes that are completely stable and will never change.
6902 The ABI for the recipes in the list are presented by output from the
6903 tasks run to build the recipe. Use of this variable is one way to
6904 remove dependencies from one recipe on another that affect task
6905 signatures and thus force rebuilds when the recipe changes.
6906
6907 .. note::
6908
6909 If you add an inappropriate variable to this list, the software
6910 might break at runtime if the interface of the recipe was changed
6911 after the other had been built.
6912
6913 SITEINFO_BITS
6914 Specifies the number of bits for the target system CPU. The value
6915 should be either "32" or "64".
6916
6917 SITEINFO_ENDIANNESS
6918 Specifies the endian byte order of the target system. The value
6919 should be either "le" for little-endian or "be" for big-endian.
6920
6921 SKIP_FILEDEPS
6922 Enables removal of all files from the "Provides" section of an RPM
6923 package. Removal of these files is required for packages containing
6924 prebuilt binaries and libraries such as ``libstdc++`` and ``glibc``.
6925
6926 To enable file removal, set the variable to "1" in your
6927 ``conf/local.conf`` configuration file in your:
6928 :term:`Build Directory`.
6929 ::
6930
6931 SKIP_FILEDEPS = "1"
6932
6933 SOC_FAMILY
6934 Groups together machines based upon the same family of SOC (System On
6935 Chip). You typically set this variable in a common ``.inc`` file that
6936 you include in the configuration files of all the machines.
6937
6938 .. note::
6939
6940 You must include
6941 conf/machine/include/soc-family.inc
6942 for this variable to appear in
6943 MACHINEOVERRIDES
6944 .
6945
6946 SOLIBS
6947 Defines the suffix for shared libraries used on the target platform.
6948 By default, this suffix is ".so.*" for all Linux-based systems and is
6949 defined in the ``meta/conf/bitbake.conf`` configuration file.
6950
6951 You will see this variable referenced in the default values of
6952 ``FILES_${PN}``.
6953
6954 SOLIBSDEV
6955 Defines the suffix for the development symbolic link (symlink) for
6956 shared libraries on the target platform. By default, this suffix is
6957 ".so" for Linux-based systems and is defined in the
6958 ``meta/conf/bitbake.conf`` configuration file.
6959
6960 You will see this variable referenced in the default values of
6961 ``FILES_${PN}-dev``.
6962
6963 SOURCE_MIRROR_FETCH
6964 When you are fetching files to create a mirror of sources (i.e.
6965 creating a source mirror), setting ``SOURCE_MIRROR_FETCH`` to "1" in
6966 your ``local.conf`` configuration file ensures the source for all
6967 recipes are fetched regardless of whether or not a recipe is
6968 compatible with the configuration. A recipe is considered
6969 incompatible with the currently configured machine when either or
6970 both the :term:`COMPATIBLE_MACHINE`
6971 variable and :term:`COMPATIBLE_HOST` variables
6972 specify compatibility with a machine other than that of the current
6973 machine or host.
6974
6975 .. note::
6976
6977 Do not set the
6978 SOURCE_MIRROR_FETCH
6979 variable unless you are creating a source mirror. In other words,
6980 do not set the variable during a normal build.
6981
6982 SOURCE_MIRROR_URL
6983 Defines your own :term:`PREMIRRORS` from which to
6984 first fetch source before attempting to fetch from the upstream
6985 specified in :term:`SRC_URI`.
6986
6987 To use this variable, you must globally inherit the
6988 :ref:`own-mirrors <ref-classes-own-mirrors>` class and then provide
6989 the URL to your mirrors. Here is the general syntax:
6990 ::
6991
6992 INHERIT += "own-mirrors"
6993 SOURCE_MIRROR_URL = "http://example.com/my_source_mirror"
6994
6995 .. note::
6996
6997 You can specify only a single URL in
6998 SOURCE_MIRROR_URL
6999 .
7000
7001 SPDXLICENSEMAP
7002 Maps commonly used license names to their SPDX counterparts found in
7003 ``meta/files/common-licenses/``. For the default ``SPDXLICENSEMAP``
7004 mappings, see the ``meta/conf/licenses.conf`` file.
7005
7006 For additional information, see the :term:`LICENSE`
7007 variable.
7008
7009 SPECIAL_PKGSUFFIX
7010 A list of prefixes for :term:`PN` used by the OpenEmbedded
7011 build system to create variants of recipes or packages. The list
7012 specifies the prefixes to strip off during certain circumstances such
7013 as the generation of the :term:`BPN` variable.
7014
7015 SPL_BINARY
7016 The file type for the Secondary Program Loader (SPL). Some devices
7017 use an SPL from which to boot (e.g. the BeagleBone development
7018 board). For such cases, you can declare the file type of the SPL
7019 binary in the ``u-boot.inc`` include file, which is used in the
7020 U-Boot recipe.
7021
7022 The SPL file type is set to "null" by default in the ``u-boot.inc``
7023 file as follows:
7024 ::
7025
7026 # Some versions of u-boot build an SPL (Second Program Loader) image that
7027 # should be packaged along with the u-boot binary as well as placed in the
7028 # deploy directory. For those versions they can set the following variables
7029 # to allow packaging the SPL.
7030 SPL_BINARY ?= ""
7031 SPL_BINARYNAME ?= "${@os.path.basename(d.getVar("SPL_BINARY"))}"
7032 SPL_IMAGE ?= "${SPL_BINARYNAME}-${MACHINE}-${PV}-${PR}"
7033 SPL_SYMLINK ?= "${SPL_BINARYNAME}-${MACHINE}"
7034
7035 The ``SPL_BINARY`` variable helps form
7036 various ``SPL_*`` variables used by the OpenEmbedded build system.
7037
7038 See the BeagleBone machine configuration example in the
7039 ":ref:`dev-manual/dev-manual-common-tasks:adding a layer using the \`\`bitbake-layers\`\` script`"
7040 section in the Yocto Project Board Support Package Developer's Guide
7041 for additional information.
7042
7043 SRC_URI
7044 The list of source files - local or remote. This variable tells the
7045 OpenEmbedded build system which bits to pull in for the build and how
7046 to pull them in. For example, if the recipe or append file only needs
7047 to fetch a tarball from the Internet, the recipe or append file uses
7048 a single ``SRC_URI`` entry. On the other hand, if the recipe or
7049 append file needs to fetch a tarball, apply two patches, and include
7050 a custom file, the recipe or append file would include four instances
7051 of the variable.
7052
7053 The following list explains the available URI protocols. URI
7054 protocols are highly dependent on particular BitBake Fetcher
7055 submodules. Depending on the fetcher BitBake uses, various URL
7056 parameters are employed. For specifics on the supported Fetchers, see
7057 the ":ref:`Fetchers <bitbake:bb-fetchers>`" section in the
7058 BitBake User Manual.
7059
7060 - ``file://`` - Fetches files, which are usually files shipped
7061 with the :term:`Metadata`, from the local machine (e.g.
7062 :ref:`patch <patching-dev-environment>` files).
7063 The path is relative to the :term:`FILESPATH`
7064 variable. Thus, the build system searches, in order, from the
7065 following directories, which are assumed to be a subdirectories of
7066 the directory in which the recipe file (``.bb``) or append file
7067 (``.bbappend``) resides:
7068
7069 - ``${BPN}`` - The base recipe name without any special suffix
7070 or version numbers.
7071
7072 - ``${BP}`` - ``${BPN}-${PV}``. The base recipe name and
7073 version but without any special package name suffix.
7074
7075 - *files -* Files within a directory, which is named ``files``
7076 and is also alongside the recipe or append file.
7077
7078 .. note::
7079
7080 If you want the build system to pick up files specified through
7081 a
7082 SRC_URI
7083 statement from your append file, you need to be sure to extend
7084 the
7085 FILESPATH
7086 variable by also using the
7087 FILESEXTRAPATHS
7088 variable from within your append file.
7089
7090 - ``bzr://`` - Fetches files from a Bazaar revision control
7091 repository.
7092
7093 - ``git://`` - Fetches files from a Git revision control
7094 repository.
7095
7096 - ``osc://`` - Fetches files from an OSC (OpenSUSE Build service)
7097 revision control repository.
7098
7099 - ``repo://`` - Fetches files from a repo (Git) repository.
7100
7101 - ``ccrc://`` - Fetches files from a ClearCase repository.
7102
7103 - ``http://`` - Fetches files from the Internet using ``http``.
7104
7105 - ``https://`` - Fetches files from the Internet using ``https``.
7106
7107 - ``ftp://`` - Fetches files from the Internet using ``ftp``.
7108
7109 - ``cvs://`` - Fetches files from a CVS revision control
7110 repository.
7111
7112 - ``hg://`` - Fetches files from a Mercurial (``hg``) revision
7113 control repository.
7114
7115 - ``p4://`` - Fetches files from a Perforce (``p4``) revision
7116 control repository.
7117
7118 - ``ssh://`` - Fetches files from a secure shell.
7119
7120 - ``svn://`` - Fetches files from a Subversion (``svn``) revision
7121 control repository.
7122
7123 - ``npm://`` - Fetches JavaScript modules from a registry.
7124
7125 Standard and recipe-specific options for ``SRC_URI`` exist. Here are
7126 standard options:
7127
7128 - ``apply`` - Whether to apply the patch or not. The default
7129 action is to apply the patch.
7130
7131 - ``striplevel`` - Which striplevel to use when applying the
7132 patch. The default level is 1.
7133
7134 - ``patchdir`` - Specifies the directory in which the patch should
7135 be applied. The default is ``${``\ :term:`S`\ ``}``.
7136
7137 Here are options specific to recipes building code from a revision
7138 control system:
7139
7140 - ``mindate`` - Apply the patch only if
7141 :term:`SRCDATE` is equal to or greater than
7142 ``mindate``.
7143
7144 - ``maxdate`` - Apply the patch only if ``SRCDATE`` is not later
7145 than ``maxdate``.
7146
7147 - ``minrev`` - Apply the patch only if ``SRCREV`` is equal to or
7148 greater than ``minrev``.
7149
7150 - ``maxrev`` - Apply the patch only if ``SRCREV`` is not later
7151 than ``maxrev``.
7152
7153 - ``rev`` - Apply the patch only if ``SRCREV`` is equal to
7154 ``rev``.
7155
7156 - ``notrev`` - Apply the patch only if ``SRCREV`` is not equal to
7157 ``rev``.
7158
7159 Here are some additional options worth mentioning:
7160
7161 - ``unpack`` - Controls whether or not to unpack the file if it is
7162 an archive. The default action is to unpack the file.
7163
7164 - ``destsuffix`` - Places the file (or extracts its contents) into
7165 the specified subdirectory of :term:`WORKDIR` when
7166 the Git fetcher is used.
7167
7168 - ``subdir`` - Places the file (or extracts its contents) into the
7169 specified subdirectory of ``WORKDIR`` when the local (``file://``)
7170 fetcher is used.
7171
7172 - ``localdir`` - Places the file (or extracts its contents) into
7173 the specified subdirectory of ``WORKDIR`` when the CVS fetcher is
7174 used.
7175
7176 - ``subpath`` - Limits the checkout to a specific subpath of the
7177 tree when using the Git fetcher is used.
7178
7179 - ``name`` - Specifies a name to be used for association with
7180 ``SRC_URI`` checksums when you have more than one file specified
7181 in ``SRC_URI``.
7182
7183 - ``downloadfilename`` - Specifies the filename used when storing
7184 the downloaded file.
7185
7186 SRC_URI_OVERRIDES_PACKAGE_ARCH
7187 By default, the OpenEmbedded build system automatically detects
7188 whether ``SRC_URI`` contains files that are machine-specific. If so,
7189 the build system automatically changes ``PACKAGE_ARCH``. Setting this
7190 variable to "0" disables this behavior.
7191
7192 SRCDATE
7193 The date of the source code used to build the package. This variable
7194 applies only if the source was fetched from a Source Code Manager
7195 (SCM).
7196
7197 SRCPV
7198 Returns the version string of the current package. This string is
7199 used to help define the value of :term:`PV`.
7200
7201 The ``SRCPV`` variable is defined in the ``meta/conf/bitbake.conf``
7202 configuration file in the :term:`Source Directory` as
7203 follows:
7204 ::
7205
7206 SRCPV = "${@bb.fetch2.get_srcrev(d)}"
7207
7208 Recipes that need to define ``PV`` do so with the help of the
7209 ``SRCPV``. For example, the ``ofono`` recipe (``ofono_git.bb``)
7210 located in ``meta/recipes-connectivity`` in the Source Directory
7211 defines ``PV`` as follows:
7212 ::
7213
7214 PV = "0.12-git${SRCPV}"
7215
7216 SRCREV
7217 The revision of the source code used to build the package. This
7218 variable applies to Subversion, Git, Mercurial, and Bazaar only. Note
7219 that if you want to build a fixed revision and you want to avoid
7220 performing a query on the remote repository every time BitBake parses
7221 your recipe, you should specify a ``SRCREV`` that is a full revision
7222 identifier and not just a tag.
7223
7224 .. note::
7225
7226 For information on limitations when inheriting the latest revision
7227 of software using
7228 SRCREV
7229 , see the
7230 AUTOREV
7231 variable description and the "
7232 Automatically Incrementing a Binary Package Revision Number
7233 " section, which is in the Yocto Project Development Tasks Manual.
7234
7235 SSTATE_DIR
7236 The directory for the shared state cache.
7237
7238 SSTATE_MIRROR_ALLOW_NETWORK
7239 If set to "1", allows fetches from mirrors that are specified in
7240 :term:`SSTATE_MIRRORS` to work even when
7241 fetching from the network is disabled by setting ``BB_NO_NETWORK`` to
7242 "1". Using the ``SSTATE_MIRROR_ALLOW_NETWORK`` variable is useful if
7243 you have set ``SSTATE_MIRRORS`` to point to an internal server for
7244 your shared state cache, but you want to disable any other fetching
7245 from the network.
7246
7247 SSTATE_MIRRORS
7248 Configures the OpenEmbedded build system to search other mirror
7249 locations for prebuilt cache data objects before building out the
7250 data. This variable works like fetcher :term:`MIRRORS`
7251 and :term:`PREMIRRORS` and points to the cache
7252 locations to check for the shared state (sstate) objects.
7253
7254 You can specify a filesystem directory or a remote URL such as HTTP
7255 or FTP. The locations you specify need to contain the shared state
7256 cache (sstate-cache) results from previous builds. The sstate-cache
7257 you point to can also be from builds on other machines.
7258
7259 When pointing to sstate build artifacts on another machine that uses
7260 a different GCC version for native builds, you must configure
7261 ``SSTATE_MIRRORS`` with a regular expression that maps local search
7262 paths to server paths. The paths need to take into account
7263 :term:`NATIVELSBSTRING` set by the
7264 :ref:`uninative <ref-classes-uninative>` class. For example, the
7265 following maps the local search path ``universal-4.9`` to the
7266 server-provided path server_url_sstate_path:
7267 ::
7268
7269 SSTATE_MIRRORS ?= "file://universal-4.9/(.*) http://server_url_sstate_path/universal-4.8/\1 \n"
7270
7271 If a mirror uses the same structure as
7272 :term:`SSTATE_DIR`, you need to add "PATH" at the
7273 end as shown in the examples below. The build system substitutes the
7274 correct path within the directory structure.
7275 ::
7276
7277 SSTATE_MIRRORS ?= "\
7278 file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \n \
7279 file://.* file:///some-local-dir/sstate/PATH"
7280
7281 SSTATE_SCAN_FILES
7282 Controls the list of files the OpenEmbedded build system scans for
7283 hardcoded installation paths. The variable uses a space-separated
7284 list of filenames (not paths) with standard wildcard characters
7285 allowed.
7286
7287 During a build, the OpenEmbedded build system creates a shared state
7288 (sstate) object during the first stage of preparing the sysroots.
7289 That object is scanned for hardcoded paths for original installation
7290 locations. The list of files that are scanned for paths is controlled
7291 by the ``SSTATE_SCAN_FILES`` variable. Typically, recipes add files
7292 they want to be scanned to the value of ``SSTATE_SCAN_FILES`` rather
7293 than the variable being comprehensively set. The
7294 :ref:`sstate <ref-classes-sstate>` class specifies the default list
7295 of files.
7296
7297 For details on the process, see the
7298 :ref:`staging <ref-classes-staging>` class.
7299
7300 STAGING_BASE_LIBDIR_NATIVE
7301 Specifies the path to the ``/lib`` subdirectory of the sysroot
7302 directory for the build host.
7303
7304 STAGING_BASELIBDIR
7305 Specifies the path to the ``/lib`` subdirectory of the sysroot
7306 directory for the target for which the current recipe is being built
7307 (:term:`STAGING_DIR_HOST`).
7308
7309 STAGING_BINDIR
7310 Specifies the path to the ``/usr/bin`` subdirectory of the sysroot
7311 directory for the target for which the current recipe is being built
7312 (:term:`STAGING_DIR_HOST`).
7313
7314 STAGING_BINDIR_CROSS
7315 Specifies the path to the directory containing binary configuration
7316 scripts. These scripts provide configuration information for other
7317 software that wants to make use of libraries or include files
7318 provided by the software associated with the script.
7319
7320 .. note::
7321
7322 This style of build configuration has been largely replaced by
7323 pkg-config
7324 . Consequently, if
7325 pkg-config
7326 is supported by the library to which you are linking, it is
7327 recommended you use
7328 pkg-config
7329 instead of a provided configuration script.
7330
7331 STAGING_BINDIR_NATIVE
7332 Specifies the path to the ``/usr/bin`` subdirectory of the sysroot
7333 directory for the build host.
7334
7335 STAGING_DATADIR
7336 Specifies the path to the ``/usr/share`` subdirectory of the sysroot
7337 directory for the target for which the current recipe is being built
7338 (:term:`STAGING_DIR_HOST`).
7339
7340 STAGING_DATADIR_NATIVE
7341 Specifies the path to the ``/usr/share`` subdirectory of the sysroot
7342 directory for the build host.
7343
7344 STAGING_DIR
7345 Helps construct the ``recipe-sysroots`` directory, which is used
7346 during packaging.
7347
7348 For information on how staging for recipe-specific sysroots occurs,
7349 see the :ref:`ref-tasks-populate_sysroot`
7350 task, the ":ref:`sdk-manual/sdk-extensible:sharing files between recipes`"
7351 section in the Yocto Project Development Tasks Manual, the
7352 ":ref:`configuration-compilation-and-staging-dev-environment`"
7353 section in the Yocto Project Overview and Concepts Manual, and the
7354 :term:`SYSROOT_DIRS` variable.
7355
7356 .. note::
7357
7358 Recipes should never write files directly under the
7359 STAGING_DIR
7360 directory because the OpenEmbedded build system manages the
7361 directory automatically. Instead, files should be installed to
7362 ${
7363 D
7364 }
7365 within your recipe's
7366 do_install
7367 task and then the OpenEmbedded build system will stage a subset of
7368 those files into the sysroot.
7369
7370 STAGING_DIR_HOST
7371 Specifies the path to the sysroot directory for the system on which
7372 the component is built to run (the system that hosts the component).
7373 For most recipes, this sysroot is the one in which that recipe's
7374 :ref:`ref-tasks-populate_sysroot` task copies
7375 files. Exceptions include ``-native`` recipes, where the
7376 ``do_populate_sysroot`` task instead uses
7377 :term:`STAGING_DIR_NATIVE`. Depending on
7378 the type of recipe and the build target, ``STAGING_DIR_HOST`` can
7379 have the following values:
7380
7381 - For recipes building for the target machine, the value is
7382 "${:term:`STAGING_DIR`}/${:term:`MACHINE`}".
7383
7384 - For native recipes building for the build host, the value is empty
7385 given the assumption that when building for the build host, the
7386 build host's own directories should be used.
7387
7388 .. note::
7389
7390 ``-native`` recipes are not installed into host paths like such
7391 as ``/usr``. Rather, these recipes are installed into
7392 ``STAGING_DIR_NATIVE``. When compiling ``-native`` recipes,
7393 standard build environment variables such as
7394 :term:`CPPFLAGS` and
7395 :term:`CFLAGS` are set up so that both host paths
7396 and ``STAGING_DIR_NATIVE`` are searched for libraries and
7397 headers using, for example, GCC's ``-isystem`` option.
7398
7399 Thus, the emphasis is that the ``STAGING_DIR*`` variables
7400 should be viewed as input variables by tasks such as
7401 :ref:`ref-tasks-configure`,
7402 :ref:`ref-tasks-compile`, and
7403 :ref:`ref-tasks-install`. Having the real system
7404 root correspond to ``STAGING_DIR_HOST`` makes conceptual sense
7405 for ``-native`` recipes, as they make use of host headers and
7406 libraries.
7407
7408 STAGING_DIR_NATIVE
7409 Specifies the path to the sysroot directory used when building
7410 components that run on the build host itself.
7411
7412 STAGING_DIR_TARGET
7413 Specifies the path to the sysroot used for the system for which the
7414 component generates code. For components that do not generate code,
7415 which is the majority, ``STAGING_DIR_TARGET`` is set to match
7416 :term:`STAGING_DIR_HOST`.
7417
7418 Some recipes build binaries that can run on the target system but
7419 those binaries in turn generate code for another different system
7420 (e.g. cross-canadian recipes). Using terminology from GNU, the
7421 primary system is referred to as the "HOST" and the secondary, or
7422 different, system is referred to as the "TARGET". Thus, the binaries
7423 run on the "HOST" system and generate binaries for the "TARGET"
7424 system. The ``STAGING_DIR_HOST`` variable points to the sysroot used
7425 for the "HOST" system, while ``STAGING_DIR_TARGET`` points to the
7426 sysroot used for the "TARGET" system.
7427
7428 STAGING_ETCDIR_NATIVE
7429 Specifies the path to the ``/etc`` subdirectory of the sysroot
7430 directory for the build host.
7431
7432 STAGING_EXECPREFIXDIR
7433 Specifies the path to the ``/usr`` subdirectory of the sysroot
7434 directory for the target for which the current recipe is being built
7435 (:term:`STAGING_DIR_HOST`).
7436
7437 STAGING_INCDIR
7438 Specifies the path to the ``/usr/include`` subdirectory of the
7439 sysroot directory for the target for which the current recipe being
7440 built (:term:`STAGING_DIR_HOST`).
7441
7442 STAGING_INCDIR_NATIVE
7443 Specifies the path to the ``/usr/include`` subdirectory of the
7444 sysroot directory for the build host.
7445
7446 STAGING_KERNEL_BUILDDIR
7447 Points to the directory containing the kernel build artifacts.
7448 Recipes building software that needs to access kernel build artifacts
7449 (e.g. ``systemtap-uprobes``) can look in the directory specified with
7450 the ``STAGING_KERNEL_BUILDDIR`` variable to find these artifacts
7451 after the kernel has been built.
7452
7453 STAGING_KERNEL_DIR
7454 The directory with kernel headers that are required to build
7455 out-of-tree modules.
7456
7457 STAGING_LIBDIR
7458 Specifies the path to the ``/usr/lib`` subdirectory of the sysroot
7459 directory for the target for which the current recipe is being built
7460 (:term:`STAGING_DIR_HOST`).
7461
7462 STAGING_LIBDIR_NATIVE
7463 Specifies the path to the ``/usr/lib`` subdirectory of the sysroot
7464 directory for the build host.
7465
7466 STAMP
7467 Specifies the base path used to create recipe stamp files. The path
7468 to an actual stamp file is constructed by evaluating this string and
7469 then appending additional information. Currently, the default
7470 assignment for ``STAMP`` as set in the ``meta/conf/bitbake.conf``
7471 file is:
7472 ::
7473
7474 STAMP = "${STAMPS_DIR}/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
7475
7476 For information on how BitBake uses stamp files to determine if a
7477 task should be rerun, see the
7478 ":ref:`overview-manual/overview-manual-concepts:stamp files and the rerunning of tasks`"
7479 section in the Yocto Project Overview and Concepts Manual.
7480
7481 See :term:`STAMPS_DIR`,
7482 :term:`MULTIMACH_TARGET_SYS`,
7483 :term:`PN`, :term:`EXTENDPE`,
7484 :term:`PV`, and :term:`PR` for related variable
7485 information.
7486
7487 STAMPS_DIR
7488 Specifies the base directory in which the OpenEmbedded build system
7489 places stamps. The default directory is ``${TMPDIR}/stamps``.
7490
7491 STRIP
7492 The minimal command and arguments to run ``strip``, which is used to
7493 strip symbols.
7494
7495 SUMMARY
7496 The short (72 characters or less) summary of the binary package for
7497 packaging systems such as ``opkg``, ``rpm``, or ``dpkg``. By default,
7498 ``SUMMARY`` is used to define the
7499 :term:`DESCRIPTION` variable if ``DESCRIPTION`` is
7500 not set in the recipe.
7501
7502 SVNDIR
7503 The directory in which files checked out of a Subversion system are
7504 stored.
7505
7506 SYSLINUX_DEFAULT_CONSOLE
7507 Specifies the kernel boot default console. If you want to use a
7508 console other than the default, set this variable in your recipe as
7509 follows where "X" is the console number you want to use:
7510 ::
7511
7512 SYSLINUX_DEFAULT_CONSOLE = "console=ttyX"
7513
7514 The :ref:`syslinux <ref-classes-syslinux>` class initially sets
7515 this variable to null but then checks for a value later.
7516
7517 SYSLINUX_OPTS
7518 Lists additional options to add to the syslinux file. You need to set
7519 this variable in your recipe. If you want to list multiple options,
7520 separate the options with a semicolon character (``;``).
7521
7522 The :ref:`syslinux <ref-classes-syslinux>` class uses this variable
7523 to create a set of options.
7524
7525 SYSLINUX_SERIAL
7526 Specifies the alternate serial port or turns it off. To turn off
7527 serial, set this variable to an empty string in your recipe. The
7528 variable's default value is set in the
7529 :ref:`syslinux <ref-classes-syslinux>` class as follows:
7530 ::
7531
7532 SYSLINUX_SERIAL ?= "0 115200"
7533
7534 The class checks for and uses the variable as needed.
7535
7536 SYSLINUX_SPLASH
7537 An ``.LSS`` file used as the background for the VGA boot menu when
7538 you use the boot menu. You need to set this variable in your recipe.
7539
7540 The :ref:`syslinux <ref-classes-syslinux>` class checks for this
7541 variable and if found, the OpenEmbedded build system installs the
7542 splash screen.
7543
7544 SYSLINUX_SERIAL_TTY
7545 Specifies the alternate console=tty... kernel boot argument. The
7546 variable's default value is set in the
7547 :ref:`syslinux <ref-classes-syslinux>` class as follows:
7548 ::
7549
7550 SYSLINUX_SERIAL_TTY ?= "console=ttyS0,115200"
7551
7552 The class checks for and uses the variable as needed.
7553
7554 SYSROOT_DESTDIR
7555 Points to the temporary directory under the work directory (default
7556 "``${``\ :term:`WORKDIR`\ ``}/sysroot-destdir``")
7557 where the files populated into the sysroot are assembled during the
7558 :ref:`ref-tasks-populate_sysroot` task.
7559
7560 SYSROOT_DIRS
7561 Directories that are staged into the sysroot by the
7562 :ref:`ref-tasks-populate_sysroot` task. By
7563 default, the following directories are staged:
7564 ::
7565
7566 SYSROOT_DIRS = " \
7567 ${includedir} \
7568 ${libdir} \
7569 ${base_libdir} \
7570 ${nonarch_base_libdir} \
7571 ${datadir} \
7572 "
7573
7574 SYSROOT_DIRS_BLACKLIST
7575 Directories that are not staged into the sysroot by the
7576 :ref:`ref-tasks-populate_sysroot` task. You
7577 can use this variable to exclude certain subdirectories of
7578 directories listed in :term:`SYSROOT_DIRS` from
7579 staging. By default, the following directories are not staged:
7580 ::
7581
7582 SYSROOT_DIRS_BLACKLIST = " \
7583 ${mandir} \
7584 ${docdir} \
7585 ${infodir} \
7586 ${datadir}/locale \
7587 ${datadir}/applications \
7588 ${datadir}/fonts \
7589 ${datadir}/pixmaps \
7590 "
7591
7592 SYSROOT_DIRS_NATIVE
7593 Extra directories staged into the sysroot by the
7594 :ref:`ref-tasks-populate_sysroot` task for
7595 ``-native`` recipes, in addition to those specified in
7596 :term:`SYSROOT_DIRS`. By default, the following
7597 extra directories are staged:
7598 ::
7599
7600 SYSROOT_DIRS_NATIVE = " \
7601 ${bindir} \
7602 ${sbindir} \
7603 ${base_bindir} \
7604 ${base_sbindir} \
7605 ${libexecdir} \
7606 ${sysconfdir} \
7607 ${localstatedir} \
7608 "
7609
7610 .. note::
7611
7612 Programs built by
7613 -native
7614 recipes run directly from the sysroot (
7615 STAGING_DIR_NATIVE
7616 ), which is why additional directories containing program
7617 executables and supporting files need to be staged.
7618
7619 SYSROOT_PREPROCESS_FUNCS
7620 A list of functions to execute after files are staged into the
7621 sysroot. These functions are usually used to apply additional
7622 processing on the staged files, or to stage additional files.
7623
7624 SYSTEMD_AUTO_ENABLE
7625 When inheriting the :ref:`systemd <ref-classes-systemd>` class,
7626 this variable specifies whether the specified service in
7627 :term:`SYSTEMD_SERVICE` should start
7628 automatically or not. By default, the service is enabled to
7629 automatically start at boot time. The default setting is in the
7630 :ref:`systemd <ref-classes-systemd>` class as follows:
7631 ::
7632
7633 SYSTEMD_AUTO_ENABLE ??= "enable"
7634
7635 You can disable the service by setting the variable to "disable".
7636
7637 SYSTEMD_BOOT_CFG
7638 When :term:`EFI_PROVIDER` is set to
7639 "systemd-boot", the ``SYSTEMD_BOOT_CFG`` variable specifies the
7640 configuration file that should be used. By default, the
7641 :ref:`systemd-boot <ref-classes-systemd-boot>` class sets the
7642 ``SYSTEMD_BOOT_CFG`` as follows:
7643 ::
7644
7645 SYSTEMD_BOOT_CFG ?= "${:term:`S`}/loader.conf"
7646
7647 For information on Systemd-boot, see the `Systemd-boot
7648 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
7649
7650 SYSTEMD_BOOT_ENTRIES
7651 When :term:`EFI_PROVIDER` is set to
7652 "systemd-boot", the ``SYSTEMD_BOOT_ENTRIES`` variable specifies a
7653 list of entry files (``*.conf``) to install that contain one boot
7654 entry per file. By default, the
7655 :ref:`systemd-boot <ref-classes-systemd-boot>` class sets the
7656 ``SYSTEMD_BOOT_ENTRIES`` as follows:
7657 ::
7658
7659 SYSTEMD_BOOT_ENTRIES ?= ""
7660
7661 For information on Systemd-boot, see the `Systemd-boot
7662 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
7663
7664 SYSTEMD_BOOT_TIMEOUT
7665 When :term:`EFI_PROVIDER` is set to
7666 "systemd-boot", the ``SYSTEMD_BOOT_TIMEOUT`` variable specifies the
7667 boot menu timeout in seconds. By default, the
7668 :ref:`systemd-boot <ref-classes-systemd-boot>` class sets the
7669 ``SYSTEMD_BOOT_TIMEOUT`` as follows:
7670 ::
7671
7672 SYSTEMD_BOOT_TIMEOUT ?= "10"
7673
7674 For information on Systemd-boot, see the `Systemd-boot
7675 documentation <http://www.freedesktop.org/wiki/Software/systemd/systemd-boot/>`__.
7676
7677 SYSTEMD_PACKAGES
7678 When inheriting the :ref:`systemd <ref-classes-systemd>` class,
7679 this variable locates the systemd unit files when they are not found
7680 in the main recipe's package. By default, the ``SYSTEMD_PACKAGES``
7681 variable is set such that the systemd unit files are assumed to
7682 reside in the recipes main package:
7683 ::
7684
7685 SYSTEMD_PACKAGES ?= "${PN}"
7686
7687 If these unit files are not in this recipe's main package, you need
7688 to use ``SYSTEMD_PACKAGES`` to list the package or packages in which
7689 the build system can find the systemd unit files.
7690
7691 SYSTEMD_SERVICE
7692 When inheriting the :ref:`systemd <ref-classes-systemd>` class,
7693 this variable specifies the systemd service name for a package.
7694
7695 When you specify this file in your recipe, use a package name
7696 override to indicate the package to which the value applies. Here is
7697 an example from the connman recipe:
7698 ::
7699
7700 SYSTEMD_SERVICE_${PN} = "connman.service"
7701
7702 SYSVINIT_ENABLED_GETTYS
7703 When using
7704 :ref:`SysVinit <dev-manual/dev-manual-common-tasks:enabling system services>`,
7705 specifies a space-separated list of the virtual terminals that should
7706 run a `getty <http://en.wikipedia.org/wiki/Getty_%28Unix%29>`__
7707 (allowing login), assuming :term:`USE_VT` is not set to
7708 "0".
7709
7710 The default value for ``SYSVINIT_ENABLED_GETTYS`` is "1" (i.e. only
7711 run a getty on the first virtual terminal).
7712
7713 T
7714 This variable points to a directory were BitBake places temporary
7715 files, which consist mostly of task logs and scripts, when building a
7716 particular recipe. The variable is typically set as follows:
7717 ::
7718
7719 T = "${WORKDIR}/temp"
7720
7721 The :term:`WORKDIR` is the directory into which
7722 BitBake unpacks and builds the recipe. The default ``bitbake.conf``
7723 file sets this variable.
7724
7725 The ``T`` variable is not to be confused with the
7726 :term:`TMPDIR` variable, which points to the root of
7727 the directory tree where BitBake places the output of an entire
7728 build.
7729
7730 TARGET_ARCH
7731 The target machine's architecture. The OpenEmbedded build system
7732 supports many architectures. Here is an example list of architectures
7733 supported. This list is by no means complete as the architecture is
7734 configurable:
7735
7736 - arm
7737 - i586
7738 - x86_64
7739 - powerpc
7740 - powerpc64
7741 - mips
7742 - mipsel
7743
7744 For additional information on machine architectures, see the
7745 :term:`TUNE_ARCH` variable.
7746
7747 TARGET_AS_ARCH
7748 Specifies architecture-specific assembler flags for the target
7749 system. ``TARGET_AS_ARCH`` is initialized from
7750 :term:`TUNE_ASARGS` by default in the BitBake
7751 configuration file (``meta/conf/bitbake.conf``):
7752 ::
7753
7754 TARGET_AS_ARCH = "${TUNE_ASARGS}"
7755
7756 TARGET_CC_ARCH
7757 Specifies architecture-specific C compiler flags for the target
7758 system. ``TARGET_CC_ARCH`` is initialized from
7759 :term:`TUNE_CCARGS` by default.
7760
7761 .. note::
7762
7763 It is a common workaround to append
7764 LDFLAGS
7765 to
7766 TARGET_CC_ARCH
7767 in recipes that build software for the target that would not
7768 otherwise respect the exported
7769 LDFLAGS
7770 variable.
7771
7772 TARGET_CC_KERNEL_ARCH
7773 This is a specific kernel compiler flag for a CPU or Application
7774 Binary Interface (ABI) tune. The flag is used rarely and only for
7775 cases where a userspace :term:`TUNE_CCARGS` is not
7776 compatible with the kernel compilation. The ``TARGET_CC_KERNEL_ARCH``
7777 variable allows the kernel (and associated modules) to use a
7778 different configuration. See the
7779 ``meta/conf/machine/include/arm/feature-arm-thumb.inc`` file in the
7780 :term:`Source Directory` for an example.
7781
7782 TARGET_CFLAGS
7783 Specifies the flags to pass to the C compiler when building for the
7784 target. When building in the target context,
7785 :term:`CFLAGS` is set to the value of this variable by
7786 default.
7787
7788 Additionally, the SDK's environment setup script sets the ``CFLAGS``
7789 variable in the environment to the ``TARGET_CFLAGS`` value so that
7790 executables built using the SDK also have the flags applied.
7791
7792 TARGET_CPPFLAGS
7793 Specifies the flags to pass to the C pre-processor (i.e. to both the
7794 C and the C++ compilers) when building for the target. When building
7795 in the target context, :term:`CPPFLAGS` is set to the
7796 value of this variable by default.
7797
7798 Additionally, the SDK's environment setup script sets the
7799 ``CPPFLAGS`` variable in the environment to the ``TARGET_CPPFLAGS``
7800 value so that executables built using the SDK also have the flags
7801 applied.
7802
7803 TARGET_CXXFLAGS
7804 Specifies the flags to pass to the C++ compiler when building for the
7805 target. When building in the target context,
7806 :term:`CXXFLAGS` is set to the value of this variable
7807 by default.
7808
7809 Additionally, the SDK's environment setup script sets the
7810 ``CXXFLAGS`` variable in the environment to the ``TARGET_CXXFLAGS``
7811 value so that executables built using the SDK also have the flags
7812 applied.
7813
7814 TARGET_FPU
7815 Specifies the method for handling FPU code. For FPU-less targets,
7816 which include most ARM CPUs, the variable must be set to "soft". If
7817 not, the kernel emulation gets used, which results in a performance
7818 penalty.
7819
7820 TARGET_LD_ARCH
7821 Specifies architecture-specific linker flags for the target system.
7822 ``TARGET_LD_ARCH`` is initialized from
7823 :term:`TUNE_LDARGS` by default in the BitBake
7824 configuration file (``meta/conf/bitbake.conf``):
7825 ::
7826
7827 TARGET_LD_ARCH = "${TUNE_LDARGS}"
7828
7829 TARGET_LDFLAGS
7830 Specifies the flags to pass to the linker when building for the
7831 target. When building in the target context,
7832 :term:`LDFLAGS` is set to the value of this variable
7833 by default.
7834
7835 Additionally, the SDK's environment setup script sets the
7836 :term:`LDFLAGS` variable in the environment to the
7837 ``TARGET_LDFLAGS`` value so that executables built using the SDK also
7838 have the flags applied.
7839
7840 TARGET_OS
7841 Specifies the target's operating system. The variable can be set to
7842 "linux" for glibc-based systems (GNU C Library) and to "linux-musl"
7843 for musl libc. For ARM/EABI targets, "linux-gnueabi" and
7844 "linux-musleabi" possible values exist.
7845
7846 TARGET_PREFIX
7847 Specifies the prefix used for the toolchain binary target tools.
7848
7849 Depending on the type of recipe and the build target,
7850 ``TARGET_PREFIX`` is set as follows:
7851
7852 - For recipes building for the target machine, the value is
7853 "${:term:`TARGET_SYS`}-".
7854
7855 - For native recipes, the build system sets the variable to the
7856 value of ``BUILD_PREFIX``.
7857
7858 - For native SDK recipes (``nativesdk``), the build system sets the
7859 variable to the value of ``SDK_PREFIX``.
7860
7861 TARGET_SYS
7862 Specifies the system, including the architecture and the operating
7863 system, for which the build is occurring in the context of the
7864 current recipe.
7865
7866 The OpenEmbedded build system automatically sets this variable based
7867 on :term:`TARGET_ARCH`,
7868 :term:`TARGET_VENDOR`, and
7869 :term:`TARGET_OS` variables.
7870
7871 .. note::
7872
7873 You do not need to set the TARGET_SYS variable yourself.
7874
7875 Consider these two examples:
7876
7877 - Given a native recipe on a 32-bit, x86 machine running Linux, the
7878 value is "i686-linux".
7879
7880 - Given a recipe being built for a little-endian, MIPS target
7881 running Linux, the value might be "mipsel-linux".
7882
7883 TARGET_VENDOR
7884 Specifies the name of the target vendor.
7885
7886 TCLIBC
7887 Specifies the GNU standard C library (``libc``) variant to use during
7888 the build process. This variable replaces ``POKYLIBC``, which is no
7889 longer supported.
7890
7891 You can select "glibc", "musl", "newlib", or "baremetal"
7892
7893 TCLIBCAPPEND
7894 Specifies a suffix to be appended onto the
7895 :term:`TMPDIR` value. The suffix identifies the
7896 ``libc`` variant for building. When you are building for multiple
7897 variants with the same :term:`Build Directory`, this
7898 mechanism ensures that output for different ``libc`` variants is kept
7899 separate to avoid potential conflicts.
7900
7901 In the ``defaultsetup.conf`` file, the default value of
7902 ``TCLIBCAPPEND`` is "-${TCLIBC}". However, distros such as poky,
7903 which normally only support one ``libc`` variant, set
7904 ``TCLIBCAPPEND`` to "" in their distro configuration file resulting
7905 in no suffix being applied.
7906
7907 TCMODE
7908 Specifies the toolchain selector. ``TCMODE`` controls the
7909 characteristics of the generated packages and images by telling the
7910 OpenEmbedded build system which toolchain profile to use. By default,
7911 the OpenEmbedded build system builds its own internal toolchain. The
7912 variable's default value is "default", which uses that internal
7913 toolchain.
7914
7915 .. note::
7916
7917 If
7918 TCMODE
7919 is set to a value other than "default", then it is your
7920 responsibility to ensure that the toolchain is compatible with the
7921 default toolchain. Using older or newer versions of these
7922 components might cause build problems. See the Release Notes for
7923 the Yocto Project release for the specific components with which
7924 the toolchain must be compatible. To access the Release Notes, go
7925 to the
7926 Downloads
7927 page on the Yocto Project website and click on the "RELEASE
7928 INFORMATION" link for the appropriate release.
7929
7930 The ``TCMODE`` variable is similar to :term:`TCLIBC`,
7931 which controls the variant of the GNU standard C library (``libc``)
7932 used during the build process: ``glibc`` or ``musl``.
7933
7934 With additional layers, it is possible to use a pre-compiled external
7935 toolchain. One example is the Sourcery G++ Toolchain. The support for
7936 this toolchain resides in the separate Mentor Graphics
7937 ``meta-sourcery`` layer at
7938 http://github.com/MentorEmbedded/meta-sourcery/.
7939
7940 The layer's ``README`` file contains information on how to use the
7941 Sourcery G++ Toolchain as an external toolchain. In summary, you must
7942 be sure to add the layer to your ``bblayers.conf`` file in front of
7943 the ``meta`` layer and then set the ``EXTERNAL_TOOLCHAIN`` variable
7944 in your ``local.conf`` file to the location in which you installed
7945 the toolchain.
7946
7947 The fundamentals used for this example apply to any external
7948 toolchain. You can use ``meta-sourcery`` as a template for adding
7949 support for other external toolchains.
7950
7951 TEST_EXPORT_DIR
7952 The location the OpenEmbedded build system uses to export tests when
7953 the :term:`TEST_EXPORT_ONLY` variable is set
7954 to "1".
7955
7956 The ``TEST_EXPORT_DIR`` variable defaults to
7957 ``"${TMPDIR}/testimage/${PN}"``.
7958
7959 TEST_EXPORT_ONLY
7960 Specifies to export the tests only. Set this variable to "1" if you
7961 do not want to run the tests but you want them to be exported in a
7962 manner that you to run them outside of the build system.
7963
7964 TEST_LOG_DIR
7965 Holds the SSH log and the boot log for QEMU machines. The
7966 ``TEST_LOG_DIR`` variable defaults to ``"${WORKDIR}/testimage"``.
7967
7968 .. note::
7969
7970 Actual test results reside in the task log (
7971 log.do_testimage
7972 ), which is in the
7973 ${WORKDIR}/temp/
7974 directory.
7975
7976 TEST_POWERCONTROL_CMD
7977 For automated hardware testing, specifies the command to use to
7978 control the power of the target machine under test. Typically, this
7979 command would point to a script that performs the appropriate action
7980 (e.g. interacting with a web-enabled power strip). The specified
7981 command should expect to receive as the last argument "off", "on" or
7982 "cycle" specifying to power off, on, or cycle (power off and then
7983 power on) the device, respectively.
7984
7985 TEST_POWERCONTROL_EXTRA_ARGS
7986 For automated hardware testing, specifies additional arguments to
7987 pass through to the command specified in
7988 :term:`TEST_POWERCONTROL_CMD`. Setting
7989 ``TEST_POWERCONTROL_EXTRA_ARGS`` is optional. You can use it if you
7990 wish, for example, to separate the machine-specific and
7991 non-machine-specific parts of the arguments.
7992
7993 TEST_QEMUBOOT_TIMEOUT
7994 The time in seconds allowed for an image to boot before automated
7995 runtime tests begin to run against an image. The default timeout
7996 period to allow the boot process to reach the login prompt is 500
7997 seconds. You can specify a different value in the ``local.conf``
7998 file.
7999
8000 For more information on testing images, see the
8001 ":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
8002 section in the Yocto Project Development Tasks Manual.
8003
8004 TEST_SERIALCONTROL_CMD
8005 For automated hardware testing, specifies the command to use to
8006 connect to the serial console of the target machine under test. This
8007 command simply needs to connect to the serial console and forward
8008 that connection to standard input and output as any normal terminal
8009 program does.
8010
8011 For example, to use the Picocom terminal program on serial device
8012 ``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows:
8013 ::
8014
8015 TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
8016
8017 TEST_SERIALCONTROL_EXTRA_ARGS
8018 For automated hardware testing, specifies additional arguments to
8019 pass through to the command specified in
8020 :term:`TEST_SERIALCONTROL_CMD`. Setting
8021 ``TEST_SERIALCONTROL_EXTRA_ARGS`` is optional. You can use it if you
8022 wish, for example, to separate the machine-specific and
8023 non-machine-specific parts of the command.
8024
8025 TEST_SERVER_IP
8026 The IP address of the build machine (host machine). This IP address
8027 is usually automatically detected. However, if detection fails, this
8028 variable needs to be set to the IP address of the build machine (i.e.
8029 where the build is taking place).
8030
8031 .. note::
8032
8033 The
8034 TEST_SERVER_IP
8035 variable is only used for a small number of tests such as the
8036 "dnf" test suite, which needs to download packages from
8037 WORKDIR/oe-rootfs-repo
8038 .
8039
8040 TEST_TARGET
8041 Specifies the target controller to use when running tests against a
8042 test image. The default controller to use is "qemu":
8043 ::
8044
8045 TEST_TARGET = "qemu"
8046
8047 A target controller is a class that defines how an image gets
8048 deployed on a target and how a target is started. A layer can extend
8049 the controllers by adding a module in the layer's
8050 ``/lib/oeqa/controllers`` directory and by inheriting the
8051 ``BaseTarget`` class, which is an abstract class that cannot be used
8052 as a value of ``TEST_TARGET``.
8053
8054 You can provide the following arguments with ``TEST_TARGET``:
8055
8056 - *"qemu":* Boots a QEMU image and runs the tests. See the
8057 ":ref:`qemu-image-enabling-tests`" section
8058 in the Yocto Project Development Tasks Manual for more
8059 information.
8060
8061 - *"simpleremote":* Runs the tests on target hardware that is
8062 already up and running. The hardware can be on the network or it
8063 can be a device running an image on QEMU. You must also set
8064 :term:`TEST_TARGET_IP` when you use
8065 "simpleremote".
8066
8067 .. note::
8068
8069 This argument is defined in
8070 meta/lib/oeqa/controllers/simpleremote.py
8071 .
8072
8073 For information on running tests on hardware, see the
8074 ":ref:`hardware-image-enabling-tests`"
8075 section in the Yocto Project Development Tasks Manual.
8076
8077 TEST_TARGET_IP
8078 The IP address of your hardware under test. The ``TEST_TARGET_IP``
8079 variable has no effect when :term:`TEST_TARGET` is
8080 set to "qemu".
8081
8082 When you specify the IP address, you can also include a port. Here is
8083 an example:
8084 ::
8085
8086 TEST_TARGET_IP = "192.168.1.4:2201"
8087
8088 Specifying a port is
8089 useful when SSH is started on a non-standard port or in cases when
8090 your hardware under test is behind a firewall or network that is not
8091 directly accessible from your host and you need to do port address
8092 translation.
8093
8094 TEST_SUITES
8095 An ordered list of tests (modules) to run against an image when
8096 performing automated runtime testing.
8097
8098 The OpenEmbedded build system provides a core set of tests that can
8099 be used against images.
8100
8101 .. note::
8102
8103 Currently, there is only support for running these tests under
8104 QEMU.
8105
8106 Tests include ``ping``, ``ssh``, ``df`` among others. You can add
8107 your own tests to the list of tests by appending ``TEST_SUITES`` as
8108 follows:
8109 ::
8110
8111 TEST_SUITES_append = " mytest"
8112
8113 Alternatively, you can
8114 provide the "auto" option to have all applicable tests run against
8115 the image.
8116 ::
8117
8118 TEST_SUITES_append = " auto"
8119
8120 Using this option causes the
8121 build system to automatically run tests that are applicable to the
8122 image. Tests that are not applicable are skipped.
8123
8124 The order in which tests are run is important. Tests that depend on
8125 another test must appear later in the list than the test on which
8126 they depend. For example, if you append the list of tests with two
8127 tests (``test_A`` and ``test_B``) where ``test_B`` is dependent on
8128 ``test_A``, then you must order the tests as follows:
8129 ::
8130
8131 TEST_SUITES = "test_A test_B"
8132
8133 For more information on testing images, see the
8134 ":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
8135 section in the Yocto Project Development Tasks Manual.
8136
8137 TESTIMAGE_AUTO
8138 Automatically runs the series of automated tests for images when an
8139 image is successfully built. Setting ``TESTIMAGE_AUTO`` to "1" causes
8140 any image that successfully builds to automatically boot under QEMU.
8141 Using the variable also adds in dependencies so that any SDK for
8142 which testing is requested is automatically built first.
8143
8144 These tests are written in Python making use of the ``unittest``
8145 module, and the majority of them run commands on the target system
8146 over ``ssh``. You can set this variable to "1" in your ``local.conf``
8147 file in the :term:`Build Directory` to have the
8148 OpenEmbedded build system automatically run these tests after an
8149 image successfully builds:
8150
8151 TESTIMAGE_AUTO = "1"
8152
8153 For more information
8154 on enabling, running, and writing these tests, see the
8155 ":ref:`dev-manual/dev-manual-common-tasks:performing automated runtime testing`"
8156 section in the Yocto Project Development Tasks Manual and the
8157 ":ref:`testimage*.bbclass <ref-classes-testimage*>`" section.
8158
8159 THISDIR
8160 The directory in which the file BitBake is currently parsing is
8161 located. Do not manually set this variable.
8162
8163 TIME
8164 The time the build was started. Times appear using the hour, minute,
8165 and second (HMS) format (e.g. "140159" for one minute and fifty-nine
8166 seconds past 1400 hours).
8167
8168 TMPDIR
8169 This variable is the base directory the OpenEmbedded build system
8170 uses for all build output and intermediate files (other than the
8171 shared state cache). By default, the ``TMPDIR`` variable points to
8172 ``tmp`` within the :term:`Build Directory`.
8173
8174 If you want to establish this directory in a location other than the
8175 default, you can uncomment and edit the following statement in the
8176 ``conf/local.conf`` file in the :term:`Source Directory`:
8177 ::
8178
8179 #TMPDIR = "${TOPDIR}/tmp"
8180
8181 An example use for this scenario is to set ``TMPDIR`` to a local disk,
8182 which does not use NFS, while having the Build Directory use NFS.
8183
8184 The filesystem used by ``TMPDIR`` must have standard filesystem
8185 semantics (i.e. mixed-case files are unique, POSIX file locking, and
8186 persistent inodes). Due to various issues with NFS and bugs in some
8187 implementations, NFS does not meet this minimum requirement.
8188 Consequently, ``TMPDIR`` cannot be on NFS.
8189
8190 TOOLCHAIN_HOST_TASK
8191 This variable lists packages the OpenEmbedded build system uses when
8192 building an SDK, which contains a cross-development environment. The
8193 packages specified by this variable are part of the toolchain set
8194 that runs on the :term:`SDKMACHINE`, and each
8195 package should usually have the prefix ``nativesdk-``. For example,
8196 consider the following command when building an SDK:
8197 ::
8198
8199 $ bitbake -c populate_sdk imagename
8200
8201 In this case, a default list of packages is
8202 set in this variable, but you can add additional packages to the
8203 list. See the
8204 ":ref:`sdk-manual/sdk-appendix-customizing-standard:adding individual packages to the standard sdk`" section
8205 in the Yocto Project Application Development and the Extensible
8206 Software Development Kit (eSDK) manual for more information.
8207
8208 For background information on cross-development toolchains in the
8209 Yocto Project development environment, see the
8210 ":ref:`sdk-manual/sdk-intro:the cross-development toolchain`"
8211 section in the Yocto Project Overview and Concepts Manual. For
8212 information on setting up a cross-development environment, see the
8213 :doc:`../sdk-manual/sdk-manual` manual.
8214
8215 TOOLCHAIN_OUTPUTNAME
8216 This variable defines the name used for the toolchain output. The
8217 :ref:`populate_sdk_base <ref-classes-populate-sdk-*>` class sets
8218 the ``TOOLCHAIN_OUTPUTNAME`` variable as follows:
8219 ::
8220
8221 TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-toolchain-${SDK_VERSION}"
8222
8223 See
8224 the :term:`SDK_NAME` and
8225 :term:`SDK_VERSION` variables for additional
8226 information.
8227
8228 TOOLCHAIN_TARGET_TASK
8229 This variable lists packages the OpenEmbedded build system uses when
8230 it creates the target part of an SDK (i.e. the part built for the
8231 target hardware), which includes libraries and headers. Use this
8232 variable to add individual packages to the part of the SDK that runs
8233 on the target. See the
8234 ":ref:`sdk-manual/sdk-appendix-customizing-standard:adding individual packages to the standard sdk`" section
8235 in the Yocto Project Application Development and the Extensible
8236 Software Development Kit (eSDK) manual for more information.
8237
8238 For background information on cross-development toolchains in the
8239 Yocto Project development environment, see the
8240 ":ref:`sdk-manual/sdk-intro:the cross-development toolchain`"
8241 section in the Yocto Project Overview and Concepts Manual. For
8242 information on setting up a cross-development environment, see the
8243 :doc:`../sdk-manual/sdk-manual` manual.
8244
8245 TOPDIR
8246 The top-level :term:`Build Directory`. BitBake
8247 automatically sets this variable when you initialize your build
8248 environment using ````` <#structure-core-script>`__.
8249
8250 TRANSLATED_TARGET_ARCH
8251 A sanitized version of :term:`TARGET_ARCH`. This
8252 variable is used where the architecture is needed in a value where
8253 underscores are not allowed, for example within package filenames. In
8254 this case, dash characters replace any underscore characters used in
8255 ``TARGET_ARCH``.
8256
8257 Do not edit this variable.
8258
8259 TUNE_ARCH
8260 The GNU canonical architecture for a specific architecture (i.e.
8261 ``arm``, ``armeb``, ``mips``, ``mips64``, and so forth). BitBake uses
8262 this value to setup configuration.
8263
8264 ``TUNE_ARCH`` definitions are specific to a given architecture. The
8265 definitions can be a single static definition, or can be dynamically
8266 adjusted. You can see details for a given CPU family by looking at
8267 the architecture's ``README`` file. For example, the
8268 ``meta/conf/machine/include/mips/README`` file in the
8269 :term:`Source Directory` provides information for
8270 ``TUNE_ARCH`` specific to the ``mips`` architecture.
8271
8272 ``TUNE_ARCH`` is tied closely to
8273 :term:`TARGET_ARCH`, which defines the target
8274 machine's architecture. The BitBake configuration file
8275 (``meta/conf/bitbake.conf``) sets ``TARGET_ARCH`` as follows:
8276 ::
8277
8278 TARGET_ARCH = "${TUNE_ARCH}"
8279
8280 The following list, which is by no means complete since architectures
8281 are configurable, shows supported machine architectures:
8282
8283 - arm
8284 - i586
8285 - x86_64
8286 - powerpc
8287 - powerpc64
8288 - mips
8289 - mipsel
8290
8291 TUNE_ASARGS
8292 Specifies architecture-specific assembler flags for the target
8293 system. The set of flags is based on the selected tune features.
8294 ``TUNE_ASARGS`` is set using the tune include files, which are
8295 typically under ``meta/conf/machine/include/`` and are influenced
8296 through :term:`TUNE_FEATURES`. For example, the
8297 ``meta/conf/machine/include/x86/arch-x86.inc`` file defines the flags
8298 for the x86 architecture as follows:
8299 ::
8300
8301 TUNE_ASARGS += "${@bb.utils.contains("TUNE_FEATURES", "mx32", "-x32", "", d)}"
8302
8303 .. note::
8304
8305 Board Support Packages (BSPs) select the tune. The selected tune,
8306 in turn, affects the tune variables themselves (i.e. the tune can
8307 supply its own set of flags).
8308
8309 TUNE_CCARGS
8310 Specifies architecture-specific C compiler flags for the target
8311 system. The set of flags is based on the selected tune features.
8312 ``TUNE_CCARGS`` is set using the tune include files, which are
8313 typically under ``meta/conf/machine/include/`` and are influenced
8314 through :term:`TUNE_FEATURES`.
8315
8316 .. note::
8317
8318 Board Support Packages (BSPs) select the tune. The selected tune,
8319 in turn, affects the tune variables themselves (i.e. the tune can
8320 supply its own set of flags).
8321
8322 TUNE_LDARGS
8323 Specifies architecture-specific linker flags for the target system.
8324 The set of flags is based on the selected tune features.
8325 ``TUNE_LDARGS`` is set using the tune include files, which are
8326 typically under ``meta/conf/machine/include/`` and are influenced
8327 through :term:`TUNE_FEATURES`. For example, the
8328 ``meta/conf/machine/include/x86/arch-x86.inc`` file defines the flags
8329 for the x86 architecture as follows:
8330 ::
8331
8332 TUNE_LDARGS += "${@bb.utils.contains("TUNE_FEATURES", "mx32", "-m elf32_x86_64", "", d)}"
8333
8334 .. note::
8335
8336 Board Support Packages (BSPs) select the tune. The selected tune,
8337 in turn, affects the tune variables themselves (i.e. the tune can
8338 supply its own set of flags).
8339
8340 TUNE_FEATURES
8341 Features used to "tune" a compiler for optimal use given a specific
8342 processor. The features are defined within the tune files and allow
8343 arguments (i.e. ``TUNE_*ARGS``) to be dynamically generated based on
8344 the features.
8345
8346 The OpenEmbedded build system verifies the features to be sure they
8347 are not conflicting and that they are supported.
8348
8349 The BitBake configuration file (``meta/conf/bitbake.conf``) defines
8350 ``TUNE_FEATURES`` as follows:
8351 ::
8352
8353 TUNE_FEATURES ??= "${TUNE_FEATURES_tune-${DEFAULTTUNE}}"
8354
8355 See the :term:`DEFAULTTUNE` variable for more information.
8356
8357 TUNE_PKGARCH
8358 The package architecture understood by the packaging system to define
8359 the architecture, ABI, and tuning of output packages. The specific
8360 tune is defined using the "_tune" override as follows:
8361 ::
8362
8363 TUNE_PKGARCH_tune-tune = "tune"
8364
8365 These tune-specific package architectures are defined in the machine
8366 include files. Here is an example of the "core2-32" tuning as used in
8367 the ``meta/conf/machine/include/tune-core2.inc`` file:
8368 ::
8369
8370 TUNE_PKGARCH_tune-core2-32 = "core2-32"
8371
8372 TUNEABI
8373 An underlying Application Binary Interface (ABI) used by a particular
8374 tuning in a given toolchain layer. Providers that use prebuilt
8375 libraries can use the ``TUNEABI``,
8376 :term:`TUNEABI_OVERRIDE`, and
8377 :term:`TUNEABI_WHITELIST` variables to check
8378 compatibility of tunings against their selection of libraries.
8379
8380 If ``TUNEABI`` is undefined, then every tuning is allowed. See the
8381 :ref:`sanity <ref-classes-sanity>` class to see how the variable is
8382 used.
8383
8384 TUNEABI_OVERRIDE
8385 If set, the OpenEmbedded system ignores the
8386 :term:`TUNEABI_WHITELIST` variable.
8387 Providers that use prebuilt libraries can use the
8388 ``TUNEABI_OVERRIDE``, ``TUNEABI_WHITELIST``, and
8389 :term:`TUNEABI` variables to check compatibility of a
8390 tuning against their selection of libraries.
8391
8392 See the :ref:`sanity <ref-classes-sanity>` class to see how the
8393 variable is used.
8394
8395 TUNEABI_WHITELIST
8396 A whitelist of permissible :term:`TUNEABI` values. If
8397 ``TUNEABI_WHITELIST`` is not set, all tunes are allowed. Providers
8398 that use prebuilt libraries can use the ``TUNEABI_WHITELIST``,
8399 :term:`TUNEABI_OVERRIDE`, and ``TUNEABI``
8400 variables to check compatibility of a tuning against their selection
8401 of libraries.
8402
8403 See the :ref:`sanity <ref-classes-sanity>` class to see how the
8404 variable is used.
8405
8406 TUNECONFLICTS[feature]
8407 Specifies CPU or Application Binary Interface (ABI) tuning features
8408 that conflict with feature.
8409
8410 Known tuning conflicts are specified in the machine include files in
8411 the :term:`Source Directory`. Here is an example from
8412 the ``meta/conf/machine/include/mips/arch-mips.inc`` include file
8413 that lists the "o32" and "n64" features as conflicting with the "n32"
8414 feature:
8415 ::
8416
8417 TUNECONFLICTS[n32] = "o32 n64"
8418
8419 TUNEVALID[feature]
8420 Specifies a valid CPU or Application Binary Interface (ABI) tuning
8421 feature. The specified feature is stored as a flag. Valid features
8422 are specified in the machine include files (e.g.
8423 ``meta/conf/machine/include/arm/arch-arm.inc``). Here is an example
8424 from that file:
8425 ::
8426
8427 TUNEVALID[bigendian] = "Enable big-endian mode."
8428
8429 See the machine include files in the :term:`Source Directory`
8430 for these features.
8431
8432 UBOOT_CONFIG
8433 Configures the :term:`UBOOT_MACHINE` and can
8434 also define :term:`IMAGE_FSTYPES` for individual
8435 cases.
8436
8437 Following is an example from the ``meta-fsl-arm`` layer. ::
8438
8439 UBOOT_CONFIG ??= "sd"
8440 UBOOT_CONFIG[sd] = "mx6qsabreauto_config,sdcard"
8441 UBOOT_CONFIG[eimnor] = "mx6qsabreauto_eimnor_config"
8442 UBOOT_CONFIG[nand] = "mx6qsabreauto_nand_config,ubifs"
8443 UBOOT_CONFIG[spinor] = "mx6qsabreauto_spinor_config"
8444
8445 In this example, "sd" is selected as the configuration of the possible four for the
8446 ``UBOOT_MACHINE``. The "sd" configuration defines
8447 "mx6qsabreauto_config" as the value for ``UBOOT_MACHINE``, while the
8448 "sdcard" specifies the ``IMAGE_FSTYPES`` to use for the U-boot image.
8449
8450 For more information on how the ``UBOOT_CONFIG`` is handled, see the
8451 :ref:`uboot-config <ref-classes-uboot-config>`
8452 class.
8453
8454 UBOOT_DTB_LOADADDRESS
8455 Specifies the load address for the dtb image used by U-boot. During FIT
8456 image creation, the ``UBOOT_DTB_LOADADDRESS`` variable is used in
8457 :ref:`kernel-fitimage <ref-classes-kernel-fitimage>` class to specify
8458 the load address to be used in
8459 creating the dtb sections of Image Tree Source for the FIT image.
8460
8461 UBOOT_DTBO_LOADADDRESS
8462 Specifies the load address for the dtbo image used by U-boot. During FIT
8463 image creation, the ``UBOOT_DTBO_LOADADDRESS`` variable is used in
8464 :ref:`kernel-fitimage <ref-classes-kernel-fitimage>` class to specify the load address to be used in
8465 creating the dtbo sections of Image Tree Source for the FIT image.
8466
8467 UBOOT_ENTRYPOINT
8468 Specifies the entry point for the U-Boot image. During U-Boot image
8469 creation, the ``UBOOT_ENTRYPOINT`` variable is passed as a
8470 command-line parameter to the ``uboot-mkimage`` utility.
8471
8472 UBOOT_LOADADDRESS
8473 Specifies the load address for the U-Boot image. During U-Boot image
8474 creation, the ``UBOOT_LOADADDRESS`` variable is passed as a
8475 command-line parameter to the ``uboot-mkimage`` utility.
8476
8477 UBOOT_LOCALVERSION
8478 Appends a string to the name of the local version of the U-Boot
8479 image. For example, assuming the version of the U-Boot image built
8480 was "2013.10", the full version string reported by U-Boot would be
8481 "2013.10-yocto" given the following statement:
8482 ::
8483
8484 UBOOT_LOCALVERSION = "-yocto"
8485
8486 UBOOT_MACHINE
8487 Specifies the value passed on the ``make`` command line when building
8488 a U-Boot image. The value indicates the target platform
8489 configuration. You typically set this variable from the machine
8490 configuration file (i.e. ``conf/machine/machine_name.conf``).
8491
8492 Please see the "Selection of Processor Architecture and Board Type"
8493 section in the U-Boot README for valid values for this variable.
8494
8495 UBOOT_MAKE_TARGET
8496 Specifies the target called in the ``Makefile``. The default target
8497 is "all".
8498
8499 UBOOT_MKIMAGE_DTCOPTS
8500 Options for the device tree compiler passed to mkimage '-D'
8501 feature while creating FIT image in :ref:`kernel-fitimage <ref-classes-kernel-fitimage>` class.
8502
8503 UBOOT_RD_LOADADDRESS
8504 Specifies the load address for the RAM disk image.
8505 During FIT image creation, the
8506 ``UBOOT_RD_LOADADDRESS`` variable is used
8507 in :ref:`kernel-fitimage <ref-classes-kernel-fitimage>` class to specify the
8508 load address to be used in creating the Image Tree Source for
8509 the FIT image.
8510
8511 UBOOT_RD_ENTRYPOINT
8512 Specifies the entrypoint for the RAM disk image.
8513 During FIT image creation, the
8514 ``UBOOT_RD_ENTRYPOINT`` variable is used
8515 in :ref:`kernel-fitimage <ref-classes-kernel-fitimage>` class to specify the
8516 entrypoint to be used in creating the Image Tree Source for
8517 the FIT image.
8518
8519 UBOOT_SIGN_ENABLE
8520 Enable signing of FIT image. The default value is "0".
8521
8522 UBOOT_SIGN_KEYDIR
8523 Location of the directory containing the RSA key and
8524 certificate used for signing FIT image.
8525
8526 UBOOT_SIGN_KEYNAME
8527 The name of keys used for signing U-boot FIT image stored in
8528 :term:`UBOOT_SIGN_KEYDIR` directory. For e.g. dev.key key and dev.crt
8529 certificate stored in :term:`UBOOT_SIGN_KEYDIR` directory will have
8530 :term:`UBOOT_SIGN_KEYNAME` set to "dev".
8531
8532 UBOOT_SUFFIX
8533 Points to the generated U-Boot extension. For example, ``u-boot.sb``
8534 has a ``.sb`` extension.
8535
8536 The default U-Boot extension is ``.bin``
8537
8538 UBOOT_TARGET
8539 Specifies the target used for building U-Boot. The target is passed
8540 directly as part of the "make" command (e.g. SPL and AIS). If you do
8541 not specifically set this variable, the OpenEmbedded build process
8542 passes and uses "all" for the target during the U-Boot building
8543 process.
8544
8545 UNKNOWN_CONFIGURE_WHITELIST
8546 Specifies a list of options that, if reported by the configure script
8547 as being invalid, should not generate a warning during the
8548 :ref:`ref-tasks-configure` task. Normally, invalid
8549 configure options are simply not passed to the configure script (e.g.
8550 should be removed from :term:`EXTRA_OECONF` or
8551 :term:`PACKAGECONFIG_CONFARGS`).
8552 However, common options, for example, exist that are passed to all
8553 configure scripts at a class level that might not be valid for some
8554 configure scripts. It follows that no benefit exists in seeing a
8555 warning about these options. For these cases, the options are added
8556 to ``UNKNOWN_CONFIGURE_WHITELIST``.
8557
8558 The configure arguments check that uses
8559 ``UNKNOWN_CONFIGURE_WHITELIST`` is part of the
8560 :ref:`insane <ref-classes-insane>` class and is only enabled if the
8561 recipe inherits the :ref:`autotools <ref-classes-autotools>` class.
8562
8563 UPDATERCPN
8564 For recipes inheriting the
8565 :ref:`update-rc.d <ref-classes-update-rc.d>` class, ``UPDATERCPN``
8566 specifies the package that contains the initscript that is enabled.
8567
8568 The default value is "${PN}". Given that almost all recipes that
8569 install initscripts package them in the main package for the recipe,
8570 you rarely need to set this variable in individual recipes.
8571
8572 UPSTREAM_CHECK_GITTAGREGEX
8573 You can perform a per-recipe check for what the latest upstream
8574 source code version is by calling ``bitbake -c checkpkg`` recipe. If
8575 the recipe source code is provided from Git repositories, the
8576 OpenEmbedded build system determines the latest upstream version by
8577 picking the latest tag from the list of all repository tags.
8578
8579 You can use the ``UPSTREAM_CHECK_GITTAGREGEX`` variable to provide a
8580 regular expression to filter only the relevant tags should the
8581 default filter not work correctly.
8582 ::
8583
8584 UPSTREAM_CHECK_GITTAGREGEX = "git_tag_regex"
8585
8586 UPSTREAM_CHECK_REGEX
8587 Use the ``UPSTREAM_CHECK_REGEX`` variable to specify a different
8588 regular expression instead of the default one when the package
8589 checking system is parsing the page found using
8590 :term:`UPSTREAM_CHECK_URI`.
8591 ::
8592
8593 UPSTREAM_CHECK_REGEX = "package_regex"
8594
8595 UPSTREAM_CHECK_URI
8596 You can perform a per-recipe check for what the latest upstream
8597 source code version is by calling ``bitbake -c checkpkg`` recipe. If
8598 the source code is provided from tarballs, the latest version is
8599 determined by fetching the directory listing where the tarball is and
8600 attempting to find a later tarball. When this approach does not work,
8601 you can use ``UPSTREAM_CHECK_URI`` to provide a different URI that
8602 contains the link to the latest tarball.
8603 ::
8604
8605 UPSTREAM_CHECK_URI = "recipe_url"
8606
8607 USE_DEVFS
8608 Determines if ``devtmpfs`` is used for ``/dev`` population. The
8609 default value used for ``USE_DEVFS`` is "1" when no value is
8610 specifically set. Typically, you would set ``USE_DEVFS`` to "0" for a
8611 statically populated ``/dev`` directory.
8612
8613 See the ":ref:`selecting-dev-manager`" section in
8614 the Yocto Project Development Tasks Manual for information on how to
8615 use this variable.
8616
8617 USE_VT
8618 When using
8619 :ref:`SysVinit <new-recipe-enabling-system-services>`,
8620 determines whether or not to run a
8621 `getty <http://en.wikipedia.org/wiki/Getty_%28Unix%29>`__ on any
8622 virtual terminals in order to enable logging in through those
8623 terminals.
8624
8625 The default value used for ``USE_VT`` is "1" when no default value is
8626 specifically set. Typically, you would set ``USE_VT`` to "0" in the
8627 machine configuration file for machines that do not have a graphical
8628 display attached and therefore do not need virtual terminal
8629 functionality.
8630
8631 USER_CLASSES
8632 A list of classes to globally inherit. These classes are used by the
8633 OpenEmbedded build system to enable extra features (e.g.
8634 ``buildstats``, ``image-mklibs``, and so forth).
8635
8636 The default list is set in your ``local.conf`` file:
8637 ::
8638
8639 USER_CLASSES ?= "buildstats image-mklibs image-prelink"
8640
8641 For more information, see
8642 ``meta-poky/conf/local.conf.sample`` in the :term:`Source Directory`.
8643
8644 USERADD_ERROR_DYNAMIC
8645 If set to ``error``, forces the OpenEmbedded build system to produce
8646 an error if the user identification (``uid``) and group
8647 identification (``gid``) values are not defined in any of the files
8648 listed in :term:`USERADD_UID_TABLES` and
8649 :term:`USERADD_GID_TABLES`. If set to
8650 ``warn``, a warning will be issued instead.
8651
8652 The default behavior for the build system is to dynamically apply
8653 ``uid`` and ``gid`` values. Consequently, the
8654 ``USERADD_ERROR_DYNAMIC`` variable is by default not set. If you plan
8655 on using statically assigned ``gid`` and ``uid`` values, you should
8656 set the ``USERADD_ERROR_DYNAMIC`` variable in your ``local.conf``
8657 file as follows:
8658 ::
8659
8660 USERADD_ERROR_DYNAMIC = "error"
8661
8662 Overriding the
8663 default behavior implies you are going to also take steps to set
8664 static ``uid`` and ``gid`` values through use of the
8665 :term:`USERADDEXTENSION`,
8666 :term:`USERADD_UID_TABLES`, and
8667 :term:`USERADD_GID_TABLES` variables.
8668
8669 .. note::
8670
8671 There is a difference in behavior between setting
8672 USERADD_ERROR_DYNAMIC
8673 to
8674 error
8675 and setting it to
8676 warn
8677 . When it is set to
8678 warn
8679 , the build system will report a warning for every undefined
8680 uid
8681 and
8682 gid
8683 in any recipe. But when it is set to
8684 error
8685 , it will only report errors for recipes that are actually built.
8686 This saves you from having to add static IDs for recipes that you
8687 know will never be built.
8688
8689 USERADD_GID_TABLES
8690 Specifies a password file to use for obtaining static group
8691 identification (``gid``) values when the OpenEmbedded build system
8692 adds a group to the system during package installation.
8693
8694 When applying static group identification (``gid``) values, the
8695 OpenEmbedded build system looks in :term:`BBPATH` for a
8696 ``files/group`` file and then applies those ``uid`` values. Set the
8697 variable as follows in your ``local.conf`` file:
8698 ::
8699
8700
8701 USERADD_GID_TABLES = "files/group"
8702
8703 .. note::
8704
8705 Setting the
8706 USERADDEXTENSION
8707 variable to "useradd-staticids" causes the build system to use
8708 static
8709 gid
8710 values.
8711
8712 USERADD_PACKAGES
8713 When inheriting the :ref:`useradd <ref-classes-useradd>` class,
8714 this variable specifies the individual packages within the recipe
8715 that require users and/or groups to be added.
8716
8717 You must set this variable if the recipe inherits the class. For
8718 example, the following enables adding a user for the main package in
8719 a recipe:
8720 ::
8721
8722 USERADD_PACKAGES = "${PN}"
8723
8724 .. note::
8725
8726 It follows that if you are going to use the
8727 USERADD_PACKAGES
8728 variable, you need to set one or more of the
8729 USERADD_PARAM
8730 ,
8731 GROUPADD_PARAM
8732 , or
8733 GROUPMEMS_PARAM
8734 variables.
8735
8736 USERADD_PARAM
8737 When inheriting the :ref:`useradd <ref-classes-useradd>` class,
8738 this variable specifies for a package what parameters should pass to
8739 the ``useradd`` command if you add a user to the system when the
8740 package is installed.
8741
8742 Here is an example from the ``dbus`` recipe:
8743 ::
8744
8745 USERADD_PARAM_${PN} = "--system --home ${localstatedir}/lib/dbus \
8746 --no-create-home --shell /bin/false \
8747 --user-group messagebus"
8748
8749 For information on the
8750 standard Linux shell command ``useradd``, see
8751 http://linux.die.net/man/8/useradd.
8752
8753 USERADD_UID_TABLES
8754 Specifies a password file to use for obtaining static user
8755 identification (``uid``) values when the OpenEmbedded build system
8756 adds a user to the system during package installation.
8757
8758 When applying static user identification (``uid``) values, the
8759 OpenEmbedded build system looks in :term:`BBPATH` for a
8760 ``files/passwd`` file and then applies those ``uid`` values. Set the
8761 variable as follows in your ``local.conf`` file:
8762 ::
8763
8764 USERADD_UID_TABLES = "files/passwd"
8765
8766 .. note::
8767
8768 Setting the
8769 USERADDEXTENSION
8770 variable to "useradd-staticids" causes the build system to use
8771 static
8772 uid
8773 values.
8774
8775 USERADDEXTENSION
8776 When set to "useradd-staticids", causes the OpenEmbedded build system
8777 to base all user and group additions on a static ``passwd`` and
8778 ``group`` files found in :term:`BBPATH`.
8779
8780 To use static user identification (``uid``) and group identification
8781 (``gid``) values, set the variable as follows in your ``local.conf``
8782 file: USERADDEXTENSION = "useradd-staticids"
8783
8784 .. note::
8785
8786 Setting this variable to use static
8787 uid
8788 and
8789 gid
8790 values causes the OpenEmbedded build system to employ the
8791 useradd-staticids
8792 class.
8793
8794 If you use static ``uid`` and ``gid`` information, you must also
8795 specify the ``files/passwd`` and ``files/group`` files by setting the
8796 :term:`USERADD_UID_TABLES` and
8797 :term:`USERADD_GID_TABLES` variables.
8798 Additionally, you should also set the
8799 :term:`USERADD_ERROR_DYNAMIC` variable.
8800
8801 VOLATILE_LOG_DIR
8802 Specifies the persistence of the target's ``/var/log`` directory,
8803 which is used to house postinstall target log files.
8804
8805 By default, ``VOLATILE_LOG_DIR`` is set to "yes", which means the
8806 file is not persistent. You can override this setting by setting the
8807 variable to "no" to make the log directory persistent.
8808
8809 WARN_QA
8810 Specifies the quality assurance checks whose failures are reported as
8811 warnings by the OpenEmbedded build system. You set this variable in
8812 your distribution configuration file. For a list of the checks you
8813 can control with this variable, see the
8814 ":ref:`insane.bbclass <ref-classes-insane>`" section.
8815
8816 WKS_FILE_DEPENDS
8817 When placed in the recipe that builds your image, this variable lists
8818 build-time dependencies. The ``WKS_FILE_DEPENDS`` variable is only
8819 applicable when Wic images are active (i.e. when
8820 :term:`IMAGE_FSTYPES` contains entries related
8821 to Wic). If your recipe does not create Wic images, the variable has
8822 no effect.
8823
8824 The ``WKS_FILE_DEPENDS`` variable is similar to the
8825 :term:`DEPENDS` variable. When you use the variable in
8826 your recipe that builds the Wic image, dependencies you list in the
8827 ``WIC_FILE_DEPENDS`` variable are added to the ``DEPENDS`` variable.
8828
8829 With the ``WKS_FILE_DEPENDS`` variable, you have the possibility to
8830 specify a list of additional dependencies (e.g. native tools,
8831 bootloaders, and so forth), that are required to build Wic images.
8832 Following is an example:
8833 ::
8834
8835 WKS_FILE_DEPENDS = "some-native-tool"
8836
8837 In the
8838 previous example, some-native-tool would be replaced with an actual
8839 native tool on which the build would depend.
8840
8841 WKS_FILE
8842 Specifies the location of the Wic kickstart file that is used by the
8843 OpenEmbedded build system to create a partitioned image
8844 (image\ ``.wic``). For information on how to create a partitioned
8845 image, see the
8846 ":ref:`dev-manual/dev-manual-common-tasks:creating partitioned images using wic`"
8847 section in the Yocto Project Development Tasks Manual. For details on
8848 the kickstart file format, see the ":doc:`../ref-manual/ref-kickstart`" Chapter.
8849
8850 WORKDIR
8851 The pathname of the work directory in which the OpenEmbedded build
8852 system builds a recipe. This directory is located within the
8853 :term:`TMPDIR` directory structure and is specific to
8854 the recipe being built and the system for which it is being built.
8855
8856 The ``WORKDIR`` directory is defined as follows:
8857 ::
8858
8859 ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}
8860
8861 The actual directory depends on several things:
8862
8863 - TMPDIR
8864 : The top-level build output directory
8865 - MULTIMACH_TARGET_SYS
8866 : The target system identifier
8867 - PN
8868 : The recipe name
8869 - EXTENDPE
8870 : The epoch - (if
8871 PE
8872 is not specified, which is usually the case for most recipes, then
8873 EXTENDPE
8874 is blank)
8875 - PV
8876 : The recipe version
8877 - PR
8878 : The recipe revision
8879
8880 As an example, assume a Source Directory top-level folder name
8881 ``poky``, a default Build Directory at ``poky/build``, and a
8882 ``qemux86-poky-linux`` machine target system. Furthermore, suppose
8883 your recipe is named ``foo_1.3.0-r0.bb``. In this case, the work
8884 directory the build system uses to build the package would be as
8885 follows:
8886 ::
8887
8888 poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
8889
8890 XSERVER
8891 Specifies the packages that should be installed to provide an X
8892 server and drivers for the current machine, assuming your image
8893 directly includes ``packagegroup-core-x11-xserver`` or, perhaps
8894 indirectly, includes "x11-base" in
8895 :term:`IMAGE_FEATURES`.
8896
8897 The default value of ``XSERVER``, if not specified in the machine
8898 configuration, is "xserver-xorg xf86-video-fbdev xf86-input-evdev".
8899
diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 364cd09eb8..a5064807e5 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<!-- Dummy chapter --> 6<!-- Dummy chapter -->
6<chapter id='ref-variables-glos'> 7<chapter id='ref-variables-glos'>
@@ -4990,6 +4991,30 @@
4990 </glossdef> 4991 </glossdef>
4991 </glossentry> 4992 </glossentry>
4992 4993
4994 <glossentry id='var-FIT_HASH_ALG'><glossterm>FIT_HASH_ALG</glossterm>
4995 <info>
4996 FIT_HASH_ALG[doc] = "Specifies the hash algorithm used in creating the FIT Image."
4997 </info>
4998 <glossdef>
4999 <para role="glossdeffirst">
5000 Specifies the hash algorithm used in creating the FIT Image.
5001 For e.g. sha256.
5002 </para>
5003 </glossdef>
5004 </glossentry>
5005
5006 <glossentry id='var-FIT_SIGN_ALG'><glossterm>FIT_SIGN_ALG</glossterm>
5007 <info>
5008 FIT_SIGN_ALG[doc] = "Specifies the signature algorithm used in creating the FIT Image."
5009 </info>
5010 <glossdef>
5011 <para role="glossdeffirst">
5012 Specifies the signature algorithm used in creating the FIT Image.
5013 For e.g. rsa2048.
5014 </para>
5015 </glossdef>
5016 </glossentry>
5017
4993 <glossentry id='var-FONT_EXTRA_RDEPENDS'><glossterm>FONT_EXTRA_RDEPENDS</glossterm> 5018 <glossentry id='var-FONT_EXTRA_RDEPENDS'><glossterm>FONT_EXTRA_RDEPENDS</glossterm>
4994 <info> 5019 <info>
4995 FONT_EXTRA_RDEPENDS[doc] = "When a recipe inherits the fontcache class, this variable specifies runtime dependencies for font packages. This variable defaults to 'fontconfig-utils'." 5020 FONT_EXTRA_RDEPENDS[doc] = "When a recipe inherits the fontcache class, this variable specifies runtime dependencies for font packages. This variable defaults to 'fontconfig-utils'."
@@ -5702,7 +5727,7 @@
5702 <para role="glossdeffirst"> 5727 <para role="glossdeffirst">
5703 A space-separated list of files installed into the 5728 A space-separated list of files installed into the
5704 boot partition when preparing an image using the Wic tool 5729 boot partition when preparing an image using the Wic tool
5705 with the <filename>bootimg-partition</filename> source 5730 with the <filename>bootimg-partition</filename> or <filename>bootimg-efi</filename> source
5706 plugin. 5731 plugin.
5707 By default, the files are installed under the same name as 5732 By default, the files are installed under the same name as
5708 the source files. 5733 the source files.
@@ -9539,6 +9564,40 @@
9539 </glossdef> 9564 </glossdef>
9540 </glossentry> 9565 </glossentry>
9541 9566
9567 <glossentry id='var-PACKAGE_ADD_METADATA'><glossterm>PACKAGE_ADD_METADATA</glossterm>
9568 <info>
9569 PACKAGE_ADD_METADATA[doc] = "This variable defines additional metadata to add to packages."
9570 </info>
9571 <glossdef>
9572 <para role="glossdeffirst">
9573<!-- <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
9574 This variable defines additional metdata to add to packages.
9575 </para>
9576
9577 <para>
9578 You may find you need to inject additional metadata into
9579 packages. This variable allows you to do that by setting
9580 the injected data as the value. Multiple fields can be
9581 added by splitting the content with the literal separator
9582 "\n".
9583 </para>
9584
9585 <para>
9586 The suffixes '_IPK', '_DEB', or '_RPM' can be applied to
9587 the variable to do package type specific settings. It can
9588 also be made package specific by using the package name as
9589 a suffix.
9590 </para>
9591
9592 <para>
9593 You can find out more about applying this variable in
9594 the
9595 "<ulink url='&YOCTO_DOCS_DEV_URL;#adding-custom-metadata-to-packages'>Adding custom metadata to packages</ulink>"
9596 section in the Yocto Project Development Tasks Manual.
9597 </para>
9598 </glossdef>
9599 </glossentry>
9600
9542 <glossentry id='var-PACKAGE_ARCH'><glossterm>PACKAGE_ARCH</glossterm> 9601 <glossentry id='var-PACKAGE_ARCH'><glossterm>PACKAGE_ARCH</glossterm>
9543 <info> 9602 <info>
9544 PACKAGE_ARCH[doc] = "The architecture of the resulting package or packages." 9603 PACKAGE_ARCH[doc] = "The architecture of the resulting package or packages."
@@ -15925,6 +15984,38 @@
15925 </glossdef> 15984 </glossdef>
15926 </glossentry> 15985 </glossentry>
15927 15986
15987 <glossentry id='var-UBOOT_DTB_LOADADDRESS'><glossterm>UBOOT_DTB_LOADADDRESS</glossterm>
15988 <info>
15989 UBOOT_DTB_LOADADDRESS[doc] = "Specifies the load address for the dtb."
15990 </info>
15991 <glossdef>
15992 <para role="glossdeffirst">
15993 Specifies the load address for the dtb image used by U-boot.
15994 During FIT image creation, the
15995 <filename>UBOOT_DTB_LOADADDRESS</filename> variable is used
15996 in <filename>kernel-fitimage</filename> class to specify the
15997 load address to be used in creating the dtb sections of
15998 Image Tree Source for the FIT image.
15999 </para>
16000 </glossdef>
16001 </glossentry>
16002
16003 <glossentry id='var-UBOOT_DTBO_LOADADDRESS'><glossterm>UBOOT_DTBO_LOADADDRESS</glossterm>
16004 <info>
16005 UBOOT_DTBO_LOADADDRESS[doc] = "Specifies the load address for the dtbo."
16006 </info>
16007 <glossdef>
16008 <para role="glossdeffirst">
16009 Specifies the load address for the dtbo image used by U-boot.
16010 During FIT image creation, the
16011 <filename>UBOOT_DTBO_LOADADDRESS</filename> variable is used
16012 in <filename>kernel-fitimage</filename> class to specify the
16013 load address to be used in creating the dtbo sections of
16014 Image Tree Source for the FIT image.
16015 </para>
16016 </glossdef>
16017 </glossentry>
16018
15928 <glossentry id='var-UBOOT_ENTRYPOINT'><glossterm>UBOOT_ENTRYPOINT</glossterm> 16019 <glossentry id='var-UBOOT_ENTRYPOINT'><glossterm>UBOOT_ENTRYPOINT</glossterm>
15929 <info> 16020 <info>
15930 UBOOT_ENTRYPOINT[doc] = "Specifies the entry point for the U-Boot image." 16021 UBOOT_ENTRYPOINT[doc] = "Specifies the entry point for the U-Boot image."
@@ -16010,6 +16101,51 @@
16010 </glossdef> 16101 </glossdef>
16011 </glossentry> 16102 </glossentry>
16012 16103
16104 <glossentry id='var-UBOOT_MKIMAGE_DTCOPTS'><glossterm>UBOOT_MKIMAGE_DTCOPTS</glossterm>
16105 <info>
16106 UBOOT_MKIMAGE_DTCOPTS[doc] = "Options for the device tree compiler passed to mkimage '-D' feature."
16107 </info>
16108 <glossdef>
16109 <para role="glossdeffirst">
16110 Options for the device tree compiler passed to mkimage '-D'
16111 feature while creating FIT image in
16112 <filename>kernel-fitimage</filename> class.
16113 </para>
16114 </glossdef>
16115 </glossentry>
16116
16117 <glossentry id='var-UBOOT_RD_LOADADDRESS'><glossterm>UBOOT_RD_LOADADDRESS</glossterm>
16118 <info>
16119 UBOOT_RD_LOADADDRESS[doc] = "Specifies the load address for the ramdisk image."
16120 </info>
16121 <glossdef>
16122 <para role="glossdeffirst">
16123 Specifies the load address for the RAM disk image.
16124 During FIT image creation, the
16125 <filename>UBOOT_RD_LOADADDRESS</filename> variable is used
16126 in <filename>kernel-fitimage</filename> class to specify the
16127 load address to be used in creating the Image Tree Source for
16128 the FIT image.
16129 </para>
16130 </glossdef>
16131 </glossentry>
16132
16133 <glossentry id='var-UBOOT_RD_ENTRYPOINT'><glossterm>UBOOT_RD_ENTRYPOINT</glossterm>
16134 <info>
16135 UBOOT_RD_ENTRYPOINT[doc] = "Specifies the entrypoint for the ramdisk image."
16136 </info>
16137 <glossdef>
16138 <para role="glossdeffirst">
16139 Specifies the entrypoint for the RAM disk image.
16140 During FIT image creation, the
16141 <filename>UBOOT_RD_ENTRYPOINT</filename> variable is used
16142 in <filename>kernel-fitimage</filename> class to specify the
16143 entrypoint to be used in creating the Image Tree Source for
16144 the FIT image.
16145 </para>
16146 </glossdef>
16147 </glossentry>
16148
16013 <glossentry id='var-UBOOT_SUFFIX'><glossterm>UBOOT_SUFFIX</glossterm> 16149 <glossentry id='var-UBOOT_SUFFIX'><glossterm>UBOOT_SUFFIX</glossterm>
16014 <info> 16150 <info>
16015 UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension." 16151 UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
@@ -16028,6 +16164,47 @@
16028 </glossdef> 16164 </glossdef>
16029 </glossentry> 16165 </glossentry>
16030 16166
16167 <glossentry id='var-UBOOT_SIGN_ENABLE'><glossterm>UBOOT_SIGN_ENABLE</glossterm>
16168 <info>
16169 UBOOT_SIGN_KEYDIR[doc] = "Enable signing of FIT image."
16170 </info>
16171 <glossdef>
16172 <para role="glossdeffirst">
16173 Enable signing of FIT image. The default value is "0".
16174 </para>
16175 </glossdef>
16176 </glossentry>
16177
16178 <glossentry id='var-UBOOT_SIGN_KEYDIR'><glossterm>UBOOT_SIGN_KEYDIR</glossterm>
16179 <info>
16180 UBOOT_SIGN_KEYDIR[doc] = "Location of the directory containing the RSA key and certificate used for signing FIT image."
16181 </info>
16182 <glossdef>
16183 <para role="glossdeffirst">
16184 Location of the directory containing the RSA key and
16185 certificate used for signing FIT image.
16186 </para>
16187 </glossdef>
16188 </glossentry>
16189
16190 <glossentry id='var-UBOOT_SIGN_KEYNAME'><glossterm>UBOOT_SIGN_KEYNAME</glossterm>
16191 <info>
16192 UBOOT_SIGN_KEYNAME[doc] = "The name of keys used for signing U-boot FIT image"
16193 </info>
16194 <glossdef>
16195 <para role="glossdeffirst">
16196 The name of keys used for signing U-boot FIT image stored in
16197 <filename><link linkend='var-UBOOT_SIGN_KEYDIR'>UBOOT_SIGN_KEYDIR</link></filename>
16198 directory. For e.g. dev.key key and dev.crt certificate
16199 stored in
16200 <filename><link linkend='var-UBOOT_SIGN_KEYDIR'>UBOOT_SIGN_KEYDIR</link></filename>
16201 directory will have
16202 <filename><link linkend='var-UBOOT_SIGN_KEYNAME'>UBOOT_SIGN_KEYNAME</link></filename>
16203 set to "dev".
16204 </para>
16205 </glossdef>
16206 </glossentry>
16207
16031 <glossentry id='var-UBOOT_TARGET'><glossterm>UBOOT_TARGET</glossterm> 16208 <glossentry id='var-UBOOT_TARGET'><glossterm>UBOOT_TARGET</glossterm>
16032 <info> 16209 <info>
16033 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot." 16210 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
diff --git a/documentation/ref-manual/ref-varlocality.rst b/documentation/ref-manual/ref-varlocality.rst
new file mode 100644
index 0000000000..a95504b571
--- /dev/null
+++ b/documentation/ref-manual/ref-varlocality.rst
@@ -0,0 +1,166 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3****************
4Variable Context
5****************
6
7While you can use most variables in almost any context such as
8``.conf``, ``.bbclass``, ``.inc``, and ``.bb`` files, some variables are
9often associated with a particular locality or context. This chapter
10describes some common associations.
11
12.. _ref-varlocality-configuration:
13
14Configuration
15=============
16
17The following subsections provide lists of variables whose context is
18configuration: distribution, machine, and local.
19
20.. _ref-varlocality-config-distro:
21
22Distribution (Distro)
23---------------------
24
25This section lists variables whose configuration context is the
26distribution, or distro.
27
28- :term:`DISTRO`
29
30- :term:`DISTRO_NAME`
31
32- :term:`DISTRO_VERSION`
33
34- :term:`MAINTAINER`
35
36- :term:`PACKAGE_CLASSES`
37
38- :term:`TARGET_OS`
39
40- :term:`TARGET_FPU`
41
42- :term:`TCMODE`
43
44- :term:`TCLIBC`
45
46.. _ref-varlocality-config-machine:
47
48Machine
49-------
50
51This section lists variables whose configuration context is the machine.
52
53- :term:`TARGET_ARCH`
54
55- :term:`SERIAL_CONSOLES`
56
57- :term:`PACKAGE_EXTRA_ARCHS`
58
59- :term:`IMAGE_FSTYPES`
60
61- :term:`MACHINE_FEATURES`
62
63- :term:`MACHINE_EXTRA_RDEPENDS`
64
65- :term:`MACHINE_EXTRA_RRECOMMENDS`
66
67- :term:`MACHINE_ESSENTIAL_EXTRA_RDEPENDS`
68
69- :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
70
71.. _ref-varlocality-config-local:
72
73Local
74-----
75
76This section lists variables whose configuration context is the local
77configuration through the ``local.conf`` file.
78
79- :term:`DISTRO`
80
81- :term:`MACHINE`
82
83- :term:`DL_DIR`
84
85- :term:`BBFILES`
86
87- :term:`EXTRA_IMAGE_FEATURES`
88
89- :term:`PACKAGE_CLASSES`
90
91- :term:`BB_NUMBER_THREADS`
92
93- :term:`BBINCLUDELOGS`
94
95- :term:`ENABLE_BINARY_LOCALE_GENERATION`
96
97.. _ref-varlocality-recipes:
98
99Recipes
100=======
101
102The following subsections provide lists of variables whose context is
103recipes: required, dependencies, path, and extra build information.
104
105.. _ref-varlocality-recipe-required:
106
107Required
108--------
109
110This section lists variables that are required for recipes.
111
112- :term:`LICENSE`
113
114- :term:`LIC_FILES_CHKSUM`
115
116- :term:`SRC_URI` - used in recipes that fetch local or remote files.
117
118.. _ref-varlocality-recipe-dependencies:
119
120Dependencies
121------------
122
123This section lists variables that define recipe dependencies.
124
125- :term:`DEPENDS`
126
127- :term:`RDEPENDS`
128
129- :term:`RRECOMMENDS`
130
131- :term:`RCONFLICTS`
132
133- :term:`RREPLACES`
134
135.. _ref-varlocality-recipe-paths:
136
137Paths
138-----
139
140This section lists variables that define recipe paths.
141
142- :term:`WORKDIR`
143
144- :term:`S`
145
146- :term:`FILES`
147
148.. _ref-varlocality-recipe-build:
149
150Extra Build Information
151-----------------------
152
153This section lists variables that define extra build information for
154recipes.
155
156- :term:`DEFAULT_PREFERENCE`
157
158- :term:`EXTRA_OECMAKE`
159
160- :term:`EXTRA_OECONF`
161
162- :term:`EXTRA_OEMAKE`
163
164- :term:`PACKAGECONFIG_CONFARGS`
165
166- :term:`PACKAGES`
diff --git a/documentation/ref-manual/ref-varlocality.xml b/documentation/ref-manual/ref-varlocality.xml
index 54524d5b60..a2436fb310 100644
--- a/documentation/ref-manual/ref-varlocality.xml
+++ b/documentation/ref-manual/ref-varlocality.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='ref-varlocality'> 6<chapter id='ref-varlocality'>
6 <title>Variable Context</title> 7 <title>Variable Context</title>
diff --git a/documentation/ref-manual/resources.rst b/documentation/ref-manual/resources.rst
new file mode 100644
index 0000000000..2b82b79102
--- /dev/null
+++ b/documentation/ref-manual/resources.rst
@@ -0,0 +1,197 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3****************************************
4Contributions and Additional Information
5****************************************
6
7.. _resources-intro:
8
9Introduction
10============
11
12The Yocto Project team is happy for people to experiment with the Yocto
13Project. A number of places exist to find help if you run into
14difficulties or find bugs. This presents information about contributing
15and participating in the Yocto Project.
16
17.. _resources-contributions:
18
19Contributions
20=============
21
22The Yocto Project gladly accepts contributions. You can submit changes
23to the project either by creating and sending pull requests, or by
24submitting patches through email. For information on how to do both as
25well as information on how to identify the maintainer for each area of
26code, see the ":ref:`how-to-submit-a-change`" section in the
27Yocto Project Development Tasks Manual.
28
29.. _resources-bugtracker:
30
31Yocto Project Bugzilla
32======================
33
34The Yocto Project uses its own implementation of
35:yocto_bugs:`Bugzilla <>` to track defects (bugs).
36Implementations of Bugzilla work well for group development because they
37track bugs and code changes, can be used to communicate changes and
38problems with developers, can be used to submit and review patches, and
39can be used to manage quality assurance.
40
41Sometimes it is helpful to submit, investigate, or track a bug against
42the Yocto Project itself (e.g. when discovering an issue with some
43component of the build system that acts contrary to the documentation or
44your expectations).
45
46A general procedure and guidelines exist for when you use Bugzilla to
47submit a bug. For information on how to use Bugzilla to submit a bug
48against the Yocto Project, see the following:
49
50- The ":ref:`dev-manual/dev-manual-common-tasks:submitting a defect against the yocto project`"
51 section in the Yocto Project Development Tasks Manual.
52
53- The Yocto Project :yocto_wiki:`Bugzilla wiki page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`
54
55For information on Bugzilla in general, see http://www.bugzilla.org/about/.
56
57.. _resources-mailinglist:
58
59Mailing lists
60=============
61
62A number of mailing lists maintained by the Yocto Project exist as well
63as related OpenEmbedded mailing lists for discussion, patch submission
64and announcements. To subscribe to one of the following mailing lists,
65click on the appropriate URL in the following list and follow the
66instructions:
67
68- https://lists.yoctoproject.org/g/yocto - General Yocto Project
69 discussion mailing list.
70
71- https://lists.openembedded.org/g/openembedded-core - Discussion mailing
72 list about OpenEmbedded-Core (the core metadata).
73
74- https://lists.openembedded.org/g/openembedded-devel - Discussion
75 mailing list about OpenEmbedded.
76
77- https://lists.openembedded.org/g/bitbake-devel - Discussion mailing
78 list about the :term:`BitBake` build tool.
79
80- https://lists.yoctoproject.org/g/poky - Discussion mailing list
81 about `Poky <#poky>`__.
82
83- https://lists.yoctoproject.org/g/yocto-announce - Mailing list to
84 receive official Yocto Project release and milestone announcements.
85
86For more Yocto Project-related mailing lists, see the
87Yocto Project Website
88.
89.. _resources-irc:
90
91Internet Relay Chat (IRC)
92=========================
93
94Two IRC channels on freenode are available for the Yocto Project and
95Poky discussions:
96
97- ``#yocto``
98
99- ``#poky``
100
101.. _resources-links-and-related-documentation:
102
103Links and Related Documentation
104===============================
105
106Here is a list of resources you might find helpful:
107
108- :yocto_home:`The Yocto Project Website <>`\ *:* The home site
109 for the Yocto Project.
110
111- :yocto_wiki:`The Yocto Project Main Wiki Page </wiki/Main_Page>`\ *:* The main wiki page for
112 the Yocto Project. This page contains information about project
113 planning, release engineering, QA & automation, a reference site map,
114 and other resources related to the Yocto Project.
115
116- `OpenEmbedded <http://www.openembedded.org/>`__\ *:* The build system used by the
117 Yocto Project. This project is the upstream, generic, embedded
118 distribution from which the Yocto Project derives its build system
119 (Poky) and to which it contributes.
120
121- `BitBake <http://www.openembedded.org/wiki/BitBake>`__\ *:* The tool
122 used to process metadata.
123
124- :doc:`BitBake User Manual <bitbake:index>`\ *:* A comprehensive
125 guide to the BitBake tool. If you want information on BitBake, see
126 this manual.
127
128- :doc:`../brief-yoctoprojectqs/brief-yoctoprojectqs` *:* This
129 short document lets you experience building an image using the Yocto
130 Project without having to understand any concepts or details.
131
132- :doc:`../overview-manual/overview-manual` *:* This manual provides overview
133 and conceptual information about the Yocto Project.
134
135- :doc:`../dev-manual/dev-manual` *:* This manual is a "how-to" guide
136 that presents procedures useful to both application and system
137 developers who use the Yocto Project.
138
139- :doc:`../sdk-manual/sdk-manual` *manual :* This
140 guide provides information that lets you get going with the standard
141 or extensible SDK. An SDK, with its cross-development toolchains,
142 allows you to develop projects inside or outside of the Yocto Project
143 environment.
144
145- :doc:`../bsp-guide/bsp` *:* This guide defines the structure
146 for BSP components. Having a commonly understood structure encourages
147 standardization.
148
149- :doc:`../kernel-dev/kernel-dev` *:* This manual describes
150 how to work with Linux Yocto kernels as well as provides a bit of
151 conceptual information on the construction of the Yocto Linux kernel
152 tree.
153
154- :doc:`../ref-manual/ref-manual` *:* This
155 manual provides reference material such as variable, task, and class
156 descriptions.
157
158- `Yocto Project Mega-Manual <https://docs.yoctoproject.org/singleindex.html>`__\ *:* This manual
159 is simply a single HTML file comprised of the bulk of the Yocto
160 Project manuals. The Mega-Manual primarily exists as a vehicle by
161 which you can easily search for phrases and terms used in the Yocto
162 Project documentation set.
163
164- :doc:`../profile-manual/profile-manual` *:* This manual presents a set of
165 common and generally useful tracing and profiling schemes along with
166 their applications (as appropriate) to each tool.
167
168- :doc:`../toaster-manual/toaster-manual` *:* This manual
169 introduces and describes how to set up and use Toaster. Toaster is an
170 Application Programming Interface (API) and web-based interface to
171 the :term:`OpenEmbedded Build System`, which uses
172 BitBake, that reports build information.
173
174- :yocto_wiki:`FAQ </wiki/FAQ>`\ *:* A list of commonly asked
175 questions and their answers.
176
177- *Release Notes:* Features, updates and known issues for the current
178 release of the Yocto Project. To access the Release Notes, go to the
179 :yocto_home:`Downloads </software-overview/downloads>` page on
180 the Yocto Project website and click on the "RELEASE INFORMATION" link
181 for the appropriate release.
182
183- `Bugzilla <https://bugzilla.yoctoproject.org>`__\ *:* The bug tracking application
184 the Yocto Project uses. If you find problems with the Yocto Project,
185 you should report them using this application.
186
187- :yocto_wiki:`Bugzilla Configuration and Bug Tracking Wiki Page </wiki/Bugzilla_Configuration_and_Bug_Tracking>`\ *:*
188 Information on how to get set up and use the Yocto Project
189 implementation of Bugzilla for logging and tracking Yocto Project
190 defects.
191
192- *Internet Relay Chat (IRC):* Two IRC channels on freenode are
193 available for Yocto Project and Poky discussions: ``#yocto`` and
194 ``#poky``, respectively.
195
196- `Quick EMUlator (QEMU) <http://wiki.qemu.org/Index.html>`__\ *:* An
197 open-source machine emulator and virtualizer.
diff --git a/documentation/ref-manual/resources.xml b/documentation/ref-manual/resources.xml
index afe8e288de..4899b2e599 100644
--- a/documentation/ref-manual/resources.xml
+++ b/documentation/ref-manual/resources.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='resources'> 6<chapter id='resources'>
6<title>Contributions and Additional Information</title> 7<title>Contributions and Additional Information</title>
diff --git a/documentation/releases.rst b/documentation/releases.rst
new file mode 100644
index 0000000000..49c33b3b5d
--- /dev/null
+++ b/documentation/releases.rst
@@ -0,0 +1,188 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=========================
4 Current Release Manuals
5=========================
6
7****************************
83.1 'dunfell' Release Series
9****************************
10
11- :yocto_docs:`3.1 Documentation </3.1>`
12- :yocto_docs:`3.1.1 Documentation </3.1.1>`
13- :yocto_docs:`3.1.2 Documentation </3.1.2>`
14
15==========================
16 Previous Release Manuals
17==========================
18
19*************************
203.0 'zeus' Release Series
21*************************
22
23- :yocto_docs:`3.0 Documentation </3.0>`
24- :yocto_docs:`3.0.1 Documentation </3.0.1>`
25- :yocto_docs:`3.0.2 Documentation </3.0.2>`
26- :yocto_docs:`3.0.3 Documentation </3.0.3>`
27
28****************************
292.7 'warrior' Release Series
30****************************
31
32- :yocto_docs:`2.7 Documentation </2.7>`
33- :yocto_docs:`2.7.1 Documentation </2.7.1>`
34- :yocto_docs:`2.7.2 Documentation </2.7.2>`
35- :yocto_docs:`2.7.3 Documentation </2.7.3>`
36- :yocto_docs:`2.7.4 Documentation </2.7.4>`
37
38*************************
392.6 'thud' Release Series
40*************************
41
42- :yocto_docs:`2.6 Documentation </2.6>`
43- :yocto_docs:`2.6.1 Documentation </2.6.1>`
44- :yocto_docs:`2.6.2 Documentation </2.6.2>`
45- :yocto_docs:`2.6.3 Documentation </2.6.3>`
46- :yocto_docs:`2.6.4 Documentation </2.6.4>`
47
48*************************
492.5 'sumo' Release Series
50*************************
51
52- :yocto_docs:`2.5 Documentation </2.5>`
53- :yocto_docs:`2.5.1 Documentation </2.5.1>`
54- :yocto_docs:`2.5.2 Documentation </2.5.2>`
55- :yocto_docs:`2.5.3 Documentation </2.5.3>`
56
57**************************
582.4 'rocko' Release Series
59**************************
60
61- :yocto_docs:`2.4 Documentation </2.4>`
62- :yocto_docs:`2.4.1 Documentation </2.4.1>`
63- :yocto_docs:`2.4.2 Documentation </2.4.2>`
64- :yocto_docs:`2.4.3 Documentation </2.4.3>`
65- :yocto_docs:`2.4.4 Documentation </2.4.4>`
66
67*************************
682.3 'pyro' Release Series
69*************************
70
71- :yocto_docs:`2.3 Documentation </2.3>`
72- :yocto_docs:`2.3.1 Documentation </2.3.1>`
73- :yocto_docs:`2.3.2 Documentation </2.3.2>`
74- :yocto_docs:`2.3.3 Documentation </2.3.3>`
75- :yocto_docs:`2.3.4 Documentation </2.3.4>`
76
77**************************
782.2 'morty' Release Series
79**************************
80
81- :yocto_docs:`2.2 Documentation </2.2>`
82- :yocto_docs:`2.2.1 Documentation </2.2.1>`
83- :yocto_docs:`2.2.2 Documentation </2.2.2>`
84- :yocto_docs:`2.2.3 Documentation </2.2.3>`
85
86****************************
872.1 'krogoth' Release Series
88****************************
89
90- :yocto_docs:`2.1 Documentation </2.1>`
91- :yocto_docs:`2.1.1 Documentation </2.1.1>`
92- :yocto_docs:`2.1.2 Documentation </2.1.2>`
93- :yocto_docs:`2.1.3 Documentation </2.1.3>`
94
95***************************
962.0 'jethro' Release Series
97***************************
98
99- :yocto_docs:`1.9 Documentation </1.9>`
100- :yocto_docs:`2.0 Documentation </2.0>`
101- :yocto_docs:`2.0.1 Documentation </2.0.1>`
102- :yocto_docs:`2.0.2 Documentation </2.0.2>`
103- :yocto_docs:`2.0.3 Documentation </2.0.3>`
104
105*************************
1061.8 'fido' Release Series
107*************************
108
109- :yocto_docs:`1.8 Documentation </1.8>`
110- :yocto_docs:`1.8.1 Documentation </1.8.1>`
111- :yocto_docs:`1.8.2 Documentation </1.8.2>`
112
113**************************
1141.7 'dizzy' Release Series
115**************************
116
117- :yocto_docs:`1.7 Documentation </1.7>`
118- :yocto_docs:`1.7.1 Documentation </1.7.1>`
119- :yocto_docs:`1.7.2 Documentation </1.7.2>`
120- :yocto_docs:`1.7.3 Documentation </1.7.3>`
121
122**************************
1231.6 'daisy' Release Series
124**************************
125
126- :yocto_docs:`1.6 Documentation </1.6>`
127- :yocto_docs:`1.6.1 Documentation </1.6.1>`
128- :yocto_docs:`1.6.2 Documentation </1.6.2>`
129- :yocto_docs:`1.6.3 Documentation </1.6.3>`
130
131*************************
1321.5 'dora' Release Series
133*************************
134
135- :yocto_docs:`1.5 Documentation </1.5>`
136- :yocto_docs:`1.5.1 Documentation </1.5.1>`
137- :yocto_docs:`1.5.2 Documentation </1.5.2>`
138- :yocto_docs:`1.5.3 Documentation </1.5.3>`
139- :yocto_docs:`1.5.4 Documentation </1.5.4>`
140
141**************************
1421.4 'dylan' Release Series
143**************************
144
145- :yocto_docs:`1.4 Documentation </1.4>`
146- :yocto_docs:`1.4.1 Documentation </1.4.1>`
147- :yocto_docs:`1.4.2 Documentation </1.4.2>`
148- :yocto_docs:`1.4.3 Documentation </1.4.3>`
149- :yocto_docs:`1.4.4 Documentation </1.4.4>`
150- :yocto_docs:`1.4.5 Documentation </1.4.5>`
151
152**************************
1531.3 'danny' Release Series
154**************************
155
156- :yocto_docs:`1.3 Documentation </1.3>`
157- :yocto_docs:`1.3.1 Documentation </1.3.1>`
158- :yocto_docs:`1.3.2 Documentation </1.3.2>`
159
160***************************
1611.2 'denzil' Release Series
162***************************
163
164- :yocto_docs:`1.2 Documentation </1.2>`
165- :yocto_docs:`1.2.1 Documentation </1.2.1>`
166- :yocto_docs:`1.2.2 Documentation </1.2.2>`
167
168***************************
1691.1 'edison' Release Series
170***************************
171
172- :yocto_docs:`1.1 Documentation </1.1>`
173- :yocto_docs:`1.1.1 Documentation </1.1.1>`
174- :yocto_docs:`1.1.2 Documentation </1.1.2>`
175
176****************************
1771.0 'bernard' Release Series
178****************************
179
180- :yocto_docs:`1.0 Documentation </1.0>`
181- :yocto_docs:`1.0.1 Documentation </1.0.1>`
182- :yocto_docs:`1.0.2 Documentation </1.0.2>`
183
184****************************
1850.9 'laverne' Release Series
186****************************
187
188- :yocto_docs:`0.9 Documentation </0.9>`
diff --git a/documentation/sdk-manual/history.rst b/documentation/sdk-manual/history.rst
new file mode 100644
index 0000000000..af027c97f8
--- /dev/null
+++ b/documentation/sdk-manual/history.rst
@@ -0,0 +1,40 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 2.1
15 - April 2016
16 - The initial document released with the Yocto Project 2.1 Release
17 * - 2.2
18 - October 2016
19 - Released with the Yocto Project 2.2 Release.
20 * - 2.3
21 - May 2017
22 - Released with the Yocto Project 2.3 Release.
23 * - 2.4
24 - October 2017
25 - Released with the Yocto Project 2.4 Release.
26 * - 2.5
27 - May 2018
28 - Released with the Yocto Project 2.5 Release.
29 * - 2.6
30 - November 2018
31 - Released with the Yocto Project 2.6 Release.
32 * - 2.7
33 - May 2019
34 - Released with the Yocto Project 2.7 Release.
35 * - 3.0
36 - October 2019
37 - Released with the Yocto Project 3.0 Release.
38 * - 3.1
39 - April 2020
40 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/sdk-manual/sdk-appendix-customizing-standard.rst b/documentation/sdk-manual/sdk-appendix-customizing-standard.rst
new file mode 100644
index 0000000000..f6f2b6640f
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-customizing-standard.rst
@@ -0,0 +1,34 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3****************************
4Customizing the Standard SDK
5****************************
6
7This appendix presents customizations you can apply to the standard SDK.
8
9Adding Individual Packages to the Standard SDK
10==============================================
11
12When you build a standard SDK using the ``bitbake -c populate_sdk``, a
13default set of packages is included in the resulting SDK. The
14:term:`TOOLCHAIN_HOST_TASK`
15and
16:term:`TOOLCHAIN_TARGET_TASK`
17variables control the set of packages adding to the SDK.
18
19If you want to add individual packages to the toolchain that runs on the
20host, simply add those packages to the ``TOOLCHAIN_HOST_TASK`` variable.
21Similarly, if you want to add packages to the default set that is part
22of the toolchain that runs on the target, add the packages to the
23``TOOLCHAIN_TARGET_TASK`` variable.
24
25Adding API Documentation to the Standard SDK
26============================================
27
28You can include API documentation as well as any other documentation
29provided by recipes with the standard SDK by adding "api-documentation"
30to the
31:term:`DISTRO_FEATURES`
32variable: DISTRO_FEATURES_append = " api-documentation" Setting this
33variable as shown here causes the OpenEmbedded build system to build the
34documentation and then include it in the standard SDK.
diff --git a/documentation/sdk-manual/sdk-appendix-customizing-standard.xml b/documentation/sdk-manual/sdk-appendix-customizing-standard.xml
index f20891c80d..3a6b4c8d82 100644
--- a/documentation/sdk-manual/sdk-appendix-customizing-standard.xml
+++ b/documentation/sdk-manual/sdk-appendix-customizing-standard.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='sdk-appendix-customizing-standard'> 6<appendix id='sdk-appendix-customizing-standard'>
6 7
diff --git a/documentation/sdk-manual/sdk-appendix-customizing.rst b/documentation/sdk-manual/sdk-appendix-customizing.rst
new file mode 100644
index 0000000000..7743e3c004
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-customizing.rst
@@ -0,0 +1,377 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3******************************
4Customizing the Extensible SDK
5******************************
6
7This appendix describes customizations you can apply to the extensible
8SDK.
9
10Configuring the Extensible SDK
11==============================
12
13The extensible SDK primarily consists of a pre-configured copy of the
14OpenEmbedded build system from which it was produced. Thus, the SDK's
15configuration is derived using that build system and the filters shown
16in the following list. When these filters are present, the OpenEmbedded
17build system applies them against ``local.conf`` and ``auto.conf``:
18
19- Variables whose values start with "/" are excluded since the
20 assumption is that those values are paths that are likely to be
21 specific to the :term:`Build Host`.
22
23- Variables listed in
24 :term:`SDK_LOCAL_CONF_BLACKLIST`
25 are excluded. These variables are not allowed through from the
26 OpenEmbedded build system configuration into the extensible SDK
27 configuration. Typically, these variables are specific to the machine
28 on which the build system is running and could be problematic as part
29 of the extensible SDK configuration.
30
31 For a list of the variables excluded by default, see the
32 :term:`SDK_LOCAL_CONF_BLACKLIST`
33 in the glossary of the Yocto Project Reference Manual.
34
35- Variables listed in
36 :term:`SDK_LOCAL_CONF_WHITELIST`
37 are included. Including a variable in the value of
38 ``SDK_LOCAL_CONF_WHITELIST`` overrides either of the previous two
39 filters. The default value is blank.
40
41- Classes inherited globally with
42 :term:`INHERIT` that are listed in
43 :term:`SDK_INHERIT_BLACKLIST`
44 are disabled. Using ``SDK_INHERIT_BLACKLIST`` to disable these
45 classes is the typical method to disable classes that are problematic
46 or unnecessary in the SDK context. The default value blacklists the
47 :ref:`buildhistory <ref-classes-buildhistory>`
48 and :ref:`icecc <ref-classes-icecc>` classes.
49
50Additionally, the contents of ``conf/sdk-extra.conf``, when present, are
51appended to the end of ``conf/local.conf`` within the produced SDK,
52without any filtering. The ``sdk-extra.conf`` file is particularly
53useful if you want to set a variable value just for the SDK and not the
54OpenEmbedded build system used to create the SDK.
55
56Adjusting the Extensible SDK to Suit Your Build Host's Setup
57============================================================
58
59In most cases, the extensible SDK defaults should work with your :term:`Build
60Host`'s setup.
61However, some cases exist for which you might consider making
62adjustments:
63
64- If your SDK configuration inherits additional classes using the
65 :term:`INHERIT` variable and you
66 do not need or want those classes enabled in the SDK, you can
67 blacklist them by adding them to the
68 :term:`SDK_INHERIT_BLACKLIST`
69 variable as described in the fourth bullet of the previous section.
70
71 .. note::
72
73 The default value of
74 SDK_INHERIT_BLACKLIST
75 is set using the "?=" operator. Consequently, you will need to
76 either define the entire list by using the "=" operator, or you
77 will need to append a value using either "_append" or the "+="
78 operator. You can learn more about these operators in the "
79 Basic Syntax
80 " section of the BitBake User Manual.
81
82 .
83
84- If you have classes or recipes that add additional tasks to the
85 standard build flow (i.e. the tasks execute as the recipe builds as
86 opposed to being called explicitly), then you need to do one of the
87 following:
88
89 - After ensuring the tasks are :ref:`shared
90 state <overview-manual/overview-manual-concepts:shared state cache>` tasks (i.e. the
91 output of the task is saved to and can be restored from the shared
92 state cache) or ensuring the tasks are able to be produced quickly
93 from a task that is a shared state task, add the task name to the
94 value of
95 :term:`SDK_RECRDEP_TASKS`.
96
97 - Disable the tasks if they are added by a class and you do not need
98 the functionality the class provides in the extensible SDK. To
99 disable the tasks, add the class to the ``SDK_INHERIT_BLACKLIST``
100 variable as described in the previous section.
101
102- Generally, you want to have a shared state mirror set up so users of
103 the SDK can add additional items to the SDK after installation
104 without needing to build the items from source. See the "`Providing
105 Additional Installable Extensible SDK
106 Content <#sdk-providing-additional-installable-extensible-sdk-content>`__"
107 section for information.
108
109- If you want users of the SDK to be able to easily update the SDK, you
110 need to set the
111 :term:`SDK_UPDATE_URL`
112 variable. For more information, see the "`Providing Updates to the
113 Extensible SDK After
114 Installation <#sdk-providing-updates-to-the-extensible-sdk-after-installation>`__"
115 section.
116
117- If you have adjusted the list of files and directories that appear in
118 :term:`COREBASE` (other than
119 layers that are enabled through ``bblayers.conf``), then you must
120 list these files in
121 :term:`COREBASE_FILES` so
122 that the files are copied into the SDK.
123
124- If your OpenEmbedded build system setup uses a different environment
125 setup script other than
126 :ref:`structure-core-script`, then you must
127 set
128 :term:`OE_INIT_ENV_SCRIPT`
129 to point to the environment setup script you use.
130
131 .. note::
132
133 You must also reflect this change in the value used for the
134 COREBASE_FILES
135 variable as previously described.
136
137Changing the Extensible SDK Installer Title
138===========================================
139
140You can change the displayed title for the SDK installer by setting the
141:term:`SDK_TITLE` variable and then
142rebuilding the the SDK installer. For information on how to build an SDK
143installer, see the "`Building an SDK
144Installer <#sdk-building-an-sdk-installer>`__" section.
145
146By default, this title is derived from
147:term:`DISTRO_NAME` when it is
148set. If the ``DISTRO_NAME`` variable is not set, the title is derived
149from the :term:`DISTRO` variable.
150
151The
152:ref:`populate_sdk_base <ref-classes-populate-sdk-*>`
153class defines the default value of the ``SDK_TITLE`` variable as
154follows:
155::
156
157 SDK_TITLE ??= "${@d.getVar('DISTRO_NAME') or d.getVar('DISTRO')} SDK"
158
159While several ways exist to change this variable, an efficient method is
160to set the variable in your distribution's configuration file. Doing so
161creates an SDK installer title that applies across your distribution. As
162an example, assume you have your own layer for your distribution named
163"meta-mydistro" and you are using the same type of file hierarchy as
164does the default "poky" distribution. If so, you could update the
165``SDK_TITLE`` variable in the
166``~/meta-mydistro/conf/distro/mydistro.conf`` file using the following
167form:
168::
169
170 SDK_TITLE = "your_title"
171
172Providing Updates to the Extensible SDK After Installation
173==========================================================
174
175When you make changes to your configuration or to the metadata and if
176you want those changes to be reflected in installed SDKs, you need to
177perform additional steps. These steps make it possible for anyone using
178the installed SDKs to update the installed SDKs by using the
179``devtool sdk-update`` command:
180
1811. Create a directory that can be shared over HTTP or HTTPS. You can do
182 this by setting up a web server such as an `Apache HTTP
183 Server <https://en.wikipedia.org/wiki/Apache_HTTP_Server>`__ or
184 `Nginx <https://en.wikipedia.org/wiki/Nginx>`__ server in the cloud
185 to host the directory. This directory must contain the published SDK.
186
1872. Set the
188 :term:`SDK_UPDATE_URL`
189 variable to point to the corresponding HTTP or HTTPS URL. Setting
190 this variable causes any SDK built to default to that URL and thus,
191 the user does not have to pass the URL to the ``devtool sdk-update``
192 command as described in the "`Applying Updates to an Installed
193 Extensible
194 SDK <#sdk-applying-updates-to-an-installed-extensible-sdk>`__"
195 section.
196
1973. Build the extensible SDK normally (i.e., use the
198 ``bitbake -c populate_sdk_ext`` imagename command).
199
2004. Publish the SDK using the following command:
201 ::
202
203 $ oe-publish-sdk some_path/sdk-installer.sh path_to_shared_http_directory
204
205 You must
206 repeat this step each time you rebuild the SDK with changes that you
207 want to make available through the update mechanism.
208
209Completing the above steps allows users of the existing installed SDKs
210to simply run ``devtool sdk-update`` to retrieve and apply the latest
211updates. See the "`Applying Updates to an Installed Extensible
212SDK <#sdk-applying-updates-to-an-installed-extensible-sdk>`__" section
213for further information.
214
215Changing the Default SDK Installation Directory
216===============================================
217
218When you build the installer for the Extensible SDK, the default
219installation directory for the SDK is based on the
220:term:`DISTRO` and
221:term:`SDKEXTPATH` variables from
222within the
223:ref:`populate_sdk_base <ref-classes-populate-sdk-*>`
224class as follows:
225::
226
227 SDKEXTPATH ??= "~/${@d.getVar('DISTRO')}_sdk"
228
229You can
230change this default installation directory by specifically setting the
231``SDKEXTPATH`` variable.
232
233While a number of ways exist through which you can set this variable,
234the method that makes the most sense is to set the variable in your
235distribution's configuration file. Doing so creates an SDK installer
236default directory that applies across your distribution. As an example,
237assume you have your own layer for your distribution named
238"meta-mydistro" and you are using the same type of file hierarchy as
239does the default "poky" distribution. If so, you could update the
240``SDKEXTPATH`` variable in the
241``~/meta-mydistro/conf/distro/mydistro.conf`` file using the following
242form:
243::
244
245 SDKEXTPATH = "some_path_for_your_installed_sdk"
246
247After building your installer, running it prompts the user for
248acceptance of the some_path_for_your_installed_sdk directory as the
249default location to install the Extensible SDK.
250
251Providing Additional Installable Extensible SDK Content
252=======================================================
253
254If you want the users of an extensible SDK you build to be able to add
255items to the SDK without requiring the users to build the items from
256source, you need to do a number of things:
257
2581. Ensure the additional items you want the user to be able to install
259 are already built:
260
261 - Build the items explicitly. You could use one or more "meta"
262 recipes that depend on lists of other recipes.
263
264 - Build the "world" target and set
265 ``EXCLUDE_FROM_WORLD_pn-``\ recipename for the recipes you do not
266 want built. See the
267 :term:`EXCLUDE_FROM_WORLD`
268 variable for additional information.
269
2702. Expose the ``sstate-cache`` directory produced by the build.
271 Typically, you expose this directory by making it available through
272 an `Apache HTTP
273 Server <https://en.wikipedia.org/wiki/Apache_HTTP_Server>`__ or
274 `Nginx <https://en.wikipedia.org/wiki/Nginx>`__ server.
275
2763. Set the appropriate configuration so that the produced SDK knows how
277 to find the configuration. The variable you need to set is
278 :term:`SSTATE_MIRRORS`:
279 ::
280
281 SSTATE_MIRRORS = "file://.* http://example.com/some_path/sstate-cache/PATH"
282
283 You can set the
284 ``SSTATE_MIRRORS`` variable in two different places:
285
286 - If the mirror value you are setting is appropriate to be set for
287 both the OpenEmbedded build system that is actually building the
288 SDK and the SDK itself (i.e. the mirror is accessible in both
289 places or it will fail quickly on the OpenEmbedded build system
290 side, and its contents will not interfere with the build), then
291 you can set the variable in your ``local.conf`` or custom distro
292 configuration file. You can then "whitelist" the variable through
293 to the SDK by adding the following:
294 ::
295
296 SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
297
298 - Alternatively, if you just want to set the ``SSTATE_MIRRORS``
299 variable's value for the SDK alone, create a
300 ``conf/sdk-extra.conf`` file either in your
301 :term:`Build Directory` or within any
302 layer and put your ``SSTATE_MIRRORS`` setting within that file.
303
304 .. note::
305
306 This second option is the safest option should you have any
307 doubts as to which method to use when setting
308 SSTATE_MIRRORS
309 .
310
311Minimizing the Size of the Extensible SDK Installer Download
312============================================================
313
314By default, the extensible SDK bundles the shared state artifacts for
315everything needed to reconstruct the image for which the SDK was built.
316This bundling can lead to an SDK installer file that is a Gigabyte or
317more in size. If the size of this file causes a problem, you can build
318an SDK that has just enough in it to install and provide access to the
319``devtool command`` by setting the following in your configuration:
320::
321
322 SDK_EXT_TYPE = "minimal"
323
324Setting
325:term:`SDK_EXT_TYPE` to
326"minimal" produces an SDK installer that is around 35 Mbytes in size,
327which downloads and installs quickly. You need to realize, though, that
328the minimal installer does not install any libraries or tools out of the
329box. These libraries and tools must be installed either "on the fly" or
330through actions you perform using ``devtool`` or explicitly with the
331``devtool sdk-install`` command.
332
333In most cases, when building a minimal SDK you need to also enable
334bringing in the information on a wider range of packages produced by the
335system. Requiring this wider range of information is particularly true
336so that ``devtool add`` is able to effectively map dependencies it
337discovers in a source tree to the appropriate recipes. Additionally, the
338information enables the ``devtool search`` command to return useful
339results.
340
341To facilitate this wider range of information, you would need to set the
342following:
343::
344
345 SDK_INCLUDE_PKGDATA = "1"
346
347See the :term:`SDK_INCLUDE_PKGDATA` variable for additional information.
348
349Setting the ``SDK_INCLUDE_PKGDATA`` variable as shown causes the "world"
350target to be built so that information for all of the recipes included
351within it are available. Having these recipes available increases build
352time significantly and increases the size of the SDK installer by 30-80
353Mbytes depending on how many recipes are included in your configuration.
354
355You can use ``EXCLUDE_FROM_WORLD_pn-``\ recipename for recipes you want
356to exclude. However, it is assumed that you would need to be building
357the "world" target if you want to provide additional items to the SDK.
358Consequently, building for "world" should not represent undue overhead
359in most cases.
360
361.. note::
362
363 If you set
364 SDK_EXT_TYPE
365 to "minimal", then providing a shared state mirror is mandatory so
366 that items can be installed as needed. See the "
367 Providing Additional Installable Extensible SDK Content
368 " section for more information.
369
370You can explicitly control whether or not to include the toolchain when
371you build an SDK by setting the
372:term:`SDK_INCLUDE_TOOLCHAIN`
373variable to "1". In particular, it is useful to include the toolchain
374when you have set ``SDK_EXT_TYPE`` to "minimal", which by default,
375excludes the toolchain. Also, it is helpful if you are building a small
376SDK for use with an IDE or some other tool where you do not want to take
377extra steps to install a toolchain.
diff --git a/documentation/sdk-manual/sdk-appendix-customizing.xml b/documentation/sdk-manual/sdk-appendix-customizing.xml
index 911658f914..08054f8b79 100644
--- a/documentation/sdk-manual/sdk-appendix-customizing.xml
+++ b/documentation/sdk-manual/sdk-appendix-customizing.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='sdk-appendix-customizing'> 6<appendix id='sdk-appendix-customizing'>
6 7
diff --git a/documentation/sdk-manual/sdk-appendix-obtain.rst b/documentation/sdk-manual/sdk-appendix-obtain.rst
new file mode 100644
index 0000000000..97ab9169ea
--- /dev/null
+++ b/documentation/sdk-manual/sdk-appendix-obtain.rst
@@ -0,0 +1,321 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************
4Obtaining the SDK
5*****************
6
7.. _sdk-locating-pre-built-sdk-installers:
8
9Locating Pre-Built SDK Installers
10=================================
11
12You can use existing, pre-built toolchains by locating and running an
13SDK installer script that ships with the Yocto Project. Using this
14method, you select and download an architecture-specific SDK installer
15and then run the script to hand-install the toolchain.
16
17Follow these steps to locate and hand-install the toolchain:
18
191. *Go to the Installers Directory:* Go to
20 :yocto_dl:`/releases/yocto/yocto-3.1.2/toolchain/`
21
222. *Open the Folder for Your Build Host:* Open the folder that matches
23 your :term:`Build Host` (i.e.
24 ``i686`` for 32-bit machines or ``x86_64`` for 64-bit machines).
25
263. *Locate and Download the SDK Installer:* You need to find and
27 download the installer appropriate for your build host, target
28 hardware, and image type.
29
30 The installer files (``*.sh``) follow this naming convention:
31 ::
32
33 poky-glibc-host_system-core-image-type-arch-toolchain[-ext]-release.sh
34
35 Where:
36 host_system is a string representing your development system:
37 "i686" or "x86_64"
38
39 type is a string representing the image:
40 "sato" or "minimal"
41
42 arch is a string representing the target architecture:
43 "aarch64", "armv5e", "core2-64", "coretexa8hf-neon", "i586", "mips32r2",
44 "mips64", or "ppc7400"
45
46 release is the version of Yocto Project.
47
48 NOTE:
49 The standard SDK installer does not have the "-ext" string as
50 part of the filename.
51
52
53 The toolchains provided by the Yocto
54 Project are based off of the ``core-image-sato`` and
55 ``core-image-minimal`` images and contain libraries appropriate for
56 developing against those images.
57
58 For example, if your build host is a 64-bit x86 system and you need
59 an extended SDK for a 64-bit core2 target, go into the ``x86_64``
60 folder and download the following installer:
61 ::
62
63 poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
64
654. *Run the Installer:* Be sure you have execution privileges and run
66 the installer. Following is an example from the ``Downloads``
67 directory:
68 ::
69
70 $ ~/Downloads/poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
71
72 During execution of the script, you choose the root location for the
73 toolchain. See the "`Installed Standard SDK Directory
74 Structure <#sdk-installed-standard-sdk-directory-structure>`__"
75 section and the "`Installed Extensible SDK Directory
76 Structure <#sdk-installed-extensible-sdk-directory-structure>`__"
77 section for more information.
78
79Building an SDK Installer
80=========================
81
82As an alternative to locating and downloading an SDK installer, you can
83build the SDK installer. Follow these steps:
84
851. *Set Up the Build Environment:* Be sure you are set up to use BitBake
86 in a shell. See the ":ref:`dev-manual/dev-manual-start:preparing the build host`" section
87 in the Yocto Project Development Tasks Manual for information on how
88 to get a build host ready that is either a native Linux machine or a
89 machine that uses CROPS.
90
912. *Clone the ``poky`` Repository:* You need to have a local copy of the
92 Yocto Project :term:`Source Directory`
93 (i.e. a local
94 ``poky`` repository). See the ":ref:`dev-manual/dev-manual-start:cloning the \`\`poky\`\` repository`" and
95 possibly the ":ref:`dev-manual/dev-manual-start:checking out by branch in poky`" and
96 ":ref:`checkout-out-by-tag-in-poky`" sections
97 all in the Yocto Project Development Tasks Manual for information on
98 how to clone the ``poky`` repository and check out the appropriate
99 branch for your work.
100
1013. *Initialize the Build Environment:* While in the root directory of
102 the Source Directory (i.e. ``poky``), run the
103 :ref:`structure-core-script` environment
104 setup script to define the OpenEmbedded build environment on your
105 build host.
106 ::
107
108 $ source oe-init-build-env
109
110 Among other things, the script
111 creates the :term:`Build Directory`,
112 which is
113 ``build`` in this case and is located in the Source Directory. After
114 the script runs, your current working directory is set to the
115 ``build`` directory.
116
1174. *Make Sure You Are Building an Installer for the Correct Machine:*
118 Check to be sure that your
119 :term:`MACHINE` variable in the
120 ``local.conf`` file in your Build Directory matches the architecture
121 for which you are building.
122
1235. *Make Sure Your SDK Machine is Correctly Set:* If you are building a
124 toolchain designed to run on an architecture that differs from your
125 current development host machine (i.e. the build host), be sure that
126 the :term:`SDKMACHINE` variable
127 in the ``local.conf`` file in your Build Directory is correctly set.
128
129 .. note::
130
131 If you are building an SDK installer for the Extensible SDK, the
132 SDKMACHINE
133 value must be set for the architecture of the machine you are
134 using to build the installer. If
135 SDKMACHINE
136 is not set appropriately, the build fails and provides an error
137 message similar to the following:
138 ::
139
140 The extensible SDK can currently only be built for the same architecture as the machine being built on - SDK_ARCH is
141 set to i686 (likely via setting SDKMACHINE) which is different from the architecture of the build machine (x86_64).
142 Unable to continue.
143
144
1456. *Build the SDK Installer:* To build the SDK installer for a standard
146 SDK and populate the SDK image, use the following command form. Be
147 sure to replace image with an image (e.g. "core-image-sato"): $
148 bitbake image -c populate_sdk You can do the same for the extensible
149 SDK using this command form:
150 ::
151
152 $ bitbake image -c populate_sdk_ext
153
154 These commands produce an SDK installer that contains the sysroot
155 that matches your target root filesystem.
156
157 When the ``bitbake`` command completes, the SDK installer will be in
158 ``tmp/deploy/sdk`` in the Build Directory.
159
160 .. note::
161
162 - By default, the previous BitBake command does not build static
163 binaries. If you want to use the toolchain to build these types
164 of libraries, you need to be sure your SDK has the appropriate
165 static development libraries. Use the
166 :term:`TOOLCHAIN_TARGET_TASK`
167 variable inside your ``local.conf`` file before building the
168 SDK installer. Doing so ensures that the eventual SDK
169 installation process installs the appropriate library packages
170 as part of the SDK. Following is an example using ``libc``
171 static development libraries: TOOLCHAIN_TARGET_TASK_append = "
172 libc-staticdev"
173
1747. *Run the Installer:* You can now run the SDK installer from
175 ``tmp/deploy/sdk`` in the Build Directory. Following is an example:
176 ::
177
178 $ cd ~/poky/build/tmp/deploy/sdk
179 $ ./poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-DISTRO.sh
180
181 During execution of the script, you choose the root location for the
182 toolchain. See the "`Installed Standard SDK Directory
183 Structure <#sdk-installed-standard-sdk-directory-structure>`__"
184 section and the "`Installed Extensible SDK Directory
185 Structure <#sdk-installed-extensible-sdk-directory-structure>`__"
186 section for more information.
187
188Extracting the Root Filesystem
189==============================
190
191After installing the toolchain, for some use cases you might need to
192separately extract a root filesystem:
193
194- You want to boot the image using NFS.
195
196- You want to use the root filesystem as the target sysroot.
197
198- You want to develop your target application using the root filesystem
199 as the target sysroot.
200
201Follow these steps to extract the root filesystem:
202
2031. *Locate and Download the Tarball for the Pre-Built Root Filesystem
204 Image File:* You need to find and download the root filesystem image
205 file that is appropriate for your target system. These files are kept
206 in machine-specific folders in the
207 :yocto_dl:`Index of Releases </releases/yocto/yocto-3.1.2/machines/>`
208 in the "machines" directory.
209
210 The machine-specific folders of the "machines" directory contain
211 tarballs (``*.tar.bz2``) for supported machines. These directories
212 also contain flattened root filesystem image files (``*.ext4``),
213 which you can use with QEMU directly.
214
215 The pre-built root filesystem image files follow these naming
216 conventions:
217 ::
218
219 core-image-profile-arch.tar.bz2
220
221 Where:
222 profile is the filesystem image's profile:
223 lsb, lsb-dev, lsb-sdk, minimal, minimal-dev, minimal-initramfs,
224 sato, sato-dev, sato-sdk, sato-sdk-ptest. For information on
225 these types of image profiles, see the "Images" chapter in
226 the Yocto Project Reference Manual.
227
228 arch is a string representing the target architecture:
229 beaglebone-yocto, beaglebone-yocto-lsb, edgerouter, edgerouter-lsb,
230 genericx86, genericx86-64, genericx86-64-lsb, genericx86-lsb and qemu*.
231
232 The root filesystems
233 provided by the Yocto Project are based off of the
234 ``core-image-sato`` and ``core-image-minimal`` images.
235
236 For example, if you plan on using a BeagleBone device as your target
237 hardware and your image is a ``core-image-sato-sdk`` image, you can
238 download the following file:
239 ::
240
241 core-image-sato-sdk-beaglebone-yocto.tar.bz2
242
2432. *Initialize the Cross-Development Environment:* You must ``source``
244 the cross-development environment setup script to establish necessary
245 environment variables.
246
247 This script is located in the top-level directory in which you
248 installed the toolchain (e.g. ``poky_sdk``).
249
250 Following is an example based on the toolchain installed in the
251 ":ref:`sdk-locating-pre-built-sdk-installers`" section:
252 ::
253
254 $ source ~/poky_sdk/environment-setup-core2-64-poky-linux
255
2563. *Extract the Root Filesystem:* Use the ``runqemu-extract-sdk``
257 command and provide the root filesystem image.
258
259 Following is an example command that extracts the root filesystem
260 from a previously built root filesystem image that was downloaded
261 from the :yocto_dl:`Index of Releases </releases/yocto/yocto-3.1.2/machines/>`.
262 This command extracts the root filesystem into the ``core2-64-sato``
263 directory:
264 ::
265
266 $ runqemu-extract-sdk ~/Downloads/core-image-sato-sdk-beaglebone-yocto.tar.bz2 ~/beaglebone-sato
267
268 You could now point to the target sysroot at ``beablebone-sato``.
269
270Installed Standard SDK Directory Structure
271==========================================
272
273The following figure shows the resulting directory structure after you
274install the Standard SDK by running the ``*.sh`` SDK installation
275script:
276
277.. image:: figures/sdk-installed-standard-sdk-directory.png
278 :scale: 80%
279 :align: center
280
281The installed SDK consists of an environment setup script for the SDK, a
282configuration file for the target, a version file for the target, and
283the root filesystem (``sysroots``) needed to develop objects for the
284target system.
285
286Within the figure, italicized text is used to indicate replaceable
287portions of the file or directory name. For example, install_dir/version
288is the directory where the SDK is installed. By default, this directory
289is ``/opt/poky/``. And, version represents the specific snapshot of the
290SDK (e.g. 3.1.2). Furthermore, target represents the target architecture
291(e.g. ``i586``) and host represents the development system's
292architecture (e.g. ``x86_64``). Thus, the complete names of the two
293directories within the ``sysroots`` could be ``i586-poky-linux`` and
294``x86_64-pokysdk-linux`` for the target and host, respectively.
295
296Installed Extensible SDK Directory Structure
297============================================
298
299The following figure shows the resulting directory structure after you
300install the Extensible SDK by running the ``*.sh`` SDK installation
301script:
302
303.. image:: figures/sdk-installed-extensible-sdk-directory.png
304 :scale: 80%
305 :align: center
306
307The installed directory structure for the extensible SDK is quite
308different than the installed structure for the standard SDK. The
309extensible SDK does not separate host and target parts in the same
310manner as does the standard SDK. The extensible SDK uses an embedded
311copy of the OpenEmbedded build system, which has its own sysroots.
312
313Of note in the directory structure are an environment setup script for
314the SDK, a configuration file for the target, a version file for the
315target, and log files for the OpenEmbedded build system preparation
316script run by the installer and BitBake.
317
318Within the figure, italicized text is used to indicate replaceable
319portions of the file or directory name. For example, install_dir is the
320directory where the SDK is installed, which is ``poky_sdk`` by default,
321and target represents the target architecture (e.g. ``i586``).
diff --git a/documentation/sdk-manual/sdk-appendix-obtain.xml b/documentation/sdk-manual/sdk-appendix-obtain.xml
index 86b6d7dd07..de7f75e2bb 100644
--- a/documentation/sdk-manual/sdk-appendix-obtain.xml
+++ b/documentation/sdk-manual/sdk-appendix-obtain.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<appendix id='sdk-appendix-obtain'> 6<appendix id='sdk-appendix-obtain'>
6 7
diff --git a/documentation/sdk-manual/sdk-extensible.rst b/documentation/sdk-manual/sdk-extensible.rst
new file mode 100644
index 0000000000..0f92ac9f0c
--- /dev/null
+++ b/documentation/sdk-manual/sdk-extensible.rst
@@ -0,0 +1,1356 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************************
4Using the Extensible SDK
5************************
6
7This chapter describes the extensible SDK and how to install it.
8Information covers the pieces of the SDK, how to install it, and
9presents a look at using the ``devtool`` functionality. The extensible
10SDK makes it easy to add new applications and libraries to an image,
11modify the source for an existing component, test changes on the target
12hardware, and ease integration into the rest of the
13:term:`OpenEmbedded Build System`.
14
15.. note::
16
17 For a side-by-side comparison of main features supported for an
18 extensible SDK as compared to a standard SDK, see the "
19 Introduction
20 " section.
21
22In addition to the functionality available through ``devtool``, you can
23alternatively make use of the toolchain directly, for example from
24Makefile and Autotools. See the "`Using the SDK Toolchain
25Directly <#sdk-working-projects>`__" chapter for more information.
26
27.. _sdk-extensible-sdk-intro:
28
29Why use the Extensible SDK and What is in It?
30=============================================
31
32The extensible SDK provides a cross-development toolchain and libraries
33tailored to the contents of a specific image. You would use the
34Extensible SDK if you want a toolchain experience supplemented with the
35powerful set of ``devtool`` commands tailored for the Yocto Project
36environment.
37
38The installed extensible SDK consists of several files and directories.
39Basically, it contains an SDK environment setup script, some
40configuration files, an internal build system, and the ``devtool``
41functionality.
42
43.. _sdk-installing-the-extensible-sdk:
44
45Installing the Extensible SDK
46=============================
47
48The first thing you need to do is install the SDK on your :term:`Build
49Host` by running the ``*.sh`` installation script.
50
51You can download a tarball installer, which includes the pre-built
52toolchain, the ``runqemu`` script, the internal build system,
53``devtool``, and support files from the appropriate
54:yocto_dl:`toolchain </releases/yocto/yocto-3.1.2/toolchain/>` directory within the Index of
55Releases. Toolchains are available for several 32-bit and 64-bit
56architectures with the ``x86_64`` directories, respectively. The
57toolchains the Yocto Project provides are based off the
58``core-image-sato`` and ``core-image-minimal`` images and contain
59libraries appropriate for developing against that image.
60
61The names of the tarball installer scripts are such that a string
62representing the host system appears first in the filename and then is
63immediately followed by a string representing the target architecture.
64An extensible SDK has the string "-ext" as part of the name. Following
65is the general form:
66::
67
68 poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh
69
70 Where:
71 host_system is a string representing your development system:
72
73 i686 or x86_64.
74
75 image_type is the image for which the SDK was built:
76
77 core-image-sato or core-image-minimal
78
79 arch is a string representing the tuned target architecture:
80
81 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon
82
83 release_version is a string representing the release number of the Yocto Project:
84
85 3.1.2, 3.1.2+snapshot
86
87For example, the following SDK installer is for a 64-bit
88development host system and a i586-tuned target architecture based off
89the SDK for ``core-image-sato`` and using the current DISTRO snapshot:
90::
91
92 poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-DISTRO.sh
93
94.. note::
95
96 As an alternative to downloading an SDK, you can build the SDK
97 installer. For information on building the installer, see the "
98 Building an SDK Installer
99 " section.
100
101The SDK and toolchains are self-contained and by default are installed
102into the ``poky_sdk`` folder in your home directory. You can choose to
103install the extensible SDK in any location when you run the installer.
104However, because files need to be written under that directory during
105the normal course of operation, the location you choose for installation
106must be writable for whichever users need to use the SDK.
107
108The following command shows how to run the installer given a toolchain
109tarball for a 64-bit x86 development host system and a 64-bit x86 target
110architecture. The example assumes the SDK installer is located in
111``~/Downloads/`` and has execution rights.
112
113.. note::
114
115 If you do not have write permissions for the directory into which you
116 are installing the SDK, the installer notifies you and exits. For
117 that case, set up the proper permissions in the directory and run the
118 installer again.
119
120::
121
122 $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
123 Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5
124 ==========================================================================
125 Enter target directory for SDK (default: ~/poky_sdk):
126 You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
127 Extracting SDK..............done
128 Setting it up...
129 Extracting buildtools...
130 Preparing build system...
131 Parsing recipes: 100% |##################################################################| Time: 0:00:52
132 Initialising tasks: 100% |###############################################################| Time: 0:00:00
133 Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00
134 Loading cache: 100% |####################################################################| Time: 0:00:00
135 Initialising tasks: 100% |###############################################################| Time: 0:00:00
136 done
137 SDK has been successfully set up and is ready to be used.
138 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
139 $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux
140
141.. _sdk-running-the-extensible-sdk-environment-setup-script:
142
143Running the Extensible SDK Environment Setup Script
144===================================================
145
146Once you have the SDK installed, you must run the SDK environment setup
147script before you can actually use the SDK. This setup script resides in
148the directory you chose when you installed the SDK, which is either the
149default ``poky_sdk`` directory or the directory you chose during
150installation.
151
152Before running the script, be sure it is the one that matches the
153architecture for which you are developing. Environment setup scripts
154begin with the string "``environment-setup``" and include as part of
155their name the tuned target architecture. As an example, the following
156commands set the working directory to where the SDK was installed and
157then source the environment setup script. In this example, the setup
158script is for an IA-based target machine using i586 tuning:
159::
160
161 $ cd /home/scottrif/poky_sdk
162 $ source environment-setup-core2-64-poky-linux
163 SDK environment now set up; additionally you may now run devtool to perform development tasks.
164 Run devtool --help for further details.
165
166Running the setup script defines many environment variables needed in
167order to use the SDK (e.g. ``PATH``,
168:term:`CC`,
169:term:`LD`, and so forth). If you want to
170see all the environment variables the script exports, examine the
171installation file itself.
172
173Using ``devtool`` in Your SDK Workflow
174======================================
175
176The cornerstone of the extensible SDK is a command-line tool called
177``devtool``. This tool provides a number of features that help you
178build, test and package software within the extensible SDK, and
179optionally integrate it into an image built by the OpenEmbedded build
180system.
181
182.. note::
183
184 The use of
185 devtool
186 is not limited to the extensible SDK. You can use
187 devtool
188 to help you easily develop any project whose build output must be
189 part of an image built using the build system.
190
191The ``devtool`` command line is organized similarly to
192:ref:`overview-manual/overview-manual-development-environment:git` in that it has a number of
193sub-commands for each function. You can run ``devtool --help`` to see
194all the commands.
195
196.. note::
197
198 See the "
199 devtool
200  Quick Reference
201 " in the Yocto Project Reference Manual for a
202 devtool
203 quick reference.
204
205Three ``devtool`` subcommands exist that provide entry-points into
206development:
207
208- *devtool add*: Assists in adding new software to be built.
209
210- *devtool modify*: Sets up an environment to enable you to modify
211 the source of an existing component.
212
213- *devtool upgrade*: Updates an existing recipe so that you can
214 build it for an updated set of source files.
215
216As with the build system, "recipes" represent software packages within
217``devtool``. When you use ``devtool add``, a recipe is automatically
218created. When you use ``devtool modify``, the specified existing recipe
219is used in order to determine where to get the source code and how to
220patch it. In both cases, an environment is set up so that when you build
221the recipe a source tree that is under your control is used in order to
222allow you to make changes to the source as desired. By default, new
223recipes and the source go into a "workspace" directory under the SDK.
224
225The remainder of this section presents the ``devtool add``,
226``devtool modify``, and ``devtool upgrade`` workflows.
227
228.. _sdk-use-devtool-to-add-an-application:
229
230Use ``devtool add`` to Add an Application
231-----------------------------------------
232
233The ``devtool add`` command generates a new recipe based on existing
234source code. This command takes advantage of the
235:ref:`devtool-the-workspace-layer-structure`
236layer that many ``devtool`` commands use. The command is flexible enough
237to allow you to extract source code into both the workspace or a
238separate local Git repository and to use existing code that does not
239need to be extracted.
240
241Depending on your particular scenario, the arguments and options you use
242with ``devtool add`` form different combinations. The following diagram
243shows common development flows you would use with the ``devtool add``
244command:
245
246.. image:: figures/sdk-devtool-add-flow.png
247 :align: center
248
2491. *Generating the New Recipe*: The top part of the flow shows three
250 scenarios by which you could use ``devtool add`` to generate a recipe
251 based on existing source code.
252
253 In a shared development environment, it is typical for other
254 developers to be responsible for various areas of source code. As a
255 developer, you are probably interested in using that source code as
256 part of your development within the Yocto Project. All you need is
257 access to the code, a recipe, and a controlled area in which to do
258 your work.
259
260 Within the diagram, three possible scenarios feed into the
261 ``devtool add`` workflow:
262
263 - *Left*: The left scenario in the figure represents a common
264 situation where the source code does not exist locally and needs
265 to be extracted. In this situation, the source code is extracted
266 to the default workspace - you do not want the files in some
267 specific location outside of the workspace. Thus, everything you
268 need will be located in the workspace:
269 ::
270
271 $ devtool add recipe fetchuri
272
273 With this command, ``devtool`` extracts the upstream
274 source files into a local Git repository within the ``sources``
275 folder. The command then creates a recipe named recipe and a
276 corresponding append file in the workspace. If you do not provide
277 recipe, the command makes an attempt to determine the recipe name.
278
279 - *Middle*: The middle scenario in the figure also represents a
280 situation where the source code does not exist locally. In this
281 case, the code is again upstream and needs to be extracted to some
282 local area - this time outside of the default workspace.
283
284 .. note::
285
286 If required,
287 devtool
288 always creates a Git repository locally during the extraction.
289
290 Furthermore, the first positional argument srctree in this case
291 identifies where the ``devtool add`` command will locate the
292 extracted code outside of the workspace. You need to specify an
293 empty directory:
294 ::
295
296 $ devtool add recipe srctree fetchuri
297
298 In summary,
299 the source code is pulled from fetchuri and extracted into the
300 location defined by srctree as a local Git repository.
301
302 Within workspace, ``devtool`` creates a recipe named recipe along
303 with an associated append file.
304
305 - *Right*: The right scenario in the figure represents a situation
306 where the srctree has been previously prepared outside of the
307 ``devtool`` workspace.
308
309 The following command provides a new recipe name and identifies
310 the existing source tree location:
311 ::
312
313 $ devtool add recipe srctree
314
315 The command examines the source code and creates a recipe named
316 recipe for the code and places the recipe into the workspace.
317
318 Because the extracted source code already exists, ``devtool`` does
319 not try to relocate the source code into the workspace - only the
320 new recipe is placed in the workspace.
321
322 Aside from a recipe folder, the command also creates an associated
323 append folder and places an initial ``*.bbappend`` file within.
324
3252. *Edit the Recipe*: You can use ``devtool edit-recipe`` to open up the
326 editor as defined by the ``$EDITOR`` environment variable and modify
327 the file:
328 ::
329
330 $ devtool edit-recipe recipe
331
332 From within the editor, you
333 can make modifications to the recipe that take affect when you build
334 it later.
335
3363. *Build the Recipe or Rebuild the Image*: The next step you take
337 depends on what you are going to do with the new code.
338
339 If you need to eventually move the build output to the target
340 hardware, use the following ``devtool`` command:
341 :;
342
343 $ devtool build recipe
344
345 On the other hand, if you want an image to contain the recipe's
346 packages from the workspace for immediate deployment onto a device
347 (e.g. for testing purposes), you can use the ``devtool build-image``
348 command:
349 ::
350
351 $ devtool build-image image
352
3534. *Deploy the Build Output*: When you use the ``devtool build`` command
354 to build out your recipe, you probably want to see if the resulting
355 build output works as expected on the target hardware.
356
357 .. note::
358
359 This step assumes you have a previously built image that is
360 already either running in QEMU or is running on actual hardware.
361 Also, it is assumed that for deployment of the image to the
362 target, SSH is installed in the image and, if the image is running
363 on real hardware, you have network access to and from your
364 development machine.
365
366 You can deploy your build output to that target hardware by using the
367 ``devtool deploy-target`` command: $ devtool deploy-target recipe
368 target The target is a live target machine running as an SSH server.
369
370 You can, of course, also deploy the image you build to actual
371 hardware by using the ``devtool build-image`` command. However,
372 ``devtool`` does not provide a specific command that allows you to
373 deploy the image to actual hardware.
374
3755. *Finish Your Work With the Recipe*: The ``devtool finish`` command
376 creates any patches corresponding to commits in the local Git
377 repository, moves the new recipe to a more permanent layer, and then
378 resets the recipe so that the recipe is built normally rather than
379 from the workspace.
380 ::
381
382 $ devtool finish recipe layer
383
384 .. note::
385
386 Any changes you want to turn into patches must be committed to the
387 Git repository in the source tree.
388
389 As mentioned, the ``devtool finish`` command moves the final recipe
390 to its permanent layer.
391
392 As a final process of the ``devtool finish`` command, the state of
393 the standard layers and the upstream source is restored so that you
394 can build the recipe from those areas rather than the workspace.
395
396 .. note::
397
398 You can use the
399 devtool reset
400 command to put things back should you decide you do not want to
401 proceed with your work. If you do use this command, realize that
402 the source tree is preserved.
403
404.. _sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component:
405
406Use ``devtool modify`` to Modify the Source of an Existing Component
407--------------------------------------------------------------------
408
409The ``devtool modify`` command prepares the way to work on existing code
410that already has a local recipe in place that is used to build the
411software. The command is flexible enough to allow you to extract code
412from an upstream source, specify the existing recipe, and keep track of
413and gather any patch files from other developers that are associated
414with the code.
415
416Depending on your particular scenario, the arguments and options you use
417with ``devtool modify`` form different combinations. The following
418diagram shows common development flows for the ``devtool modify``
419command:
420
421.. image:: figures/sdk-devtool-modify-flow.png
422 :align: center
423
4241. *Preparing to Modify the Code*: The top part of the flow shows three
425 scenarios by which you could use ``devtool modify`` to prepare to
426 work on source files. Each scenario assumes the following:
427
428 - The recipe exists locally in a layer external to the ``devtool``
429 workspace.
430
431 - The source files exist either upstream in an un-extracted state or
432 locally in a previously extracted state.
433
434 The typical situation is where another developer has created a layer
435 for use with the Yocto Project and their recipe already resides in
436 that layer. Furthermore, their source code is readily available
437 either upstream or locally.
438
439 - *Left*: The left scenario in the figure represents a common
440 situation where the source code does not exist locally and it
441 needs to be extracted from an upstream source. In this situation,
442 the source is extracted into the default ``devtool`` workspace
443 location. The recipe, in this scenario, is in its own layer
444 outside the workspace (i.e. ``meta-``\ layername).
445
446 The following command identifies the recipe and, by default,
447 extracts the source files:
448 ::
449
450 $ devtool modify recipe
451
452 Once
453 ``devtool``\ locates the recipe, ``devtool`` uses the recipe's
454 :term:`SRC_URI` statements to
455 locate the source code and any local patch files from other
456 developers.
457
458 With this scenario, no srctree argument exists. Consequently, the
459 default behavior of the ``devtool modify`` command is to extract
460 the source files pointed to by the ``SRC_URI`` statements into a
461 local Git structure. Furthermore, the location for the extracted
462 source is the default area within the ``devtool`` workspace. The
463 result is that the command sets up both the source code and an
464 append file within the workspace while the recipe remains in its
465 original location.
466
467 Additionally, if you have any non-patch local files (i.e. files
468 referred to with ``file://`` entries in ``SRC_URI`` statement
469 excluding ``*.patch/`` or ``*.diff``), these files are copied to
470 an ``oe-local-files`` folder under the newly created source tree.
471 Copying the files here gives you a convenient area from which you
472 can modify the files. Any changes or additions you make to those
473 files are incorporated into the build the next time you build the
474 software just as are other changes you might have made to the
475 source.
476
477 - *Middle*: The middle scenario in the figure represents a situation
478 where the source code also does not exist locally. In this case,
479 the code is again upstream and needs to be extracted to some local
480 area as a Git repository. The recipe, in this scenario, is again
481 local and in its own layer outside the workspace.
482
483 The following command tells ``devtool`` the recipe with which to
484 work and, in this case, identifies a local area for the extracted
485 source files that exists outside of the default ``devtool``
486 workspace:
487 ::
488
489 $ devtool modify recipe srctree
490
491 .. note::
492
493 You cannot provide a URL for
494 srctree
495 using the
496 devtool
497 command.
498
499 As with all extractions, the command uses the recipe's ``SRC_URI``
500 statements to locate the source files and any associated patch
501 files. Non-patch files are copied to an ``oe-local-files`` folder
502 under the newly created source tree.
503
504 Once the files are located, the command by default extracts them
505 into srctree.
506
507 Within workspace, ``devtool`` creates an append file for the
508 recipe. The recipe remains in its original location but the source
509 files are extracted to the location you provide with srctree.
510
511 - *Right*: The right scenario in the figure represents a situation
512 where the source tree (srctree) already exists locally as a
513 previously extracted Git structure outside of the ``devtool``
514 workspace. In this example, the recipe also exists elsewhere
515 locally in its own layer.
516
517 The following command tells ``devtool`` the recipe with which to
518 work, uses the "-n" option to indicate source does not need to be
519 extracted, and uses srctree to point to the previously extracted
520 source files:
521 ::
522
523 $ devtool modify -n recipe srctree
524
525 If an ``oe-local-files`` subdirectory happens to exist and it
526 contains non-patch files, the files are used. However, if the
527 subdirectory does not exist and you run the ``devtool finish``
528 command, any non-patch files that might exist next to the recipe
529 are removed because it appears to ``devtool`` that you have
530 deleted those files.
531
532 Once the ``devtool modify`` command finishes, it creates only an
533 append file for the recipe in the ``devtool`` workspace. The
534 recipe and the source code remain in their original locations.
535
5362. *Edit the Source*: Once you have used the ``devtool modify`` command,
537 you are free to make changes to the source files. You can use any
538 editor you like to make and save your source code modifications.
539
5403. *Build the Recipe or Rebuild the Image*: The next step you take
541 depends on what you are going to do with the new code.
542
543 If you need to eventually move the build output to the target
544 hardware, use the following ``devtool`` command:
545 ::
546
547 $ devtool build recipe
548
549 On the other hand, if you want an image to contain the recipe's
550 packages from the workspace for immediate deployment onto a device
551 (e.g. for testing purposes), you can use the ``devtool build-image``
552 command: $ devtool build-image image
553
5544. *Deploy the Build Output*: When you use the ``devtool build`` command
555 to build out your recipe, you probably want to see if the resulting
556 build output works as expected on target hardware.
557
558 .. note::
559
560 This step assumes you have a previously built image that is
561 already either running in QEMU or running on actual hardware.
562 Also, it is assumed that for deployment of the image to the
563 target, SSH is installed in the image and if the image is running
564 on real hardware that you have network access to and from your
565 development machine.
566
567 You can deploy your build output to that target hardware by using the
568 ``devtool deploy-target`` command:
569 ::
570
571 $ devtool deploy-target recipe target
572
573 The target is a live target machine running as an SSH server.
574
575 You can, of course, use other methods to deploy the image you built
576 using the ``devtool build-image`` command to actual hardware.
577 ``devtool`` does not provide a specific command to deploy the image
578 to actual hardware.
579
5805. *Finish Your Work With the Recipe*: The ``devtool finish`` command
581 creates any patches corresponding to commits in the local Git
582 repository, updates the recipe to point to them (or creates a
583 ``.bbappend`` file to do so, depending on the specified destination
584 layer), and then resets the recipe so that the recipe is built
585 normally rather than from the workspace.
586 ::
587
588 $ devtool finish recipe layer
589
590 .. note::
591
592 Any changes you want to turn into patches must be staged and
593 committed within the local Git repository before you use the
594 devtool finish
595 command.
596
597 Because there is no need to move the recipe, ``devtool finish``
598 either updates the original recipe in the original layer or the
599 command creates a ``.bbappend`` file in a different layer as provided
600 by layer. Any work you did in the ``oe-local-files`` directory is
601 preserved in the original files next to the recipe during the
602 ``devtool finish`` command.
603
604 As a final process of the ``devtool finish`` command, the state of
605 the standard layers and the upstream source is restored so that you
606 can build the recipe from those areas rather than from the workspace.
607
608 .. note::
609
610 You can use the
611 devtool reset
612 command to put things back should you decide you do not want to
613 proceed with your work. If you do use this command, realize that
614 the source tree is preserved.
615
616.. _sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software:
617
618Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software
619-------------------------------------------------------------------------------------------------------
620
621The ``devtool upgrade`` command upgrades an existing recipe to that of a
622more up-to-date version found upstream. Throughout the life of software,
623recipes continually undergo version upgrades by their upstream
624publishers. You can use the ``devtool upgrade`` workflow to make sure
625your recipes you are using for builds are up-to-date with their upstream
626counterparts.
627
628.. note::
629
630 Several methods exist by which you can upgrade recipes -
631 devtool upgrade
632 happens to be one. You can read about all the methods by which you
633 can upgrade recipes in the "
634 Upgrading Recipes
635 " section of the Yocto Project Development Tasks Manual.
636
637The ``devtool upgrade`` command is flexible enough to allow you to
638specify source code revision and versioning schemes, extract code into
639or out of the ``devtool``
640:ref:`devtool-the-workspace-layer-structure`,
641and work with any source file forms that the
642:ref:`fetchers <bitbake:bb-fetchers>` support.
643
644The following diagram shows the common development flow used with the
645``devtool upgrade`` command:
646
647.. image:: figures/sdk-devtool-upgrade-flow.png
648 :align: center
649
6501. *Initiate the Upgrade*: The top part of the flow shows the typical
651 scenario by which you use the ``devtool upgrade`` command. The
652 following conditions exist:
653
654 - The recipe exists in a local layer external to the ``devtool``
655 workspace.
656
657 - The source files for the new release exist in the same location
658 pointed to by :term:`SRC_URI`
659 in the recipe (e.g. a tarball with the new version number in the
660 name, or as a different revision in the upstream Git repository).
661
662 A common situation is where third-party software has undergone a
663 revision so that it has been upgraded. The recipe you have access to
664 is likely in your own layer. Thus, you need to upgrade the recipe to
665 use the newer version of the software:
666 ::
667
668 $ devtool upgrade -V version recipe
669
670 By default, the ``devtool upgrade`` command extracts source
671 code into the ``sources`` directory in the
672 :ref:`devtool-the-workspace-layer-structure`.
673 If you want the code extracted to any other location, you need to
674 provide the srctree positional argument with the command as follows:
675 $ devtool upgrade -V version recipe srctree
676
677 .. note::
678
679 In this example, the "-V" option specifies the new version. If you
680 don't use "-V", the command upgrades the recipe to the latest
681 version.
682
683 If the source files pointed to by the ``SRC_URI`` statement in the
684 recipe are in a Git repository, you must provide the "-S" option and
685 specify a revision for the software.
686
687 Once ``devtool`` locates the recipe, it uses the ``SRC_URI`` variable
688 to locate the source code and any local patch files from other
689 developers. The result is that the command sets up the source code,
690 the new version of the recipe, and an append file all within the
691 workspace.
692
693 Additionally, if you have any non-patch local files (i.e. files
694 referred to with ``file://`` entries in ``SRC_URI`` statement
695 excluding ``*.patch/`` or ``*.diff``), these files are copied to an
696 ``oe-local-files`` folder under the newly created source tree.
697 Copying the files here gives you a convenient area from which you can
698 modify the files. Any changes or additions you make to those files
699 are incorporated into the build the next time you build the software
700 just as are other changes you might have made to the source.
701
7022. *Resolve any Conflicts created by the Upgrade*: Conflicts could exist
703 due to the software being upgraded to a new version. Conflicts occur
704 if your recipe specifies some patch files in ``SRC_URI`` that
705 conflict with changes made in the new version of the software. For
706 such cases, you need to resolve the conflicts by editing the source
707 and following the normal ``git rebase`` conflict resolution process.
708
709 Before moving onto the next step, be sure to resolve any such
710 conflicts created through use of a newer or different version of the
711 software.
712
7133. *Build the Recipe or Rebuild the Image*: The next step you take
714 depends on what you are going to do with the new code.
715
716 If you need to eventually move the build output to the target
717 hardware, use the following ``devtool`` command:
718 ::
719
720 $ devtool build recipe
721
722 On the other hand, if you want an image to contain the recipe's
723 packages from the workspace for immediate deployment onto a device
724 (e.g. for testing purposes), you can use the ``devtool build-image``
725 command:
726 ::
727
728 $ devtool build-image image
729
7304. *Deploy the Build Output*: When you use the ``devtool build`` command
731 or ``bitbake`` to build your recipe, you probably want to see if the
732 resulting build output works as expected on target hardware.
733
734 .. note::
735
736 This step assumes you have a previously built image that is
737 already either running in QEMU or running on actual hardware.
738 Also, it is assumed that for deployment of the image to the
739 target, SSH is installed in the image and if the image is running
740 on real hardware that you have network access to and from your
741 development machine.
742
743 You can deploy your build output to that target hardware by using the
744 ``devtool deploy-target`` command: $ devtool deploy-target recipe
745 target The target is a live target machine running as an SSH server.
746
747 You can, of course, also deploy the image you build using the
748 ``devtool build-image`` command to actual hardware. However,
749 ``devtool`` does not provide a specific command that allows you to do
750 this.
751
7525. *Finish Your Work With the Recipe*: The ``devtool finish`` command
753 creates any patches corresponding to commits in the local Git
754 repository, moves the new recipe to a more permanent layer, and then
755 resets the recipe so that the recipe is built normally rather than
756 from the workspace.
757
758 Any work you did in the ``oe-local-files`` directory is preserved in
759 the original files next to the recipe during the ``devtool finish``
760 command.
761
762 If you specify a destination layer that is the same as the original
763 source, then the old version of the recipe and associated files are
764 removed prior to adding the new version.
765 ::
766
767 $ devtool finish recipe layer
768
769 .. note::
770
771 Any changes you want to turn into patches must be committed to the
772 Git repository in the source tree.
773
774 As a final process of the ``devtool finish`` command, the state of
775 the standard layers and the upstream source is restored so that you
776 can build the recipe from those areas rather than the workspace.
777
778 .. note::
779
780 You can use the
781 devtool reset
782 command to put things back should you decide you do not want to
783 proceed with your work. If you do use this command, realize that
784 the source tree is preserved.
785
786.. _sdk-a-closer-look-at-devtool-add:
787
788A Closer Look at ``devtool add``
789================================
790
791The ``devtool add`` command automatically creates a recipe based on the
792source tree you provide with the command. Currently, the command has
793support for the following:
794
795- Autotools (``autoconf`` and ``automake``)
796
797- CMake
798
799- Scons
800
801- ``qmake``
802
803- Plain ``Makefile``
804
805- Out-of-tree kernel module
806
807- Binary package (i.e. "-b" option)
808
809- Node.js module
810
811- Python modules that use ``setuptools`` or ``distutils``
812
813Apart from binary packages, the determination of how a source tree
814should be treated is automatic based on the files present within that
815source tree. For example, if a ``CMakeLists.txt`` file is found, then
816the source tree is assumed to be using CMake and is treated accordingly.
817
818.. note::
819
820 In most cases, you need to edit the automatically generated recipe in
821 order to make it build properly. Typically, you would go through
822 several edit and build cycles until the recipe successfully builds.
823 Once the recipe builds, you could use possible further iterations to
824 test the recipe on the target device.
825
826The remainder of this section covers specifics regarding how parts of
827the recipe are generated.
828
829.. _sdk-name-and-version:
830
831Name and Version
832----------------
833
834If you do not specify a name and version on the command line,
835``devtool add`` uses various metadata within the source tree in an
836attempt to determine the name and version of the software being built.
837Based on what the tool determines, ``devtool`` sets the name of the
838created recipe file accordingly.
839
840If ``devtool`` cannot determine the name and version, the command prints
841an error. For such cases, you must re-run the command and provide the
842name and version, just the name, or just the version as part of the
843command line.
844
845Sometimes the name or version determined from the source tree might be
846incorrect. For such a case, you must reset the recipe:
847::
848
849 $ devtool reset -n recipename
850
851After running the ``devtool reset`` command, you need to
852run ``devtool add`` again and provide the name or the version.
853
854.. _sdk-dependency-detection-and-mapping:
855
856Dependency Detection and Mapping
857--------------------------------
858
859The ``devtool add`` command attempts to detect build-time dependencies
860and map them to other recipes in the system. During this mapping, the
861command fills in the names of those recipes as part of the
862:term:`DEPENDS` variable within the
863recipe. If a dependency cannot be mapped, ``devtool`` places a comment
864in the recipe indicating such. The inability to map a dependency can
865result from naming not being recognized or because the dependency simply
866is not available. For cases where the dependency is not available, you
867must use the ``devtool add`` command to add an additional recipe that
868satisfies the dependency. Once you add that recipe, you need to update
869the ``DEPENDS`` variable in the original recipe to include the new
870recipe.
871
872If you need to add runtime dependencies, you can do so by adding the
873following to your recipe:
874::
875
876 RDEPENDS_${PN} += "dependency1 dependency2 ..."
877
878.. note::
879
880 The
881 devtool add
882 command often cannot distinguish between mandatory and optional
883 dependencies. Consequently, some of the detected dependencies might
884 in fact be optional. When in doubt, consult the documentation or the
885 configure script for the software the recipe is building for further
886 details. In some cases, you might find you can substitute the
887 dependency with an option that disables the associated functionality
888 passed to the configure script.
889
890.. _sdk-license-detection:
891
892License Detection
893-----------------
894
895The ``devtool add`` command attempts to determine if the software you
896are adding is able to be distributed under a common, open-source
897license. If so, the command sets the
898:term:`LICENSE` value accordingly.
899You should double-check the value added by the command against the
900documentation or source files for the software you are building and, if
901necessary, update that ``LICENSE`` value.
902
903The ``devtool add`` command also sets the
904:term:`LIC_FILES_CHKSUM`
905value to point to all files that appear to be license-related. Realize
906that license statements often appear in comments at the top of source
907files or within the documentation. In such cases, the command does not
908recognize those license statements. Consequently, you might need to
909amend the ``LIC_FILES_CHKSUM`` variable to point to one or more of those
910comments if present. Setting ``LIC_FILES_CHKSUM`` is particularly
911important for third-party software. The mechanism attempts to ensure
912correct licensing should you upgrade the recipe to a newer upstream
913version in future. Any change in licensing is detected and you receive
914an error prompting you to check the license text again.
915
916If the ``devtool add`` command cannot determine licensing information,
917``devtool`` sets the ``LICENSE`` value to "CLOSED" and leaves the
918``LIC_FILES_CHKSUM`` value unset. This behavior allows you to continue
919with development even though the settings are unlikely to be correct in
920all cases. You should check the documentation or source files for the
921software you are building to determine the actual license.
922
923.. _sdk-adding-makefile-only-software:
924
925Adding Makefile-Only Software
926-----------------------------
927
928The use of Make by itself is very common in both proprietary and
929open-source software. Unfortunately, Makefiles are often not written
930with cross-compilation in mind. Thus, ``devtool add`` often cannot do
931very much to ensure that these Makefiles build correctly. It is very
932common, for example, to explicitly call ``gcc`` instead of using the
933:term:`CC` variable. Usually, in a
934cross-compilation environment, ``gcc`` is the compiler for the build
935host and the cross-compiler is named something similar to
936``arm-poky-linux-gnueabi-gcc`` and might require arguments (e.g. to
937point to the associated sysroot for the target machine).
938
939When writing a recipe for Makefile-only software, keep the following in
940mind:
941
942- You probably need to patch the Makefile to use variables instead of
943 hardcoding tools within the toolchain such as ``gcc`` and ``g++``.
944
945- The environment in which Make runs is set up with various standard
946 variables for compilation (e.g. ``CC``, ``CXX``, and so forth) in a
947 similar manner to the environment set up by the SDK's environment
948 setup script. One easy way to see these variables is to run the
949 ``devtool build`` command on the recipe and then look in
950 ``oe-logs/run.do_compile``. Towards the top of this file, a list of
951 environment variables exists that are being set. You can take
952 advantage of these variables within the Makefile.
953
954- If the Makefile sets a default for a variable using "=", that default
955 overrides the value set in the environment, which is usually not
956 desirable. For this case, you can either patch the Makefile so it
957 sets the default using the "?=" operator, or you can alternatively
958 force the value on the ``make`` command line. To force the value on
959 the command line, add the variable setting to
960 :term:`EXTRA_OEMAKE` or
961 :term:`PACKAGECONFIG_CONFARGS`
962 within the recipe. Here is an example using ``EXTRA_OEMAKE``:
963 ::
964
965 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
966
967 In the above example,
968 single quotes are used around the variable settings as the values are
969 likely to contain spaces because required default options are passed
970 to the compiler.
971
972- Hardcoding paths inside Makefiles is often problematic in a
973 cross-compilation environment. This is particularly true because
974 those hardcoded paths often point to locations on the build host and
975 thus will either be read-only or will introduce contamination into
976 the cross-compilation because they are specific to the build host
977 rather than the target. Patching the Makefile to use prefix variables
978 or other path variables is usually the way to handle this situation.
979
980- Sometimes a Makefile runs target-specific commands such as
981 ``ldconfig``. For such cases, you might be able to apply patches that
982 remove these commands from the Makefile.
983
984.. _sdk-adding-native-tools:
985
986Adding Native Tools
987-------------------
988
989Often, you need to build additional tools that run on the :term:`Build
990Host` as opposed to
991the target. You should indicate this requirement by using one of the
992following methods when you run ``devtool add``:
993
994- Specify the name of the recipe such that it ends with "-native".
995 Specifying the name like this produces a recipe that only builds for
996 the build host.
997
998- Specify the "DASHDASHalso-native" option with the ``devtool add``
999 command. Specifying this option creates a recipe file that still
1000 builds for the target but also creates a variant with a "-native"
1001 suffix that builds for the build host.
1002
1003.. note::
1004
1005 If you need to add a tool that is shipped as part of a source tree
1006 that builds code for the target, you can typically accomplish this by
1007 building the native and target parts separately rather than within
1008 the same compilation process. Realize though that with the
1009 "DASHDASHalso-native" option, you can add the tool using just one
1010 recipe file.
1011
1012.. _sdk-adding-node-js-modules:
1013
1014Adding Node.js Modules
1015----------------------
1016
1017You can use the ``devtool add`` command two different ways to add
1018Node.js modules: 1) Through ``npm`` and, 2) from a repository or local
1019source.
1020
1021Use the following form to add Node.js modules through ``npm``:
1022::
1023
1024 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
1025
1026The name and
1027version parameters are mandatory. Lockdown and shrinkwrap files are
1028generated and pointed to by the recipe in order to freeze the version
1029that is fetched for the dependencies according to the first time. This
1030also saves checksums that are verified on future fetches. Together,
1031these behaviors ensure the reproducibility and integrity of the build.
1032
1033.. note::
1034
1035 - You must use quotes around the URL. The ``devtool add`` does not
1036 require the quotes, but the shell considers ";" as a splitter
1037 between multiple commands. Thus, without the quotes,
1038 ``devtool add`` does not receive the other parts, which results in
1039 several "command not found" errors.
1040
1041 - In order to support adding Node.js modules, a ``nodejs`` recipe
1042 must be part of your SDK.
1043
1044As mentioned earlier, you can also add Node.js modules directly from a
1045repository or local source tree. To add modules this way, use
1046``devtool add`` in the following form:
1047::
1048
1049 $ devtool add https://github.com/diversario/node-ssdp
1050
1051In this example, ``devtool``
1052fetches the specified Git repository, detects the code as Node.js code,
1053fetches dependencies using ``npm``, and sets
1054:term:`SRC_URI` accordingly.
1055
1056.. _sdk-working-with-recipes:
1057
1058Working With Recipes
1059====================
1060
1061When building a recipe using the ``devtool build`` command, the typical
1062build progresses as follows:
1063
10641. Fetch the source
1065
10662. Unpack the source
1067
10683. Configure the source
1069
10704. Compile the source
1071
10725. Install the build output
1073
10746. Package the installed output
1075
1076For recipes in the workspace, fetching and unpacking is disabled as the
1077source tree has already been prepared and is persistent. Each of these
1078build steps is defined as a function (task), usually with a "do\_" prefix
1079(e.g. :ref:`ref-tasks-fetch`,
1080:ref:`ref-tasks-unpack`, and so
1081forth). These functions are typically shell scripts but can instead be
1082written in Python.
1083
1084If you look at the contents of a recipe, you will see that the recipe
1085does not include complete instructions for building the software.
1086Instead, common functionality is encapsulated in classes inherited with
1087the ``inherit`` directive. This technique leaves the recipe to describe
1088just the things that are specific to the software being built. A
1089:ref:`base <ref-classes-base>` class exists that
1090is implicitly inherited by all recipes and provides the functionality
1091that most recipes typically need.
1092
1093The remainder of this section presents information useful when working
1094with recipes.
1095
1096.. _sdk-finding-logs-and-work-files:
1097
1098Finding Logs and Work Files
1099---------------------------
1100
1101After the first run of the ``devtool build`` command, recipes that were
1102previously created using the ``devtool add`` command or whose sources
1103were modified using the ``devtool modify`` command contain symbolic
1104links created within the source tree:
1105
1106- ``oe-logs``: This link points to the directory in which log files and
1107 run scripts for each build step are created.
1108
1109- ``oe-workdir``: This link points to the temporary work area for the
1110 recipe. The following locations under ``oe-workdir`` are particularly
1111 useful:
1112
1113 - ``image/``: Contains all of the files installed during the
1114 :ref:`ref-tasks-install` stage.
1115 Within a recipe, this directory is referred to by the expression
1116 ``${``\ :term:`D`\ ``}``.
1117
1118 - ``sysroot-destdir/``: Contains a subset of files installed within
1119 ``do_install`` that have been put into the shared sysroot. For
1120 more information, see the "`Sharing Files Between
1121 Recipes <#sdk-sharing-files-between-recipes>`__" section.
1122
1123 - ``packages-split/``: Contains subdirectories for each package
1124 produced by the recipe. For more information, see the
1125 "`Packaging <#sdk-packaging>`__" section.
1126
1127You can use these links to get more information on what is happening at
1128each build step.
1129
1130.. _sdk-setting-configure-arguments:
1131
1132Setting Configure Arguments
1133---------------------------
1134
1135If the software your recipe is building uses GNU autoconf, then a fixed
1136set of arguments is passed to it to enable cross-compilation plus any
1137extras specified by
1138:term:`EXTRA_OECONF` or
1139:term:`PACKAGECONFIG_CONFARGS`
1140set within the recipe. If you wish to pass additional options, add them
1141to ``EXTRA_OECONF`` or ``PACKAGECONFIG_CONFARGS``. Other supported build
1142tools have similar variables (e.g.
1143:term:`EXTRA_OECMAKE` for
1144CMake, :term:`EXTRA_OESCONS`
1145for Scons, and so forth). If you need to pass anything on the ``make``
1146command line, you can use ``EXTRA_OEMAKE`` or the
1147:term:`PACKAGECONFIG_CONFARGS`
1148variables to do so.
1149
1150You can use the ``devtool configure-help`` command to help you set the
1151arguments listed in the previous paragraph. The command determines the
1152exact options being passed, and shows them to you along with any custom
1153arguments specified through ``EXTRA_OECONF`` or
1154``PACKAGECONFIG_CONFARGS``. If applicable, the command also shows you
1155the output of the configure script's "DASHDASHhelp" option as a
1156reference.
1157
1158.. _sdk-sharing-files-between-recipes:
1159
1160Sharing Files Between Recipes
1161-----------------------------
1162
1163Recipes often need to use files provided by other recipes on the
1164:term:`Build Host`. For example,
1165an application linking to a common library needs access to the library
1166itself and its associated headers. The way this access is accomplished
1167within the extensible SDK is through the sysroot. One sysroot exists per
1168"machine" for which the SDK is being built. In practical terms, this
1169means a sysroot exists for the target machine, and a sysroot exists for
1170the build host.
1171
1172Recipes should never write files directly into the sysroot. Instead,
1173files should be installed into standard locations during the
1174:ref:`ref-tasks-install` task within
1175the ``${``\ :term:`D`\ ``}`` directory. A
1176subset of these files automatically goes into the sysroot. The reason
1177for this limitation is that almost all files that go into the sysroot
1178are cataloged in manifests in order to ensure they can be removed later
1179when a recipe is modified or removed. Thus, the sysroot is able to
1180remain free from stale files.
1181
1182.. _sdk-packaging:
1183
1184Packaging
1185---------
1186
1187Packaging is not always particularly relevant within the extensible SDK.
1188However, if you examine how build output gets into the final image on
1189the target device, it is important to understand packaging because the
1190contents of the image are expressed in terms of packages and not
1191recipes.
1192
1193During the :ref:`ref-tasks-package`
1194task, files installed during the
1195:ref:`ref-tasks-install` task are
1196split into one main package, which is almost always named the same as
1197the recipe, and into several other packages. This separation exists
1198because not all of those installed files are useful in every image. For
1199example, you probably do not need any of the documentation installed in
1200a production image. Consequently, for each recipe the documentation
1201files are separated into a ``-doc`` package. Recipes that package
1202software containing optional modules or plugins might undergo additional
1203package splitting as well.
1204
1205After building a recipe, you can see where files have gone by looking in
1206the ``oe-workdir/packages-split`` directory, which contains a
1207subdirectory for each package. Apart from some advanced cases, the
1208:term:`PACKAGES` and
1209:term:`FILES` variables controls
1210splitting. The ``PACKAGES`` variable lists all of the packages to be
1211produced, while the ``FILES`` variable specifies which files to include
1212in each package by using an override to specify the package. For
1213example, ``FILES_${PN}`` specifies the files to go into the main package
1214(i.e. the main package has the same name as the recipe and
1215``${``\ :term:`PN`\ ``}`` evaluates to the
1216recipe name). The order of the ``PACKAGES`` value is significant. For
1217each installed file, the first package whose ``FILES`` value matches the
1218file is the package into which the file goes. Defaults exist for both
1219the ``PACKAGES`` and ``FILES`` variables. Consequently, you might find
1220you do not even need to set these variables in your recipe unless the
1221software the recipe is building installs files into non-standard
1222locations.
1223
1224.. _sdk-restoring-the-target-device-to-its-original-state:
1225
1226Restoring the Target Device to its Original State
1227=================================================
1228
1229If you use the ``devtool deploy-target`` command to write a recipe's
1230build output to the target, and you are working on an existing component
1231of the system, then you might find yourself in a situation where you
1232need to restore the original files that existed prior to running the
1233``devtool deploy-target`` command. Because the ``devtool deploy-target``
1234command backs up any files it overwrites, you can use the
1235``devtool undeploy-target`` command to restore those files and remove
1236any other files the recipe deployed. Consider the following example:
1237::
1238
1239 $ devtool undeploy-target lighttpd root@192.168.7.2
1240
1241If you have deployed
1242multiple applications, you can remove them all using the "-a" option
1243thus restoring the target device to its original state:
1244::
1245
1246 $ devtool undeploy-target -a root@192.168.7.2
1247
1248Information about files deployed to
1249the target as well as any backed up files are stored on the target
1250itself. This storage, of course, requires some additional space on the
1251target machine.
1252
1253.. note::
1254
1255 The
1256 devtool deploy-target
1257 and
1258 devtool undeploy-target
1259 commands do not currently interact with any package management system
1260 on the target device (e.g. RPM or OPKG). Consequently, you should not
1261 intermingle
1262 devtool deploy-target
1263 and package manager operations on the target device. Doing so could
1264 result in a conflicting set of files.
1265
1266.. _sdk-installing-additional-items-into-the-extensible-sdk:
1267
1268Installing Additional Items Into the Extensible SDK
1269===================================================
1270
1271Out of the box the extensible SDK typically only comes with a small
1272number of tools and libraries. A minimal SDK starts mostly empty and is
1273populated on-demand. Sometimes you must explicitly install extra items
1274into the SDK. If you need these extra items, you can first search for
1275the items using the ``devtool search`` command. For example, suppose you
1276need to link to libGL but you are not sure which recipe provides libGL.
1277You can use the following command to find out:
1278::
1279
1280 $ devtool search libGL mesa
1281
1282A free implementation of the OpenGL API Once you know the recipe
1283(i.e. ``mesa`` in this example), you can install it:
1284::
1285
1286 $ devtool sdk-install mesa
1287
1288By default, the ``devtool sdk-install`` command assumes
1289the item is available in pre-built form from your SDK provider. If the
1290item is not available and it is acceptable to build the item from
1291source, you can add the "-s" option as follows:
1292::
1293
1294 $ devtool sdk-install -s mesa
1295
1296It is important to remember that building the item from source
1297takes significantly longer than installing the pre-built artifact. Also,
1298if no recipe exists for the item you want to add to the SDK, you must
1299instead add the item using the ``devtool add`` command.
1300
1301.. _sdk-applying-updates-to-an-installed-extensible-sdk:
1302
1303Applying Updates to an Installed Extensible SDK
1304===============================================
1305
1306If you are working with an installed extensible SDK that gets
1307occasionally updated (e.g. a third-party SDK), then you will need to
1308manually "pull down" the updates into the installed SDK.
1309
1310To update your installed SDK, use ``devtool`` as follows:
1311::
1312
1313 $ devtool sdk-update
1314
1315The previous command assumes your SDK provider has set the
1316default update URL for you through the
1317:term:`SDK_UPDATE_URL`
1318variable as described in the "`Providing Updates to the Extensible SDK
1319After
1320Installation <#sdk-providing-updates-to-the-extensible-sdk-after-installation>`__"
1321section. If the SDK provider has not set that default URL, you need to
1322specify it yourself in the command as follows: $ devtool sdk-update
1323path_to_update_directory
1324
1325.. note::
1326
1327 The URL needs to point specifically to a published SDK and not to an
1328 SDK installer that you would download and install.
1329
1330.. _sdk-creating-a-derivative-sdk-with-additional-components:
1331
1332Creating a Derivative SDK With Additional Components
1333====================================================
1334
1335You might need to produce an SDK that contains your own custom
1336libraries. A good example would be if you were a vendor with customers
1337that use your SDK to build their own platform-specific software and
1338those customers need an SDK that has custom libraries. In such a case,
1339you can produce a derivative SDK based on the currently installed SDK
1340fairly easily by following these steps:
1341
13421. If necessary, install an extensible SDK that you want to use as a
1343 base for your derivative SDK.
1344
13452. Source the environment script for the SDK.
1346
13473. Add the extra libraries or other components you want by using the
1348 ``devtool add`` command.
1349
13504. Run the ``devtool build-sdk`` command.
1351
1352The previous steps take the recipes added to the workspace and construct
1353a new SDK installer that contains those recipes and the resulting binary
1354artifacts. The recipes go into their own separate layer in the
1355constructed derivative SDK, which leaves the workspace clean and ready
1356for users to add their own recipes.
diff --git a/documentation/sdk-manual/sdk-extensible.xml b/documentation/sdk-manual/sdk-extensible.xml
index 94d2a241fe..a73a07a7b9 100644
--- a/documentation/sdk-manual/sdk-extensible.xml
+++ b/documentation/sdk-manual/sdk-extensible.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='sdk-extensible'> 6<chapter id='sdk-extensible'>
6 7
diff --git a/documentation/sdk-manual/sdk-intro.rst b/documentation/sdk-manual/sdk-intro.rst
new file mode 100644
index 0000000000..82b7bcf3cf
--- /dev/null
+++ b/documentation/sdk-manual/sdk-intro.rst
@@ -0,0 +1,231 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Introduction
5************
6
7.. _sdk-manual-intro:
8
9eSDK Introduction
10=================
11
12Welcome to the Yocto Project Application Development and the Extensible
13Software Development Kit (eSDK) manual. This manual provides information
14that explains how to use both the Yocto Project extensible and standard
15SDKs to develop applications and images.
16
17.. note::
18
19 Prior to the 2.0 Release of the Yocto Project, application
20 development was primarily accomplished through the use of the
21 Application Development Toolkit (ADT) and the availability of
22 stand-alone cross-development toolchains and other tools. With the
23 2.1 Release of the Yocto Project, application development has
24 transitioned to within a tool-rich extensible SDK and the more
25 traditional standard SDK.
26
27All SDKs consist of the following:
28
29- *Cross-Development Toolchain*: This toolchain contains a compiler,
30 debugger, and various miscellaneous tools.
31
32- *Libraries, Headers, and Symbols*: The libraries, headers, and
33 symbols are specific to the image (i.e. they match the image).
34
35- *Environment Setup Script*: This ``*.sh`` file, once run, sets up the
36 cross-development environment by defining variables and preparing for
37 SDK use.
38
39Additionally, an extensible SDK has tools that allow you to easily add
40new applications and libraries to an image, modify the source of an
41existing component, test changes on the target hardware, and easily
42integrate an application into the :term:`OpenEmbedded Build System`.
43
44You can use an SDK to independently develop and test code that is
45destined to run on some target machine. SDKs are completely
46self-contained. The binaries are linked against their own copy of
47``libc``, which results in no dependencies on the target system. To
48achieve this, the pointer to the dynamic loader is configured at install
49time since that path cannot be dynamically altered. This is the reason
50for a wrapper around the ``populate_sdk`` and ``populate_sdk_ext``
51archives.
52
53Another feature for the SDKs is that only one set of cross-compiler
54toolchain binaries are produced for any given architecture. This feature
55takes advantage of the fact that the target hardware can be passed to
56``gcc`` as a set of compiler options. Those options are set up by the
57environment script and contained in variables such as
58:term:`CC` and
59:term:`LD`. This reduces the space needed
60for the tools. Understand, however, that every target still needs a
61sysroot because those binaries are target-specific.
62
63The SDK development environment consists of the following:
64
65- The self-contained SDK, which is an architecture-specific
66 cross-toolchain and matching sysroots (target and native) all built
67 by the OpenEmbedded build system (e.g. the SDK). The toolchain and
68 sysroots are based on a :term:`Metadata`
69 configuration and extensions, which allows you to cross-develop on
70 the host machine for the target hardware. Additionally, the
71 extensible SDK contains the ``devtool`` functionality.
72
73- The Quick EMUlator (QEMU), which lets you simulate target hardware.
74 QEMU is not literally part of the SDK. You must build and include
75 this emulator separately. However, QEMU plays an important role in
76 the development process that revolves around use of the SDK.
77
78In summary, the extensible and standard SDK share many features.
79However, the extensible SDK has powerful development tools to help you
80more quickly develop applications. Following is a table that summarizes
81the primary differences between the standard and extensible SDK types
82when considering which to build:
83
84+-----------------------+-----------------------+-----------------------+
85| *Feature* | *Standard SDK* | *Extensible SDK* |
86+=======================+=======================+=======================+
87| Toolchain | Yes | Yes\* |
88+-----------------------+-----------------------+-----------------------+
89| Debugger | Yes | Yes\* |
90+-----------------------+-----------------------+-----------------------+
91| Size | 100+ MBytes | 1+ GBytes (or 300+ |
92| | | MBytes for minimal |
93| | | w/toolchain) |
94+-----------------------+-----------------------+-----------------------+
95| ``devtool`` | No | Yes |
96+-----------------------+-----------------------+-----------------------+
97| Build Images | No | Yes |
98+-----------------------+-----------------------+-----------------------+
99| Updateable | No | Yes |
100+-----------------------+-----------------------+-----------------------+
101| Managed Sysroot*\* | No | Yes |
102+-----------------------+-----------------------+-----------------------+
103| Installed Packages | No**\* | Yes***\* |
104+-----------------------+-----------------------+-----------------------+
105| Construction | Packages | Shared State |
106+-----------------------+-----------------------+-----------------------+
107
108\* Extensible SDK contains the toolchain and debugger if
109:term:`SDK_EXT_TYPE` is "full"
110or
111:term:`SDK_INCLUDE_TOOLCHAIN`
112is "1", which is the default.
113
114\*\* Sysroot is managed through the use of
115``devtool``. Thus, it is less likely that you will corrupt your SDK
116sysroot when you try to add additional libraries.
117
118\*\*\* You can add
119runtime package management to the standard SDK but it is not supported
120by default.
121
122\*\*\*\* You must build and make the shared state available to
123extensible SDK users for "packages" you want to enable users to install.
124
125The Cross-Development Toolchain
126-------------------------------
127
128The :term:`Cross-Development Toolchain` consists
129of a cross-compiler, cross-linker, and cross-debugger that are used to
130develop user-space applications for targeted hardware. Additionally, for
131an extensible SDK, the toolchain also has built-in ``devtool``
132functionality. This toolchain is created by running a SDK installer
133script or through a :term:`Build Directory` that is based on
134your metadata configuration or extension for your targeted device. The
135cross-toolchain works with a matching target sysroot.
136
137.. _sysroot:
138
139Sysroots
140--------
141
142The native and target sysroots contain needed headers and libraries for
143generating binaries that run on the target architecture. The target
144sysroot is based on the target root filesystem image that is built by
145the OpenEmbedded build system and uses the same metadata configuration
146used to build the cross-toolchain.
147
148The QEMU Emulator
149-----------------
150
151The QEMU emulator allows you to simulate your hardware while running
152your application or image. QEMU is not part of the SDK but is made
153available a number of different ways:
154
155- If you have cloned the ``poky`` Git repository to create a
156 :term:`Source Directory` and you have
157 sourced the environment setup script, QEMU is installed and
158 automatically available.
159
160- If you have downloaded a Yocto Project release and unpacked it to
161 create a Source Directory and you have sourced the environment setup
162 script, QEMU is installed and automatically available.
163
164- If you have installed the cross-toolchain tarball and you have
165 sourced the toolchain's setup environment script, QEMU is also
166 installed and automatically available.
167
168SDK Development Model
169=====================
170
171Fundamentally, the SDK fits into the development process as follows:
172
173.. image:: figures/sdk-environment.png
174 :align: center
175
176The SDK is installed on any machine and can be used to develop applications,
177images, and kernels. An SDK can even be used by a QA Engineer or Release
178Engineer. The fundamental concept is that the machine that has the SDK
179installed does not have to be associated with the machine that has the
180Yocto Project installed. A developer can independently compile and test
181an object on their machine and then, when the object is ready for
182integration into an image, they can simply make it available to the
183machine that has the Yocto Project. Once the object is available, the
184image can be rebuilt using the Yocto Project to produce the modified
185image.
186
187You just need to follow these general steps:
188
1891. *Install the SDK for your target hardware:* For information on how to
190 install the SDK, see the "`Installing the
191 SDK <#sdk-installing-the-sdk>`__" section.
192
1932. *Download or Build the Target Image:* The Yocto Project supports
194 several target architectures and has many pre-built kernel images and
195 root filesystem images.
196
197 If you are going to develop your application on hardware, go to the
198 :yocto_dl:`machines </releases/yocto/yocto-3.1.2/machines/>` download area and choose a
199 target machine area from which to download the kernel image and root
200 filesystem. This download area could have several files in it that
201 support development using actual hardware. For example, the area
202 might contain ``.hddimg`` files that combine the kernel image with
203 the filesystem, boot loaders, and so forth. Be sure to get the files
204 you need for your particular development process.
205
206 If you are going to develop your application and then run and test it
207 using the QEMU emulator, go to the
208 :yocto_dl:`machines/qemu </releases/yocto/yocto-3.1.2/machines/qemu>` download area. From this
209 area, go down into the directory for your target architecture (e.g.
210 ``qemux86_64`` for an Intel-based 64-bit architecture). Download the
211 kernel, root filesystem, and any other files you need for your
212 process.
213
214 .. note::
215
216 To use the root filesystem in QEMU, you need to extract it. See
217 the "
218 Extracting the Root Filesystem
219 " section for information on how to extract the root filesystem.
220
2213. *Develop and Test your Application:* At this point, you have the
222 tools to develop your application. If you need to separately install
223 and use the QEMU emulator, you can go to `QEMU Home
224 Page <http://wiki.qemu.org/Main_Page>`__ to download and learn about
225 the emulator. See the ":doc:`../dev-manual/dev-manual-qemu`" chapter in the
226 Yocto Project Development Tasks Manual for information on using QEMU
227 within the Yocto Project.
228
229The remainder of this manual describes how to use the extensible and
230standard SDKs. Information also exists in appendix form that describes
231how you can build, install, and modify an SDK.
diff --git a/documentation/sdk-manual/sdk-intro.xml b/documentation/sdk-manual/sdk-intro.xml
index 9169fe9c05..f42670ea6e 100644
--- a/documentation/sdk-manual/sdk-intro.xml
+++ b/documentation/sdk-manual/sdk-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='sdk-intro'> 6<chapter id='sdk-intro'>
6<title>Introduction</title> 7<title>Introduction</title>
diff --git a/documentation/sdk-manual/sdk-manual-customization.xsl b/documentation/sdk-manual/sdk-manual-customization.xsl
index efa8a84bbb..4f8816f950 100644
--- a/documentation/sdk-manual/sdk-manual-customization.xsl
+++ b/documentation/sdk-manual/sdk-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/sdk-manual/sdk-manual.rst b/documentation/sdk-manual/sdk-manual.rst
new file mode 100644
index 0000000000..d7776b7c40
--- /dev/null
+++ b/documentation/sdk-manual/sdk-manual.rst
@@ -0,0 +1,22 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3========================================================================================
4Yocto Project Application Development and the Extensible Software Development Kit (eSDK)
5========================================================================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 sdk-intro
14 sdk-extensible
15 sdk-using
16 sdk-working-projects
17 sdk-appendix-obtain
18 sdk-appendix-customizing
19 sdk-appendix-customizing-standard
20 history
21
22.. include:: /boilerplate.rst
diff --git a/documentation/sdk-manual/sdk-manual.xml b/documentation/sdk-manual/sdk-manual.xml
index 1bcc0c853f..6344478fb0 100755
--- a/documentation/sdk-manual/sdk-manual.xml
+++ b/documentation/sdk-manual/sdk-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='sdk-manual' lang='en' 6<book id='sdk-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -72,28 +73,8 @@
72 </revision> 73 </revision>
73 <revision> 74 <revision>
74 <revnumber>3.1</revnumber> 75 <revnumber>3.1</revnumber>
75 <date>April 2020</date>
76 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
77 </revision>
78 <revision>
79 <revnumber>3.1.1</revnumber>
80 <date>June 2020</date>
81 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
82 </revision>
83 <revision>
84 <revnumber>3.1.2</revnumber>
85 <date>August 2020</date>
86 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
87 </revision>
88 <revision>
89 <revnumber>3.1.3</revnumber>
90 <date>October 2020</date>
91 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
92 </revision>
93 <revision>
94 <revnumber>3.1.4</revnumber>
95 <date>&REL_MONTH_YEAR;</date> 76 <date>&REL_MONTH_YEAR;</date>
96 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 77 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
97 </revision> 78 </revision>
98 </revhistory> 79 </revhistory>
99 80
diff --git a/documentation/sdk-manual/sdk-style.css b/documentation/sdk-manual/sdk-style.css
index 52518964ca..e0c4416a15 100644
--- a/documentation/sdk-manual/sdk-style.css
+++ b/documentation/sdk-manual/sdk-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/sdk-manual/sdk-using.rst b/documentation/sdk-manual/sdk-using.rst
new file mode 100644
index 0000000000..09a194cab5
--- /dev/null
+++ b/documentation/sdk-manual/sdk-using.rst
@@ -0,0 +1,159 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************
4Using the Standard SDK
5**********************
6
7This chapter describes the standard SDK and how to install it.
8Information includes unique installation and setup aspects for the
9standard SDK.
10
11.. note::
12
13 For a side-by-side comparison of main features supported for a
14 standard SDK as compared to an extensible SDK, see the "
15 Introduction
16 " section.
17
18You can use a standard SDK to work on Makefile and Autotools-based
19projects. See the "`Using the SDK Toolchain
20Directly <#sdk-working-projects>`__" chapter for more information.
21
22.. _sdk-standard-sdk-intro:
23
24Why use the Standard SDK and What is in It?
25===========================================
26
27The Standard SDK provides a cross-development toolchain and libraries
28tailored to the contents of a specific image. You would use the Standard
29SDK if you want a more traditional toolchain experience as compared to
30the extensible SDK, which provides an internal build system and the
31``devtool`` functionality.
32
33The installed Standard SDK consists of several files and directories.
34Basically, it contains an SDK environment setup script, some
35configuration files, and host and target root filesystems to support
36usage. You can see the directory structure in the "`Installed Standard
37SDK Directory
38Structure <#sdk-installed-standard-sdk-directory-structure>`__" section.
39
40.. _sdk-installing-the-sdk:
41
42Installing the SDK
43==================
44
45The first thing you need to do is install the SDK on your :term:`Build
46Host` by running the ``*.sh`` installation script.
47
48You can download a tarball installer, which includes the pre-built
49toolchain, the ``runqemu`` script, and support files from the
50appropriate :yocto_dl:`toolchain </releases/yocto/yocto-3.1.2/toolchain/>` directory within
51the Index of Releases. Toolchains are available for several 32-bit and
5264-bit architectures with the ``x86_64`` directories, respectively. The
53toolchains the Yocto Project provides are based off the
54``core-image-sato`` and ``core-image-minimal`` images and contain
55libraries appropriate for developing against that image.
56
57The names of the tarball installer scripts are such that a string
58representing the host system appears first in the filename and then is
59immediately followed by a string representing the target architecture.
60::
61
62 poky-glibc-host_system-image_type-arch-toolchain-release_version.sh
63
64 Where:
65 host_system is a string representing your development system:
66
67 i686 or x86_64.
68
69 image_type is the image for which the SDK was built:
70
71 core-image-minimal or core-image-sato.
72
73 arch is a string representing the tuned target architecture:
74
75 aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon.
76
77 release_version is a string representing the release number of the Yocto Project:
78
79 3.1.2, 3.1.2+snapshot
80
81For example, the following SDK installer is for a 64-bit
82development host system and a i586-tuned target architecture based off
83the SDK for ``core-image-sato`` and using the current DISTRO snapshot:
84::
85
86 poky-glibc-x86_64-core-image-sato-i586-toolchain-DISTRO.sh
87
88.. note::
89
90 As an alternative to downloading an SDK, you can build the SDK
91 installer. For information on building the installer, see the "
92 Building an SDK Installer
93 " section.
94
95The SDK and toolchains are self-contained and by default are installed
96into the ``poky_sdk`` folder in your home directory. You can choose to
97install the extensible SDK in any location when you run the installer.
98However, because files need to be written under that directory during
99the normal course of operation, the location you choose for installation
100must be writable for whichever users need to use the SDK.
101
102The following command shows how to run the installer given a toolchain
103tarball for a 64-bit x86 development host system and a 64-bit x86 target
104architecture. The example assumes the SDK installer is located in
105``~/Downloads/`` and has execution rights.
106
107.. note::
108
109 If you do not have write permissions for the directory into which you
110 are installing the SDK, the installer notifies you and exits. For
111 that case, set up the proper permissions in the directory and run the
112 installer again.
113
114::
115
116 $ ./Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-3.1.2.sh
117 Poky (Yocto Project Reference Distro) SDK installer version 3.1.2
118 ===============================================================
119 Enter target directory for SDK (default: /opt/poky/3.1.2):
120 You are about to install the SDK to "/opt/poky/3.1.2". Proceed [Y/n]? Y
121 Extracting SDK........................................ ..............................done
122 Setting it up...done
123 SDK has been successfully set up and is ready to be used.
124 Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
125 $ . /opt/poky/3.1.2/environment-setup-i586-poky-linux
126
127Again, reference the "`Installed Standard SDK Directory
128Structure <#sdk-installed-standard-sdk-directory-structure>`__" section
129for more details on the resulting directory structure of the installed
130SDK.
131
132.. _sdk-running-the-sdk-environment-setup-script:
133
134Running the SDK Environment Setup Script
135========================================
136
137Once you have the SDK installed, you must run the SDK environment setup
138script before you can actually use the SDK. This setup script resides in
139the directory you chose when you installed the SDK, which is either the
140default ``/opt/poky/3.1.2`` directory or the directory you chose during
141installation.
142
143Before running the script, be sure it is the one that matches the
144architecture for which you are developing. Environment setup scripts
145begin with the string "``environment-setup``" and include as part of
146their name the tuned target architecture. As an example, the following
147commands set the working directory to where the SDK was installed and
148then source the environment setup script. In this example, the setup
149script is for an IA-based target machine using i586 tuning:
150::
151
152 $ source /opt/poky/3.1.2/environment-setup-i586-poky-linux
153
154When you run the
155setup script, the same environment variables are defined as are when you
156run the setup script for an extensible SDK. See the "`Running the
157Extensible SDK Environment Setup
158Script <#sdk-running-the-extensible-sdk-environment-setup-script>`__"
159section for more information.
diff --git a/documentation/sdk-manual/sdk-using.xml b/documentation/sdk-manual/sdk-using.xml
index 66b15cd6ce..28ee50d0b7 100644
--- a/documentation/sdk-manual/sdk-using.xml
+++ b/documentation/sdk-manual/sdk-using.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='sdk-using-the-standard-sdk'> 6<chapter id='sdk-using-the-standard-sdk'>
6 <title>Using the Standard SDK</title> 7 <title>Using the Standard SDK</title>
diff --git a/documentation/sdk-manual/sdk-working-projects.rst b/documentation/sdk-manual/sdk-working-projects.rst
new file mode 100644
index 0000000000..2c20a1ec57
--- /dev/null
+++ b/documentation/sdk-manual/sdk-working-projects.rst
@@ -0,0 +1,423 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3********************************
4Using the SDK Toolchain Directly
5********************************
6
7You can use the SDK toolchain directly with Makefile and Autotools-based
8projects.
9
10Autotools-Based Projects
11========================
12
13Once you have a suitable :ref:`sdk-manual/sdk-intro:the cross-development toolchain`
14installed, it is very easy to develop a project using the `GNU
15Autotools-based <https://en.wikipedia.org/wiki/GNU_Build_System>`__
16workflow, which is outside of the :term:`OpenEmbedded Build System`.
17
18The following figure presents a simple Autotools workflow.
19
20.. image:: figures/sdk-autotools-flow.png
21 :align: center
22
23Follow these steps to create a simple Autotools-based "Hello World"
24project:
25
26.. note::
27
28 For more information on the GNU Autotools workflow, see the same
29 example on the
30 GNOME Developer
31 site.
32
331. *Create a Working Directory and Populate It:* Create a clean
34 directory for your project and then make that directory your working
35 location.
36 ::
37
38 $ mkdir $HOME/helloworld
39 $ cd $HOME/helloworld
40
41 After setting up the directory, populate it with files needed for the flow.
42 You need a project source file, a file to help with configuration,
43 and a file to help create the Makefile, and a README file:
44 ``hello.c``, ``configure.ac``, ``Makefile.am``, and ``README``,
45 respectively.
46
47 Use the following command to create an empty README file, which is
48 required by GNU Coding Standards:
49 ::
50
51 $ touch README
52
53 Create the remaining
54 three files as follows:
55
56 - ``hello.c``:
57 ::
58
59 #include <stdio.h>
60
61 main()
62 {
63 printf("Hello World!\n");
64 }
65
66 - ``configure.ac``:
67 ::
68
69 AC_INIT(hello,0.1)
70 AM_INIT_AUTOMAKE([foreign])
71 AC_PROG_CC
72 AC_CONFIG_FILES(Makefile)
73 AC_OUTPUT
74
75 - ``Makefile.am``:
76 ::
77
78 bin_PROGRAMS = hello
79 hello_SOURCES = hello.c
80
812. *Source the Cross-Toolchain Environment Setup File:* As described
82 earlier in the manual, installing the cross-toolchain creates a
83 cross-toolchain environment setup script in the directory that the
84 SDK was installed. Before you can use the tools to develop your
85 project, you must source this setup script. The script begins with
86 the string "environment-setup" and contains the machine architecture,
87 which is followed by the string "poky-linux". For this example, the
88 command sources a script from the default SDK installation directory
89 that uses the 32-bit Intel x86 Architecture and the 3.1.2 Yocto
90 Project release:
91 ::
92
93 $ source /opt/poky/3.1.2/environment-setup-i586-poky-linux
94
953. *Create the configure Script:* Use the ``autoreconf`` command to
96 generate the ``configure`` script.
97 ::
98
99 $ autoreconf
100
101 The ``autoreconf``
102 tool takes care of running the other Autotools such as ``aclocal``,
103 ``autoconf``, and ``automake``.
104
105 .. note::
106
107 If you get errors from
108 configure.ac
109 , which
110 autoreconf
111 runs, that indicate missing files, you can use the "-i" option,
112 which ensures missing auxiliary files are copied to the build
113 host.
114
1154. *Cross-Compile the Project:* This command compiles the project using
116 the cross-compiler. The
117 :term:`CONFIGURE_FLAGS`
118 environment variable provides the minimal arguments for GNU
119 configure:
120 ::
121
122 $ ./configure ${CONFIGURE_FLAGS}
123
124 For an Autotools-based
125 project, you can use the cross-toolchain by just passing the
126 appropriate host option to ``configure.sh``. The host option you use
127 is derived from the name of the environment setup script found in the
128 directory in which you installed the cross-toolchain. For example,
129 the host option for an ARM-based target that uses the GNU EABI is
130 ``armv5te-poky-linux-gnueabi``. You will notice that the name of the
131 script is ``environment-setup-armv5te-poky-linux-gnueabi``. Thus, the
132 following command works to update your project and rebuild it using
133 the appropriate cross-toolchain tools:
134 ::
135
136 $ ./configure --host=armv5te-poky-linux-gnueabi --with-libtool-sysroot=sysroot_dir
137
1385. *Make and Install the Project:* These two commands generate and
139 install the project into the destination directory:
140 ::
141
142 $ make
143 $ make install DESTDIR=./tmp
144
145 .. note::
146
147 To learn about environment variables established when you run the
148 cross-toolchain environment setup script and how they are used or
149 overridden when the Makefile, see the "
150 Makefile-Based Projects
151 " section.
152
153 This next command is a simple way to verify the installation of your
154 project. Running the command prints the architecture on which the
155 binary file can run. This architecture should be the same
156 architecture that the installed cross-toolchain supports.
157 ::
158
159 $ file ./tmp/usr/local/bin/hello
160
1616. *Execute Your Project:* To execute the project, you would need to run
162 it on your target hardware. If your target hardware happens to be
163 your build host, you could run the project as follows:
164 ::
165
166 $ ./tmp/usr/local/bin/hello
167
168 As expected, the project displays the "Hello World!" message.
169
170Makefile-Based Projects
171=======================
172
173Simple Makefile-based projects use and interact with the cross-toolchain
174environment variables established when you run the cross-toolchain
175environment setup script. The environment variables are subject to
176general ``make`` rules.
177
178This section presents a simple Makefile development flow and provides an
179example that lets you see how you can use cross-toolchain environment
180variables and Makefile variables during development.
181
182.. image:: figures/sdk-makefile-flow.png
183 :align: center
184
185The main point of this section is to explain the following three cases
186regarding variable behavior:
187
188- *Case 1 - No Variables Set in the Makefile Map to Equivalent
189 Environment Variables Set in the SDK Setup Script:* Because matching
190 variables are not specifically set in the ``Makefile``, the variables
191 retain their values based on the environment setup script.
192
193- *Case 2 - Variables Are Set in the Makefile that Map to Equivalent
194 Environment Variables from the SDK Setup Script:* Specifically
195 setting matching variables in the ``Makefile`` during the build
196 results in the environment settings of the variables being
197 overwritten. In this case, the variables you set in the ``Makefile``
198 are used.
199
200- *Case 3 - Variables Are Set Using the Command Line that Map to
201 Equivalent Environment Variables from the SDK Setup Script:*
202 Executing the ``Makefile`` from the command line results in the
203 environment variables being overwritten. In this case, the
204 command-line content is used.
205
206.. note::
207
208 Regardless of how you set your variables, if you use the "-e" option
209 with
210 make
211 , the variables from the SDK setup script take precedence:
212 ::
213
214 $ make -e target
215
216
217The remainder of this section presents a simple Makefile example that
218demonstrates these variable behaviors.
219
220In a new shell environment variables are not established for the SDK
221until you run the setup script. For example, the following commands show
222a null value for the compiler variable (i.e.
223:term:`CC`).
224::
225
226 $ echo ${CC}
227
228 $
229
230Running the
231SDK setup script for a 64-bit build host and an i586-tuned target
232architecture for a ``core-image-sato`` image using the current 3.1.2
233Yocto Project release and then echoing that variable shows the value
234established through the script:
235::
236
237 $ source /opt/poky/3.1.2/environment-setup-i586-poky-linux
238 $ echo ${CC}
239 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/3.1.2/sysroots/i586-poky-linux
240
241To illustrate variable use, work through this simple "Hello World!"
242example:
243
2441. *Create a Working Directory and Populate It:* Create a clean
245 directory for your project and then make that directory your working
246 location.
247 ::
248
249 $ mkdir $HOME/helloworld
250 $ cd $HOME/helloworld
251
252 After
253 setting up the directory, populate it with files needed for the flow.
254 You need a ``main.c`` file from which you call your function, a
255 ``module.h`` file to contain headers, and a ``module.c`` that defines
256 your function.
257
258 Create the three files as follows:
259
260 - ``main.c``:
261 ::
262
263 #include "module.h"
264 void sample_func();
265 int main()
266 {
267 sample_func();
268 return 0;
269 }
270
271 - ``module.h``:
272 ::
273
274 #include <stdio.h>
275 void sample_func();
276
277 - ``module.c``:
278 ::
279
280 #include "module.h"
281 void sample_func()
282 {
283 printf("Hello World!");
284 printf("\n");
285 }
286
2872. *Source the Cross-Toolchain Environment Setup File:* As described
288 earlier in the manual, installing the cross-toolchain creates a
289 cross-toolchain environment setup script in the directory that the
290 SDK was installed. Before you can use the tools to develop your
291 project, you must source this setup script. The script begins with
292 the string "environment-setup" and contains the machine architecture,
293 which is followed by the string "poky-linux". For this example, the
294 command sources a script from the default SDK installation directory
295 that uses the 32-bit Intel x86 Architecture and the DISTRO_NAME Yocto
296 Project release:
297 ::
298
299 $ source /opt/poky/DISTRO/environment-setup-i586-poky-linux
300
3013. *Create the Makefile:* For this example, the Makefile contains
302 two lines that can be used to set the ``CC`` variable. One line is
303 identical to the value that is set when you run the SDK environment
304 setup script, and the other line sets ``CC`` to "gcc", the default
305 GNU compiler on the build host:
306 ::
307
308 # CC=i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux
309 # CC="gcc"
310 all: main.o module.o
311 ${CC} main.o module.o -o target_bin
312 main.o: main.c module.h
313 ${CC} -I . -c main.c
314 module.o: module.c
315 module.h ${CC} -I . -c module.c
316 clean:
317 rm -rf *.o
318 rm target_bin
319
3204. *Make the Project:* Use the ``make`` command to create the binary
321 output file. Because variables are commented out in the Makefile, the
322 value used for ``CC`` is the value set when the SDK environment setup
323 file was run:
324 ::
325
326 $ make
327 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
328 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
329 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o target_bin
330
331 From the results of the previous command, you can see that
332 the compiler used was the compiler established through the ``CC``
333 variable defined in the setup script.
334
335 You can override the ``CC`` environment variable with the same
336 variable as set from the Makefile by uncommenting the line in the
337 Makefile and running ``make`` again.
338 ::
339
340 $ make clean
341 rm -rf *.o
342 rm target_bin
343 #
344 # Edit the Makefile by uncommenting the line that sets CC to "gcc"
345 #
346 $ make
347 gcc -I . -c main.c
348 gcc -I . -c module.c
349 gcc main.o module.o -o target_bin
350
351 As shown in the previous example, the
352 cross-toolchain compiler is not used. Rather, the default compiler is
353 used.
354
355 This next case shows how to override a variable by providing the
356 variable as part of the command line. Go into the Makefile and
357 re-insert the comment character so that running ``make`` uses the
358 established SDK compiler. However, when you run ``make``, use a
359 command-line argument to set ``CC`` to "gcc":
360 ::
361
362 $ make clean
363 rm -rf *.o
364 rm target_bin
365 #
366 # Edit the Makefile to comment out the line setting CC to "gcc"
367 #
368 $ make
369 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
370 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
371 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o target_bin
372 $ make clean
373 rm -rf *.o
374 rm target_bin
375 $ make CC="gcc"
376 gcc -I . -c main.c
377 gcc -I . -c module.c
378 gcc main.o module.o -o target_bin
379
380 In the previous case, the command-line argument overrides the SDK
381 environment variable.
382
383 In this last case, edit Makefile again to use the "gcc" compiler but
384 then use the "-e" option on the ``make`` command line:
385 ::
386
387 $ make clean
388 rm -rf *.o
389 rm target_bin
390 #
391 # Edit the Makefile to use "gcc"
392 #
393 $ make
394 gcc -I . -c main.c
395 gcc -I . -c module.c
396 gcc main.o module.o -o target_bin
397 $ make clean
398 rm -rf *.o
399 rm target_bin
400 $ make -e
401 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c main.c
402 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux -I . -c module.c
403 i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux main.o module.o -o target_bin
404
405 In the previous case, the "-e" option forces ``make`` to
406 use the SDK environment variables regardless of the values in the
407 Makefile.
408
4095. *Execute Your Project:* To execute the project (i.e. ``target_bin``),
410 use the following command:
411 ::
412
413 $ ./target_bin
414 Hello World!
415
416 .. note::
417
418 If you used the cross-toolchain compiler to build
419 target_bin
420 and your build host differs in architecture from that of the
421 target machine, you need to run your project on the target device.
422
423 As expected, the project displays the "Hello World!" message.
diff --git a/documentation/sdk-manual/sdk-working-projects.xml b/documentation/sdk-manual/sdk-working-projects.xml
index 521271d54c..070d903c73 100644
--- a/documentation/sdk-manual/sdk-working-projects.xml
+++ b/documentation/sdk-manual/sdk-working-projects.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='sdk-working-projects'> 6<chapter id='sdk-working-projects'>
6 7
diff --git a/documentation/sphinx-static/YoctoProject_Logo_RGB.jpg b/documentation/sphinx-static/YoctoProject_Logo_RGB.jpg
new file mode 100644
index 0000000000..8ab47d49f7
--- /dev/null
+++ b/documentation/sphinx-static/YoctoProject_Logo_RGB.jpg
Binary files differ
diff --git a/documentation/sphinx-static/switchers.js b/documentation/sphinx-static/switchers.js
new file mode 100644
index 0000000000..32113cfa96
--- /dev/null
+++ b/documentation/sphinx-static/switchers.js
@@ -0,0 +1,233 @@
1(function() {
2 'use strict';
3
4 var all_versions = {
5 'dev': 'dev (3.2)',
6 '3.1.2': '3.1.2',
7 '3.0.3': '3.0.3',
8 '2.7.4': '2.7.4',
9 };
10
11 var all_doctypes = {
12 'single': 'Individual Webpages',
13 'mega': "All-in-one 'Mega' Manual",
14 };
15
16 // Simple version comparision
17 // Return 1 if a > b
18 // Return -1 if a < b
19 // Return 0 if a == b
20 function ver_compare(a, b) {
21 if (a == "dev") {
22 return 1;
23 }
24
25 if (a === b) {
26 return 0;
27 }
28
29 var a_components = a.split(".");
30 var b_components = b.split(".");
31
32 var len = Math.min(a_components.length, b_components.length);
33
34 // loop while the components are equal
35 for (var i = 0; i < len; i++) {
36 // A bigger than B
37 if (parseInt(a_components[i]) > parseInt(b_components[i])) {
38 return 1;
39 }
40
41 // B bigger than A
42 if (parseInt(a_components[i]) < parseInt(b_components[i])) {
43 return -1;
44 }
45 }
46
47 // If one's a prefix of the other, the longer one is greater.
48 if (a_components.length > b_components.length) {
49 return 1;
50 }
51
52 if (a_components.length < b_components.length) {
53 return -1;
54 }
55
56 // Otherwise they are the same.
57 return 0;
58 }
59
60 function build_version_select(current_series, current_version) {
61 var buf = ['<select>'];
62
63 $.each(all_versions, function(version, title) {
64 var series = version.substr(0, 3);
65 if (series == current_series) {
66 if (version == current_version)
67 buf.push('<option value="' + version + '" selected="selected">' + title + '</option>');
68 else
69 buf.push('<option value="' + version + '">' + title + '</option>');
70
71 if (version != current_version)
72 buf.push('<option value="' + current_version + '" selected="selected">' + current_version + '</option>');
73 } else {
74 buf.push('<option value="' + version + '">' + title + '</option>');
75 }
76 });
77
78 buf.push('</select>');
79 return buf.join('');
80 }
81
82 function build_doctype_select(current_doctype) {
83 var buf = ['<select>'];
84
85 $.each(all_doctypes, function(doctype, title) {
86 if (doctype == current_doctype)
87 buf.push('<option value="' + doctype + '" selected="selected">' +
88 all_doctypes[current_doctype] + '</option>');
89 else
90 buf.push('<option value="' + doctype + '">' + title + '</option>');
91 });
92 if (!(current_doctype in all_doctypes)) {
93 // In case we're browsing a doctype that is not yet in all_doctypes.
94 buf.push('<option value="' + current_doctype + '" selected="selected">' +
95 current_doctype + '</option>');
96 all_doctypes[current_doctype] = current_doctype;
97 }
98 buf.push('</select>');
99 return buf.join('');
100 }
101
102 function navigate_to_first_existing(urls) {
103 // Navigate to the first existing URL in urls.
104 var url = urls.shift();
105
106 // Web browsers won't redirect file:// urls to file urls using ajax but
107 // its useful for local testing
108 if (url.startsWith("file://")) {
109 window.location.href = url;
110 return;
111 }
112
113 if (urls.length == 0) {
114 window.location.href = url;
115 return;
116 }
117 $.ajax({
118 url: url,
119 success: function() {
120 window.location.href = url;
121 },
122 error: function() {
123 navigate_to_first_existing(urls);
124 }
125 });
126 }
127
128 function get_docroot_url() {
129 var url = window.location.href;
130 var root = DOCUMENTATION_OPTIONS.URL_ROOT;
131
132 var urlarray = url.split('/');
133 // Trim off anything after '/'
134 urlarray.pop();
135 var depth = (root.match(/\.\.\//g) || []).length;
136 for (var i = 0; i < depth; i++) {
137 urlarray.pop();
138 }
139
140 return urlarray.join('/') + '/';
141 }
142
143 function on_version_switch() {
144 var selected_version = $(this).children('option:selected').attr('value');
145 var url = window.location.href;
146 var current_version = DOCUMENTATION_OPTIONS.VERSION;
147 var docroot = get_docroot_url()
148
149 var new_versionpath = selected_version + '/';
150 if (selected_version == "dev")
151 new_versionpath = '';
152
153 // dev versions have no version prefix
154 if (current_version == "dev") {
155 var new_url = docroot + new_versionpath + url.replace(docroot, "");
156 var fallback_url = docroot + new_versionpath;
157 } else {
158 var new_url = url.replace('/' + current_version + '/', '/' + new_versionpath);
159 var fallback_url = new_url.replace(url.replace(docroot, ""), "");
160 }
161
162 console.log(get_docroot_url())
163 console.log(url + " to url " + new_url);
164 console.log(url + " to fallback " + fallback_url);
165
166 if (new_url != url) {
167 navigate_to_first_existing([
168 new_url,
169 fallback_url,
170 'https://www.yoctoproject.org/docs/',
171 ]);
172 }
173 }
174
175 function on_doctype_switch() {
176 var selected_doctype = $(this).children('option:selected').attr('value');
177 var url = window.location.href;
178 if (selected_doctype == 'mega') {
179 var docroot = get_docroot_url()
180 var current_version = DOCUMENTATION_OPTIONS.VERSION;
181 // Assume manuals before 3.2 are using old docbook mega-manual
182 if (ver_compare(current_version, "3.2") < 0) {
183 var new_url = docroot + "mega-manual/mega-manual.html";
184 } else {
185 var new_url = docroot + "singleindex.html";
186 }
187 } else {
188 var new_url = url.replace("singleindex.html", "index.html")
189 }
190
191 if (new_url != url) {
192 navigate_to_first_existing([
193 new_url,
194 'https://www.yoctoproject.org/docs/',
195 ]);
196 }
197 }
198
199 // Returns the current doctype based upon the url
200 function doctype_segment_from_url(url) {
201 if (url.includes("singleindex") || url.includes("mega-manual"))
202 return "mega";
203 return "single";
204 }
205
206 $(document).ready(function() {
207 var release = DOCUMENTATION_OPTIONS.VERSION;
208 var current_doctype = doctype_segment_from_url(window.location.href);
209 var current_series = release.substr(0, 3);
210 var version_select = build_version_select(current_series, release);
211
212 $('.version_switcher_placeholder').html(version_select);
213 $('.version_switcher_placeholder select').bind('change', on_version_switch);
214
215 var doctype_select = build_doctype_select(current_doctype);
216
217 $('.doctype_switcher_placeholder').html(doctype_select);
218 $('.doctype_switcher_placeholder select').bind('change', on_doctype_switch);
219
220 if (ver_compare(release, "3.1") < 0) {
221 $('#outdated-warning').html('Version ' + release + ' of the project is now considered obsolete, please select and use a more recent version');
222 $('#outdated-warning').css('padding', '.5em');
223 } else if (release != "dev") {
224 $.each(all_versions, function(version, title) {
225 var series = version.substr(0, 3);
226 if (series == current_series && version != release) {
227 $('#outdated-warning').html('This document is for outdated version ' + release + ', you should select the latest release version in this series, ' + version + '.');
228 $('#outdated-warning').css('padding', '.5em');
229 }
230 });
231 }
232 });
233})();
diff --git a/documentation/sphinx-static/theme_overrides.css b/documentation/sphinx-static/theme_overrides.css
new file mode 100644
index 0000000000..55da38a2b8
--- /dev/null
+++ b/documentation/sphinx-static/theme_overrides.css
@@ -0,0 +1,164 @@
1/*
2 SPDX-License-Identifier: CC-BY-2.0-UK
3*/
4
5body {
6 font-family: Verdana, Sans, sans-serif;
7 margin: 0em auto;
8 color: #333;
9}
10
11h1,h2,h3,h4,h5,h6,h7 {
12 font-family: Arial, Sans;
13 color: #00557D;
14 clear: both;
15}
16
17h1 {
18 font-size: 2em;
19 text-align: left;
20 padding: 0em 0em 0em 0em;
21 margin: 2em 0em 0em 0em;
22}
23
24h2.subtitle {
25 margin: 0.10em 0em 3.0em 0em;
26 padding: 0em 0em 0em 0em;
27 font-size: 1.8em;
28 padding-left: 20%;
29 font-weight: normal;
30 font-style: italic;
31}
32
33h2 {
34 margin: 2em 0em 0.66em 0em;
35 padding: 0.5em 0em 0em 0em;
36 font-size: 1.5em;
37 font-weight: bold;
38}
39
40h3.subtitle {
41 margin: 0em 0em 1em 0em;
42 padding: 0em 0em 0em 0em;
43 font-size: 142.14%;
44 text-align: right;
45}
46
47h3 {
48 margin: 1em 0em 0.5em 0em;
49 padding: 1em 0em 0em 0em;
50 font-size: 140%;
51 font-weight: bold;
52}
53
54h4 {
55 margin: 1em 0em 0.5em 0em;
56 padding: 1em 0em 0em 0em;
57 font-size: 120%;
58 font-weight: bold;
59}
60
61h5 {
62 margin: 1em 0em 0.5em 0em;
63 padding: 1em 0em 0em 0em;
64 font-size: 110%;
65 font-weight: bold;
66}
67
68h6 {
69 margin: 1em 0em 0em 0em;
70 padding: 1em 0em 0em 0em;
71 font-size: 110%;
72 font-weight: bold;
73}
74
75em {
76 font-weight: bold;
77}
78
79.pre {
80 font-size: medium;
81 font-family: Courier, monospace;
82}
83
84.wy-nav-content a {
85 text-decoration: underline;
86 color: #444;
87 background: transparent;
88}
89
90.wy-nav-content a:hover {
91 text-decoration: underline;
92 background-color: #dedede;
93}
94
95.wy-nav-content a:visited {
96 color: #444;
97}
98
99[alt='Permalink'] { color: #eee; }
100[alt='Permalink']:hover { color: black; }
101
102@media screen {
103 /* content column
104 *
105 * RTD theme's default is 800px as max width for the content, but we have
106 * tables with tons of columns, which need the full width of the view-port.
107 */
108
109 .wy-nav-content{max-width: none; }
110
111 /* inline literal: drop the borderbox, padding and red color */
112 code, .rst-content tt, .rst-content code {
113 color: inherit;
114 border: none;
115 padding: unset;
116 background: inherit;
117 font-size: 85%;
118 }
119
120 .rst-content tt.literal,.rst-content tt.literal,.rst-content code.literal {
121 color: inherit;
122 }
123
124 /* Admonition should be gray, not blue or green */
125 .rst-content .note .admonition-title,
126 .rst-content .tip .admonition-title,
127 .rst-content .warning .admonition-title,
128 .rst-content .caution .admonition-title,
129 .rst-content .admonition-tying-it-together .admonition-title,
130 .rst-content .important .admonition-title {
131 background: #f0f0f2;
132 color: #00557D;
133
134 }
135
136 .rst-content .note,
137 .rst-content .tip,
138 .rst-content .important,
139 .rst-content .warning,
140 .rst-content .admonition-tying-it-together,
141 .rst-content .caution {
142 background: #f0f0f2;
143 }
144
145 /* Remove the icon in front of note/tip element, and before the logo */
146 .icon-home:before, .rst-content .admonition-title:before {
147 display: none
148 }
149
150 /* a custom informalexample container is used in some doc */
151 .informalexample {
152 border: 1px solid;
153 border-color: #aaa;
154 margin: 1em 0em;
155 padding: 1em;
156 page-break-inside: avoid;
157 }
158
159 /* Remove the blue background in the top left corner, around the logo */
160 .wy-side-nav-search {
161 background: inherit;
162 }
163
164}
diff --git a/documentation/sphinx/yocto-vars.py b/documentation/sphinx/yocto-vars.py
new file mode 100644
index 0000000000..8083d7da19
--- /dev/null
+++ b/documentation/sphinx/yocto-vars.py
@@ -0,0 +1,47 @@
1#!/usr/bin/env python
2import re
3import sys
4
5import sphinx
6from sphinx.application import Sphinx
7
8# This extension uses pyyaml, report an explicit
9# error message if it's not installed
10try:
11 import yaml
12except ImportError:
13 sys.stderr.write("The Yocto Project Sphinx documentation requires PyYAML.\
14 \nPlease make sure to install pyyaml python package.\n")
15 sys.exit(1)
16
17__version__ = '1.0'
18
19# Variables substitutions. Uses {VAR} subst using variables defined in poky.yaml
20# Each .rst file is processed after source-read event (subst_vars_replace runs once per file)
21subst_vars = {}
22
23def subst_vars_replace(app: Sphinx, docname, source):
24 result = source[0]
25 for k in subst_vars:
26 result = result.replace("&"+k+";", subst_vars[k])
27 source[0] = result
28
29PATTERN = re.compile(r'&(.*?);')
30def expand(val, src):
31 return PATTERN.sub(lambda m: expand(src.get(m.group(1), ''), src), val)
32
33def setup(app: Sphinx):
34 #FIXME: if poky.yaml changes, files are not reprocessed.
35 with open("poky.yaml") as file:
36 subst_vars.update(yaml.load(file, Loader=yaml.FullLoader))
37
38 for k in subst_vars:
39 subst_vars[k] = expand(subst_vars[k], subst_vars)
40
41 app.connect('source-read', subst_vars_replace)
42
43 return dict(
44 version = __version__,
45 parallel_read_safe = True,
46 parallel_write_safe = True
47 )
diff --git a/documentation/test-manual/figures/ab-test-cluster.png b/documentation/test-manual/figures/ab-test-cluster.png
new file mode 100644
index 0000000000..6a6a7882b4
--- /dev/null
+++ b/documentation/test-manual/figures/ab-test-cluster.png
Binary files differ
diff --git a/documentation/test-manual/figures/test-manual-title.png b/documentation/test-manual/figures/test-manual-title.png
new file mode 100644
index 0000000000..c709cb9d09
--- /dev/null
+++ b/documentation/test-manual/figures/test-manual-title.png
Binary files differ
diff --git a/documentation/test-manual/history.rst b/documentation/test-manual/history.rst
new file mode 100644
index 0000000000..76d43091a5
--- /dev/null
+++ b/documentation/test-manual/history.rst
@@ -0,0 +1,16 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 3.2
15 - October 2020
16 - The initial document released with the Yocto Project 3.2 Release
diff --git a/documentation/test-manual/test-manual-customization.xsl b/documentation/test-manual/test-manual-customization.xsl
new file mode 100644
index 0000000000..17bf57c2d1
--- /dev/null
+++ b/documentation/test-manual/test-manual-customization.xsl
@@ -0,0 +1,29 @@
1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
5
6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
7
8<!--
9
10 <xsl:import href="../template/1.76.1/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
11
12 <xsl:import href="http://docbook.sourceforge.net/release/xsl/1.76.1/xhtml/docbook.xsl" />
13
14-->
15
16 <xsl:include href="../template/permalinks.xsl"/>
17 <xsl:include href="../template/section.title.xsl"/>
18 <xsl:include href="../template/component.title.xsl"/>
19 <xsl:include href="../template/division.title.xsl"/>
20 <xsl:include href="../template/formal.object.heading.xsl"/>
21
22 <xsl:param name="html.stylesheet" select="'test-manual-style.css'" />
23 <xsl:param name="chapter.autolabel" select="1" />
24 <xsl:param name="appendix.autolabel" select="A" />
25 <xsl:param name="section.autolabel" select="1" />
26 <xsl:param name="section.label.includes.component.label" select="1" />
27 <xsl:param name="generate.id.attributes" select="1" />
28
29</xsl:stylesheet>
diff --git a/documentation/test-manual/test-manual-intro.rst b/documentation/test-manual/test-manual-intro.rst
new file mode 100644
index 0000000000..53ad650b35
--- /dev/null
+++ b/documentation/test-manual/test-manual-intro.rst
@@ -0,0 +1,550 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*****************************************
4The Yocto Project Test Environment Manual
5*****************************************
6
7.. _test-welcome:
8
9Welcome
10=======
11
12Welcome to the Yocto Project Test Environment Manual! This manual is a
13work in progress. The manual contains information about the testing
14environment used by the Yocto Project to make sure each major and minor
15release works as intended. All the project's testing infrastructure and
16processes are publicly visible and available so that the community can
17see what testing is being performed, how it's being done and the current
18status of the tests and the project at any given time. It is intended
19that Other organizations can leverage off the process and testing
20environment used by the Yocto Project to create their own automated,
21production test environment, building upon the foundations from the
22project core.
23
24Currently, the Yocto Project Test Environment Manual has no projected
25release date. This manual is a work-in-progress and is being initially
26loaded with information from the README files and notes from key
27engineers:
28
29- *yocto-autobuilder2:* This
30 :yocto_git:`README.md </cgit.cgi/yocto-autobuilder2/tree/README.md>`
31 is the main README which detials how to set up the Yocto Project
32 Autobuilder. The ``yocto-autobuilder2`` repository represents the
33 Yocto Project's console UI plugin to Buildbot and the configuration
34 necessary to configure Buildbot to perform the testing the project
35 requires.
36
37- *yocto-autobuilder-helper:* This :yocto_git:`README </cgit.cgi/yocto-autobuilder-helper/tree/README/>`
38 and repository contains Yocto Project Autobuilder Helper scripts and
39 configuration. The ``yocto-autobuilder-helper`` repository contains
40 the "glue" logic that defines which tests to run and how to run them.
41 As a result, it can be used by any Continuous Improvement (CI) system
42 to run builds, support getting the correct code revisions, configure
43 builds and layers, run builds, and collect results. The code is
44 independent of any CI system, which means the code can work `Buildbot <https://docs.buildbot.net/0.9.15.post1/>`__,
45 Jenkins, or others. This repository has a branch per release of the
46 project defining the tests to run on a per release basis.
47
48.. _test-yocto-project-autobuilder-overview:
49
50Yocto Project Autobuilder Overview
51==================================
52
53The Yocto Project Autobuilder collectively refers to the software,
54tools, scripts, and procedures used by the Yocto Project to test
55released software across supported hardware in an automated and regular
56fashion. Basically, during the development of a Yocto Project release,
57the Autobuilder tests if things work. The Autobuilder builds all test
58targets and runs all the tests.
59
60The Yocto Project uses now uses standard upstream
61`Buildbot <https://docs.buildbot.net/0.9.15.post1/>`__ (version 9) to
62drive its integration and testing. Buildbot Nine has a plug-in interface
63that the Yocto Project customizes using code from the
64``yocto-autobuilder2`` repository, adding its own console UI plugin. The
65resulting UI plug-in allows you to visualize builds in a way suited to
66the project's needs.
67
68A ``helper`` layer provides configuration and job management through
69scripts found in the ``yocto-autobuilder-helper`` repository. The
70``helper`` layer contains the bulk of the build configuration
71information and is release-specific, which makes it highly customizable
72on a per-project basis. The layer is CI system-agnostic and contains a
73number of Helper scripts that can generate build configurations from
74simple JSON files.
75
76.. note::
77
78 The project uses Buildbot for historical reasons but also because
79 many of the project developers have knowledge of python. It is
80 possible to use the outer layers from another Continuous Integration
81 (CI) system such as
82 `Jenkins <https://en.wikipedia.org/wiki/Jenkins_(software)>`__
83 instead of Buildbot.
84
85The following figure shows the Yocto Project Autobuilder stack with a
86topology that includes a controller and a cluster of workers:
87
88.. image:: figures/ab-test-cluster.png
89 :align: center
90
91.. _test-project-tests:
92
93Yocto Project Tests - Types of Testing Overview
94===============================================
95
96The Autobuilder tests different elements of the project by using
97thefollowing types of tests:
98
99- *Build Testing:* Tests whether specific configurations build by
100 varying :term:`MACHINE`,
101 :term:`DISTRO`, other configuration
102 options, and the specific target images being built (or world). Used
103 to trigger builds of all the different test configurations on the
104 Autobuilder. Builds usually cover many different targets for
105 different architectures, machines, and distributions, as well as
106 different configurations, such as different init systems. The
107 Autobuilder tests literally hundreds of configurations and targets.
108
109 - *Sanity Checks During the Build Process:* Tests initiated through
110 the :ref:`insane <ref-classes-insane>`
111 class. These checks ensure the output of the builds are correct.
112 For example, does the ELF architecture in the generated binaries
113 match the target system? ARM binaries would not work in a MIPS
114 system!
115
116- *Build Performance Testing:* Tests whether or not commonly used steps
117 during builds work efficiently and avoid regressions. Tests to time
118 commonly used usage scenarios are run through ``oe-build-perf-test``.
119 These tests are run on isolated machines so that the time
120 measurements of the tests are accurate and no other processes
121 interfere with the timing results. The project currently tests
122 performance on two different distributions, Fedora and Ubuntu, to
123 ensure we have no single point of failure and can ensure the
124 different distros work effectively.
125
126- *eSDK Testing:* Image tests initiated through the following command::
127
128 $ bitbake image -c testsdkext
129
130 The tests utilize the ``testsdkext`` class and the ``do_testsdkext`` task.
131
132- *Feature Testing:* Various scenario-based tests are run through the
133 :ref:`OpenEmbedded Self test (oe-selftest) <ref-manual/ref-release-process:Testing and Quality Assurance>`. We test oe-selftest on each of the main distrubutions
134 we support.
135
136- *Image Testing:* Image tests initiated through the following command::
137
138 $ bitbake image -c testimage
139
140 The tests utilize the :ref:`testimage* <ref-classes-testimage*>`
141 classes and the :ref:`ref-tasks-testimage` task.
142
143- *Layer Testing:* The Autobuilder has the possibility to test whether
144 specific layers work with the test of the system. The layers tested
145 may be selected by members of the project. Some key community layers
146 are also tested periodically.
147
148- *Package Testing:* A Package Test (ptest) runs tests against packages
149 built by the OpenEmbedded build system on the target machine. See the
150 :ref:`Testing Packages With
151 ptest <dev-manual/dev-manual-common-tasks:Testing Packages With ptest>` section
152 in the Yocto Project Development Tasks Manual and the
153 ":yocto_wiki:`Ptest </wiki/Ptest>`" Wiki page for more
154 information on Ptest.
155
156- *SDK Testing:* Image tests initiated through the following command::
157
158 $ bitbake image -c testsdk
159
160 The tests utilize the :ref:`testsdk <ref-classes-testsdk>` class and
161 the ``do_testsdk`` task.
162
163- *Unit Testing:* Unit tests on various components of the system run
164 through :ref:`bitbake-selftest <ref-manual/ref-release-process:Testing and Quality Assurance>` and
165 :ref:`oe-selftest <ref-manual/ref-release-process:Testing and Quality Assurance>`.
166
167- *Automatic Upgrade Helper:* This target tests whether new versions of
168 software are available and whether we can automatically upgrade to
169 those new versions. If so, this target emails the maintainers with a
170 patch to let them know this is possible.
171
172.. _test-test-mapping:
173
174How Tests Map to Areas of Code
175==============================
176
177Tests map into the codebase as follows:
178
179- *bitbake-selftest:*
180
181 These tests are self-contained and test BitBake as well as its APIs,
182 which include the fetchers. The tests are located in
183 ``bitbake/lib/*/tests``.
184
185 From within the BitBake repository, run the following::
186
187 $ bitbake-selftest
188
189 To skip tests that access the Internet, use the ``BB_SKIP_NETTEST``
190 variable when running "bitbake-selftest" as follows::
191
192 $ BB_SKIP_NETTEST=yes bitbake-selftest
193
194 The default output is quiet and just prints a summary of what was
195 run. To see more information, there is a verbose option::
196
197 $ bitbake-selftest -v
198
199 Use this option when you wish to skip tests that access the network,
200 which are mostly necessary to test the fetcher modules. To specify
201 individual test modules to run, append the test module name to the
202 "bitbake-selftest" command. For example, to specify the tests for the
203 bb.data.module, run::
204
205 $ bitbake-selftest bb.test.data.module
206
207 You can also specify individual tests by defining the full name and module
208 plus the class path of the test, for example::
209
210 $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override
211
212 The tests are based on `Python
213 unittest <https://docs.python.org/3/library/unittest.html>`__.
214
215- *oe-selftest:*
216
217 - These tests use OE to test the workflows, which include testing
218 specific features, behaviors of tasks, and API unit tests.
219
220 - The tests can take advantage of parallelism through the "-j"
221 option, which can specify a number of threads to spread the tests
222 across. Note that all tests from a given class of tests will run
223 in the same thread. To parallelize large numbers of tests you can
224 split the class into multiple units.
225
226 - The tests are based on Python unittest.
227
228 - The code for the tests resides in
229 ``meta/lib/oeqa/selftest/cases/``.
230
231 - To run all the tests, enter the following command::
232
233 $ oe-selftest -a
234
235 - To run a specific test, use the following command form where
236 testname is the name of the specific test::
237
238 $ oe-selftest -r <testname>
239
240 For example, the following command would run the tinfoil
241 getVar API test::
242
243 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
244
245 It is also possible to run a set
246 of tests. For example the following command will run all of the
247 tinfoil tests::
248
249 $ oe-selftest -r tinfoil
250
251- *testimage:*
252
253 - These tests build an image, boot it, and run tests against the
254 image's content.
255
256 - The code for these tests resides in ``meta/lib/oeqa/runtime/cases/``.
257
258 - You need to set the :term:`IMAGE_CLASSES` variable as follows::
259
260 IMAGE_CLASSES += "testimage"
261
262 - Run the tests using the following command form::
263
264 $ bitbake image -c testimage
265
266- *testsdk:*
267
268 - These tests build an SDK, install it, and then run tests against
269 that SDK.
270
271 - The code for these tests resides in ``meta/lib/oeqa/sdk/cases/``.
272
273 - Run the test using the following command form::
274
275 $ bitbake image -c testsdk
276
277- *testsdk_ext:*
278
279 - These tests build an extended SDK (eSDK), install that eSDK, and
280 run tests against the eSDK.
281
282 - The code for these tests resides in ``meta/lib/oeqa/esdk``.
283
284 - To run the tests, use the following command form::
285
286 $ bitbake image -c testsdkext
287
288- *oe-build-perf-test:*
289
290 - These tests run through commonly used usage scenarios and measure
291 the performance times.
292
293 - The code for these tests resides in ``meta/lib/oeqa/buildperf``.
294
295 - To run the tests, use the following command form::
296
297 $ oe-build-perf-test <options>
298
299 The command takes a number of options,
300 such as where to place the test results. The Autobuilder Helper
301 Scripts include the ``build-perf-test-wrapper`` script with
302 examples of how to use the oe-build-perf-test from the command
303 line.
304
305 Use the ``oe-git-archive`` command to store test results into a
306 Git repository.
307
308 Use the ``oe-build-perf-report`` command to generate text reports
309 and HTML reports with graphs of the performance data. For
310 examples, see
311 :yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html`
312 and
313 :yocto_dl:`/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt`.
314
315 - The tests are contained in ``lib/oeqa/buildperf/test_basic.py``.
316
317Test Examples
318=============
319
320This section provides example tests for each of the tests listed in the
321:ref:`test-manual/test-manual-intro:How Tests Map to Areas of Code` section.
322
323For oeqa tests, testcases for each area reside in the main test
324directory at ``meta/lib/oeqa/selftest/cases`` directory.
325
326For oe-selftest. bitbake testcases reside in the ``lib/bb/tests/``
327directory.
328
329.. _bitbake-selftest-example:
330
331``bitbake-selftest``
332--------------------
333
334A simple test example from ``lib/bb/tests/data.py`` is::
335
336 class DataExpansions(unittest.TestCase):
337 def setUp(self):
338 self.d = bb.data.init()
339 self.d["foo"] = "value_of_foo"
340 self.d["bar"] = "value_of_bar"
341 self.d["value_of_foo"] = "value_of_'value_of_foo'"
342
343 def test_one_var(self):
344 val = self.d.expand("${foo}")
345 self.assertEqual(str(val), "value_of_foo")
346
347In this example, a ``DataExpansions`` class of tests is created,
348derived from standard python unittest. The class has a common ``setUp``
349function which is shared by all the tests in the class. A simple test is
350then added to test that when a variable is expanded, the correct value
351is found.
352
353Bitbake selftests are straightforward python unittest. Refer to the
354Python unittest documentation for additional information on writing
355these tests at: https://docs.python.org/3/library/unittest.html.
356
357.. _oe-selftest-example:
358
359``oe-selftest``
360---------------
361
362These tests are more complex due to the setup required behind the scenes
363for full builds. Rather than directly using Python's unittest, the code
364wraps most of the standard objects. The tests can be simple, such as
365testing a command from within the OE build environment using the
366following example::
367
368 class BitbakeLayers(OESelftestTestCase):
369 def test_bitbakelayers_showcrossdepends(self):
370 result = runCmd('bitbake-layers show-cross-depends')
371 self.assertTrue('aspell' in result.output, msg = "No dependencies were shown. bitbake-layers show-cross-depends output: %s"% result.output)
372
373This example, taken from ``meta/lib/oeqa/selftest/cases/bblayers.py``,
374creates a testcase from the ``OESelftestTestCase`` class, derived
375from ``unittest.TestCase``, which runs the ``bitbake-layers`` command
376and checks the output to ensure it contains something we know should be
377here.
378
379The ``oeqa.utils.commands`` module contains Helpers which can assist
380with common tasks, including:
381
382- *Obtaining the value of a bitbake variable:* Use
383 ``oeqa.utils.commands.get_bb_var()`` or use
384 ``oeqa.utils.commands.get_bb_vars()`` for more than one variable
385
386- *Running a bitbake invocation for a build:* Use
387 ``oeqa.utils.commands.bitbake()``
388
389- *Running a command:* Use ``oeqa.utils.commandsrunCmd()``
390
391There is also a ``oeqa.utils.commands.runqemu()`` function for launching
392the ``runqemu`` command for testing things within a running, virtualized
393image.
394
395You can run these tests in parallel. Parallelism works per test class,
396so tests within a given test class should always run in the same build,
397while tests in different classes or modules may be split into different
398builds. There is no data store available for these tests since the tests
399launch the ``bitbake`` command and exist outside of its context. As a
400result, common bitbake library functions (bb.\*) are also unavailable.
401
402.. _testimage-example:
403
404``testimage``
405-------------
406
407These tests are run once an image is up and running, either on target
408hardware or under QEMU. As a result, they are assumed to be running in a
409target image environment, as opposed to a host build environment. A
410simple example from ``meta/lib/oeqa/runtime/cases/python.py`` contains
411the following::
412
413 class PythonTest(OERuntimeTestCase):
414 @OETestDepends(['ssh.SSHTest.test_ssh'])
415 @OEHasPackage(['python3-core'])
416 def test_python3(self):
417 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
418 status, output = self.target.run(cmd)
419 msg = 'Exit status was not 0. Output: %s' % output
420 self.assertEqual(status, 0, msg=msg)
421
422In this example, the ``OERuntimeTestCase`` class wraps
423``unittest.TestCase``. Within the test, ``self.target`` represents the
424target system, where commands can be run on it using the ``run()``
425method.
426
427To ensure certain test or package dependencies are met, you can use the
428``OETestDepends`` and ``OEHasPackage`` decorators. For example, the test
429in this example would only make sense if python3-core is installed in
430the image.
431
432.. _testsdk_ext-example:
433
434``testsdk_ext``
435---------------
436
437These tests are run against built extensible SDKs (eSDKs). The tests can
438assume that the eSDK environment has already been setup. An example from
439``meta/lib/oeqa/sdk/cases/devtool.py`` contains the following::
440
441 class DevtoolTest(OESDKExtTestCase):
442 @classmethod def setUpClass(cls):
443 myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp")
444 cls.myapp_dst = os.path.join(cls.tc.sdk_dir, "myapp")
445 shutil.copytree(myapp_src, cls.myapp_dst)
446 subprocess.check_output(['git', 'init', '.'], cwd=cls.myapp_dst)
447 subprocess.check_output(['git', 'add', '.'], cwd=cls.myapp_dst)
448 subprocess.check_output(['git', 'commit', '-m', "'test commit'"], cwd=cls.myapp_dst)
449
450 @classmethod
451 def tearDownClass(cls):
452 shutil.rmtree(cls.myapp_dst)
453 def _test_devtool_build(self, directory):
454 self._run('devtool add myapp %s' % directory)
455 try:
456 self._run('devtool build myapp')
457 finally:
458 self._run('devtool reset myapp')
459 def test_devtool_build_make(self):
460 self._test_devtool_build(self.myapp_dst)
461
462In this example, the ``devtool``
463command is tested to see whether a sample application can be built with
464the ``devtool build`` command within the eSDK.
465
466.. _testsdk-example:
467
468``testsdk``
469-----------
470
471These tests are run against built SDKs. The tests can assume that an SDK
472has already been extracted and its environment file has been sourced. A
473simple example from ``meta/lib/oeqa/sdk/cases/python2.py`` contains the
474following::
475
476 class Python3Test(OESDKTestCase):
477 def setUp(self):
478 if not (self.tc.hasHostPackage("nativesdk-python3-core") or
479 self.tc.hasHostPackage("python3-core-native")):
480 raise unittest.SkipTest("No python3 package in the SDK")
481
482 def test_python3(self):
483 cmd = "python3 -c \\"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
484 output = self._run(cmd)
485 self.assertEqual(output, "Hello, world\n")
486
487In this example, if nativesdk-python3-core has been installed into the SDK, the code runs
488the python3 interpreter with a basic command to check it is working
489correctly. The test would only run if python3 is installed in the SDK.
490
491.. _oe-build-perf-test-example:
492
493``oe-build-perf-test``
494----------------------
495
496The performance tests usually measure how long operations take and the
497resource utilisation as that happens. An example from
498``meta/lib/oeqa/buildperf/test_basic.py`` contains the following::
499
500 class Test3(BuildPerfTestCase):
501 def test3(self):
502 """Bitbake parsing (bitbake -p)"""
503 # Drop all caches and parse
504 self.rm_cache()
505 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
506 self.measure_cmd_resources(['bitbake', '-p'], 'parse_1',
507 'bitbake -p (no caches)')
508 # Drop tmp/cache
509 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
510 self.measure_cmd_resources(['bitbake', '-p'], 'parse_2',
511 'bitbake -p (no tmp/cache)')
512 # Parse with fully cached data
513 self.measure_cmd_resources(['bitbake', '-p'], 'parse_3',
514 'bitbake -p (cached)')
515
516This example shows how three specific parsing timings are
517measured, with and without various caches, to show how BitBake's parsing
518performance trends over time.
519
520.. _test-writing-considerations:
521
522Considerations When Writing Tests
523=================================
524
525When writing good tests, there are several things to keep in mind. Since
526things running on the Autobuilder are accessed concurrently by multiple
527workers, consider the following:
528
529**Running "cleanall" is not permitted.**
530
531This can delete files from DL_DIR which would potentially break other
532builds running in parallel. If this is required, DL_DIR must be set to
533an isolated directory.
534
535**Running "cleansstate" is not permitted.**
536
537This can delete files from SSTATE_DIR which would potentially break
538other builds running in parallel. If this is required, SSTATE_DIR must
539be set to an isolated directory. Alternatively, you can use the "-f"
540option with the ``bitbake`` command to "taint" tasks by changing the
541sstate checksums to ensure sstate cache items will not be reused.
542
543**Tests should not change the metadata.**
544
545This is particularly true for oe-selftests since these can run in
546parallel and changing metadata leads to changing checksums, which
547confuses BitBake while running in parallel. If this is necessary, copy
548layers to a temporary location and modify them. Some tests need to
549change metadata, such as the devtool tests. To prevent the metadate from
550changes, set up temporary copies of that data first.
diff --git a/documentation/test-manual/test-manual-intro.xml b/documentation/test-manual/test-manual-intro.xml
new file mode 100644
index 0000000000..0cdbee4d8e
--- /dev/null
+++ b/documentation/test-manual/test-manual-intro.xml
@@ -0,0 +1,624 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-intro'>
7
8<title>The Yocto Project Test Environment Manual</title>
9 <section id='test-welcome'>
10 <title>Welcome</title>
11
12 <para> Welcome to the Yocto Project Test Environment Manual! This manual is a work in
13 progress. The manual contains information about the testing environment used by the
14 Yocto Project to make sure each major and minor release works as intended. All the
15 project's testing infrastructure and processes are publicly visible and available so
16 that the community can see what testing is being performed, how it's being done and the
17 current status of the tests and the project at any given time. It is intended that Other
18 organizations can leverage off the process and testing environment used by the Yocto
19 Project to create their own automated, production test environment, building upon the
20 foundations from the project core. </para>
21
22 <para> Currently, the Yocto Project Test Environment Manual has no projected release date.
23 This manual is a work-in-progress and is being initially loaded with information from
24 the <ulink url="">README</ulink> files and notes from key engineers: <itemizedlist>
25 <listitem>
26 <para>
27 <emphasis><filename>yocto-autobuilder2</filename>:</emphasis> This <ulink
28 url="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder2/tree/README.md"
29 ><filename>README.md</filename></ulink> is the main README which
30 detials how to set up the Yocto Project Autobuilder. The
31 <filename>yocto-autobuilder2</filename> repository represents the Yocto
32 Project's console UI plugin to Buildbot and the configuration necessary to
33 configure Buildbot to perform the testing the project requires. </para>
34 </listitem>
35 <listitem>
36 <para>
37 <emphasis><filename>yocto-autobuilder-helper</filename>:</emphasis> This
38 <ulink
39 url="http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder-helper/tree/README"
40 ><filename>README</filename></ulink> and repository contains Yocto
41 Project Autobuilder Helper scripts and configuration. The
42 <filename>yocto-autobuilder-helper</filename> repository contains the
43 "glue" logic that defines which tests to run and how to run them. As a
44 result, it can be used by any Continuous Improvement (CI) system to run
45 builds, support getting the correct code revisions, configure builds and
46 layers, run builds, and collect results. The code is independent of any CI
47 system, which means the code can work Buildbot, Jenkins, or others. This
48 repository has a branch per release of the project defining the tests to run
49 on a per release basis.</para>
50 </listitem>
51 </itemizedlist>
52 </para>
53 </section>
54
55 <section id='test-yocto-project-autobuilder-overview'>
56 <title>Yocto Project Autobuilder Overview</title>
57
58 <para>The Yocto Project Autobuilder collectively refers to the software, tools, scripts, and
59 procedures used by the Yocto Project to test released software across supported hardware
60 in an automated and regular fashion. Basically, during the development of a Yocto
61 Project release, the Autobuilder tests if things work. The Autobuilder builds all test
62 targets and runs all the tests. </para>
63
64 <para>The Yocto Project uses now uses standard upstream <ulink
65 url="https://docs.buildbot.net/0.9.15.post1/">Buildbot</ulink> (version 9) to drive
66 its integration and testing. Buildbot Nine has a plug-in interface that the Yocto
67 Project customizes using code from the <filename>yocto-autobuilder2</filename>
68 repository, adding its own console UI plugin. The resulting UI plug-in allows you to
69 visualize builds in a way suited to the project's needs.</para>
70
71 <para>A <filename>helper</filename> layer provides configuration and job management through
72 scripts found in the <filename>yocto-autobuilder-helper</filename> repository. The
73 <filename>helper</filename> layer contains the bulk of the build configuration
74 information and is release-specific, which makes it highly customizable on a per-project
75 basis. The layer is CI system-agnostic and contains a number of Helper scripts that can
76 generate build configurations from simple JSON files. <note>
77 <para>The project uses Buildbot for historical reasons but also because many of the
78 project developers have knowledge of python. It is possible to use the outer
79 layers from another Continuous Integration (CI) system such as <ulink
80 url="https://en.wikipedia.org/wiki/Jenkins_(software)">Jenkins</ulink>
81 instead of Buildbot. </para>
82 </note>
83 </para>
84
85 <para> The following figure shows the Yocto Project Autobuilder stack with a topology that
86 includes a controller and a cluster of workers: <imagedata
87 fileref="figures/ab-test-cluster.png" width="4.6in" depth="4.35in" align="center"
88 scalefit="1"/>
89 </para>
90 </section>
91
92 <section id='test-project-tests'>
93 <title>Yocto Project Tests - Types of Testing Overview</title>
94
95 <para>The Autobuilder tests different elements of the project by using thefollowing types of
96 tests: <itemizedlist>
97 <listitem>
98 <para>
99 <emphasis>Build Testing:</emphasis> Tests whether specific configurations
100 build by varying <ulink url="&YOCTO_DOCS_REF_URL;#var-MACHINE"
101 ><filename>MACHINE</filename></ulink>, <ulink
102 url="&YOCTO_DOCS_REF_URL;#var-DISTRO"
103 ><filename>DISTRO</filename></ulink>, other configuration options, and
104 the specific target images being built (or world). Used to trigger builds of
105 all the different test configurations on the Autobuilder. Builds usually
106 cover many different targets for different architectures, machines, and
107 distributions, as well as different configurations, such as different init
108 systems. The Autobuilder tests literally hundreds of configurations and
109 targets. <itemizedlist>
110 <listitem>
111 <para>
112 <emphasis>Sanity Checks During the Build Process:</emphasis>
113 Tests initiated through the <ulink
114 url="&YOCTO_DOCS_REF_URL;#ref-classes-insane"
115 ><filename>insane</filename></ulink> class. These checks
116 ensure the output of the builds are correct. For example, does
117 the ELF architecture in the generated binaries match the target
118 system? ARM binaries would not work in a MIPS system! </para>
119 </listitem>
120 </itemizedlist></para>
121 </listitem>
122 <listitem>
123 <para>
124 <emphasis>Build Performance Testing:</emphasis> Tests whether or not
125 commonly used steps during builds work efficiently and avoid regressions.
126 Tests to time commonly used usage scenarios are run through
127 <filename>oe-build-perf-test</filename>. These tests are run on isolated
128 machines so that the time measurements of the tests are accurate and no
129 other processes interfere with the timing results. The project currently
130 tests performance on two different distributions, Fedora and Ubuntu, to
131 ensure we have no single point of failure and can ensure the different
132 distros work effectively. </para>
133 </listitem>
134 <listitem>
135 <para>
136 <emphasis>eSDK Testing:</emphasis> Image tests initiated through the
137 following command:
138 <literallayout class="monospaced">
139 $ bitbake <replaceable>image</replaceable> -c testsdkext
140 </literallayout>
141 The tests utilize the <filename>testsdkext</filename> class and the
142 <filename>do_testsdkext</filename> task. </para>
143 </listitem>
144 <listitem>
145 <para>
146 <emphasis>Feature Testing:</emphasis> Various scenario-based tests are run
147 through the <ulink url="&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance"
148 >OpenEmbedded Self-Test</ulink> (oe-selftest). We test oe-selftest on
149 each of the main distrubutions we support. </para>
150 </listitem>
151 <listitem>
152 <para>
153 <emphasis>Image Testing:</emphasis> Image tests initiated through the
154 following command:
155 <literallayout class="monospaced">
156 $ bitbake <replaceable>image</replaceable> -c testimage
157 </literallayout>
158 The tests utilize the <ulink
159 url="&YOCTO_DOCS_REF_URL;#ref-classes-testimage*"
160 ><filename>testimage*</filename></ulink> classes and the <ulink
161 url="&YOCTO_DOCS_REF_URL;#ref-tasks-testimage"
162 ><filename>do_testimage</filename></ulink> task. </para>
163 </listitem>
164 <listitem>
165 <para>
166 <emphasis>Layer Testing:</emphasis> The Autobuilder has the possibility to
167 test whether specific layers work with the test of the system. The layers
168 tested may be selected by members of the project. Some key community layers
169 are also tested periodically.</para>
170 </listitem>
171 <listitem>
172 <para>
173 <emphasis>Package Testing:</emphasis> A Package Test (ptest) runs tests
174 against packages built by the OpenEmbedded build system on the target
175 machine. See the "<ulink
176 url="&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest">Testing Packages
177 With ptest</ulink>" section in the Yocto Project Development Tasks
178 Manual and the "<ulink url="&YOCTO_WIKI_URL;/wiki/Ptest">Ptest</ulink>" Wiki
179 page for more information on Ptest. </para>
180 </listitem>
181 <listitem>
182 <para>
183 <emphasis>SDK Testing:</emphasis> Image tests initiated through the
184 following command:
185 <literallayout class="monospaced">
186 $ bitbake <replaceable>image</replaceable> -c testsdk
187 </literallayout>
188 The tests utilize the <ulink url="&YOCTO_DOCS_REF_URL;#ref-classes-testsdk"
189 ><filename>testsdk</filename></ulink> class and the
190 <filename>do_testsdk</filename> task. </para>
191 </listitem>
192 <listitem>
193 <para>
194 <emphasis>Unit Testing:</emphasis> Unit tests on various components of the
195 system run through <filename>oe-selftest</filename> and <ulink
196 url="&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance"
197 ><filename>bitbake-selftest</filename></ulink>. </para>
198 </listitem>
199 <listitem>
200 <para>
201 <emphasis>Automatic Upgrade Helper:</emphasis> This target tests whether new
202 versions of software are available and whether we can automatically upgrade
203 to those new versions. If so, this target emails the maintainers with a
204 patch to let them know this is possible.</para>
205 </listitem>
206 </itemizedlist>
207 </para>
208 </section>
209
210 <section id='test-test-mapping'>
211 <title>How Tests Map to Areas of Code</title>
212
213 <para>
214 Tests map into the codebase as follows:
215 <itemizedlist>
216 <listitem><para>
217 <emphasis>bitbake-selftest</emphasis>: </para>
218 <para>These tests are self-contained and test BitBake as well as its APIs, which
219 include the fetchers. The tests are located in
220 <filename>bitbake/lib/*/tests</filename>. </para>
221 <para>From within the BitBake repository, run the following:
222 <literallayout class="monospaced">
223 $ bitbake-selftest
224 </literallayout>
225 </para>
226 <para>To skip tests that access the Internet, use the
227 <filename>BB_SKIP_NETTEST</filename> variable when running
228 "bitbake-selftest" as follows:
229 <literallayout class="monospaced">
230 $ BB_SKIP_NETTEST=yes bitbake-selftest
231 </literallayout></para>
232 <para>The default output is quiet and just prints a summary of what was run. To
233 see more information, there is a verbose
234 option:<literallayout class="monospaced">
235 $ bitbake-selftest -v
236 </literallayout></para>
237 <para>Use this option when you wish to skip tests that access the network, which
238 are mostly necessary to test the fetcher modules. To specify individual test
239 modules to run, append the test module name to the "bitbake-selftest"
240 command. For example, to specify the tests for the bb.data.module, run:
241 <literallayout class="monospaced">
242 $ bitbake-selftest bb.test.data.module
243 </literallayout>You
244 can also specify individual tests by defining the full name and module plus
245 the class path of the test, for example:
246 <literallayout class="monospaced">
247 $ bitbake-selftest bb.tests.data.TestOverrides.test_one_override
248 </literallayout></para>
249 <para>The tests are based on <ulink
250 url="https://docs.python.org/3/library/unittest.html">Python
251 unittest</ulink>. </para></listitem>
252 <listitem><para>
253 <emphasis>oe-selftest</emphasis>: <itemizedlist>
254 <listitem>
255 <para>These tests use OE to test the workflows, which include
256 testing specific features, behaviors of tasks, and API unit
257 tests. </para>
258 </listitem>
259 <listitem>
260 <para>The tests can take advantage of parallelism through the "-j"
261 option, which can specify a number of threads to spread the
262 tests across. Note that all tests from a given class of tests
263 will run in the same thread. To parallelize large numbers of
264 tests you can split the class into multiple units.</para>
265 </listitem>
266 <listitem>
267 <para>The tests are based on Python unittest. </para>
268 </listitem>
269 <listitem>
270 <para>The code for the tests resides in
271 <filename>meta/lib/oeqa/selftest/cases/</filename>. </para>
272 </listitem>
273 <listitem>
274 <para>To run all the tests, enter the following command:
275 <literallayout class="monospaced">
276 $ oe-selftest -a
277 </literallayout>
278 </para>
279 </listitem>
280 <listitem>
281 <para>To run a specific test, use the following command form where
282 <replaceable>testname</replaceable> is the name of the
283 specific test:
284 <literallayout class="monospaced">
285 $ oe-selftest -r <replaceable>testname</replaceable>
286 </literallayout>
287 For example, the following command would run the tinfoil getVar
288 API
289 test:<literallayout class="monospaced">
290 $ oe-selftest -r tinfoil.TinfoilTests.test_getvar
291 </literallayout>It
292 is also possible to run a set of tests. For example the
293 following command will run all of the tinfoil
294 tests:<literallayout class="monospaced">
295 $ oe-selftest -r tinfoil
296 </literallayout></para>
297 </listitem>
298 </itemizedlist>
299 </para></listitem>
300 <listitem><para>
301 <emphasis>testimage:</emphasis>
302 <itemizedlist>
303 <listitem><para>
304 These tests build an image, boot it, and run tests
305 against the image's content.
306 </para></listitem>
307 <listitem><para> The code for these tests resides in <filename>meta/lib/oeqa/runtime/cases/</filename>. </para></listitem>
308 <listitem><para>
309 You need to set the
310 <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_CLASSES'><filename>IMAGE_CLASSES</filename></ulink>
311 variable as follows:
312 <literallayout class='monospaced'>
313 IMAGE_CLASSES += "testimage"
314 </literallayout>
315 </para></listitem>
316 <listitem><para>
317 Run the tests using the following command form:
318 <literallayout class='monospaced'>
319 $ bitbake <replaceable>image</replaceable> -c testimage
320 </literallayout>
321 </para></listitem>
322 </itemizedlist>
323 </para></listitem>
324 <listitem><para>
325 <emphasis>testsdk:</emphasis>
326 <itemizedlist>
327 <listitem><para>These tests build an SDK, install it, and then run tests against that SDK. </para></listitem>
328 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/sdk/cases/</filename>. </para></listitem>
329 <listitem><para>Run the test using the following command form:
330 <literallayout class="monospaced">
331 $ bitbake <replaceable>image</replaceable> -c testsdk
332 </literallayout>
333 </para></listitem>
334 </itemizedlist>
335 </para></listitem>
336 <listitem><para>
337 <emphasis>testsdk_ext:</emphasis>
338 <itemizedlist>
339 <listitem><para>These tests build an extended SDK (eSDK), install that eSDK, and run tests against the eSDK. </para></listitem>
340 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/esdk</filename>. </para></listitem>
341 <listitem><para>To run the tests, use the following command form:
342 <literallayout class="monospaced">
343 $ bitbake <replaceable>image</replaceable> -c testsdkext
344 </literallayout>
345 </para></listitem>
346 </itemizedlist>
347 </para></listitem>
348
349
350 <listitem><para>
351 <emphasis>oe-build-perf-test:</emphasis>
352 <itemizedlist>
353 <listitem><para>These tests run through commonly used usage scenarios and measure the performance times. </para></listitem>
354 <listitem><para>The code for these tests resides in <filename>meta/lib/oeqa/buildperf</filename>. </para></listitem>
355 <listitem><para>To run the tests, use the following command form:
356 <literallayout class="monospaced">
357 $ oe-build-perf-test <replaceable>options</replaceable>
358 </literallayout>The
359 command takes a number of options, such as where to place the
360 test results. The Autobuilder Helper Scripts include the
361 <filename>build-perf-test-wrapper</filename> script with
362 examples of how to use the oe-build-perf-test from the command
363 line.</para>
364 <para>Use the <filename>oe-git-archive</filename> command to store
365 test results into a Git repository. </para>
366 <para>Use the <filename>oe-build-perf-report</filename> command to
367 generate text reports and HTML reports with graphs of the
368 performance data. For examples, see <link linkend=""
369 >http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.html</link>
370 and <link linkend=""
371 >http://downloads.yoctoproject.org/releases/yocto/yocto-2.7/testresults/buildperf-centos7/perf-centos7.yoctoproject.org_warrior_20190414204758_0e39202.txt</link>.</para></listitem>
372 <listitem>
373 <para>The tests are contained in
374 <filename>lib/oeqa/buildperf/test_basic.py</filename>.</para>
375 </listitem>
376 </itemizedlist>
377 </para></listitem>
378
379
380
381
382 </itemizedlist>
383 </para>
384 </section>
385
386 <section id='test-examples'>
387 <title>Test Examples</title>
388
389 <para>This section provides example tests for each of the tests listed in the <link
390 linkend="test-test-mapping">How Tests Map to Areas of Code</link> section. </para>
391 <para>For oeqa tests, testcases for each area reside in the main test directory at
392 <filename>meta/lib/oeqa/selftest/cases</filename> directory.</para>
393 <para>For oe-selftest. bitbake testcases reside in the <filename>lib/bb/tests/</filename>
394 directory. </para>
395
396 <section id='bitbake-selftest-example'>
397 <title><filename>bitbake-selftest</filename></title>
398
399 <para>A simple test example from <filename>lib/bb/tests/data.py</filename> is:
400 <literallayout class="monospaced">
401 class DataExpansions(unittest.TestCase):
402 def setUp(self):
403 self.d = bb.data.init()
404 self.d["foo"] = "value_of_foo"
405 self.d["bar"] = "value_of_bar"
406 self.d["value_of_foo"] = "value_of_'value_of_foo'"
407
408 def test_one_var(self):
409 val = self.d.expand("${foo}")
410 self.assertEqual(str(val), "value_of_foo")
411 </literallayout>
412 </para>
413 <para>In this example, a <ulink url=""><filename>DataExpansions</filename></ulink> class
414 of tests is created, derived from standard python unittest. The class has a common
415 <filename>setUp</filename> function which is shared by all the tests in the
416 class. A simple test is then added to test that when a variable is expanded, the
417 correct value is found.</para>
418 <para>Bitbake selftests are straightforward python unittest. Refer to the Python
419 unittest documentation for additional information on writing these tests at: <link
420 linkend="">https://docs.python.org/3/library/unittest.html</link>.</para>
421 </section>
422
423 <section id='oe-selftest-example'>
424 <title><filename>oe-selftest</filename></title>
425
426 <para>These tests are more complex due to the setup required behind the scenes for full
427 builds. Rather than directly using Python's unittest, the code wraps most of the
428 standard objects. The tests can be simple, such as testing a command from within the
429 OE build environment using the following
430 example:<literallayout class="monospaced">
431 class BitbakeLayers(OESelftestTestCase):
432 def test_bitbakelayers_showcrossdepends(self):
433 result = runCmd('bitbake-layers show-cross-depends')
434 self.assertTrue('aspell' in result.output, msg = "No dependencies
435 were shown. bitbake-layers show-cross-depends output:
436 %s"% result.output)
437 </literallayout></para>
438 <para>This example, taken from
439 <filename>meta/lib/oeqa/selftest/cases/bblayers.py</filename>, creates a
440 testcase from the <ulink url=""><filename>OESelftestTestCase</filename></ulink>
441 class, derived from <filename>unittest.TestCase</filename>, which runs the
442 <filename>bitbake-layers</filename> command and checks the output to ensure it
443 contains something we know should be here.</para>
444 <para>The <filename>oeqa.utils.commands</filename> module contains Helpers which can
445 assist with common tasks, including:<itemizedlist>
446 <listitem>
447 <para><emphasis>Obtaining the value of a bitbake variable:</emphasis> Use
448 <filename>oeqa.utils.commands.get_bb_var()</filename> or use
449 <filename>oeqa.utils.commands.get_bb_vars()</filename> for more than
450 one variable</para>
451 </listitem>
452 <listitem>
453 <para><emphasis>Running a bitbake invocation for a build:</emphasis> Use
454 <filename>oeqa.utils.commands.bitbake()</filename></para>
455 </listitem>
456 <listitem>
457 <para><emphasis>Running a command:</emphasis> Use
458 <filename>oeqa.utils.commandsrunCmd()</filename></para>
459 </listitem>
460 </itemizedlist></para>
461 <para>There is also a <filename>oeqa.utils.commands.runqemu()</filename> function for
462 launching the <filename>runqemu</filename> command for testing things within a
463 running, virtualized image.</para>
464 <para>You can run these tests in parallel. Parallelism works per test class, so tests
465 within a given test class should always run in the same build, while tests in
466 different classes or modules may be split into different builds. There is no data
467 store available for these tests since the tests launch the
468 <filename>bitbake</filename> command and exist outside of its context. As a
469 result, common bitbake library functions (bb.*) are also unavailable.</para>
470 </section>
471
472 <section id='testimage-example'>
473 <title><filename>testimage</filename></title>
474
475 <para>These tests are run once an image is up and running, either on target hardware or
476 under QEMU. As a result, they are assumed to be running in a target image
477 environment, as opposed to a host build environment. A simple example from
478 <filename>meta/lib/oeqa/runtime/cases/python.py</filename> contains the
479 following:<literallayout class="monospaced">
480 class PythonTest(OERuntimeTestCase):
481 @OETestDepends(['ssh.SSHTest.test_ssh'])
482 @OEHasPackage(['python3-core'])
483 def test_python3(self):
484 cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb,
485 jbeyq', 'rot13'))\""
486 status, output = self.target.run(cmd)
487 msg = 'Exit status was not 0. Output: %s' % output
488 self.assertEqual(status, 0, msg=msg)
489 </literallayout></para>
490 <para>In this example, the <ulink url=""><filename>OERuntimeTestCase</filename></ulink>
491 class wraps <filename>unittest.TestCase</filename>. Within the test,
492 <filename>self.target</filename> represents the target system, where commands
493 can be run on it using the <filename>run()</filename> method. </para>
494 <para>To ensure certain test or package dependencies are met, you can use the
495 <filename>OETestDepends</filename> and <filename>OEHasPackage</filename>
496 decorators. For example, the test in this example would only make sense if
497 python3-core is installed in the image.</para>
498 </section>
499
500 <section id='testsdk_ext-example'>
501 <title><filename>testsdk_ext</filename></title>
502
503 <para>These tests are run against built extensible SDKs (eSDKs). The tests can assume
504 that the eSDK environment has already been setup. An example from
505 <filename>meta/lib/oeqa/sdk/cases/devtool.py</filename> contains the
506 following:<literallayout class="monospaced">
507 class DevtoolTest(OESDKExtTestCase):
508 @classmethod
509 def setUpClass(cls):
510 myapp_src = os.path.join(cls.tc.esdk_files_dir, "myapp")
511 cls.myapp_dst = os.path.join(cls.tc.sdk_dir, "myapp")
512 shutil.copytree(myapp_src, cls.myapp_dst)
513 subprocess.check_output(['git', 'init', '.'], cwd=cls.myapp_dst)
514 subprocess.check_output(['git', 'add', '.'], cwd=cls.myapp_dst)
515 subprocess.check_output(['git', 'commit', '-m', "'test commit'"], cwd=cls.myapp_dst)
516
517 @classmethod
518 def tearDownClass(cls):
519 shutil.rmtree(cls.myapp_dst)
520 def _test_devtool_build(self, directory):
521 self._run('devtool add myapp %s' % directory)
522 try:
523 self._run('devtool build myapp')
524 finally:
525 self._run('devtool reset myapp')
526 def test_devtool_build_make(self):
527 self._test_devtool_build(self.myapp_dst)
528 </literallayout>In
529 this example, the <filename>devtool</filename> command is tested to see whether a
530 sample application can be built with the <filename>devtool build</filename> command
531 within the eSDK.</para>
532 </section>
533
534 <section id='testsdk-example'>
535 <title><filename>testsdk</filename></title>
536
537 <para>These tests are run against built SDKs. The tests can assume that an SDK has
538 already been extracted and its environment file has been sourced. A simple example
539 from <filename>meta/lib/oeqa/sdk/cases/python2.py</filename> contains the
540 following:<literallayout class="monospaced">
541 class Python3Test(OESDKTestCase):
542 def setUp(self):
543 if not (self.tc.hasHostPackage("nativesdk-python3-core") or
544 self.tc.hasHostPackage("python3-core-native")):
545 raise unittest.SkipTest("No python3 package in the SDK")
546
547 def test_python3(self):
548 cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
549 output = self._run(cmd)
550 self.assertEqual(output, "Hello, world\n")
551 </literallayout>In
552 this example, if nativesdk-python3-core has been installed into the SDK, the code
553 runs the python3 interpreter with a basic command to check it is working correctly.
554 The test would only run if python3 is installed in the SDK.</para>
555 </section>
556
557 <section id='oe-build-perf-test-example'>
558 <title><filename>oe-build-perf-test</filename></title>
559
560 <para>The performance tests usually measure how long operations take and the resource
561 utilisation as that happens. An example from
562 <filename>meta/lib/oeqa/buildperf/test_basic.py</filename> contains the
563 following:<literallayout class="monospaced">
564 class Test3(BuildPerfTestCase):
565
566 def test3(self):
567 """Bitbake parsing (bitbake -p)"""
568 # Drop all caches and parse
569 self.rm_cache()
570 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
571 self.measure_cmd_resources(['bitbake', '-p'], 'parse_1',
572 'bitbake -p (no caches)')
573 # Drop tmp/cache
574 oe.path.remove(os.path.join(self.bb_vars['TMPDIR'], 'cache'), True)
575 self.measure_cmd_resources(['bitbake', '-p'], 'parse_2',
576 'bitbake -p (no tmp/cache)')
577 # Parse with fully cached data
578 self.measure_cmd_resources(['bitbake', '-p'], 'parse_3',
579 'bitbake -p (cached)')
580 </literallayout>This
581 example shows how three specific parsing timings are measured, with and without
582 various caches, to show how BitBake's parsing performance trends over time.</para>
583 </section>
584 </section>
585 <section id='test-writing-considerations'>
586 <title>Considerations When Writing Tests</title>
587 <para>When writing good tests, there are several things to keep in mind. Since things
588 running on the Autobuilder are accessed concurrently by multiple workers, consider the
589 following:</para>
590 <formalpara>
591 <title>Running "cleanall" is not permitted</title>
592 <para>This can delete files from DL_DIR which would potentially break other builds
593 running in parallel. If this is required, DL_DIR must be set to an isolated
594 directory.</para>
595 </formalpara>
596 <formalpara>
597 <title>Running "cleansstate" is not permitted</title>
598 <para>This can delete files from SSTATE_DIR which would potentially break other builds
599 running in parallel. If this is required, SSTATE_DIR must be set to an isolated
600 directory. Alternatively, you can use the "-f" option with the
601 <filename>bitbake</filename> command to "taint" tasks by changing the sstate
602 checksums to ensure sstate cache items will not be reused.</para>
603 </formalpara>
604 <formalpara>
605 <title>Tests should not change the metadata</title>
606 <para>This is particularly true for oe-selftests since these can run in parallel and
607 changing metadata leads to changing checksums, which confuses BitBake while running
608 in parallel. If this is necessary, copy layers to a temporary location and modify
609 them. Some tests need to change metadata, such as the devtool tests. To prevent the
610 metadate from changes, set up temporary copies of that data first.</para>
611 </formalpara>
612 </section>
613
614
615
616
617
618
619
620
621</chapter>
622<!--
623vim: expandtab tw=80 ts=4
624-->
diff --git a/documentation/test-manual/test-manual-style.css b/documentation/test-manual/test-manual-style.css
new file mode 100644
index 0000000000..10ee4c79cf
--- /dev/null
+++ b/documentation/test-manual/test-manual-style.css
@@ -0,0 +1,991 @@
1/*
2 Generic XHTML / DocBook XHTML CSS Stylesheet.
3
4 SPDX-License-Identifier: CC-BY-2.0-UK
5
6 Browser wrangling and typographic design by
7 Oyvind Kolas / pippin@gimp.org
8
9 Customised for Poky by
10 Matthew Allum / mallum@o-hand.com
11
12 Thanks to:
13 Liam R. E. Quin
14 William Skaggs
15 Jakub Steiner
16
17 Structure
18 ---------
19
20 The stylesheet is divided into the following sections:
21
22 Positioning
23 Margins, paddings, width, font-size, clearing.
24 Decorations
25 Borders, style
26 Colors
27 Colors
28 Graphics
29 Graphical backgrounds
30 Nasty IE tweaks
31 Workarounds needed to make it work in internet explorer,
32 currently makes the stylesheet non validating, but up until
33 this point it is validating.
34 Mozilla extensions
35 Transparency for footer
36 Rounded corners on boxes
37
38*/
39
40
41 /*************** /
42 / Positioning /
43/ ***************/
44
45body {
46 font-family: Verdana, Sans, sans-serif;
47
48 min-width: 640px;
49 width: 80%;
50 margin: 0em auto;
51 padding: 2em 5em 5em 5em;
52 color: #333;
53}
54
55h1,h2,h3,h4,h5,h6,h7 {
56 font-family: Arial, Sans;
57 color: #00557D;
58 clear: both;
59}
60
61h1 {
62 font-size: 2em;
63 text-align: left;
64 padding: 0em 0em 0em 0em;
65 margin: 2em 0em 0em 0em;
66}
67
68h2.subtitle {
69 margin: 0.10em 0em 3.0em 0em;
70 padding: 0em 0em 0em 0em;
71 font-size: 1.8em;
72 padding-left: 20%;
73 font-weight: normal;
74 font-style: italic;
75}
76
77h2 {
78 margin: 2em 0em 0.66em 0em;
79 padding: 0.5em 0em 0em 0em;
80 font-size: 1.5em;
81 font-weight: bold;
82}
83
84h3.subtitle {
85 margin: 0em 0em 1em 0em;
86 padding: 0em 0em 0em 0em;
87 font-size: 142.14%;
88 text-align: right;
89}
90
91h3 {
92 margin: 1em 0em 0.5em 0em;
93 padding: 1em 0em 0em 0em;
94 font-size: 140%;
95 font-weight: bold;
96}
97
98h4 {
99 margin: 1em 0em 0.5em 0em;
100 padding: 1em 0em 0em 0em;
101 font-size: 120%;
102 font-weight: bold;
103}
104
105h5 {
106 margin: 1em 0em 0.5em 0em;
107 padding: 1em 0em 0em 0em;
108 font-size: 110%;
109 font-weight: bold;
110}
111
112h6 {
113 margin: 1em 0em 0em 0em;
114 padding: 1em 0em 0em 0em;
115 font-size: 110%;
116 font-weight: bold;
117}
118
119.authorgroup {
120 background-color: transparent;
121 background-repeat: no-repeat;
122 padding-top: 256px;
123 background-image: url("figures/test-manual-title.png");
124 background-position: left top;
125 margin-top: -256px;
126 padding-right: 50px;
127 margin-left: 0px;
128 text-align: right;
129 width: 740px;
130}
131
132h3.author {
133 margin: 0em 0me 0em 0em;
134 padding: 0em 0em 0em 0em;
135 font-weight: normal;
136 font-size: 100%;
137 color: #333;
138 clear: both;
139}
140
141.author tt.email {
142 font-size: 66%;
143}
144
145.titlepage hr {
146 width: 0em;
147 clear: both;
148}
149
150.revhistory {
151 padding-top: 2em;
152 clear: both;
153}
154
155.toc,
156.list-of-tables,
157.list-of-examples,
158.list-of-figures {
159 padding: 1.33em 0em 2.5em 0em;
160 color: #00557D;
161}
162
163.toc p,
164.list-of-tables p,
165.list-of-figures p,
166.list-of-examples p {
167 padding: 0em 0em 0em 0em;
168 padding: 0em 0em 0.3em;
169 margin: 1.5em 0em 0em 0em;
170}
171
172.toc p b,
173.list-of-tables p b,
174.list-of-figures p b,
175.list-of-examples p b{
176 font-size: 100.0%;
177 font-weight: bold;
178}
179
180.toc dl,
181.list-of-tables dl,
182.list-of-figures dl,
183.list-of-examples dl {
184 margin: 0em 0em 0.5em 0em;
185 padding: 0em 0em 0em 0em;
186}
187
188.toc dt {
189 margin: 0em 0em 0em 0em;
190 padding: 0em 0em 0em 0em;
191}
192
193.toc dd {
194 margin: 0em 0em 0em 2.6em;
195 padding: 0em 0em 0em 0em;
196}
197
198div.glossary dl,
199div.variablelist dl {
200}
201
202.glossary dl dt,
203.variablelist dl dt,
204.variablelist dl dt span.term {
205 font-weight: normal;
206 width: 20em;
207 text-align: right;
208}
209
210.variablelist dl dt {
211 margin-top: 0.5em;
212}
213
214.glossary dl dd,
215.variablelist dl dd {
216 margin-top: 0em;
217 margin-left: 25.5em;
218}
219
220.glossary dd p,
221.variablelist dd p {
222 margin-top: 0em;
223 margin-bottom: 1em;
224}
225
226
227div.calloutlist table td {
228 padding: 0em 0em 0em 0em;
229 margin: 0em 0em 0em 0em;
230}
231
232div.calloutlist table td p {
233 margin-top: 0em;
234 margin-bottom: 1em;
235}
236
237div p.copyright {
238 text-align: left;
239}
240
241div.legalnotice p.legalnotice-title {
242 margin-bottom: 0em;
243}
244
245p {
246 line-height: 1.5em;
247 margin-top: 0em;
248
249}
250
251dl {
252 padding-top: 0em;
253}
254
255hr {
256 border: solid 1px;
257}
258
259
260.mediaobject,
261.mediaobjectco {
262 text-align: center;
263}
264
265img {
266 border: none;
267}
268
269ul {
270 padding: 0em 0em 0em 1.5em;
271}
272
273ul li {
274 padding: 0em 0em 0em 0em;
275}
276
277ul li p {
278 text-align: left;
279}
280
281table {
282 width :100%;
283}
284
285th {
286 padding: 0.25em;
287 text-align: left;
288 font-weight: normal;
289 vertical-align: top;
290}
291
292td {
293 padding: 0.25em;
294 vertical-align: top;
295}
296
297p a[id] {
298 margin: 0px;
299 padding: 0px;
300 display: inline;
301 background-image: none;
302}
303
304a {
305 text-decoration: underline;
306 color: #444;
307}
308
309pre {
310 overflow: auto;
311}
312
313a:hover {
314 text-decoration: underline;
315 /*font-weight: bold;*/
316}
317
318/* This style defines how the permalink character
319 appears by itself and when hovered over with
320 the mouse. */
321
322[alt='Permalink'] { color: #eee; }
323[alt='Permalink']:hover { color: black; }
324
325
326div.informalfigure,
327div.informalexample,
328div.informaltable,
329div.figure,
330div.table,
331div.example {
332 margin: 1em 0em;
333 padding: 1em;
334 page-break-inside: avoid;
335}
336
337
338div.informalfigure p.title b,
339div.informalexample p.title b,
340div.informaltable p.title b,
341div.figure p.title b,
342div.example p.title b,
343div.table p.title b{
344 padding-top: 0em;
345 margin-top: 0em;
346 font-size: 100%;
347 font-weight: normal;
348}
349
350.mediaobject .caption,
351.mediaobject .caption p {
352 text-align: center;
353 font-size: 80%;
354 padding-top: 0.5em;
355 padding-bottom: 0.5em;
356}
357
358.epigraph {
359 padding-left: 55%;
360 margin-bottom: 1em;
361}
362
363.epigraph p {
364 text-align: left;
365}
366
367.epigraph .quote {
368 font-style: italic;
369}
370.epigraph .attribution {
371 font-style: normal;
372 text-align: right;
373}
374
375span.application {
376 font-style: italic;
377}
378
379.programlisting {
380 font-family: monospace;
381 font-size: 80%;
382 white-space: pre;
383 margin: 1.33em 0em;
384 padding: 1.33em;
385}
386
387.tip,
388.warning,
389.caution,
390.note {
391 margin-top: 1em;
392 margin-bottom: 1em;
393
394}
395
396/* force full width of table within div */
397.tip table,
398.warning table,
399.caution table,
400.note table {
401 border: none;
402 width: 100%;
403}
404
405
406.tip table th,
407.warning table th,
408.caution table th,
409.note table th {
410 padding: 0.8em 0.0em 0.0em 0.0em;
411 margin : 0em 0em 0em 0em;
412}
413
414.tip p,
415.warning p,
416.caution p,
417.note p {
418 margin-top: 0.5em;
419 margin-bottom: 0.5em;
420 padding-right: 1em;
421 text-align: left;
422}
423
424.acronym {
425 text-transform: uppercase;
426}
427
428b.keycap,
429.keycap {
430 padding: 0.09em 0.3em;
431 margin: 0em;
432}
433
434.itemizedlist li {
435 clear: none;
436}
437
438.filename {
439 font-size: medium;
440 font-family: Courier, monospace;
441}
442
443
444div.navheader, div.heading{
445 position: absolute;
446 left: 0em;
447 top: 0em;
448 width: 100%;
449 background-color: #cdf;
450 width: 100%;
451}
452
453div.navfooter, div.footing{
454 position: fixed;
455 left: 0em;
456 bottom: 0em;
457 background-color: #eee;
458 width: 100%;
459}
460
461
462div.navheader td,
463div.navfooter td {
464 font-size: 66%;
465}
466
467div.navheader table th {
468 /*font-family: Georgia, Times, serif;*/
469 /*font-size: x-large;*/
470 font-size: 80%;
471}
472
473div.navheader table {
474 border-left: 0em;
475 border-right: 0em;
476 border-top: 0em;
477 width: 100%;
478}
479
480div.navfooter table {
481 border-left: 0em;
482 border-right: 0em;
483 border-bottom: 0em;
484 width: 100%;
485}
486
487div.navheader table td a,
488div.navfooter table td a {
489 color: #777;
490 text-decoration: none;
491}
492
493/* normal text in the footer */
494div.navfooter table td {
495 color: black;
496}
497
498div.navheader table td a:visited,
499div.navfooter table td a:visited {
500 color: #444;
501}
502
503
504/* links in header and footer */
505div.navheader table td a:hover,
506div.navfooter table td a:hover {
507 text-decoration: underline;
508 background-color: transparent;
509 color: #33a;
510}
511
512div.navheader hr,
513div.navfooter hr {
514 display: none;
515}
516
517
518.qandaset tr.question td p {
519 margin: 0em 0em 1em 0em;
520 padding: 0em 0em 0em 0em;
521}
522
523.qandaset tr.answer td p {
524 margin: 0em 0em 1em 0em;
525 padding: 0em 0em 0em 0em;
526}
527.answer td {
528 padding-bottom: 1.5em;
529}
530
531.emphasis {
532 font-weight: bold;
533}
534
535
536 /************* /
537 / decorations /
538/ *************/
539
540.titlepage {
541}
542
543.part .title {
544}
545
546.subtitle {
547 border: none;
548}
549
550/*
551h1 {
552 border: none;
553}
554
555h2 {
556 border-top: solid 0.2em;
557 border-bottom: solid 0.06em;
558}
559
560h3 {
561 border-top: 0em;
562 border-bottom: solid 0.06em;
563}
564
565h4 {
566 border: 0em;
567 border-bottom: solid 0.06em;
568}
569
570h5 {
571 border: 0em;
572}
573*/
574
575.programlisting {
576 border: solid 1px;
577}
578
579div.figure,
580div.table,
581div.informalfigure,
582div.informaltable,
583div.informalexample,
584div.example {
585 border: 1px solid;
586}
587
588
589
590.tip,
591.warning,
592.caution,
593.note {
594 border: 1px solid;
595}
596
597.tip table th,
598.warning table th,
599.caution table th,
600.note table th {
601 border-bottom: 1px solid;
602}
603
604.question td {
605 border-top: 1px solid black;
606}
607
608.answer {
609}
610
611
612b.keycap,
613.keycap {
614 border: 1px solid;
615}
616
617
618div.navheader, div.heading{
619 border-bottom: 1px solid;
620}
621
622
623div.navfooter, div.footing{
624 border-top: 1px solid;
625}
626
627 /********* /
628 / colors /
629/ *********/
630
631body {
632 color: #333;
633 background: white;
634}
635
636a {
637 background: transparent;
638}
639
640a:hover {
641 background-color: #dedede;
642}
643
644
645h1,
646h2,
647h3,
648h4,
649h5,
650h6,
651h7,
652h8 {
653 background-color: transparent;
654}
655
656hr {
657 border-color: #aaa;
658}
659
660
661.tip, .warning, .caution, .note {
662 border-color: #fff;
663}
664
665
666.tip table th,
667.warning table th,
668.caution table th,
669.note table th {
670 border-bottom-color: #fff;
671}
672
673
674.warning {
675 background-color: #f0f0f2;
676}
677
678.caution {
679 background-color: #f0f0f2;
680}
681
682.tip {
683 background-color: #f0f0f2;
684}
685
686.note {
687 background-color: #f0f0f2;
688}
689
690.glossary dl dt,
691.variablelist dl dt,
692.variablelist dl dt span.term {
693 color: #044;
694}
695
696div.figure,
697div.table,
698div.example,
699div.informalfigure,
700div.informaltable,
701div.informalexample {
702 border-color: #aaa;
703}
704
705pre.programlisting {
706 color: black;
707 background-color: #fff;
708 border-color: #aaa;
709 border-width: 2px;
710}
711
712.guimenu,
713.guilabel,
714.guimenuitem {
715 background-color: #eee;
716}
717
718
719b.keycap,
720.keycap {
721 background-color: #eee;
722 border-color: #999;
723}
724
725
726div.navheader {
727 border-color: black;
728}
729
730
731div.navfooter {
732 border-color: black;
733}
734
735.writernotes {
736 color: red;
737}
738
739
740 /*********** /
741 / graphics /
742/ ***********/
743
744/*
745body {
746 background-image: url("images/body_bg.jpg");
747 background-attachment: fixed;
748}
749
750.navheader,
751.note,
752.tip {
753 background-image: url("images/note_bg.jpg");
754 background-attachment: fixed;
755}
756
757.warning,
758.caution {
759 background-image: url("images/warning_bg.jpg");
760 background-attachment: fixed;
761}
762
763.figure,
764.informalfigure,
765.example,
766.informalexample,
767.table,
768.informaltable {
769 background-image: url("images/figure_bg.jpg");
770 background-attachment: fixed;
771}
772
773*/
774h1,
775h2,
776h3,
777h4,
778h5,
779h6,
780h7{
781}
782
783/*
784Example of how to stick an image as part of the title.
785
786div.article .titlepage .title
787{
788 background-image: url("figures/white-on-black.png");
789 background-position: center;
790 background-repeat: repeat-x;
791}
792*/
793
794div.preface .titlepage .title,
795div.colophon .title,
796div.chapter .titlepage .title,
797div.article .titlepage .title
798{
799}
800
801div.section div.section .titlepage .title,
802div.sect2 .titlepage .title {
803 background: none;
804}
805
806
807h1.title {
808 background-color: transparent;
809 background-image: url("figures/test-title.png");
810 background-repeat: no-repeat;
811 height: 256px;
812 text-indent: -9000px;
813 overflow:hidden;
814}
815
816h2.subtitle {
817 background-color: transparent;
818 text-indent: -9000px;
819 overflow:hidden;
820 width: 0px;
821 display: none;
822}
823
824 /*************************************** /
825 / pippin.gimp.org specific alterations /
826/ ***************************************/
827
828/*
829div.heading, div.navheader {
830 color: #777;
831 font-size: 80%;
832 padding: 0;
833 margin: 0;
834 text-align: left;
835 position: absolute;
836 top: 0px;
837 left: 0px;
838 width: 100%;
839 height: 50px;
840 background: url('/gfx/heading_bg.png') transparent;
841 background-repeat: repeat-x;
842 background-attachment: fixed;
843 border: none;
844}
845
846div.heading a {
847 color: #444;
848}
849
850div.footing, div.navfooter {
851 border: none;
852 color: #ddd;
853 font-size: 80%;
854 text-align:right;
855
856 width: 100%;
857 padding-top: 10px;
858 position: absolute;
859 bottom: 0px;
860 left: 0px;
861
862 background: url('/gfx/footing_bg.png') transparent;
863}
864*/
865
866
867
868 /****************** /
869 / nasty ie tweaks /
870/ ******************/
871
872/*
873div.heading, div.navheader {
874 width:expression(document.body.clientWidth + "px");
875}
876
877div.footing, div.navfooter {
878 width:expression(document.body.clientWidth + "px");
879 margin-left:expression("-5em");
880}
881body {
882 padding:expression("4em 5em 0em 5em");
883}
884*/
885
886 /**************************************** /
887 / mozilla vendor specific css extensions /
888/ ****************************************/
889/*
890div.navfooter, div.footing{
891 -moz-opacity: 0.8em;
892}
893
894div.figure,
895div.table,
896div.informalfigure,
897div.informaltable,
898div.informalexample,
899div.example,
900.tip,
901.warning,
902.caution,
903.note {
904 -moz-border-radius: 0.5em;
905}
906
907b.keycap,
908.keycap {
909 -moz-border-radius: 0.3em;
910}
911*/
912
913table tr td table tr td {
914 display: none;
915}
916
917
918hr {
919 display: none;
920}
921
922table {
923 border: 0em;
924}
925
926 .photo {
927 float: right;
928 margin-left: 1.5em;
929 margin-bottom: 1.5em;
930 margin-top: 0em;
931 max-width: 17em;
932 border: 1px solid gray;
933 padding: 3px;
934 background: white;
935}
936 .seperator {
937 padding-top: 2em;
938 clear: both;
939 }
940
941 #validators {
942 margin-top: 5em;
943 text-align: right;
944 color: #777;
945 }
946 @media print {
947 body {
948 font-size: 8pt;
949 }
950 .noprint {
951 display: none;
952 }
953 }
954
955
956.tip,
957.note {
958 background: #f0f0f2;
959 color: #333;
960 padding: 20px;
961 margin: 20px;
962}
963
964.tip h3,
965.note h3 {
966 padding: 0em;
967 margin: 0em;
968 font-size: 2em;
969 font-weight: bold;
970 color: #333;
971}
972
973.tip a,
974.note a {
975 color: #333;
976 text-decoration: underline;
977}
978
979.footnote {
980 font-size: small;
981 color: #333;
982}
983
984/* Changes the announcement text */
985.tip h3,
986.warning h3,
987.caution h3,
988.note h3 {
989 font-size:large;
990 color: #00557D;
991}
diff --git a/documentation/test-manual/test-manual-test-process.rst b/documentation/test-manual/test-manual-test-process.rst
new file mode 100644
index 0000000000..96e71bf314
--- /dev/null
+++ b/documentation/test-manual/test-manual-test-process.rst
@@ -0,0 +1,103 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************************
4Project Testing and Release Process
5***********************************
6
7.. _test-daily-devel:
8
9Day to Day Development
10======================
11
12This section details how the project tests changes, through automation
13on the Autobuilder or with the assistance of QA teams, through to making
14releases.
15
16The project aims to test changes against our test matrix before those
17changes are merged into the master branch. As such, changes are queued
18up in batches either in the ``master-next`` branch in the main trees, or
19in user trees such as ``ross/mut`` in ``poky-contrib`` (Ross Burton
20helps review and test patches and this is his testing tree).
21
22We have two broad categories of test builds, including "full" and
23"quick". On the Autobuilder, these can be seen as "a-quick" and
24"a-full", simply for ease of sorting in the UI. Use our Autobuilder
25console view to see where me manage most test-related items, available
26at: :yocto_ab:`/typhoon/#/console`.
27
28Builds are triggered manually when the test branches are ready. The
29builds are monitored by the SWAT team. For additional information, see
30:yocto_wiki:`/wiki/Yocto_Build_Failure_Swat_Team`.
31If successful, the changes would usually be merged to the ``master``
32branch. If not successful, someone would respond to the changes on the
33mailing list explaining that there was a failure in testing. The choice
34of quick or full would depend on the type of changes and the speed with
35which the result was required.
36
37The Autobuilder does build the ``master`` branch once daily for several
38reasons, in particular, to ensure the current ``master`` branch does
39build, but also to keep ``yocto-testresults``
40(:yocto_git:`/cgit.cgi/yocto-testresults/`),
41buildhistory
42(:yocto_git:`/cgit.cgi/poky-buildhistory/`), and
43our sstate up to date. On the weekend, there is a master-next build
44instead to ensure the test results are updated for the less frequently
45run targets.
46
47Performance builds (buildperf-\* targets in the console) are triggered
48separately every six hours and automatically push their results to the
49buildstats repository at:
50:yocto_git:`/cgit.cgi/yocto-buildstats/`.
51
52The 'quick' targets have been selected to be the ones which catch the
53most failures or give the most valuable data. We run 'fast' ptests in
54this case for example but not the ones which take a long time. The quick
55target doesn't include \*-lsb builds for all architectures, some world
56builds and doesn't trigger performance tests or ltp testing. The full
57build includes all these things and is slower but more comprehensive.
58
59Release Builds
60==============
61
62The project typically has two major releases a year with a six month
63cadence in April and October. Between these there would be a number of
64milestone releases (usually four) with the final one being stablization
65only along with point releases of our stable branches.
66
67The build and release process for these project releases is similar to
68that in `Day to Day Development <#test-daily-devel>`__, in that the
69a-full target of the Autobuilder is used but in addition the form is
70configured to generate and publish artefacts and the milestone number,
71version, release candidate number and other information is entered. The
72box to "generate an email to QA"is also checked.
73
74When the build completes, an email is sent out using the send-qa-email
75script in the ``yocto-autobuilder-helper`` repository to the list of
76people configured for that release. Release builds are placed into a
77directory in https://autobuilder.yocto.io/pub/releases on the
78Autobuilder which is included in the email. The process from here is
79more manual and control is effectively passed to release engineering.
80The next steps include:
81
82- QA teams respond to the email saying which tests they plan to run and
83 when the results will be available.
84
85- QA teams run their tests and share their results in the yocto-
86 testresults-contrib repository, along with a summary of their
87 findings.
88
89- Release engineering prepare the release as per their process.
90
91- Test results from the QA teams are included into the release in
92 separate directories and also uploaded to the yocto-testresults
93 repository alongside the other test results for the given revision.
94
95- The QA report in the final release is regenerated using resulttool to
96 include the new test results and the test summaries from the teams
97 (as headers to the generated report).
98
99- The release is checked against the release checklist and release
100 readiness criteria.
101
102- A final decision on whether to release is made by the YP TSC who have
103 final oversight on release readiness.
diff --git a/documentation/test-manual/test-manual-test-process.xml b/documentation/test-manual/test-manual-test-process.xml
new file mode 100644
index 0000000000..6e2157c621
--- /dev/null
+++ b/documentation/test-manual/test-manual-test-process.xml
@@ -0,0 +1,110 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-test-process'>
7
8<title>Project Testing and Release Process</title>
9 <section id='test-daily-devel'>
10 <title>Day to Day Development</title>
11
12 <para>This section details how the project tests changes, through automation on the
13 Autobuilder or with the assistance of QA teams, through to making releases.</para>
14
15 <para>The project aims to test changes against our test matrix before those changes are
16 merged into the master branch. As such, changes are queued up in batches either in the
17 <filename>master-next</filename> branch in the main trees, or in user trees such as
18 <filename>ross/mut</filename> in <filename>poky-contrib</filename> (Ross Burton
19 helps review and test patches and this is his testing tree).</para>
20 <para>We have two broad categories of test builds, including "full" and "quick". On the
21 Autobuilder, these can be seen as "a-quick" and "a-full", simply for ease of sorting in
22 the UI. Use our Autobuilder console view to see where me manage most test-related items,
23 available at: <link linkend=""
24 >https://autobuilder.yoctoproject.org/typhoon/#/console</link>.</para>
25 <para>Builds are triggered manually when the test branches are ready. The builds are
26 monitored by the SWAT team. For additional information, see <link linkend=""
27 >https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team</link>. If
28 successful, the changes would usually be merged to the <filename>master</filename>
29 branch. If not successful, someone would respond to the changes on the mailing list
30 explaining that there was a failure in testing. The choice of quick or full would depend
31 on the type of changes and the speed with which the result was required.</para>
32 <para>The Autobuilder does build the <filename>master</filename> branch once daily for
33 several reasons, in particular, to ensure the current <filename>master</filename> branch
34 does build, but also to keep <filename>yocto-testresults</filename> (<link linkend=""
35 >http://git.yoctoproject.org/cgit.cgi/yocto-testresults/</link>), buildhistory
36 (<link linkend="">http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/</link>),
37 and our sstate up to date. On the weekend, there is a master-next build instead to
38 ensure the test results are updated for the less frequently run targets.</para>
39 <para>Performance builds (buildperf-* targets in the console) are triggered separately every
40 six hours and automatically push their results to the buildstats repository at: <link
41 linkend="">http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/</link>. </para>
42 <para>The 'quick' targets have been selected to be the ones which catch the most failures or
43 give the most valuable data. We run 'fast' ptests in this case for example but not the
44 ones which take a long time. The quick target doesn't include *-lsb builds for all
45 architectures, some world builds and doesn't trigger performance tests or ltp testing.
46 The full build includes all these things and is slower but more comprehensive.</para>
47 </section>
48
49 <section id='test-yocto-project-autobuilder-overview'>
50 <title>Release Builds</title>
51
52 <para>The project typically has two major releases a year with a six month cadence in April
53 and October. Between these there would be a number of milestone releases (usually four)
54 with the final one being stablization only along with point releases of our stable
55 branches.</para>
56 <para>The build and release process for these project releases is similar to that in <link
57 linkend="test-daily-devel">Day to Day Development</link>, in that the a-full target
58 of the Autobuilder is used but in addition the form is configured to generate and
59 publish artefacts and the milestone number, version, release candidate number and other
60 information is entered. The box to "generate an email to QA"is also checked.</para>
61 <para>When the build completes, an email is sent out using the send-qa-email script in the
62 <filename>yocto-autobuilder-helper</filename> repository to the list of people
63 configured for that release. Release builds are placed into a directory in <link
64 linkend="">https://autobuilder.yocto.io/pub/releases</link> on the Autobuilder which
65 is included in the email. The process from here is more manual and control is
66 effectively passed to release engineering. The next steps include:<itemizedlist>
67 <listitem>
68 <para>QA teams respond to the email saying which tests they plan to run and when
69 the results will be available.</para>
70 </listitem>
71 <listitem>
72 <para>QA teams run their tests and share their results in the yocto-
73 testresults-contrib repository, along with a summary of their findings.
74 </para>
75 </listitem>
76 <listitem>
77 <para>Release engineering prepare the release as per their process. </para>
78 </listitem>
79 <listitem>
80 <para>Test results from the QA teams are included into the release in separate
81 directories and also uploaded to the yocto-testresults repository alongside
82 the other test results for the given revision.</para>
83 </listitem>
84 <listitem>
85 <para>The QA report in the final release is regenerated using resulttool to
86 include the new test results and the test summaries from the teams (as
87 headers to the generated report).</para>
88 </listitem>
89 <listitem>
90 <para>The release is checked against the release checklist and release readiness
91 criteria.</para>
92 </listitem>
93 <listitem>
94 <para>A final decision on whether to release is made by the YP TSC who have
95 final oversight on release readiness.</para>
96 </listitem>
97 </itemizedlist></para>
98 </section>
99
100
101
102
103
104
105
106
107</chapter>
108<!--
109vim: expandtab tw=80 ts=4
110-->
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.rst b/documentation/test-manual/test-manual-understand-autobuilder.rst
new file mode 100644
index 0000000000..2fcae5000e
--- /dev/null
+++ b/documentation/test-manual/test-manual-understand-autobuilder.rst
@@ -0,0 +1,305 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3*******************************************
4Understanding the Yocto Project Autobuilder
5*******************************************
6
7Execution Flow within the Autobuilder
8=====================================
9
10The "a-full" and "a-quick" targets are the usual entry points into the
11Autobuilder and it makes sense to follow the process through the system
12starting there. This is best visualised from the Autobuilder Console
13view (:yocto_ab:`/typhoon/#/console`).
14
15Each item along the top of that view represents some "target build" and
16these targets are all run in parallel. The 'full' build will trigger the
17majority of them, the "quick" build will trigger some subset of them.
18The Autobuilder effectively runs whichever configuration is defined for
19each of those targets on a seperate buildbot worker. To understand the
20configuration, you need to look at the entry on ``config.json`` file
21within the ``yocto-autobuilder-helper`` repository. The targets are
22defined in the ‘overrides' section, a quick example could be qemux86-64
23which looks like::
24
25 "qemux86-64" : {
26 "MACHINE" : "qemux86-64",
27 "TEMPLATE" : "arch-qemu",
28 "step1" : {
29 "extravars" : [
30 "IMAGE_FSTYPES_append = ' wic wic.bmap'"
31 ]
32 }
33 },
34
35And to expand that, you need the "arch-qemu" entry from
36the "templates" section, which looks like::
37
38 "arch-qemu" : {
39 "BUILDINFO" : true,
40 "BUILDHISTORY" : true,
41 "step1" : {
42 "BBTARGETS" : "core-image-sato core-image-sato-dev core-image-sato-sdk core-image-minimal core-image-minimal-dev core-image-sato:do_populate_sdk",
43 "SANITYTARGETS" : "core-image-minimal:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage core-image-sato:do_testsdk"
44 },
45 "step2" : {
46 "SDKMACHINE" : "x86_64",
47 "BBTARGETS" : "core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext core-image-sato:do_populate_sdk_ext",
48 "SANITYTARGETS" : "core-image-sato:do_testsdk core-image-minimal:do_testsdkext core-image-sato:do_testsdkext"
49 },
50 "step3" : {
51 "BUILDHISTORY" : false,
52 "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest ${HELPERSTMACHTARGS} -j 15"],
53 "ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
54 }
55 },
56
57Combining these two entries you can see that "qemux86-64" is a three step build where the
58``bitbake BBTARGETS`` would be run, then ``bitbake SANITYTARGETS`` for each step; all for
59``MACHINE="qemx86-64"`` but with differing SDKMACHINE settings. In step
601 an extra variable is added to the ``auto.conf`` file to enable wic
61image generation.
62
63While not every detail of this is covered here, you can see how the
64template mechanism allows quite complex configurations to be built up
65yet allows duplication and repetition to be kept to a minimum.
66
67The different build targets are designed to allow for parallelisation,
68so different machines are usually built in parallel, operations using
69the same machine and metadata are built sequentially, with the aim of
70trying to optimise build efficiency as much as possible.
71
72The ``config.json`` file is processed by the scripts in the Helper
73repository in the ``scripts`` directory. The following section details
74how this works.
75
76.. _test-autobuilder-target-exec-overview:
77
78Autobuilder Target Execution Overview
79=====================================
80
81For each given target in a build, the Autobuilder executes several
82steps. These are configured in ``yocto-autobuilder2/builders.py`` and
83roughly consist of:
84
85#. *Run clobberdir*.
86
87 This cleans out any previous build. Old builds are left around to
88 allow easier debugging of failed builds. For additional information,
89 see :ref:`test-manual/test-manual-understand-autobuilder:clobberdir`.
90
91#. *Obtain yocto-autobuilder-helper*
92
93 This step clones the ``yocto-autobuilder-helper`` git repository.
94 This is necessary to prevent the requirement to maintain all the
95 release or project-specific code within Buildbot. The branch chosen
96 matches the release being built so we can support older releases and
97 still make changes in newer ones.
98
99#. *Write layerinfo.json*
100
101 This transfers data in the Buildbot UI when the build was configured
102 to the Helper.
103
104#. *Call scripts/shared-repo-unpack*
105
106 This is a call into the Helper scripts to set up a checkout of all
107 the pieces this build might need. It might clone the BitBake
108 repository and the OpenEmbedded-Core repository. It may clone the
109 Poky repository, as well as additional layers. It will use the data
110 from the ``layerinfo.json`` file to help understand the
111 configuration. It will also use a local cache of repositories to
112 speed up the clone checkouts. For additional information, see
113 :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Clone Cache`.
114
115 This step has two possible modes of operation. If the build is part
116 of a parent build, its possible that all the repositories needed may
117 already be available, ready in a pre-prepared directory. An "a-quick"
118 or "a-full" build would prepare this before starting the other
119 sub-target builds. This is done for two reasons:
120
121 - the upstream may change during a build, for example, from a forced
122 push and this ensures we have matching content for the whole build
123
124 - if 15 Workers all tried to pull the same data from the same repos,
125 we can hit resource limits on upstream servers as they can think
126 they are under some kind of network attack
127
128 This pre-prepared directory is shared among the Workers over NFS. If
129 the build is an individual build and there is no "shared" directory
130 available, it would clone from the cache and the upstreams as
131 necessary. This is considered the fallback mode.
132
133#. *Call scripts/run-config*
134
135 This is another call into the Helper scripts where its expected that
136 the main functionality of this target will be executed.
137
138.. _test-autobuilder-tech:
139
140Autobuilder Technology
141======================
142
143The Autobuilder has Yocto Project-specific functionality to allow builds
144to operate with increased efficiency and speed.
145
146.. _test-clobberdir:
147
148clobberdir
149----------
150
151When deleting files, the Autobuilder uses ``clobberdir``, which is a
152special script that moves files to a special location, rather than
153deleting them. Files in this location are deleted by an ``rm`` command,
154which is run under ``ionice -c 3``. For example, the deletion only
155happens when there is idle IO capacity on the Worker. The Autobuilder
156Worker Janitor runs this deletion. See :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Worker Janitor`.
157
158.. _test-autobuilder-clone-cache:
159
160Autobuilder Clone Cache
161-----------------------
162
163Cloning repositories from scratch each time they are required was slow
164on the Autobuilder. We therefore have a stash of commonly used
165repositories pre-cloned on the Workers. Data is fetched from these
166during clones first, then "topped up" with later revisions from any
167upstream when necesary. The cache is maintained by the Autobuilder
168Worker Janitor. See :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Worker Janitor`.
169
170.. _test-autobuilder-worker-janitor:
171
172Autobuilder Worker Janitor
173--------------------------
174
175This is a process running on each Worker that performs two basic
176operations, including background file deletion at IO idle (see :ref:`test-manual/test-manual-understand-autobuilder:Autobuilder Target Execution Overview`: Run clobberdir) and
177maintainenance of a cache of cloned repositories to improve the speed
178the system can checkout repositories.
179
180.. _test-shared-dl-dir:
181
182Shared DL_DIR
183-------------
184
185The Workers are all connected over NFS which allows DL_DIR to be shared
186between them. This reduces network accesses from the system and allows
187the build to be sped up. Usage of the directory within the build system
188is designed to be able to be shared over NFS.
189
190.. _test-shared-sstate-cache:
191
192Shared SSTATE_DIR
193-----------------
194
195The Workers are all connected over NFS which allows the ``sstate``
196directory to be shared between them. This means once a Worker has built
197an artifact, all the others can benefit from it. Usage of the directory
198within the directory is designed for sharing over NFS.
199
200.. _test-resulttool:
201
202Resulttool
203----------
204
205All of the different tests run as part of the build generate output into
206``testresults.json`` files. This allows us to determine which tests ran
207in a given build and their status. Additional information, such as
208failure logs or the time taken to run the tests, may also be included.
209
210Resulttool is part of OpenEmbedded-Core and is used to manipulate these
211json results files. It has the ability to merge files together, display
212reports of the test results and compare different result files.
213
214For details, see :yocto_wiki:`/wiki/Resulttool`.
215
216.. _test-run-config-tgt-execution:
217
218run-config Target Execution
219===========================
220
221The ``scripts/run-config`` execution is where most of the work within
222the Autobuilder happens. It runs through a number of steps; the first
223are general setup steps that are run once and include:
224
225#. Set up any ``buildtools-tarball`` if configured.
226
227#. Call "buildhistory-init" if buildhistory is configured.
228
229For each step that is configured in ``config.json``, it will perform the
230following:
231
232#. Add any layers that are specified using the
233 ``bitbake-layers add-layer`` command (logging as stepXa)
234
235#. Call the ``scripts/setup-config`` script to generate the necessary
236 ``auto.conf`` configuration file for the build
237
238#. Run the ``bitbake BBTARGETS`` command (logging as stepXb)
239
240#. Run the ``bitbake SANITYTARGETS`` command (logging as stepXc)
241
242#. Run the ``EXTRACMDS`` command, which are run within the BitBake build
243 environment (logging as stepXd)
244
245#. Run the ``EXTRAPLAINCMDS`` command(s), which are run outside the
246 BitBake build environment (logging as stepXd)
247
248#. Remove any layers added in step
249 1 using the ``bitbake-layers remove-layer`` command (logging as stepXa)
250
251Once the execution steps above complete, ``run-config`` executes a set
252of post-build steps, including:
253
254#. Call ``scripts/publish-artifacts`` to collect any output which is to
255 be saved from the build.
256
257#. Call ``scripts/collect-results`` to collect any test results to be
258 saved from the build.
259
260#. Call ``scripts/upload-error-reports`` to send any error reports
261 generated to the remote server.
262
263#. Cleanup the build directory using
264 :ref:`test-manual/test-manual-understand-autobuilder:clobberdir` if the build was successful,
265 else rename it to "build-renamed" for potential future debugging.
266
267.. _test-deploying-yp-autobuilder:
268
269Deploying Yocto Autobuilder
270===========================
271
272The most up to date information about how to setup and deploy your own
273Autbuilder can be found in README.md in the ``yocto-autobuilder2``
274repository.
275
276We hope that people can use the ``yocto-autobuilder2`` code directly but
277it is inevitable that users will end up needing to heavily customise the
278``yocto-autobuilder-helper`` repository, particularly the
279``config.json`` file as they will want to define their own test matrix.
280
281The Autobuilder supports wo customization options:
282
283- variable substitution
284
285- overlaying configuration files
286
287The standard ``config.json`` minimally attempts to allow substitution of
288the paths. The Helper script repository includes a
289``local-example.json`` file to show how you could override these from a
290separate configuration file. Pass the following into the environment of
291the Autobuilder::
292
293 $ ABHELPER_JSON="config.json local-example.json"
294
295As another example, you could also pass the following into the
296environment::
297
298 $ ABHELPER_JSON="config.json /some/location/local.json"
299
300One issue users often run into is validation of the ``config.json`` files. A
301tip for minimizing issues from invalid json files is to use a Git
302``pre-commit-hook.sh`` script to verify the JSON file before committing
303it. Create a symbolic link as follows::
304
305 $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
diff --git a/documentation/test-manual/test-manual-understand-autobuilder.xml b/documentation/test-manual/test-manual-understand-autobuilder.xml
new file mode 100644
index 0000000000..8600367be7
--- /dev/null
+++ b/documentation/test-manual/test-manual-understand-autobuilder.xml
@@ -0,0 +1,314 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<chapter id='test-manual-understand-autobuilder'>
7
8<title>Understanding the Yocto Project Autobuilder</title>
9 <section>
10 <title>Execution Flow within the Autobuilder</title>
11 <para>The "a-full" and "a-quick" targets are the usual entry points into the Autobuilder and
12 it makes sense to follow the process through the system starting there. This is best
13 visualised from the Autobuilder Console view (<link linkend=""
14 >https://autobuilder.yoctoproject.org/typhoon/#/console</link>). </para>
15 <para>Each item along the top of that view represents some "target build" and these targets
16 are all run in parallel. The 'full' build will trigger the majority of them, the "quick"
17 build will trigger some subset of them. The Autobuilder effectively runs whichever
18 configuration is defined for each of those targets on a seperate buildbot worker. To
19 understand the configuration, you need to look at the entry on
20 <filename>config.json</filename> file within the
21 <filename>yocto-autobuilder-helper</filename> repository. The targets are defined in
22 the ‘overrides' section, a quick example could be qemux86-64 which looks
23 like:<literallayout class="monospaced">
24 "qemux86-64" : {
25 "MACHINE" : "qemux86-64",
26 "TEMPLATE" : "arch-qemu",
27 "step1" : {
28 "extravars" : [
29 "IMAGE_FSTYPES_append = ' wic wic.bmap'"
30 ]
31 }
32 },
33 </literallayout>And
34 to expand that, you need the "arch-qemu" entry from the "templates" section, which looks
35 like:<literallayout class="monospaced">
36 "arch-qemu" : {
37 "BUILDINFO" : true,
38 "BUILDHISTORY" : true,
39 "step1" : {
40 "BBTARGETS" : "core-image-sato core-image-sato-dev core-image-sato-sdk core-image-minimal core-image-minimal-dev core-image-sato:do_populate_sdk",
41 "SANITYTARGETS" : "core-image-minimal:do_testimage core-image-sato:do_testimage core-image-sato-sdk:do_testimage core-image-sato:do_testsdk"
42 },
43 "step2" : {
44 "SDKMACHINE" : "x86_64",
45 "BBTARGETS" : "core-image-sato:do_populate_sdk core-image-minimal:do_populate_sdk_ext core-image-sato:do_populate_sdk_ext",
46 "SANITYTARGETS" : "core-image-sato:do_testsdk core-image-minimal:do_testsdkext core-image-sato:do_testsdkext"
47 },
48 "step3" : {
49 "BUILDHISTORY" : false,
50 "EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1 oe-selftest ${HELPERSTMACHTARGS} -j 15"],
51 "ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
52 }
53 },
54 </literallayout>Combining
55 these two entries you can see that "qemux86-64" is a three step build where the
56 <filename>bitbake BBTARGETS</filename> would be run, then <filename>bitbake
57 SANITYTARGETS</filename> for each step; all for
58 <filename>MACHINE="qemx86-64"</filename> but with differing SDKMACHINE settings. In
59 step 1 an extra variable is added to the <filename>auto.conf</filename> file to enable
60 wic image generation.</para>
61 <para>While not every detail of this is covered here, you can see how the templating
62 mechanism allows quite complex configurations to be built up yet allows duplication and
63 repetition to be kept to a minimum.</para>
64 <para>The different build targets are designed to allow for parallelisation, so different
65 machines are usually built in parallel, operations using the same machine and metadata
66 are built sequentially, with the aim of trying to optimise build efficiency as much as
67 possible.</para>
68 <para>The <filename>config.json</filename> file is processed by the scripts in the Helper
69 repository in the <filename>scripts</filename> directory. The following section details
70 how this works.</para>
71 </section>
72
73 <section id='test-autobuilder-target-exec-overview'>
74 <title>Autobuilder Target Execution Overview</title>
75
76 <para>For each given target in a build, the Autobuilder executes several steps. These are
77 configured in <filename>yocto-autobuilder2/builders.py</filename> and roughly consist
78 of: <orderedlist>
79 <listitem id='test-list-tgt-exec-clobberdir'>
80 <para><emphasis>Run <filename>clobberdir</filename></emphasis></para>
81 <para>This cleans out any previous build. Old builds are left around to allow
82 easier debugging of failed builds. For additional information, see <link
83 linkend="test-clobberdir"><filename>clobberdir</filename></link>.</para>
84 </listitem>
85 <listitem>
86 <para><emphasis>Obtain yocto-autobuilder-helper</emphasis></para>
87 <para>This step clones the <filename>yocto-autobuilder-helper</filename> git
88 repository. This is necessary to prevent the requirement to maintain all the
89 release or project-specific code within Buildbot. The branch chosen matches
90 the release being built so we can support older releases and still make
91 changes in newer ones.</para>
92 </listitem>
93 <listitem>
94 <para><emphasis>Write layerinfo.json</emphasis></para>
95 <para>This transfers data in the Buildbot UI when the build was configured to
96 the Helper.</para>
97 </listitem>
98 <listitem>
99 <para><emphasis>Call scripts/shared-repo-unpack</emphasis></para>
100 <para>This is a call into the Helper scripts to set up a checkout of all the
101 pieces this build might need. It might clone the BitBake repository and the
102 OpenEmbedded-Core repository. It may clone the Poky repository, as well as
103 additional layers. It will use the data from the
104 <filename>layerinfo.json</filename> file to help understand the
105 configuration. It will also use a local cache of repositories to speed up
106 the clone checkouts. For additional information, see <link
107 linkend="test-autobuilder-clone-cache">Autobuilder Clone
108 Cache</link>.</para>
109 <para>This step has two possible modes of operation. If the build is part of a
110 parent build, its possible that all the repositories needed may already be
111 available, ready in a pre-prepared directory. An "a-quick" or "a-full" build
112 would prepare this before starting the other sub-target builds. This is done
113 for two reasons:<itemizedlist>
114 <listitem>
115 <para>the upstream may change during a build, for example, from a
116 forced push and this ensures we have matching content for the
117 whole build</para>
118 </listitem>
119 <listitem>
120 <para>if 15 Workers all tried to pull the same data from the same
121 repos, we can hit resource limits on upstream servers as they
122 can think they are under some kind of network attack</para>
123 </listitem>
124 </itemizedlist>This pre-prepared directory is shared among the Workers over
125 NFS. If the build is an individual build and there is no "shared" directory
126 available, it would clone from the cache and the upstreams as necessary.
127 This is considered the fallback mode.</para>
128 </listitem>
129 <listitem>
130 <para><emphasis>Call scripts/run-config</emphasis></para>
131 <para>This is another call into the Helper scripts where its expected that the
132 main functionality of this target will be executed.</para>
133 </listitem>
134 </orderedlist></para>
135 </section>
136 <section id='test-autobuilder-tech'>
137 <title>Autobuilder Technology</title>
138 <para>The Autobuilder has Yocto Project-specific functionality to allow builds to operate
139 with increased efficiency and speed.</para>
140 <section id='test-clobberdir'>
141 <title>clobberdir</title>
142 <para>When deleting files, the Autobuilder uses <filename>clobberdir</filename>, which
143 is a special script that moves files to a special location, rather than deleting
144 them. Files in this location are deleted by an <filename>rm</filename> command,
145 which is run under <filename>ionice -c 3</filename>. For example, the deletion only
146 happens when there is idle IO capacity on the Worker. The Autobuilder Worker Janitor
147 runs this deletion. See <link linkend="test-autobuilder-worker-janitor">Autobuilder
148 Worker Janitor</link>.</para>
149 </section>
150 <section id='test-autobuilder-clone-cache'>
151 <title>Autobuilder Clone Cache</title>
152 <para>Cloning repositories from scratch each time they are required was slow on the
153 Autobuilder. We therefore have a stash of commonly used repositories pre-cloned on
154 the Workers. Data is fetched from these during clones first, then "topped up" with
155 later revisions from any upstream when necesary. The cache is maintained by the
156 Autobuilder Worker Janitor. See <link linkend="test-autobuilder-worker-janitor"
157 >Autobuilder Worker Janitor</link>.</para>
158 </section>
159 <section id='test-autobuilder-worker-janitor'>
160 <title>Autobuilder Worker Janitor</title>
161 <para>This is a process running on each Worker that performs two basic operations,
162 including background file deletion at IO idle (see <link
163 linkend="test-list-tgt-exec-clobberdir">Target Execution: clobberdir</link>) and
164 maintainenance of a cache of cloned repositories to improve the speed the system can
165 checkout repositories.</para>
166 </section>
167 <section id='test-shared-dl-dir'>
168 <title>Shared DL_DIR</title>
169 <para>The Workers are all connected over NFS which allows DL_DIR to be shared between
170 them. This reduces network accesses from the system and allows the build to be sped
171 up. Usage of the directory within the build system is designed to be able to be
172 shared over NFS.</para>
173 </section>
174 <section id='test-shared-sstate-cache'>
175 <title>Shared SSTATE_DIR</title>
176 <para>The Workers are all connected over NFS which allows the
177 <filename>sstate</filename> directory to be shared between them. This means once
178 a Worker has built an artefact, all the others can benefit from it. Usage of the
179 directory within the directory is designed for sharing over NFS.</para>
180 </section>
181 <section id='test-resulttool'>
182 <title>Resulttool</title>
183 <para>All of the different tests run as part of the build generate output into
184 <filename>testresults.json</filename> files. This allows us to determine which
185 tests ran in a given build and their status. Additional information, such as failure
186 logs or the time taken to run the tests, may also be included.</para>
187 <para>Resulttool is part of OpenEmbedded-Core and is used to manipulate these json
188 results files. It has the ability to merge files together, display reports of the
189 test results and compare different result files.</para>
190 <para>For details, see <link linkend=""
191 >https://wiki.yoctoproject.org/wiki/Resulttool</link>.</para>
192 </section>
193 </section>
194 <section id='test-run-config-tgt-execution'>
195 <title>run-config Target Execution</title>
196 <para>The <filename>scripts/run-config</filename> execution is where most of the work within
197 the Autobuilder happens. It runs through a number of steps; the first are general setup
198 steps that are run once and include:<orderedlist>
199 <listitem>
200 <para>Set up any <filename>buildtools-tarball</filename> if configured.</para>
201 </listitem>
202 <listitem>
203 <para>Call "buildhistory-init" if buildhistory is configured.</para>
204 </listitem>
205 </orderedlist></para>
206 <para>For each step that is configured in <filename>config.json</filename>, it will perform
207 the following:</para>
208 <para>
209 <remark>## WRITER's question: What does "logging in as stepXa" and others refer to
210 below? ##</remark>
211 <orderedlist>
212 <listitem id="test-run-config-add-layers-step">
213 <para dir="ltr">Add any layers that are specified using the
214 <filename>bitbake-layers add-layer</filename> command (logging as
215 stepXa)</para>
216 </listitem>
217 <listitem>
218 <para dir="ltr">Call the <filename>scripts/setup-config</filename> script to
219 generate the necessary <filename>auto.conf</filename> configuration file for
220 the build</para>
221 </listitem>
222 <listitem>
223 <para dir="ltr">Run the <filename>bitbake BBTARGETS</filename> command (logging
224 as stepXb)</para>
225 </listitem>
226 <listitem>
227 <para dir="ltr">Run the <filename>bitbake SANITYTARGETS</filename> command
228 (logging as stepXc)</para>
229 </listitem>
230 <listitem>
231 <para dir="ltr">Run the <filename>EXTRACMDS</filename> command, which are run
232 within the BitBake build environment (logging as stepXd)</para>
233 </listitem>
234 <listitem>
235 <para dir="ltr">Run the <filename>EXTRAPLAINCMDS</filename> command(s), which
236 are run outside the BitBake build environment (logging as stepXd)</para>
237 </listitem>
238 <listitem>
239 <para dir="ltr">Remove any layers added in <link
240 linkend="test-run-config-add-layers-step">step 1</link> using the
241 <filename>bitbake-layers remove-layer</filename> command (logging as
242 stepXa)</para>
243 </listitem>
244 </orderedlist>
245 </para>
246 <para>Once the execution steps above complete, <filename>run-config</filename> executes a
247 set of post-build steps, including:<orderedlist>
248 <listitem>
249 <para dir="ltr">Call <filename>scripts/publish-artifacts</filename> to collect
250 any output which is to be saved from the build.</para>
251 </listitem>
252 <listitem>
253 <para dir="ltr">Call <filename>scripts/collect-results</filename> to collect any
254 test results to be saved from the build.</para>
255 </listitem>
256 <listitem>
257 <para dir="ltr">Call <filename>scripts/upload-error-reports</filename> to send
258 any error reports generated to the remote server.</para>
259 </listitem>
260 <listitem>
261 <para dir="ltr">Cleanup the build directory using <link
262 linkend="test-clobberdir"><filename>clobberdir</filename></link> if the
263 build was successful, else rename it to "build-renamed" for potential future
264 debugging.</para>
265 </listitem>
266 </orderedlist></para>
267 </section>
268 <section id='test-deploying-yp-autobuilder'>
269 <title>Deploying Yocto Autobuilder</title>
270 <para>The most up to date information about how to setup and deploy your own Autbuilder can
271 be found in README.md in the <filename>yocto-autobuilder2</filename> repository.</para>
272 <para>We hope that people can use the <filename>yocto-autobuilder2</filename> code directly
273 but it is inevitable that users will end up needing to heavily customise the
274 <filename>yocto-autobuilder-helper</filename> repository, particularly the
275 <filename>config.json</filename> file as they will want to define their own test
276 matrix.</para>
277 <para>The Autobuilder supports wo customization options: <itemizedlist>
278 <listitem>
279 <para>variable substitution</para>
280 </listitem>
281 <listitem>
282 <para>overlaying configuration files</para>
283 </listitem>
284 </itemizedlist>The standard <filename>config.json</filename> minimally attempts to allow
285 substitution of the paths. The Helper script repository includes a
286 <filename>local-example.json</filename> file to show how you could override these
287 from a separate configuration file. Pass the following into the environment of the
288 Autobuilder:<literallayout class="monospaced">
289 $ ABHELPER_JSON="config.json local-example.json"
290 </literallayout>As
291 another example, you could also pass the following into the
292 environment:<literallayout class="monospaced">
293 $ ABHELPER_JSON="config.json <replaceable>/some/location/</replaceable>local.json"
294 </literallayout>One
295 issue users often run into is validation of the <filename>config.json</filename> files.
296 A tip for minimizing issues from invalid json files is to use a Git
297 <filename>pre-commit-hook.sh</filename> script to verify the JSON file before
298 committing it. Create a symbolic link as
299 follows:<literallayout class="monospaced">
300 $ ln -s ../../scripts/pre-commit-hook.sh .git/hooks/pre-commit
301 </literallayout></para>
302 </section>
303
304
305
306
307
308
309
310
311</chapter>
312<!--
313vim: expandtab tw=80 ts=4
314-->
diff --git a/documentation/test-manual/test-manual.rst b/documentation/test-manual/test-manual.rst
new file mode 100644
index 0000000000..bd5b1b0968
--- /dev/null
+++ b/documentation/test-manual/test-manual.rst
@@ -0,0 +1,18 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=====================================
4Yocto Project Test Environment Manual
5=====================================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 test-manual-intro
14 test-manual-test-process
15 test-manual-understand-autobuilder
16 history
17
18.. include:: /boilerplate.rst
diff --git a/documentation/test-manual/test-manual.xml b/documentation/test-manual/test-manual.xml
new file mode 100755
index 0000000000..d454566c5f
--- /dev/null
+++ b/documentation/test-manual/test-manual.xml
@@ -0,0 +1,104 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
5
6<book id='test-manual' lang='en'
7 xmlns:xi="http://www.w3.org/2003/XInclude"
8 xmlns="http://docbook.org/ns/docbook"
9 >
10 <bookinfo>
11
12 <mediaobject>
13 <imageobject>
14 <imagedata fileref='figures/test-manual-title.png'
15 format='SVG'
16 align='left' scalefit='1' width='100%'/>
17 </imageobject>
18 </mediaobject>
19
20 <title>
21 Yocto Project Test Environment Manual
22 </title>
23
24 <authorgroup>
25 <author>
26 <affiliation>
27 <orgname>&ORGNAME;</orgname>
28 </affiliation>
29 <email>&ORGEMAIL;</email>
30 </author>
31 </authorgroup>
32
33 <revhistory>
34 <revision>
35 <revnumber>3.1.1</revnumber>
36 <date>TBD</date>
37 <revremark>DRAFT - Work-in-Progress - posted June 16, 2020</revremark>
38 </revision>
39 </revhistory>
40
41 <copyright>
42 <year>&COPYRIGHT_YEAR;</year>
43 <holder>Linux Foundation</holder>
44 </copyright>
45
46 <legalnotice>
47 <para>
48 Permission is granted to copy, distribute and/or modify this document under
49 the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">
50 Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by
51 Creative Commons.
52 </para>
53 <note><title>Manual Notes</title>
54 <itemizedlist>
55 <listitem><para>
56 This version of the
57 <emphasis>Yocto Project Test Environment Manual</emphasis>
58 is for the &YOCTO_DOC_VERSION; release of the
59 Yocto Project.
60 To be sure you have the latest version of the manual
61 for this release, go to the
62 <ulink url='&YOCTO_HOME_URL;/documentation'>Yocto Project documentation page</ulink>
63 and select the manual from that site.
64 Manuals from the site are more up-to-date than manuals
65 derived from the Yocto Project released TAR files.
66 </para></listitem>
67 <listitem><para>
68 If you located this manual through a web search, the
69 version of the manual might not be the one you want
70 (e.g. the search might have returned a manual much
71 older than the Yocto Project version with which you
72 are working).
73 You can see all Yocto Project major releases by
74 visiting the
75 <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Releases</ulink>
76 page.
77 If you need a version of this manual for a different
78 Yocto Project release, visit the
79 <ulink url='&YOCTO_HOME_URL;/documentation'>Yocto Project documentation page</ulink>
80 and select the manual set by using the
81 "ACTIVE RELEASES DOCUMENTATION" or "DOCUMENTS ARCHIVE"
82 pull-down menus.
83 </para></listitem>
84 <listitem><para>
85 To report any inaccuracies or problems with this
86 manual, send an email to the Yocto Project
87 discussion group at
88 <filename>yocto@yoctoproject.com</filename> or log into
89 the freenode <filename>#yocto</filename> channel.
90 </para></listitem>
91 </itemizedlist>
92 </note>
93 </legalnotice>
94
95 </bookinfo>
96
97 <xi:include href="test-manual-intro.xml"/>
98 <xi:include href="test-manual-test-process.xml"/>
99 <xi:include href="test-manual-understand-autobuilder.xml"/>
100
101</book>
102<!--
103vim: expandtab tw=80 ts=4
104-->
diff --git a/documentation/toaster-manual/history.rst b/documentation/toaster-manual/history.rst
new file mode 100644
index 0000000000..027b343d00
--- /dev/null
+++ b/documentation/toaster-manual/history.rst
@@ -0,0 +1,46 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3***********************
4Manual Revision History
5***********************
6
7.. list-table::
8 :widths: 10 15 40
9 :header-rows: 1
10
11 * - Revision
12 - Date
13 - Note
14 * - 1.8
15 - April 2015
16 - The initial document released with the Yocto Project 1.8 Release
17 * - 2.0
18 - October 2015
19 - Released with the Yocto Project 2.0 Release.
20 * - 2.1
21 - April 2016
22 - Released with the Yocto Project 2.1 Release.
23 * - 2.2
24 - October 2016
25 - Released with the Yocto Project 2.2 Release.
26 * - 2.3
27 - May 2017
28 - Released with the Yocto Project 2.3 Release.
29 * - 2.4
30 - October 2017
31 - Released with the Yocto Project 2.4 Release.
32 * - 2.5
33 - May 2018
34 - Released with the Yocto Project 2.5 Release.
35 * - 2.6
36 - November 2018
37 - Released with the Yocto Project 2.6 Release.
38 * - 2.7
39 - May 2019
40 - Released with the Yocto Project 2.7 Release.
41 * - 3.0
42 - October 2019
43 - Released with the Yocto Project 3.0 Release.
44 * - 3.1
45 - April 2020
46 - Released with the Yocto Project 3.1 Release.
diff --git a/documentation/toaster-manual/toaster-manual-customization.xsl b/documentation/toaster-manual/toaster-manual-customization.xsl
index d78694ac14..3a9b22eb8e 100644
--- a/documentation/toaster-manual/toaster-manual-customization.xsl
+++ b/documentation/toaster-manual/toaster-manual-customization.xsl
@@ -1,4 +1,6 @@
1<?xml version='1.0'?> 1<?xml version='1.0'?>
2<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
3
2<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0"> 4<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
3 5
4 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" /> 6 <xsl:import href="http://downloads.yoctoproject.org/mirror/docbook-mirror/docbook-xsl-1.76.1/xhtml/docbook.xsl" />
diff --git a/documentation/toaster-manual/toaster-manual-intro.rst b/documentation/toaster-manual/toaster-manual-intro.rst
new file mode 100644
index 0000000000..0b7cd41c8f
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-intro.rst
@@ -0,0 +1,105 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3************
4Introduction
5************
6
7Toaster is a web interface to the Yocto Project's
8:term:`OpenEmbedded Build System`. The interface
9enables you to configure and run your builds. Information about builds
10is collected and stored in a database. You can use Toaster to configure
11and start builds on multiple remote build servers.
12
13.. _intro-features:
14
15Toaster Features
16================
17
18Toaster allows you to configure and run builds, and it provides
19extensive information about the build process.
20
21- *Configure and Run Builds:* You can use the Toaster web interface to
22 configure and start your builds. Builds started using the Toaster web
23 interface are organized into projects. When you create a project, you
24 are asked to select a release, or version of the build system you
25 want to use for the project builds. As shipped, Toaster supports
26 Yocto Project releases 1.8 and beyond. With the Toaster web
27 interface, you can:
28
29 - Browse layers listed in the various
30 :ref:`layer sources <toaster-manual/toaster-manual-reference:layer source>`
31 that are available in your project (e.g. the OpenEmbedded Layer Index at
32 http://layers.openembedded.org/layerindex/).
33
34 - Browse images, recipes, and machines provided by those layers.
35
36 - Import your own layers for building.
37
38 - Add and remove layers from your configuration.
39
40 - Set configuration variables.
41
42 - Select a target or multiple targets to build.
43
44 - Start your builds.
45
46 Toaster also allows you to configure and run your builds from the
47 command line, and switch between the command line and the web
48 interface at any time. Builds started from the command line appear
49 within a special Toaster project called "Command line builds".
50
51- *Information About the Build Process:* Toaster also records extensive
52 information about your builds. Toaster collects data for builds you
53 start from the web interface and from the command line as long as
54 Toaster is running.
55
56 .. note::
57
58 You must start Toaster before the build or it will not collect
59 build data.
60
61 With Toaster you can:
62
63 - See what was built (recipes and packages) and what packages were
64 installed into your final image.
65
66 - Browse the directory structure of your image.
67
68 - See the value of all variables in your build configuration, and
69 which files set each value.
70
71 - Examine error, warning, and trace messages to aid in debugging.
72
73 - See information about the BitBake tasks executed and reused during
74 your build, including those that used shared state.
75
76 - See dependency relationships between recipes, packages, and tasks.
77
78 - See performance information such as build time, task time, CPU
79 usage, and disk I/O.
80
81For an overview of Toaster shipped with the Yocto Project &DISTRO;
82Release, see the "`Toaster - Yocto Project
832.2 <https://youtu.be/BlXdOYLgPxA>`__" video.
84
85.. _toaster-installation-options:
86
87Installation Options
88====================
89
90You can set Toaster up to run as a local instance or as a shared hosted
91service.
92
93When Toaster is set up as a local instance, all the components reside on
94a single build host. Fundamentally, a local instance of Toaster is
95suited for a single user developing on a single build host.
96
97.. image:: figures/simple-configuration.png
98 :align: center
99
100Toaster as a hosted service is suited for multiple users developing
101across several build hosts. When Toaster is set up as a hosted service,
102its components can be spread across several machines:
103
104.. image:: figures/hosted-service.png
105 :align: center
diff --git a/documentation/toaster-manual/toaster-manual-intro.xml b/documentation/toaster-manual/toaster-manual-intro.xml
index e84964c4e1..6ee9ec720a 100644
--- a/documentation/toaster-manual/toaster-manual-intro.xml
+++ b/documentation/toaster-manual/toaster-manual-intro.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='toaster-manual-intro'> 6<chapter id='toaster-manual-intro'>
6<title>Introduction</title> 7<title>Introduction</title>
diff --git a/documentation/toaster-manual/toaster-manual-reference.rst b/documentation/toaster-manual/toaster-manual-reference.rst
new file mode 100644
index 0000000000..e95536e052
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-reference.rst
@@ -0,0 +1,662 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3**********************
4Concepts and Reference
5**********************
6
7In order to configure and use Toaster, you should understand some
8concepts and have some basic command reference material available. This
9final chapter provides conceptual information on layer sources,
10releases, and JSON configuration files. Also provided is a quick look at
11some useful ``manage.py`` commands that are Toaster-specific.
12Information on ``manage.py`` commands does exist across the Web and the
13information in this manual by no means attempts to provide a command
14comprehensive reference.
15
16Layer Source
17============
18
19In general, a "layer source" is a source of information about existing
20layers. In particular, we are concerned with layers that you can use
21with the Yocto Project and Toaster. This chapter describes a particular
22type of layer source called a "layer index."
23
24A layer index is a web application that contains information about a set
25of custom layers. A good example of an existing layer index is the
26OpenEmbedded Layer Index. A public instance of this layer index exists
27at http://layers.openembedded.org. You can find the code for this
28layer index's web application at
29http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/.
30
31When you tie a layer source into Toaster, it can query the layer source
32through a
33`REST <http://en.wikipedia.org/wiki/Representational_state_transfer>`__
34API, store the information about the layers in the Toaster database, and
35then show the information to users. Users are then able to view that
36information and build layers from Toaster itself without worrying about
37cloning or editing the BitBake layers configuration file
38``bblayers.conf``.
39
40Tying a layer source into Toaster is convenient when you have many
41custom layers that need to be built on a regular basis by a community of
42developers. In fact, Toaster comes pre-configured with the OpenEmbedded
43Metadata Index.
44
45.. note::
46
47 You do not have to use a layer source to use Toaster. Tying into a
48 layer source is optional.
49
50.. _layer-source-using-with-toaster:
51
52Setting Up and Using a Layer Source
53-----------------------------------
54
55To use your own layer source, you need to set up the layer source and
56then tie it into Toaster. This section describes how to tie into a layer
57index in a manner similar to the way Toaster ties into the OpenEmbedded
58Metadata Index.
59
60Understanding Your Layers
61~~~~~~~~~~~~~~~~~~~~~~~~~
62
63The obvious first step for using a layer index is to have several custom
64layers that developers build and access using the Yocto Project on a
65regular basis. This set of layers needs to exist and you need to be
66familiar with where they reside. You will need that information when you
67set up the code for the web application that "hooks" into your set of
68layers.
69
70For general information on layers, see the
71":ref:`overview-manual/overview-manual-yp-intro:the yocto project layer model`"
72section in the Yocto Project Overview and Concepts Manual. For information on how
73to create layers, see the ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`"
74section in the Yocto Project Development Tasks Manual.
75
76.. _configuring-toaster-to-hook-into-your-layer-source:
77
78Configuring Toaster to Hook Into Your Layer Index
79~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
80
81If you want Toaster to use your layer index, you must host the web
82application in a server to which Toaster can connect. You also need to
83give Toaster the information about your layer index. In other words, you
84have to configure Toaster to use your layer index. This section
85describes two methods by which you can configure and use your layer
86index.
87
88In the previous section, the code for the OpenEmbedded Metadata Index
89(i.e. http://layers.openembedded.org) was referenced. You can use
90this code, which is at
91http://git.yoctoproject.org/cgit/cgit.cgi/layerindex-web/, as a
92base to create your own layer index.
93
94Use the Administration Interface
95^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
96
97Access the administration interface through a browser by entering the
98URL of your Toaster instance and adding "``/admin``" to the end of the
99URL. As an example, if you are running Toaster locally, use the
100following URL::
101
102 http://127.0.0.1:8000/admin
103
104The administration interface has a "Layer sources" section that includes
105an "Add layer source" button. Click that button and provide the required
106information. Make sure you select "layerindex" as the layer source type.
107
108Use the Fixture Feature
109^^^^^^^^^^^^^^^^^^^^^^^
110
111The Django fixture feature overrides the default layer server when you
112use it to specify a custom URL. To use the fixture feature, create (or
113edit) the file ``bitbake/lib/toaster.orm/fixtures/custom.xml``, and then
114set the following Toaster setting to your custom URL:
115
116.. code-block:: xml
117
118 <?xml version="1.0" ?>
119 <django-objects version="1.0">
120 <object model="orm.toastersetting" pk="100">
121 <field name="name" type="CharField">CUSTOM_LAYERINDEX_SERVER</field>
122 <field name="value" type="CharField">https://layers.my_organization.org/layerindex/branch/master/layers/</field>
123 </object>
124 <django-objects>
125
126When you start Toaster for the first time, or
127if you delete the file ``toaster.sqlite`` and restart, the database will
128populate cleanly from this layer index server.
129
130Once the information has been updated, verify the new layer information
131is available by using the Toaster web interface. To do that, visit the
132"All compatible layers" page inside a Toaster project. The layers from
133your layer source should be listed there.
134
135If you change the information in your layer index server, refresh the
136Toaster database by running the following command:
137
138.. code-block:: shell
139
140 $ bitbake/lib/toaster/manage.py lsupdates
141
142
143If Toaster can reach the API URL, you should see a message telling you that
144Toaster is updating the layer source information.
145
146.. _toaster-releases:
147
148Releases
149========
150
151When you create a Toaster project using the web interface, you are asked
152to choose a "Release." In the context of Toaster, the term "Release"
153refers to a set of layers and a BitBake version the OpenEmbedded build
154system uses to build something. As shipped, Toaster is pre-configured
155with releases that correspond to Yocto Project release branches.
156However, you can modify, delete, and create new releases according to
157your needs. This section provides some background information on
158releases.
159
160.. _toaster-releases-supported:
161
162Pre-Configured Releases
163-----------------------
164
165As shipped, Toaster is configured to use a specific set of releases. Of
166course, you can always configure Toaster to use any release. For
167example, you might want your project to build against a specific commit
168of any of the "out-of-the-box" releases. Or, you might want your project
169to build against different revisions of OpenEmbedded and BitBake.
170
171As shipped, Toaster is configured to work with the following releases:
172
173- *Yocto Project &DISTRO; "&DISTRO_NAME;" or OpenEmbedded "&DISTRO_NAME;":*
174 This release causes your Toaster projects to build against the head
175 of the &DISTRO_NAME_NO_CAP; branch at
176 https://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=&DISTRO_NAME_NO_CAP; or
177 http://git.openembedded.org/openembedded-core/commit/?h=&DISTRO_NAME_NO_CAP;.
178
179- *Yocto Project "Master" or OpenEmbedded "Master":* This release
180 causes your Toaster Projects to build against the head of the master
181 branch, which is where active development takes place, at
182 https://git.yoctoproject.org/cgit/cgit.cgi/poky/log/ or
183 http://git.openembedded.org/openembedded-core/log/.
184
185- *Local Yocto Project or Local OpenEmbedded:* This release causes your
186 Toaster Projects to build against the head of the ``poky`` or
187 ``openembedded-core`` clone you have local to the machine running
188 Toaster.
189
190Configuring Toaster
191===================
192
193In order to use Toaster, you must configure the database with the
194default content. The following subsections describe various aspects of
195Toaster configuration.
196
197Configuring the Workflow
198------------------------
199
200The ``bldcontrol/management/commands/checksettings.py`` file controls
201workflow configuration. The following steps outline the process to
202initially populate this database.
203
2041. The default project settings are set from
205 ``orm/fixtures/settings.xml``.
206
2072. The default project distro and layers are added from
208 ``orm/fixtures/poky.xml`` if poky is installed. If poky is not
209 installed, they are added from ``orm/fixtures/oe-core.xml``.
210
2113. If the ``orm/fixtures/custom.xml`` file exists, then its values are
212 added.
213
2144. The layer index is then scanned and added to the database.
215
216Once these steps complete, Toaster is set up and ready to use.
217
218Customizing Pre-Set Data
219------------------------
220
221The pre-set data for Toaster is easily customizable. You can create the
222``orm/fixtures/custom.xml`` file to customize the values that go into to
223the database. Customization is additive, and can either extend or
224completely replace the existing values.
225
226You use the ``orm/fixtures/custom.xml`` file to change the default
227project settings for the machine, distro, file images, and layers. When
228creating a new project, you can use the file to define the offered
229alternate project release selections. For example, you can add one or
230more additional selections that present custom layer sets or distros,
231and any other local or proprietary content.
232
233Additionally, you can completely disable the content from the
234``oe-core.xml`` and ``poky.xml`` files by defining the section shown
235below in the ``settings.xml`` file. For example, this option is
236particularly useful if your custom configuration defines fewer releases
237or layers than the default fixture files.
238
239The following example sets "name" to "CUSTOM_XML_ONLY" and its value to
240"True".
241
242.. code-block:: xml
243
244 <object model="orm.toastersetting" pk="99">
245 <field type="CharField" name="name">CUSTOM_XML_ONLY</field>
246 <field type="CharField" name="value">True</field>
247 </object>
248
249Understanding Fixture File Format
250---------------------------------
251
252The following is an overview of the file format used by the
253``oe-core.xml``, ``poky.xml``, and ``custom.xml`` files.
254
255The following subsections describe each of the sections in the fixture
256files, and outline an example section of the XML code. you can use to
257help understand this information and create a local ``custom.xml`` file.
258
259Defining the Default Distro and Other Values
260~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
261
262This section defines the default distro value for new projects. By
263default, it reserves the first Toaster Setting record "1". The following
264demonstrates how to set the project default value for
265:term:`DISTRO`:
266
267.. code-block:: xml
268
269 <!-- Set the project default value for DISTRO -->
270 <object model="orm.toastersetting" pk="1">
271 <field type="CharField" name="name">DEFCONF_DISTRO</field>
272 <field type="CharField" name="value">poky</field>
273 </object>
274
275You can override
276other default project values by adding additional Toaster Setting
277sections such as any of the settings coming from the ``settings.xml``
278file. Also, you can add custom values that are included in the BitBake
279environment. The "pk" values must be unique. By convention, values that
280set default project values have a "DEFCONF" prefix.
281
282Defining BitBake Version
283~~~~~~~~~~~~~~~~~~~~~~~~
284
285The following defines which version of BitBake is used for the following
286release selection:
287
288.. code-block:: xml
289
290 <!-- Bitbake versions which correspond to the metadata release -->
291 <object model="orm.bitbakeversion" pk="1">
292 <field type="CharField" name="name">&DISTRO_NAME_NO_CAP;</field>
293 <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
294 <field type="CharField" name="branch">&DISTRO_NAME_NO_CAP;</field>
295 <field type="CharField" name="dirpath">bitbake</field>
296 </object>
297
298.. _defining-releases:
299
300Defining Release
301~~~~~~~~~~~~~~~~
302
303The following defines the releases when you create a new project:
304
305.. code-block:: xml
306
307 <!-- Releases available -->
308 <object model="orm.release" pk="1">
309 <field type="CharField" name="name">&DISTRO_NAME_NO_CAP;</field>
310 <field type="CharField" name="description">Yocto Project &DISTRO; "&DISTRO_NAME;"</field>
311 <field rel="ManyToOneRel" to="orm.bitbakeversion" name="bitbake_version">1</field>
312 <field type="CharField" name="branch_name">&DISTRO_NAME_NO_CAP;</field>
313 <field type="TextField" name="helptext">Toaster will run your builds using the tip of the <a href="http://git.yoctoproject.org/cgit/cgit.cgi/poky/log/?h=&DISTRO_NAME_NO_CAP;">Yocto Project &DISTRO_NAME; branch</a>.</field>
314 </object>
315
316The "pk" value must match the above respective BitBake version record.
317
318Defining the Release Default Layer Names
319~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
320
321The following defines the default layers for each release:
322
323.. code-block:: xml
324
325 <!-- Default project layers for each release -->
326 <object model="orm.releasedefaultlayer" pk="1">
327 <field rel="ManyToOneRel" to="orm.release" name="release">1</field>
328 <field type="CharField" name="layer_name">openembedded-core</field>
329 </object>
330
331The 'pk' values in the example above should start at "1" and increment
332uniquely. You can use the same layer name in multiple releases.
333
334Defining Layer Definitions
335~~~~~~~~~~~~~~~~~~~~~~~~~~
336
337Layer definitions are the most complex. The following defines each of
338the layers, and then defines the exact layer version of the layer used
339for each respective release. You must have one ``orm.layer`` entry for
340each layer. Then, with each entry you need a set of
341``orm.layer_version`` entries that connects the layer with each release
342that includes the layer. In general all releases include the layer.
343
344.. code-block:: xml
345
346 <object model="orm.layer" pk="1">
347 <field type="CharField" name="name">openembedded-core</field>
348 <field type="CharField" name="layer_index_url"></field>
349 <field type="CharField" name="vcs_url">git://git.yoctoproject.org/poky</field>
350 <field type="CharField" name="vcs_web_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky</field>
351 <field type="CharField" name="vcs_web_tree_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field>
352 <field type="CharField" name="vcs_web_file_base_url">http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/%path%?h=%branch%</field>
353 </object>
354 <object model="orm.layer_version" pk="1">
355 <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
356 <field type="IntegerField" name="layer_source">0</field>
357 <field rel="ManyToOneRel" to="orm.release" name="release">1</field>
358 <field type="CharField" name="branch">&DISTRO_NAME_NO_CAP;</field>
359 <field type="CharField" name="dirpath">meta</field>
360 </object> <object model="orm.layer_version" pk="2">
361 <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
362 <field type="IntegerField" name="layer_source">0</field>
363 <field rel="ManyToOneRel" to="orm.release" name="release">2</field>
364 <field type="CharField" name="branch">HEAD</field>
365 <field type="CharField" name="commit">HEAD</field>
366 <field type="CharField" name="dirpath">meta</field>
367 </object>
368 <object model="orm.layer_version" pk="3">
369 <field rel="ManyToOneRel" to="orm.layer" name="layer">1</field>
370 <field type="IntegerField" name="layer_source">0</field>
371 <field rel="ManyToOneRel" to="orm.release" name="release">3</field>
372 <field type="CharField" name="branch">master</field>
373 <field type="CharField" name="dirpath">meta</field>
374 </object>
375
376The layer "pk" values above must be unique, and typically start at "1". The
377layer version "pk" values must also be unique across all layers, and typically
378start at "1".
379
380Remote Toaster Monitoring
381=========================
382
383Toaster has an API that allows remote management applications to
384directly query the state of the Toaster server and its builds in a
385machine-to-machine manner. This API uses the
386`REST <http://en.wikipedia.org/wiki/Representational_state_transfer>`__
387interface and the transfer of JSON files. For example, you might monitor
388a build inside a container through well supported known HTTP ports in
389order to easily access a Toaster server inside the container. In this
390example, when you use this direct JSON API, you avoid having web page
391parsing against the display the user sees.
392
393Checking Health
394---------------
395
396Before you use remote Toaster monitoring, you should do a health check.
397To do this, ping the Toaster server using the following call to see if
398it is still alive::
399
400 http://host:port/health
401
402Be sure to provide values for host and port. If the server is alive, you will
403get the response HTML:
404
405.. code-block:: html
406
407 <!DOCTYPE html>
408 <html lang="en">
409 <head><title>Toaster Health</title></head>
410 <body>Ok</body>
411 </html>
412
413Determining Status of Builds in Progress
414----------------------------------------
415
416Sometimes it is useful to determine the status of a build in progress.
417To get the status of pending builds, use the following call::
418
419 http://host:port/toastergui/api/building
420
421Be sure to provide values for host and port. The output is a JSON file that
422itemizes all builds in progress. This file includes the time in seconds since
423each respective build started as well as the progress of the cloning, parsing,
424and task execution. The following is sample output for a build in progress:
425
426.. code-block:: JSON
427
428 {"count": 1,
429 "building": [
430 {"machine": "beaglebone",
431 "seconds": "463.869",
432 "task": "927:2384",
433 "distro": "poky",
434 "clone": "1:1",
435 "id": 2,
436 "start": "2017-09-22T09:31:44.887Z",
437 "name": "20170922093200",
438 "parse": "818:818",
439 "project": "my_rocko",
440 "target": "core-image-minimal"
441 }]
442 }
443
444The JSON data for this query is returned in a
445single line. In the previous example the line has been artificially
446split for readability.
447
448Checking Status of Builds Completed
449-----------------------------------
450
451Once a build is completed, you get the status when you use the following
452call::
453
454 http://host:port/toastergui/api/builds
455
456Be sure to provide values for host and port. The output is a JSON file that
457itemizes all complete builds, and includes build summary information. The
458following is sample output for a completed build:
459
460.. code-block:: JSON
461
462 {"count": 1,
463 "builds": [
464 {"distro": "poky",
465 "errors": 0,
466 "machine": "beaglebone",
467 "project": "my_rocko",
468 "stop": "2017-09-22T09:26:36.017Z",
469 "target": "quilt-native",
470 "seconds": "78.193",
471 "outcome": "Succeeded",
472 "id": 1,
473 "start": "2017-09-22T09:25:17.824Z",
474 "warnings": 1,
475 "name": "20170922092618"
476 }]
477 }
478
479The JSON data for this query is returned in a single line. In the
480previous example the line has been artificially split for readability.
481
482Determining Status of a Specific Build
483--------------------------------------
484
485Sometimes it is useful to determine the status of a specific build. To
486get the status of a specific build, use the following call::
487
488 http://host:port/toastergui/api/build/ID
489
490Be sure to provide values for
491host, port, and ID. You can find the value for ID from the Builds
492Completed query. See the ":ref:`toaster-manual/toaster-manual-reference:checking status of builds completed`"
493section for more information.
494
495The output is a JSON file that itemizes the specific build and includes
496build summary information. The following is sample output for a specific
497build:
498
499.. code-block:: JSON
500
501 {"build":
502 {"distro": "poky",
503 "errors": 0,
504 "machine": "beaglebone",
505 "project": "my_rocko",
506 "stop": "2017-09-22T09:26:36.017Z",
507 "target": "quilt-native",
508 "seconds": "78.193",
509 "outcome": "Succeeded",
510 "id": 1,
511 "start": "2017-09-22T09:25:17.824Z",
512 "warnings": 1,
513 "name": "20170922092618",
514 "cooker_log": "/opt/user/poky/build-toaster-2/tmp/log/cooker/beaglebone/build_20170922_022607.991.log"
515 }
516 }
517
518The JSON data for this query is returned in a single line. In the
519previous example the line has been artificially split for readability.
520
521.. _toaster-useful-commands:
522
523Useful Commands
524===============
525
526In addition to the web user interface and the scripts that start and
527stop Toaster, command-line commands exist through the ``manage.py``
528management script. You can find general documentation on ``manage.py``
529at the
530`Django <https://docs.djangoproject.com/en/2.2/topics/settings/>`__
531site. However, several ``manage.py`` commands have been created that are
532specific to Toaster and are used to control configuration and back-end
533tasks. You can locate these commands in the
534:term:`Source Directory` (e.g. ``poky``) at
535``bitbake/lib/manage.py``. This section documents those commands.
536
537.. note::
538
539 - When using ``manage.py`` commands given a default configuration,
540 you must be sure that your working directory is set to the
541 :term:`Build Directory`. Using
542 ``manage.py`` commands from the Build Directory allows Toaster to
543 find the ``toaster.sqlite`` file, which is located in the Build
544 Directory.
545
546 - For non-default database configurations, it is possible that you
547 can use ``manage.py`` commands from a directory other than the
548 Build Directory. To do so, the ``toastermain/settings.py`` file
549 must be configured to point to the correct database backend.
550
551.. _toaster-command-buildslist:
552
553``buildslist``
554--------------
555
556The ``buildslist`` command lists all builds that Toaster has recorded.
557Access the command as follows:
558
559.. code-block:: shell
560
561 $ bitbake/lib/toaster/manage.py buildslist
562
563The command returns a list, which includes numeric
564identifications, of the builds that Toaster has recorded in the current
565database.
566
567You need to run the ``buildslist`` command first to identify existing
568builds in the database before using the
569:ref:`toaster-manual/toaster-manual-reference:\`\`builddelete\`\`` command. Here is an
570example that assumes default repository and build directory names:
571
572.. code-block:: shell
573
574 $ cd ~/poky/build
575 $ python ../bitbake/lib/toaster/manage.py buildslist
576
577If your Toaster database had only one build, the above
578:ref:`toaster-manual/toaster-manual-reference:\`\`buildslist\`\``
579command would return something like the following::
580
581 1: qemux86 poky core-image-minimal
582
583.. _toaster-command-builddelete:
584
585``builddelete``
586---------------
587
588The ``builddelete`` command deletes data associated with a build. Access
589the command as follows:
590
591.. code-block::
592
593 $ bitbake/lib/toaster/manage.py builddelete build_id
594
595The command deletes all the build data for the specified
596build_id. This command is useful for removing old and unused data from
597the database.
598
599Prior to running the ``builddelete`` command, you need to get the ID
600associated with builds by using the
601:ref:`toaster-manual/toaster-manual-reference:\`\`buildslist\`\`` command.
602
603.. _toaster-command-perf:
604
605``perf``
606--------
607
608The ``perf`` command measures Toaster performance. Access the command as
609follows:
610
611.. code-block:: shell
612
613 $ bitbake/lib/toaster/manage.py perf
614
615The command is a sanity check that returns page loading times in order to
616identify performance problems.
617
618.. _toaster-command-checksettings:
619
620``checksettings``
621-----------------
622
623The ``checksettings`` command verifies existing Toaster settings. Access
624the command as follows:
625
626.. code-block:: shell
627
628 $ bitbake/lib/toaster/manage.py checksettings
629
630Toaster uses settings that are based on the database to configure the
631building tasks. The ``checksettings`` command verifies that the database
632settings are valid in the sense that they have the minimal information
633needed to start a build.
634
635In order for the ``checksettings`` command to work, the database must be
636correctly set up and not have existing data. To be sure the database is
637ready, you can run the following:
638
639.. code-block:: shell
640
641 $ bitbake/lib/toaster/manage.py syncdb
642 $ bitbake/lib/toaster/manage.py migrate orm
643 $ bitbake/lib/toaster/manage.py migrate bldcontrol
644
645After running these commands, you can run the ``checksettings`` command.
646
647.. _toaster-command-runbuilds:
648
649``runbuilds``
650-------------
651
652The ``runbuilds`` command launches scheduled builds. Access the command
653as follows:
654
655.. code-block:: shell
656
657 $ bitbake/lib/toaster/manage.py runbuilds
658
659The ``runbuilds`` command checks if scheduled builds exist in the database
660and then launches them per schedule. The command returns after the builds
661start but before they complete. The Toaster Logging Interface records and
662updates the database when the builds complete.
diff --git a/documentation/toaster-manual/toaster-manual-reference.xml b/documentation/toaster-manual/toaster-manual-reference.xml
index 7440580e7c..ae267f4184 100644
--- a/documentation/toaster-manual/toaster-manual-reference.xml
+++ b/documentation/toaster-manual/toaster-manual-reference.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='toaster-manual-reference'> 6<chapter id='toaster-manual-reference'>
6 7
diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.rst b/documentation/toaster-manual/toaster-manual-setup-and-use.rst
new file mode 100644
index 0000000000..01c0dce41f
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.rst
@@ -0,0 +1,651 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2.. Set default pygment highlighting to 'shell' for this document
3.. highlight:: shell
4
5****************************
6Setting Up and Using Toaster
7****************************
8
9Starting Toaster for Local Development
10======================================
11
12Once you have set up the Yocto Project and installed the Toaster system
13dependencies as described in the ":ref:`toaster-manual/toaster-manual-start:Preparing to Use
14Toaster`" chapter, you are ready to start
15Toaster.
16
17Navigate to the root of your
18:term:`Source Directory` (e.g. ``poky``)::
19
20 $ cd poky
21
22Once in that directory, source the build environment script::
23
24 $ source oe-init-build-env
25
26Next, from the build directory (e.g.
27``poky/build``), start Toaster using this command::
28
29 $ source toaster start
30
31You can now run your builds from the command line, or with Toaster
32as explained in section
33":ref:`toaster-manual/toaster-manual-setup-and-use:using the toaster web interface`".
34
35To access the Toaster web interface, open your favorite browser and
36enter the following::
37
38 http://127.0.0.1:8000
39
40Setting a Different Port
41========================
42
43By default, Toaster starts on port 8000. You can use the ``WEBPORT``
44parameter to set a different port. For example, the following command
45sets the port to "8400"::
46
47 $ source toaster start webport=8400
48
49Setting Up Toaster Without a Web Server
50=======================================
51
52You can start a Toaster environment without starting its web server.
53This is useful for the following:
54
55- Capturing a command-line build's statistics into the Toaster database
56 for examination later.
57
58- Capturing a command-line build's statistics when the Toaster server
59 is already running.
60
61- Having one instance of the Toaster web server track and capture
62 multiple command-line builds, where each build is started in its own
63 "noweb" Toaster environment.
64
65The following commands show how to start a Toaster environment without
66starting its web server, perform BitBake operations, and then shut down
67the Toaster environment. Once the build is complete, you can close the
68Toaster environment. Before closing the environment, however, you should
69allow a few minutes to ensure the complete transfer of its BitBake build
70statistics to the Toaster database. If you have a separate Toaster web
71server instance running, you can watch this command-line build's
72progress and examine the results as soon as they are posted::
73
74 $ source toaster start noweb
75 $ bitbake target
76 $ source toaster stop
77
78Setting Up Toaster Without a Build Server
79=========================================
80
81You can start a Toaster environment with the "New Projects" feature
82disabled. Doing so is useful for the following:
83
84- Sharing your build results over the web server while blocking others
85 from starting builds on your host.
86
87- Allowing only local command-line builds to be captured into the
88 Toaster database.
89
90Use the following command to set up Toaster without a build server::
91
92 $ source toaster start nobuild webport=port
93
94Setting up External Access
95==========================
96
97By default, Toaster binds to the loop back address (i.e. ``localhost``),
98which does not allow access from external hosts. To allow external
99access, use the ``WEBPORT`` parameter to open an address that connects
100to the network, specifically the IP address that your NIC uses to
101connect to the network. You can also bind to all IP addresses the
102computer supports by using the shortcut "0.0.0.0:port".
103
104The following example binds to all IP addresses on the host::
105
106 $ source toaster start webport=0.0.0.0:8400
107
108This example binds to a specific IP address on the host's NIC::
109
110 $ source toaster start webport=192.168.1.1:8400
111
112The Directory for Cloning Layers
113================================
114
115Toaster creates a ``_toaster_clones`` directory inside your Source
116Directory (i.e. ``poky``) to clone any layers needed for your builds.
117
118Alternatively, if you would like all of your Toaster related files and
119directories to be in a particular location other than the default, you
120can set the ``TOASTER_DIR`` environment variable, which takes precedence
121over your current working directory. Setting this environment variable
122causes Toaster to create and use ``$TOASTER_DIR./_toaster_clones``.
123
124.. _toaster-the-build-directory:
125
126The Build Directory
127===================
128
129Toaster creates a build directory within your Source Directory (e.g.
130``poky``) to execute the builds.
131
132Alternatively, if you would like all of your Toaster related files and
133directories to be in a particular location, you can set the
134``TOASTER_DIR`` environment variable, which takes precedence over your
135current working directory. Setting this environment variable causes
136Toaster to use ``$TOASTER_DIR/build`` as the build directory.
137
138.. _toaster-creating-a-django-super-user:
139
140Creating a Django Superuser
141===========================
142
143Toaster is built on the `Django
144framework <https://www.djangoproject.com/>`__. Django provides an
145administration interface you can use to edit Toaster configuration
146parameters.
147
148To access the Django administration interface, you must create a
149superuser by following these steps:
150
151#. If you used ``pip3``, which is recommended, to set up the Toaster
152 system dependencies, you need be sure the local user path is in your
153 ``PATH`` list. To append the pip3 local user path, use the following
154 command::
155
156 $ export PATH=$PATH:$HOME/.local/bin
157
158#. From the directory containing the Toaster database, which by default
159 is the :term:`Build Directory`,
160 invoke the ``createsuperuser`` command from ``manage.py``::
161
162 $ cd ~/poky/build
163 $ ../bitbake/lib/toaster/manage.py createsuperuser
164
165#. Django prompts you for the username, which you need to provide.
166
167#. Django prompts you for an email address, which is optional.
168
169#. Django prompts you for a password, which you must provide.
170
171#. Django prompts you to re-enter your password for verification.
172
173After completing these steps, the following confirmation message
174appears::
175
176 Superuser created successfully.
177
178Creating a superuser allows you to access the Django administration
179interface through a browser. The URL for this interface is the same as
180the URL used for the Toaster instance with "/admin" on the end. For
181example, if you are running Toaster locally, use the following URL::
182
183 http://127.0.0.1:8000/admin
184
185You can use the Django administration interface to set Toaster configuration
186parameters such as the build directory, layer sources, default variable
187values, and BitBake versions.
188
189.. _toaster-setting-up-a-production-instance-of-toaster:
190
191Setting Up a Production Instance of Toaster
192===========================================
193
194You can use a production instance of Toaster to share the Toaster
195instance with remote users, multiple users, or both. The production
196instance is also the setup that can handle heavier loads on the web
197service. Use the instructions in the following sections to set up
198Toaster to run builds through the Toaster web interface.
199
200.. _toaster-production-instance-requirements:
201
202Requirements
203------------
204
205Be sure you meet the following requirements:
206
207.. note::
208
209 You must comply with all Apache, ``mod-wsgi``, and Mysql requirements.
210
211- Have all the build requirements as described in the ":ref:`toaster-manual/toaster-manual-start:Preparing to
212 Use Toaster`" chapter.
213
214- Have an Apache webserver.
215
216- Have ``mod-wsgi`` for the Apache webserver.
217
218- Use the Mysql database server.
219
220- If you are using Ubuntu, run the following::
221
222 $ sudo apt-get install apache2 libapache2-mod-wsgi-py3 mysql-server python3-pip libmysqlclient-dev
223
224- If you are using Fedora or a RedHat distribution, run the
225 following::
226
227 $ sudo dnf install httpd python3-mod_wsgi python3-pip mariadb-server mariadb-devel python3-devel
228
229- If you are using openSUSE, run the following::
230
231 $ sudo zypper install apache2 apache2-mod_wsgi-python3 python3-pip mariadb mariadb-client python3-devel
232
233.. _toaster-installation-steps:
234
235Installation
236------------
237
238Perform the following steps to install Toaster:
239
240#. Create toaster user and set its home directory to
241 ``/var/www/toaster``::
242
243 $ sudo /usr/sbin/useradd toaster -md /var/www/toaster -s /bin/false
244 $ sudo su - toaster -s /bin/bash
245
246#. Checkout a copy of ``poky`` into the web server directory. You will
247 be using ``/var/www/toaster``::
248
249 $ git clone git://git.yoctoproject.org/poky
250 $ git checkout &DISTRO_NAME_NO_CAP;
251
252#. Install Toaster dependencies using the --user flag which keeps the
253 Python packages isolated from your system-provided packages::
254
255 $ cd /var/www/toaster/
256 $ pip3 install --user -r ./poky/bitbake/toaster-requirements.txt
257 $ pip3 install --user mysqlclient
258
259 .. note::
260
261 Isolating these packages is not required but is recommended.
262 Alternatively, you can use your operating system's package
263 manager to install the packages.
264
265#. Configure Toaster by editing
266 ``/var/www/toaster/poky/bitbake/lib/toaster/toastermain/settings.py``
267 as follows:
268
269 - Edit the
270 `DATABASES <https://docs.djangoproject.com/en/2.2/ref/settings/#databases>`__
271 settings:
272
273 .. code-block:: python
274
275 DATABASES = {
276 'default': {
277 'ENGINE': 'django.db.backends.mysql',
278 'NAME': 'toaster_data',
279 'USER': 'toaster',
280 'PASSWORD': 'yourpasswordhere',
281 'HOST': 'localhost',
282 'PORT': '3306',
283 }
284 }
285
286 - Edit the
287 `SECRET_KEY <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-SECRET_KEY>`__:
288
289 .. code-block:: python
290
291 SECRET_KEY = 'your_secret_key'
292
293 - Edit the
294 `STATIC_ROOT <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-STATIC_ROOT>`__:
295
296 .. code-block:: python
297
298 STATIC_ROOT = '/var/www/toaster/static_files/'
299
300#. Add the database and user to the ``mysql`` server defined earlier::
301
302 $ mysql -u root -p
303 mysql> CREATE DATABASE toaster_data;
304 mysql> CREATE USER 'toaster'@'localhost' identified by 'yourpasswordhere';
305 mysql> GRANT all on toaster_data.\* to 'toaster'@'localhost';
306 mysql> quit
307
308#. Get Toaster to create the database schema, default data, and gather
309 the statically-served files::
310
311 $ cd /var/www/toaster/poky/
312 $ ./bitbake/lib/toaster/manage.py migrate
313 $ TOASTER_DIR=`pwd\` TEMPLATECONF='poky' \
314 ./bitbake/lib/toaster/manage.py checksettings
315 $ ./bitbake/lib/toaster/manage.py collectstatic
316
317
318 In the previous
319 example, from the ``poky`` directory, the ``migrate`` command
320 ensures the database schema changes have propagated correctly (i.e.
321 migrations). The next line sets the Toaster root directory
322 ``TOASTER_DIR`` and the location of the Toaster configuration file
323 ``TOASTER_CONF``, which is relative to ``TOASTER_DIR``. The
324 ``TEMPLATECONF`` value reflects the contents of
325 ``poky/.templateconf``, and by default, should include the string
326 "poky". For more information on the Toaster configuration file, see
327 the ":ref:`toaster-manual/toaster-manual-reference:Configuring Toaster`" section.
328
329 This line also runs the ``checksettings`` command, which configures
330 the location of the Toaster :term:`Build Directory`.
331 The Toaster
332 root directory ``TOASTER_DIR`` determines where the Toaster build
333 directory is created on the file system. In the example above,
334 ``TOASTER_DIR`` is set as follows::
335
336 /var/www/toaster/poky
337
338
339 This setting causes the Toaster build directory to be::
340
341 /var/www/toaster/poky/build
342
343 Finally, the ``collectstatic`` command is a Django framework command
344 that collects all the statically served files into a designated
345 directory to be served up by the Apache web server as defined by
346 ``STATIC_ROOT``.
347
348#. Test and/or use the Mysql integration with Toaster's Django web
349 server. At this point, you can start up the normal Toaster Django
350 web server with the Toaster database in Mysql. You can use this web
351 server to confirm that the database migration and data population
352 from the Layer Index is complete.
353
354 To start the default Toaster Django web server with the Toaster
355 database now in Mysql, use the standard start commands::
356
357 $ source oe-init-build-env
358 $ source toaster start
359
360 Additionally, if Django is sufficient for your requirements, you can use
361 it for your release system and migrate later to Apache as your
362 requirements change.
363
364#. Add an Apache configuration file for Toaster to your Apache web
365 server's configuration directory. If you are using Ubuntu or Debian,
366 put the file here::
367
368 /etc/apache2/conf-available/toaster.conf
369
370
371 If you are using Fedora or RedHat, put it here::
372
373 /etc/httpd/conf.d/toaster.conf
374
375 If you are using OpenSUSE, put it here::
376
377 /etc/apache2/conf.d/toaster.conf
378
379 Following is a sample Apache configuration for Toaster you can follow:
380
381 .. code-block:: apache
382
383 Alias /static /var/www/toaster/static_files
384 <Directory /var/www/toaster/static_files>
385 <IfModule mod_access_compat.c>
386 Order allow,deny
387 Allow from all
388 </IfModule>
389 <IfModule !mod_access_compat.c>
390 Require all granted
391 </IfModule>
392 </Directory>
393
394 <Directory /var/www/toaster/poky/bitbake/lib/toaster/toastermain>
395 <Files "wsgi.py">
396 Require all granted
397 </Files>
398 </Directory>
399
400 WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/.local/lib/python3.4/site-packages
401 WSGIScriptAlias / "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py"
402 <Location />
403 WSGIProcessGroup toaster_wsgi
404 </Location>
405
406
407 If you are using Ubuntu or Debian, you will need to enable the config and
408 module for Apache::
409
410 $ sudo a2enmod wsgi
411 $ sudo a2enconf toaster
412 $ chmod +x bitbake/lib/toaster/toastermain/wsgi.py
413
414 Finally, restart Apache to make sure all new configuration is loaded. For Ubuntu,
415 Debian, and OpenSUSE use::
416
417 $ sudo service apache2 restart
418
419 For Fedora and RedHat use::
420
421 $ sudo service httpd restart
422
423#. Prepare the systemd service to run Toaster builds. Here is a sample
424 configuration file for the service:
425
426 .. code-block:: ini
427
428 [Unit]
429 Description=Toaster runbuilds
430
431 [Service]
432 Type=forking User=toaster
433 ExecStart=/usr/bin/screen -d -m -S runbuilds /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
434 ExecStop=/usr/bin/screen -S runbuilds -X quit
435 WorkingDirectory=/var/www/toaster/poky
436
437 [Install]
438 WantedBy=multi-user.target
439
440
441 Prepare the ``runbuilds-service.sh`` script that you need to place in the
442 ``/var/www/toaster/poky/bitbake/lib/toaster/`` directory by setting
443 up executable permissions::
444
445 #!/bin/bash
446
447 #export http_proxy=http://proxy.host.com:8080
448 #export https_proxy=http://proxy.host.com:8080
449 #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy
450 cd ~/poky/
451 source ./oe-init-build-env build
452 source ../bitbake/bin/toaster $1 noweb
453 [ "$1" == 'start' ] && /bin/bash
454
455#. Run the service::
456
457 $ sudo service runbuilds start
458
459 Since the service is running in a detached screen session, you can
460 attach to it using this command::
461
462 $ sudo su - toaster
463 $ screen -rS runbuilds
464
465 You can detach from the service again using "Ctrl-a" followed by "d" key
466 combination.
467
468You can now open up a browser and start using Toaster.
469
470Using the Toaster Web Interface
471===============================
472
473The Toaster web interface allows you to do the following:
474
475- Browse published layers in the `OpenEmbedded Layer
476 Index <http://layers.openembedded.org>`__ that are available for your
477 selected version of the build system.
478
479- Import your own layers for building.
480
481- Add and remove layers from your configuration.
482
483- Set configuration variables.
484
485- Select a target or multiple targets to build.
486
487- Start your builds.
488
489- See what was built (recipes and packages) and what packages were
490 installed into your final image.
491
492- Browse the directory structure of your image.
493
494- See the value of all variables in your build configuration, and which
495 files set each value.
496
497- Examine error, warning and trace messages to aid in debugging.
498
499- See information about the BitBake tasks executed and reused during
500 your build, including those that used shared state.
501
502- See dependency relationships between recipes, packages and tasks.
503
504- See performance information such as build time, task time, CPU usage,
505 and disk I/O.
506
507.. _web-interface-videos:
508
509Toaster Web Interface Videos
510----------------------------
511
512Following are several videos that show how to use the Toaster GUI:
513
514- *Build Configuration:* This
515 `video <https://www.youtube.com/watch?v=qYgDZ8YzV6w>`__ overviews and
516 demonstrates build configuration for Toaster.
517
518- *Build Custom Layers:* This
519 `video <https://www.youtube.com/watch?v=QJzaE_XjX5c>`__ shows you how
520 to build custom layers that are used with Toaster.
521
522- *Toaster Homepage and Table Controls:* This
523 `video <https://www.youtube.com/watch?v=QEARDnrR1Xw>`__ goes over the
524 Toaster entry page, and provides an overview of the data manipulation
525 capabilities of Toaster, which include search, sorting and filtering
526 by different criteria.
527
528- *Build Dashboard:* This
529 `video <https://www.youtube.com/watch?v=KKqHYcnp2gE>`__ shows you the
530 build dashboard, a page providing an overview of the information
531 available for a selected build.
532
533- *Image Information:* This
534 `video <https://www.youtube.com/watch?v=XqYGFsmA0Rw>`__ walks through
535 the information Toaster provides about images: packages installed and
536 root file system.
537
538- *Configuration:* This
539 `video <https://www.youtube.com/watch?v=UW-j-T2TzIg>`__ provides
540 Toaster build configuration information.
541
542- *Tasks:* This `video <https://www.youtube.com/watch?v=D4-9vGSxQtw>`__
543 shows the information Toaster provides about the tasks run by the
544 build system.
545
546- *Recipes and Packages Built:* This
547 `video <https://www.youtube.com/watch?v=x-6dx4huNnw>`__ shows the
548 information Toaster provides about recipes and packages built.
549
550- *Performance Data:* This
551 `video <https://www.youtube.com/watch?v=qWGMrJoqusQ>`__ shows the
552 build performance data provided by Toaster.
553
554.. _a-note-on-the-local-yocto-project-release:
555
556Additional Information About the Local Yocto Project Release
557------------------------------------------------------------
558
559This section only applies if you have set up Toaster for local
560development, as explained in the
561":ref:`toaster-manual/toaster-manual-setup-and-use:starting toaster for local development`"
562section.
563
564When you create a project in Toaster, you will be asked to provide a
565name and to select a Yocto Project release. One of the release options
566you will find is called "Local Yocto Project".
567
568.. image:: figures/new-project.png
569 :align: center
570 :scale: 75%
571
572When you select the "Local Yocto Project" release, Toaster will run your
573builds using the local Yocto Project clone you have in your computer:
574the same clone you are using to run Toaster. Unless you manually update
575this clone, your builds will always use the same Git revision.
576
577If you select any of the other release options, Toaster will fetch the
578tip of your selected release from the upstream `Yocto Project
579repository <https://git.yoctoproject.org>`__ every time you run a build.
580Fetching this tip effectively means that if your selected release is
581updated upstream, the Git revision you are using for your builds will
582change. If you are doing development locally, you might not want this
583change to happen. In that case, the "Local Yocto Project" release might
584be the right choice.
585
586However, the "Local Yocto Project" release will not provide you with any
587compatible layers, other than the three core layers that come with the
588Yocto Project:
589
590- `openembedded-core <http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/>`__
591
592- `meta-poky <http://layers.openembedded.org/layerindex/branch/master/layer/meta-poky/>`__
593
594- `meta-yocto-bsp <http://layers.openembedded.org/layerindex/branch/master/layer/meta-yocto-bsp/>`__
595
596.. image:: figures/compatible-layers.png
597 :align: center
598 :scale: 75%
599
600If you want to build any other layers, you will need to manually import
601them into your Toaster project, using the "Import layer" page.
602
603.. image:: figures/import-layer.png
604 :align: center
605 :scale: 75%
606
607.. _toaster-web-interface-preferred-version:
608
609Building a Specific Recipe Given Multiple Versions
610--------------------------------------------------
611
612Occasionally, a layer might provide more than one version of the same
613recipe. For example, the ``openembedded-core`` layer provides two
614versions of the ``bash`` recipe (i.e. 3.2.48 and 4.3.30-r0) and two
615versions of the ``which`` recipe (i.e. 2.21 and 2.18). The following
616figure shows this exact scenario:
617
618.. image:: figures/bash-oecore.png
619 :align: center
620 :scale: 75%
621
622By default, the OpenEmbedded build system builds one of the two recipes.
623For the ``bash`` case, version 4.3.30-r0 is built by default.
624Unfortunately, Toaster as it exists, is not able to override the default
625recipe version. If you would like to build bash 3.2.48, you need to set
626the
627:term:`PREFERRED_VERSION`
628variable. You can do so from Toaster, using the "Add variable" form,
629which is available in the "BitBake variables" page of the project
630configuration section as shown in the following screen:
631
632.. image:: figures/add-variable.png
633 :align: center
634 :scale: 75%
635
636To specify ``bash`` 3.2.48 as the version to build, enter
637"PREFERRED_VERSION_bash" in the "Variable" field, and "3.2.48" in the
638"Value" field. Next, click the "Add variable" button:
639
640.. image:: figures/set-variable.png
641 :align: center
642 :scale: 75%
643
644After clicking the "Add variable" button, the settings for
645``PREFERRED_VERSION`` are added to the bottom of the BitBake variables
646list. With these settings, the OpenEmbedded build system builds the
647desired version of the recipe rather than the default version:
648
649.. image:: figures/variable-added.png
650 :align: center
651 :scale: 75%
diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index b4caebbe0f..f555745923 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='toaster-manual-setup-and-use'> 6<chapter id='toaster-manual-setup-and-use'>
6 7
@@ -69,17 +70,17 @@
69 web server. This is useful for the following: 70 web server. This is useful for the following:
70 <itemizedlist> 71 <itemizedlist>
71 <listitem><para> 72 <listitem><para>
72 Capturing a command-line builds statistics into 73 Capturing a command-line build's statistics into
73 the Toaster database for examination later. 74 the Toaster database for examination later.
74 </para></listitem> 75 </para></listitem>
75 <listitem><para> 76 <listitem><para>
76 Capturing a command-line builds statistics when 77 Capturing a command-line build's statistics when
77 the Toaster server is already running. 78 the Toaster server is already running.
78 </para></listitem> 79 </para></listitem>
79 <listitem><para> 80 <listitem><para>
80 Having one instance of the Toaster web server 81 Having one instance of the Toaster web server
81 track and capture multiple command-line builds, 82 track and capture multiple command-line builds,
82 where each build is started in its own noweb 83 where each build is started in its own "noweb"
83 Toaster environment. 84 Toaster environment.
84 </para></listitem> 85 </para></listitem>
85 </itemizedlist> 86 </itemizedlist>
@@ -91,7 +92,7 @@
91 minutes to ensure the complete transfer of its BitBake build 92 minutes to ensure the complete transfer of its BitBake build
92 statistics to the Toaster database. 93 statistics to the Toaster database.
93 If you have a separate Toaster web server instance running, you 94 If you have a separate Toaster web server instance running, you
94 can watch this command-line builds progress and examine the 95 can watch this command-line build's progress and examine the
95 results as soon as they are posted: 96 results as soon as they are posted:
96 <literallayout class='monospaced'> 97 <literallayout class='monospaced'>
97 $ source toaster start noweb 98 $ source toaster start noweb
@@ -106,7 +107,7 @@
106 107
107 <para> 108 <para>
108 You can start a Toaster environment with the 109 You can start a Toaster environment with the
109 New Projects feature disabled. 110 "New Projects" feature disabled.
110 Doing so is useful for the following: 111 Doing so is useful for the following:
111 <itemizedlist> 112 <itemizedlist>
112 <listitem><para> 113 <listitem><para>
@@ -469,7 +470,7 @@
469 <filename>STATIC_ROOT</filename>. 470 <filename>STATIC_ROOT</filename>.
470 </para></listitem> 471 </para></listitem>
471 <listitem><para> 472 <listitem><para>
472 Test and/or use the Mysql integration with Toasters 473 Test and/or use the Mysql integration with Toaster's
473 Django web server. 474 Django web server.
474 At this point, you can start up the normal Toaster 475 At this point, you can start up the normal Toaster
475 Django web server with the Toaster database in Mysql. 476 Django web server with the Toaster database in Mysql.
diff --git a/documentation/toaster-manual/toaster-manual-start.rst b/documentation/toaster-manual/toaster-manual-start.rst
new file mode 100644
index 0000000000..2d612b8938
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual-start.rst
@@ -0,0 +1,57 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2.. Set default pygments highlighting to shell for this document
3.. highlight:: shell
4
5************************
6Preparing to Use Toaster
7************************
8
9This chapter describes how you need to prepare your system in order to
10use Toaster.
11
12.. _toaster-setting-up-the-basic-system-requirements:
13
14Setting Up the Basic System Requirements
15========================================
16
17Before you can use Toaster, you need to first set up your build system
18to run the Yocto Project. To do this, follow the instructions in the
19":ref:`dev-manual/dev-manual-start:preparing the build host`" section of
20the Yocto Project Development Tasks Manual. For Ubuntu/Debian, you might
21also need to do an additional install of pip3. ::
22
23 $ sudo apt-get install python3-pip
24
25.. _toaster-establishing-toaster-system-dependencies:
26
27Establishing Toaster System Dependencies
28========================================
29
30Toaster requires extra Python dependencies in order to run. A Toaster
31requirements file named ``toaster-requirements.txt`` defines the Python
32dependencies. The requirements file is located in the ``bitbake``
33directory, which is located in the root directory of the
34:term:`Source Directory` (e.g.
35``poky/bitbake/toaster-requirements.txt``). The dependencies appear in a
36``pip``, install-compatible format.
37
38.. _toaster-load-packages:
39
40Install Toaster Packages
41------------------------
42
43You need to install the packages that Toaster requires. Use this
44command::
45
46 $ pip3 install --user -r bitbake/toaster-requirements.txt
47
48The previous command installs the necessary Toaster modules into a local
49python 3 cache in your ``$HOME`` directory. The caches is actually
50located in ``$HOME/.local``. To see what packages have been installed
51into your ``$HOME`` directory, do the following::
52
53 $ pip3 list installed --local
54
55If you need to remove something, the following works::
56
57 $ pip3 uninstall PackageNameToUninstall
diff --git a/documentation/toaster-manual/toaster-manual-start.xml b/documentation/toaster-manual/toaster-manual-start.xml
index fc187ecd5e..8a857006e5 100644
--- a/documentation/toaster-manual/toaster-manual-start.xml
+++ b/documentation/toaster-manual/toaster-manual-start.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<chapter id='toaster-manual-start'> 6<chapter id='toaster-manual-start'>
6 7
diff --git a/documentation/toaster-manual/toaster-manual-style.css b/documentation/toaster-manual/toaster-manual-style.css
index 6d6b9fb65d..a7f430df05 100644
--- a/documentation/toaster-manual/toaster-manual-style.css
+++ b/documentation/toaster-manual/toaster-manual-style.css
@@ -1,4 +1,7 @@
1/* 1/*
2
3 SPDX-License-Identifier: CC-BY-2.0-UK
4
2 Generic XHTML / DocBook XHTML CSS Stylesheet. 5 Generic XHTML / DocBook XHTML CSS Stylesheet.
3 6
4 Browser wrangling and typographic design by 7 Browser wrangling and typographic design by
diff --git a/documentation/toaster-manual/toaster-manual.rst b/documentation/toaster-manual/toaster-manual.rst
new file mode 100644
index 0000000000..f6f59411b6
--- /dev/null
+++ b/documentation/toaster-manual/toaster-manual.rst
@@ -0,0 +1,19 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3===================
4Toaster User Manual
5===================
6
7|
8
9.. toctree::
10 :caption: Table of Contents
11 :numbered:
12
13 toaster-manual-intro
14 toaster-manual-start
15 toaster-manual-setup-and-use
16 toaster-manual-reference
17 history
18
19.. include:: /boilerplate.rst
diff --git a/documentation/toaster-manual/toaster-manual.xml b/documentation/toaster-manual/toaster-manual.xml
index 03db6bed3a..136b4df964 100755
--- a/documentation/toaster-manual/toaster-manual.xml
+++ b/documentation/toaster-manual/toaster-manual.xml
@@ -1,6 +1,7 @@
1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > 3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4<!--SPDX-License-Identifier: CC-BY-2.0-UK-->
4 5
5<book id='toaster-manual' lang='en' 6<book id='toaster-manual' lang='en'
6 xmlns:xi="http://www.w3.org/2003/XInclude" 7 xmlns:xi="http://www.w3.org/2003/XInclude"
@@ -82,28 +83,8 @@
82 </revision> 83 </revision>
83 <revision> 84 <revision>
84 <revnumber>3.1</revnumber> 85 <revnumber>3.1</revnumber>
85 <date>April 2020</date>
86 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
87 </revision>
88 <revision>
89 <revnumber>3.1.1</revnumber>
90 <date>June 2020</date>
91 <revremark>Released with the Yocto Project 3.1.1 Release.</revremark>
92 </revision>
93 <revision>
94 <revnumber>3.1.2</revnumber>
95 <date>August 2020</date>
96 <revremark>Released with the Yocto Project 3.1.2 Release.</revremark>
97 </revision>
98 <revision>
99 <revnumber>3.1.3</revnumber>
100 <date>October 2020</date>
101 <revremark>Released with the Yocto Project 3.1.3 Release.</revremark>
102 </revision>
103 <revision>
104 <revnumber>3.1.4</revnumber>
105 <date>&REL_MONTH_YEAR;</date> 86 <date>&REL_MONTH_YEAR;</date>
106 <revremark>Released with the Yocto Project 3.1.4 Release.</revremark> 87 <revremark>Released with the Yocto Project 3.1 Release.</revremark>
107 </revision> 88 </revision>
108 </revhistory> 89 </revhistory>
109 90
diff --git a/documentation/tools/eclipse-help.sed b/documentation/tools/eclipse-help.sed
index ab5c9affd4..9716ea42bc 100644
--- a/documentation/tools/eclipse-help.sed
+++ b/documentation/tools/eclipse-help.sed
@@ -1,3 +1,6 @@
1#
2# SPDX-License-Identifier: CC-BY-2.0-UK
3#
1# Process poky-ref-manual and yocto-project-qs manual (<word>-<word>-<word> style) 4# Process poky-ref-manual and yocto-project-qs manual (<word>-<word>-<word> style)
2# For example: 5# For example:
3# "ulink" href="http://www.yoctoproject.org/docs/1.3/poky-ref-manual/poky-ref-manual.html#faq" 6# "ulink" href="http://www.yoctoproject.org/docs/1.3/poky-ref-manual/poky-ref-manual.html#faq"
diff --git a/documentation/tools/mega-manual.sed b/documentation/tools/mega-manual.sed
index 12e0e6145b..c525ab46a4 100644
--- a/documentation/tools/mega-manual.sed
+++ b/documentation/tools/mega-manual.sed
@@ -1,36 +1,39 @@
1#
2# SPDX-License-Identifier: CC-BY-2.0-UK
3#
1# Processes bitbake-user-manual (<word>-<word>-<word> style). 4# Processes bitbake-user-manual (<word>-<word>-<word> style).
2# This style is for manual three-word folders, which currently is only the BitBake User Manual. 5# This style is for manual three-word folders, which currently is only the BitBake User Manual.
3# We used to have the "yocto-project-qs" and "poky-ref-manual" folders but no longer do. 6# We used to have the "yocto-project-qs" and "poky-ref-manual" folders but no longer do.
4# s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/[a-z]*-[a-z]*-[a-z]*/[a-z]*-[a-z]*-[a-z]*.html#@"link" href="#@g 7# s@"ulink" href="http://www.yoctoproject.org/docs/3.1/[a-z]*-[a-z]*-[a-z]*/[a-z]*-[a-z]*-[a-z]*.html#@"link" href="#@g
5s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/bitbake-user-manual/bitbake-user-manual.html#@"link" href="#@g 8s@"ulink" href="http://www.yoctoproject.org/docs/3.1/bitbake-user-manual/bitbake-user-manual.html#@"link" href="#@g
6 9
7# Processes all other manuals (<word>-<word> style). 10# Processes all other manuals (<word>-<word> style).
8# This style is for manual folders that use two word, which is the standard now (e.g. "ref-manual"). 11# This style is for manual folders that use two word, which is the standard now (e.g. "ref-manual").
9# Here is the one-liner: 12# Here is the one-liner:
10# s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/[a-z]*-[a-z]*/[a-z]*-[a-z]*.html#@"link" href="#@g 13# s@"ulink" href="http://www.yoctoproject.org/docs/3.1/[a-z]*-[a-z]*/[a-z]*-[a-z]*.html#@"link" href="#@g
11 14
12s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/sdk-manual/sdk-manual.html#@"link" href="#@g 15s@"ulink" href="http://www.yoctoproject.org/docs/3.1/sdk-manual/sdk-manual.html#@"link" href="#@g
13s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/bsp-guide/bsp-guide.html#@"link" href="#@g 16s@"ulink" href="http://www.yoctoproject.org/docs/3.1/bsp-guide/bsp-guide.html#@"link" href="#@g
14s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/dev-manual/dev-manual.html#@"link" href="#@g 17s@"ulink" href="http://www.yoctoproject.org/docs/3.1/dev-manual/dev-manual.html#@"link" href="#@g
15s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/overview-manual/overview-manual.html#@"link" href="#@g 18s@"ulink" href="http://www.yoctoproject.org/docs/3.1/overview-manual/overview-manual.html#@"link" href="#@g
16s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/brief-yoctoprojectqs/brief-yoctoprojectqs.html#@"link" href="#@g 19s@"ulink" href="http://www.yoctoproject.org/docs/3.1/brief-yoctoprojectqs/brief-yoctoprojectqs.html#@"link" href="#@g
17s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/kernel-dev/kernel-dev.html#@"link" href="#@g 20s@"ulink" href="http://www.yoctoproject.org/docs/3.1/kernel-dev/kernel-dev.html#@"link" href="#@g
18s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/profile-manual/profile-manual.html#@"link" href="#@g 21s@"ulink" href="http://www.yoctoproject.org/docs/3.1/profile-manual/profile-manual.html#@"link" href="#@g
19s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/ref-manual/ref-manual.html#@"link" href="#@g 22s@"ulink" href="http://www.yoctoproject.org/docs/3.1/ref-manual/ref-manual.html#@"link" href="#@g
20s@"ulink" href="http://www.yoctoproject.org/docs/3.1.4/toaster-manual/toaster-manual.html#@"link" href="#@g 23s@"ulink" href="http://www.yoctoproject.org/docs/3.1/toaster-manual/toaster-manual.html#@"link" href="#@g
21 24
22# Process cases where just an external manual is referenced without an id anchor 25# Process cases where just an external manual is referenced without an id anchor
23s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/brief-yoctoprojectqs/brief-yoctoprojectqs.html" target="_top">Yocto Project Quick Build</a>@Yocto Project Quick Build@g 26s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/brief-yoctoprojectqs/brief-yoctoprojectqs.html" target="_top">Yocto Project Quick Build</a>@Yocto Project Quick Build@g
24s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/bitbake-user-manual/bitbake-user-manual.html" target="_top">BitBake User Manual</a>@BitBake User Manual@g 27s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/bitbake-user-manual/bitbake-user-manual.html" target="_top">BitBake User Manual</a>@BitBake User Manual@g
25s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/dev-manual/dev-manual.html" target="_top">Yocto Project Development Tasks Manual</a>@Yocto Project Development Tasks Manual@g 28s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/dev-manual/dev-manual.html" target="_top">Yocto Project Development Tasks Manual</a>@Yocto Project Development Tasks Manual@g
26s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/overview-manual/overview-manual.html" target="_top">Yocto Project Overview and Concepts Manual</a>@Yocto project Overview and Concepts Manual@g 29s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/overview-manual/overview-manual.html" target="_top">Yocto Project Overview and Concepts Manual</a>@Yocto project Overview and Concepts Manual@g
27s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/sdk-manual/sdk-manual.html" target="_top">Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</a>@Yocto Project Application Development and the Extensible Software Development Kit (eSDK)@g 30s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/sdk-manual/sdk-manual.html" target="_top">Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</a>@Yocto Project Application Development and the Extensible Software Development Kit (eSDK)@g
28s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/bsp-guide/bsp-guide.html" target="_top">Yocto Project Board Support Package (BSP) Developer's Guide</a>@Yocto Project Board Support Package (BSP) Developer's Guide@g 31s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/bsp-guide/bsp-guide.html" target="_top">Yocto Project Board Support Package (BSP) Developer's Guide</a>@Yocto Project Board Support Package (BSP) Developer's Guide@g
29s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/profile-manual/profile-manual.html" target="_top">Yocto Project Profiling and Tracing Manual</a>@Yocto Project Profiling and Tracing Manual@g 32s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/profile-manual/profile-manual.html" target="_top">Yocto Project Profiling and Tracing Manual</a>@Yocto Project Profiling and Tracing Manual@g
30s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/kernel-dev/kernel-dev.html" target="_top">Yocto Project Linux Kernel Development Manual</a>@Yocto Project Linux Kernel Development Manual@g 33s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/kernel-dev/kernel-dev.html" target="_top">Yocto Project Linux Kernel Development Manual</a>@Yocto Project Linux Kernel Development Manual@g
31s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/ref-manual/ref-manual.html" target="_top">Yocto Project Reference Manual</a>@Yocto Project Reference Manual@g 34s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/ref-manual/ref-manual.html" target="_top">Yocto Project Reference Manual</a>@Yocto Project Reference Manual@g
32s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/toaster-manual/toaster-manual.html" target="_top">Toaster User Manual</a>@Toaster User Manual@g 35s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/toaster-manual/toaster-manual.html" target="_top">Toaster User Manual</a>@Toaster User Manual@g
33 36
34# Process a single, rouge occurrence of a linked reference to the Mega-Manual. 37# Process a single, rouge occurrence of a linked reference to the Mega-Manual.
35s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1.4/mega-manual/mega-manual.html" target="_top">Yocto Project Mega-Manual</a>@Yocto Project Mega-Manual@g 38s@<a class="ulink" href="http://www.yoctoproject.org/docs/3.1/mega-manual/mega-manual.html" target="_top">Yocto Project Mega-Manual</a>@Yocto Project Mega-Manual@g
36 39
diff --git a/documentation/tools/poky-docbook-to-pdf b/documentation/tools/poky-docbook-to-pdf
index f55fd278af..b36e74b6be 100755
--- a/documentation/tools/poky-docbook-to-pdf
+++ b/documentation/tools/poky-docbook-to-pdf
@@ -1,5 +1,8 @@
1#!/bin/sh 1#!/bin/sh
2 2#
3# SPDX-License-Identifier: CC-BY-2.0-UK
4#
5
3if [ -z "$1" -o -z "$2" ]; then 6if [ -z "$1" -o -z "$2" ]; then
4 echo "usage: [-v] $0 <docbook file> <templatedir>" 7 echo "usage: [-v] $0 <docbook file> <templatedir>"
5 echo 8 echo
diff --git a/documentation/tools/update-documentation-conf b/documentation/tools/update-documentation-conf
index 3f8d280093..adfca3ca50 100644
--- a/documentation/tools/update-documentation-conf
+++ b/documentation/tools/update-documentation-conf
@@ -1,23 +1,13 @@
1#!/usr/bin/env python 1#!/usr/bin/env python
2 2#
3# SPDX-License-Identifier: GPL-2.0-only
4#
3# documentation.conf update script 5# documentation.conf update script
4# 6#
5# Author: Paul Eggleton <paul.eggleton@linux.intel.com> 7# Author: Paul Eggleton <paul.eggleton@linux.intel.com>
6# 8#
7# Copyright (C) 2015 Intel Corporation 9# Copyright (C) 2015 Intel Corporation
8# 10#
9# This program is free software; you can redistribute it and/or modify
10# it under the terms of the GNU General Public License version 2 as
11# published by the Free Software Foundation.
12#
13# This program is distributed in the hope that it will be useful,
14# but WITHOUT ANY WARRANTY; without even the implied warranty of
15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16# GNU General Public License for more details.
17#
18# You should have received a copy of the GNU General Public License along
19# with this program; if not, write to the Free Software Foundation, Inc.,
20# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
21 11
22 12
23import sys 13import sys
diff --git a/documentation/transitioning-to-a-custom-environment.rst b/documentation/transitioning-to-a-custom-environment.rst
new file mode 100644
index 0000000000..160152b096
--- /dev/null
+++ b/documentation/transitioning-to-a-custom-environment.rst
@@ -0,0 +1,116 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=============================================================
4Transitioning to a custom environment for systems development
5=============================================================
6
7|
8
9.. note::
10
11 So you've finished the :doc:`brief-yoctoprojectqs/brief-yoctoprojectqs` and
12 glanced over the document :doc:`what-i-wish-id-known`, the latter contains
13 important information learned from other users. You're well prepared. But
14 now, as you are starting your own project, it isn't exactly straightforward what
15 to do. And, the documentation is daunting. We've put together a few hints to
16 get you started.
17
18#. **Make a list of the processor, target board, technologies, and capabilities
19 that will be part of your project**.
20 You will be finding layers with recipes and other metadata that support these
21 things, and adding them to your configuration. (See #3)
22
23#. **Set up your board support**.
24 Even if you're using custom hardware, it might be easier to start with an
25 existing target board that uses the same processor or at least the same
26 architecture as your custom hardware. Knowing the board already has a
27 functioning Board Support Package (BSP) within the project makes it easier
28 for you to get comfortable with project concepts.
29
30#. **Find and acquire the best BSP for your target**.
31 Use the :yocto_home:`Yocto Project curated layer index
32 </software-overview/layers/>` or even the `OpenEmbedded layer index
33 <https://layers.openembedded.org>`_ to find and acquire the best BSP for your
34 target board. The Yocto Project layer index BSPs are regularly validated. The
35 best place to get your first BSP is from your silicon manufacturer or board
36 vendor – they can point you to their most qualified efforts. In general, for
37 Intel silicon use meta-intel, for Texas Instruments use meta-ti, and so
38 forth. Choose a BSP that has been tested with the same Yocto Project release
39 that you've downloaded. Be aware that some BSPs may not be immediately
40 supported on the very latest release, but they will be eventually.
41
42 You might want to start with the build specification that Poky provides
43 (which is reference embedded distribution) and then add your newly chosen
44 layers to that. Here is the information :ref:`about adding layers
45 <dev-manual/dev-manual-common-tasks:Understanding and Creating Layers>`.
46
47#. **Based on the layers you've chosen, make needed changes in your
48 configuration**.
49 For instance, you've chosen a machine type and added in the corresponding BSP
50 layer. You'll then need to change the value of the ``MACHINE`` variable in your
51 configuration file (build/local.conf) to point to that same machine
52 type. There could be other layer-specific settings you need to change as
53 well. Each layer has a ``README`` document that you can look at for this type of
54 usage information.
55
56#. **Add a new layer for any custom recipes and metadata you create**.
57 Use the ``bitbake-layers create-layer`` tool for Yocto Project 2.4+
58 releases. If you are using a Yocto Project release earlier than 2.4, use the
59 ``yocto-layer create`` tool. The ``bitbake-layers`` tool also provides a number
60 of other useful layer-related commands. See
61 :ref:`dev-manual/dev-manual-common-tasks:creating a general layer using the
62 \`\`bitbake-layers\`\` script` section.
63
64#. **Create your own layer for the BSP you're going to use**.
65 It is not common that you would need to create an entire BSP from scratch
66 unless you have a *really* special device. Even if you are using an existing
67 BSP, :ref:`create your own layer for the BSP <bsp-guide/bsp:creating a new
68 bsp layer using the \`\`bitbake-layers\`\` script>`. For example, given a
69 64-bit x86-based machine, copy the conf/intel-corei7-64 definition and give
70 the machine a relevant name (think board name, not product name). Make sure
71 the layer configuration is dependent on the meta-intel layer (or at least,
72 meta-intel remains in your bblayers.conf). Now you can put your custom BSP
73 settings into your layer and you can re-use it for different applications.
74
75#. **Write your own recipe to build additional software support that isn't
76 already available in the form of a recipe**.
77 Creating your own recipe is especially important for custom application
78 software that you want to run on your device. Writing new recipes is a
79 process of refinement. Start by getting each step of the build process
80 working beginning with fetching all the way through packaging. Next, run the
81 software on your target and refine further as needed. See :ref:`Writing a New
82 Recipe <dev-manual/dev-manual-common-tasks:writing a new recipe>` in the
83 Yocto Project Development Tasks Manual for more information.
84
85#. **Now you're ready to create an image recipe**.
86 There are a number of ways to do this. However, it is strongly recommended
87 that you have your own image recipe - don't try appending to existing image
88 recipes. Recipes for images are trivial to create and you usually want to
89 fully customize their contents.
90
91#. **Build your image and refine it**.
92 Add what's missing and fix anything that's broken using your knowledge of the
93 :ref:`workflow <sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk
94 workflow>` to identify where issues might be occurring.
95
96#. **Consider creating your own distribution**.
97 When you get to a certain level of customization, consider creating your own
98 distribution rather than using the default reference distribution.
99
100 Distribution settings define the packaging back-end (e.g. rpm or other) as
101 well as the package feed and possibly the update solution. You would create
102 your own distribution in a new layer inheriting from Poky but overriding what
103 needs to change for your distribution. If you find yourself adding a lot of
104 configuration to your local.conf file aside from paths and other typical
105 local settings, it's time to :ref:`consider creating your own distribution
106 <dev-manual/dev-manual-common-tasks:creating your own distribution>`.
107
108 You can add product specifications that can customize the distribution if
109 needed in other layers. You can also add other functionality specific to the
110 product. But to update the distribution, not individual products, you update
111 the distribution feature through that layer.
112
113#. **Congratulations! You're well on your way.**
114 Welcome to the Yocto Project community.
115
116.. include:: /boilerplate.rst
diff --git a/documentation/what-i-wish-id-known.rst b/documentation/what-i-wish-id-known.rst
new file mode 100644
index 0000000000..495ebdc200
--- /dev/null
+++ b/documentation/what-i-wish-id-known.rst
@@ -0,0 +1,226 @@
1.. SPDX-License-Identifier: CC-BY-2.0-UK
2
3=========================================
4What I wish I'd known about Yocto Project
5=========================================
6
7|
8
9.. note::
10
11 Before reading further, make sure you've taken a look at the
12 :yocto_home:`Software Overview</software-overview>` page which presents the
13 definitions for many of the terms referenced here. Also, know that some of the
14 information here won't make sense now, but as you start developing, it is the
15 information you'll want to keep close at hand. These are best known methods for
16 working with Yocto Project and they are updated regularly.
17
18Using the Yocto Project is fairly easy, *until something goes wrong*. Without an
19understanding of how the build process works, you'll find yourself trying to
20troubleshoot "a black box". Here are a few items that new users wished they had
21known before embarking on their first build with Yocto Project. Feel free to
22contact us with other suggestions.
23
24#. **Use Git, not the tarball download:**
25 If you use git the software will be automatically updated with bug updates
26 because of how git works. If you download the tarball instead, you will need
27 to be responsible for your own updates.
28
29#. **Get to know the layer index:**
30 All layers can be found in the `layer index
31 <https://layers.openembedded.org/>`_. Layers which have applied for Yocto
32 Project Compatible status (structure continuity assurance and testing) can be
33 found in the :yocto_home:`Yocto Project Compatible index
34 </software-over/layer/>`. Generally check the Compatible layer index first,
35 and if you don't find the necessary layer check the general layer index. The
36 layer index is an original artifact from the Open Embedded Project. As such,
37 that index doesn't have the curating and testing that the Yocto Project
38 provides on Yocto Project Compatible layer list, but the latter has fewer
39 entries. Know that when you start searching in the layer index that not all
40 layers have the same level of maturity, validation, or usability. Nor do
41 searches prioritize displayed results. There is no easy way to help you
42 through the process of choosing the best layer to suit your needs.
43 Consequently, it is often trial and error, checking the mailing lists, or
44 working with other developers through collaboration rooms that can help you
45 make good choices.
46
47#. **Use existing BSP layers from silicon vendors when possible:**
48 Intel, TI, NXP and others have information on what BSP layers to use with
49 their silicon. These layers have names such as "meta-intel" or "meta-ti". Try
50 not to build layers from scratch. If you do have custom silicon, use one of
51 these layers as a guide or template and familiarize yourself with the
52 :doc:`bsp-guide/bsp-guide`.
53
54#. **Do not put everything into one layer:**
55 Use different layers to logically separate information in your build. As an
56 example, you could have a BSP layer, a GUI layer, a distro configuration,
57 middleware, or an application (e.g. "meta-filesystems", "meta-python",
58 "meta-intel", and so forth). Putting your entire build into one layer limits
59 and complicates future customization and reuse. Isolating information into
60 layers, on the other hand, helps keep simplify future customizations and
61 reuse.
62
63#. **Never modify the POKY layer. Never. Ever. When you update to the next
64 release, you'll lose all of your work. ALL OF IT.**
65
66#. **Don't be fooled by documentation searching results:**
67 Yocto Project documentation is always being updated. Unfortunately, when you
68 use Google to search for Yocto Project concepts or terms, Google consistently
69 searches and retrieves older versions of Yocto Project manuals. For example,
70 searching for a particular topic using Google could result in a "hit" on a
71 Yocto Project manual that is several releases old. To be sure that you are
72 using the most current Yocto Project documentation, use the drop-down menu at
73 the top of any of its page.
74
75 Many developers look through the :yocto_docs:`All-in-one 'Mega' Manual </singleindex.html>`
76 for a concept or term by doing a search through the whole page. This manual
77 is a concatenation of the core set of Yocto Project manual. Thus, a simple
78 string search using Ctrl-F in this manual produces all the "hits" for a
79 desired term or concept. Once you find the area in which you are
80 interested, you can display the actual manual, if desired. It is also
81 possible to use the search bar in the menu or in the left navigation pane.
82
83#. **Understand the basic concepts of how the build system works: the workflow:**
84 Understanding the Yocto Project workflow is important as it can help you both
85 pinpoint where trouble is occurring and how the build is breaking. The
86 workflow breaks down into the following steps:
87
88 #. Fetch – get the source code
89 #. Extract – unpack the sources
90 #. Patch – apply patches for bug fixes and new capability
91 #. Configure – set up your environment specifications
92 #. Build – compile and link
93 #. Install – copy files to target directories
94 #. Package – bundle files for installation
95
96 During "fetch", there may be an inability to find code. During "extract",
97 there is likely an invalid zip or something similar. In other words, the
98 function of a particular part of the workflow gives you an idea of what might
99 be going wrong.
100
101 .. image:: figures/yp-how-it-works-new-diagram.png
102
103#. **Know that you can generate a dependency graph and learn how to do it:**
104 A dependency graph shows dependencies between recipes, tasks, and targets.
105 You can use the "-g" option with BitBake to generate this graph. When you
106 start a build and the build breaks, you could see packages you have no clue
107 about or have any idea why the build system has included them. The
108 dependency graph can clarify that confusion. You can learn more about
109 dependency graphs and how to generate them in the
110 :ref:`bitbake-user-manual/bitbake-user-manual-intro:generating dependency
111 graphs` section in the BitBake User Manual.
112
113#. **Here's how you decode "magic" folder names in tmp/work:**
114 The build system fetches, unpacks, preprocesses, and builds. If something
115 goes wrong, the build system reports to you directly the path to a folder
116 where the temporary (build/tmp) files and packages reside resulting from the
117 build. For a detailed example of this process, see the :yocto_wiki:`example
118 </Cookbook:Example:Adding_packages_to_your_OS_image>`. Unfortunately this
119 example is on an earlier release of Yocto Project.
120
121 When you perform a build, you can use the "-u" BitBake command-line option to
122 specify a user interface viewer into the dependency graph (e.g. knotty,
123 ncurses, or taskexp) that helps you understand the build dependencies better.
124
125#. **You can build more than just images:**
126 You can build and run a specific task for a specific package (including
127 devshell) or even a single recipe. When developers first start using the
128 Yocto Project, the instructions found in the
129 :doc:`brief-yoctoprojectqs/brief-yoctoprojectqs` show how to create an image
130 and then run or flash that image. However, you can actually build just a
131 single recipe. Thus, if some dependency or recipe isn't working, you can just
132 say "bitbake foo" where "foo" is the name for a specific recipe. As you
133 become more advanced using the Yocto Project, and if builds are failing, it
134 can be useful to make sure the fetch itself works as desired. Here are some
135 valuable links: :ref:`dev-manual/dev-manual-common-tasks:Using a Development
136 Shell` for information on how to build and run a specific task using
137 devshell. Also, the :ref:`SDK manual shows how to build out a specific recipe
138 <sdk-devtool-use-devtool-modify-to-modify-the-source-of-an-existing-component>`.
139
140#. **An ambiguous definition: Package vs Recipe:**
141 A recipe contains instructions the build system uses to create
142 packages. Recipes and Packages are the difference between the front end and
143 the result of the build process.
144
145 As mentioned, the build system takes the recipe and creates packages from the
146 recipe's instructions. The resulting packages are related to the one thing
147 the recipe is building but are different parts (packages) of the build
148 (i.e. the main package, the doc package, the debug symbols package, the
149 separate utilities package, and so forth). The build system splits out the
150 packages so that you don't need to install the packages you don't want or
151 need, which is advantageous because you are building for small devices when
152 developing for embedded and IoT.
153
154#. **You will want to learn about and know what's packaged in rootfs.**
155
156#. **Create your own image recipe:**
157 There are a number of ways to create your own image recipe. We suggest you
158 create your own image recipe as opposed to appending an existing recipe. It
159 is trivial and easy to write an image recipe. Again, do not try appending to
160 an existing image recipe. Create your own and do it right from the start.
161
162#. **Finally, here is a list of the basic skills you will need as a systems
163 developer. You must be able to:**
164
165 * deal with corporate proxies
166 * add a package to an image
167 * understand the difference between a recipe and package
168 * build a package by itself and why that's useful
169 * find out what packages are created by a recipe
170 * find out what files are in a package
171 * find out what files are in an image
172 * add an ssh server to an image (enable transferring of files to target)
173 * know the anatomy of a recipe
174 * know how to create and use layers
175 * find recipes (with the `OpenEmbedded Layer index <https://layers.openembedded.org>`_)
176 * understand difference between machine and distro settings
177 * find and use the right BSP (machine) for your hardware
178 * find examples of distro features and know where to set them
179 * understanding the task pipeline and executing individual tasks
180 * understand devtool and how it simplifies your workflow
181 * improve build speeds with shared downloads and shared state cache
182 * generate and understand a dependency graph
183 * generate and understand bitbake environment
184 * build an Extensible SDK for applications development
185
186#. **Depending on what you primary interests are with the Yocto Project, you
187 could consider any of the following reading:**
188
189 * **Look Through the Yocto Project Development Tasks Manual**: This manual
190 contains procedural information grouped to help you get set up, work with
191 layers, customize images, write new recipes, work with libraries, and use
192 QEMU. The information is task-based and spans the breadth of the Yocto
193 Project. See the :doc:`../dev-manual/dev-manual`.
194
195 * **Look Through the Yocto Project Application Development and the Extensible
196 Software Development Kit (eSDK) manual**: This manual describes how to use
197 both the standard SDK and the extensible SDK, which are used primarily for
198 application development. The :doc:`../sdk-manual/sdk-extensible` also provides
199 example workflows that use devtool. See the section
200 :ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`
201 for more information.
202
203 * **Learn About Kernel Development**: If you want to see how to work with the
204 kernel and understand Yocto Linux kernels, see the :doc:`../kernel-dev/kernel-dev`.
205 This manual provides information on how to patch the kernel, modify kernel
206 recipes, and configure the kernel.
207
208 * **Learn About Board Support Packages (BSPs)**: If you want to learn about
209 BSPs, see the :doc:`../bsp-guide/bsp-guide`. This manual also provides an
210 example BSP creation workflow. See the :doc:`../bsp-guide/bsp` section.
211
212 * **Learn About Toaster**: Toaster is a web interface to the Yocto Project's
213 OpenEmbedded build system. If you are interested in using this type of
214 interface to create images, see the :doc:`../toaster-manual/toaster-manual`.
215
216 * **Have Available the Yocto Project Reference Manual**: Unlike the rest of
217 the Yocto Project manual set, this manual is comprised of material suited
218 for reference rather than procedures. You can get build details, a closer
219 look at how the pieces of the Yocto Project development environment work
220 together, information on various technical details, guidance on migrating
221 to a newer Yocto Project release, reference material on the directory
222 structure, classes, and tasks. The :doc:`../ref-manual/ref-manual` also
223 contains a fairly comprehensive glossary of variables used within the Yocto
224 Project.
225
226.. include:: /boilerplate.rst