summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/ref-development-environment.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/ref-manual/ref-development-environment.xml')
-rw-r--r--documentation/ref-manual/ref-development-environment.xml2760
1 files changed, 0 insertions, 2760 deletions
diff --git a/documentation/ref-manual/ref-development-environment.xml b/documentation/ref-manual/ref-development-environment.xml
deleted file mode 100644
index 543a669e05..0000000000
--- a/documentation/ref-manual/ref-development-environment.xml
+++ /dev/null
@@ -1,2760 +0,0 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='ref-development-environment'>
6<title>The Yocto Project Development Environment</title>
7
8<para>
9 This chapter takes a look at the Yocto Project development
10 environment and also provides a detailed look at what goes on during
11 development in that environment.
12 The chapter provides Yocto Project Development environment concepts that
13 help you understand how work is accomplished in an open source environment,
14 which is very different as compared to work accomplished in a closed,
15 proprietary environment.
16</para>
17
18<para>
19 Specifically, this chapter addresses open source philosophy, workflows,
20 Git, source repositories, licensing, recipe syntax, and development
21 syntax.
22</para>
23
24<section id='open-source-philosophy'>
25 <title>Open Source Philosophy</title>
26
27 <para>
28 Open source philosophy is characterized by software development
29 directed by peer production and collaboration through an active
30 community of developers.
31 Contrast this to the more standard centralized development models
32 used by commercial software companies where a finite set of developers
33 produces a product for sale using a defined set of procedures that
34 ultimately result in an end product whose architecture and source
35 material are closed to the public.
36 </para>
37
38 <para>
39 Open source projects conceptually have differing concurrent agendas,
40 approaches, and production.
41 These facets of the development process can come from anyone in the
42 public (community) that has a stake in the software project.
43 The open source environment contains new copyright, licensing, domain,
44 and consumer issues that differ from the more traditional development
45 environment.
46 In an open source environment, the end product, source material,
47 and documentation are all available to the public at no cost.
48 </para>
49
50 <para>
51 A benchmark example of an open source project is the Linux kernel,
52 which was initially conceived and created by Finnish computer science
53 student Linus Torvalds in 1991.
54 Conversely, a good example of a non-open source project is the
55 <trademark class='registered'>Windows</trademark> family of operating
56 systems developed by
57 <trademark class='registered'>Microsoft</trademark> Corporation.
58 </para>
59
60 <para>
61 Wikipedia has a good historical description of the Open Source
62 Philosophy
63 <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>.
64 You can also find helpful information on how to participate in the
65 Linux Community
66 <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>.
67 </para>
68</section>
69
70<section id='workflows'>
71 <title>Workflows</title>
72
73 <para>
74 This section provides workflow concepts using the Yocto Project and
75 Git.
76 In particular, the information covers basic practices that describe
77 roles and actions in a collaborative development environment.
78 <note>
79 If you are familiar with this type of development environment, you
80 might not want to read this section.
81 </note>
82 </para>
83
84 <para>
85 The Yocto Project files are maintained using Git in "master"
86 branches whose Git histories track every change and whose structures
87 provides branches for all diverging functionality.
88 Although there is no need to use Git, many open source projects do so.
89 <para>
90
91 </para>
92 For the Yocto Project, a key individual called the "maintainer" is
93 responsible for the "master" branch of a given Git repository.
94 The "master" branch is the “upstream” repository from which final or
95 most recent builds of the project occur.
96 The maintainer is responsible for accepting changes from other
97 developers and for organizing the underlying branch structure to
98 reflect release strategies and so forth.
99 <note>For information on finding out who is responsible for (maintains)
100 a particular area of code, see the
101 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
102 section of the Yocto Project Development Tasks Manual.
103 </note>
104 </para>
105
106 <para>
107 The Yocto Project <filename>poky</filename> Git repository also has an
108 upstream contribution Git repository named
109 <filename>poky-contrib</filename>.
110 You can see all the branches in this repository using the web interface
111 of the
112 <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized
113 within the "Poky Support" area.
114 These branches temporarily hold changes to the project that have been
115 submitted or committed by the Yocto Project development team and by
116 community members who contribute to the project.
117 The maintainer determines if the changes are qualified to be moved
118 from the "contrib" branches into the "master" branch of the Git
119 repository.
120 </para>
121
122 <para>
123 Developers (including contributing community members) create and
124 maintain cloned repositories of the upstream "master" branch.
125 The cloned repositories are local to their development platforms and
126 are used to develop changes.
127 When a developer is satisfied with a particular feature or change,
128 they "push" the changes to the appropriate "contrib" repository.
129 </para>
130
131 <para>
132 Developers are responsible for keeping their local repository
133 up-to-date with "master".
134 They are also responsible for straightening out any conflicts that
135 might arise within files that are being worked on simultaneously by
136 more than one person.
137 All this work is done locally on the developer’s machine before
138 anything is pushed to a "contrib" area and examined at the maintainer’s
139 level.
140 </para>
141
142 <para>
143 A somewhat formal method exists by which developers commit changes
144 and push them into the "contrib" area and subsequently request that
145 the maintainer include them into "master".
146 This process is called “submitting a patch” or "submitting a change."
147 For information on submitting patches and changes, see the
148 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
149 section in the Yocto Project Development Tasks Manual.
150 </para>
151
152 <para>
153 To summarize the development workflow: a single point of entry
154 exists for changes into the project’s "master" branch of the
155 Git repository, which is controlled by the project’s maintainer.
156 And, a set of developers exist who independently develop, test, and
157 submit changes to "contrib" areas for the maintainer to examine.
158 The maintainer then chooses which changes are going to become a
159 permanent part of the project.
160 </para>
161
162 <para>
163 <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
164 </para>
165
166 <para>
167 While each development environment is unique, there are some best
168 practices or methods that help development run smoothly.
169 The following list describes some of these practices.
170 For more information about Git workflows, see the workflow topics in
171 the
172 <ulink url='http://book.git-scm.com'>Git Community Book</ulink>.
173 <itemizedlist>
174 <listitem><para>
175 <emphasis>Make Small Changes:</emphasis>
176 It is best to keep the changes you commit small as compared to
177 bundling many disparate changes into a single commit.
178 This practice not only keeps things manageable but also allows
179 the maintainer to more easily include or refuse changes.</para>
180
181 <para>It is also good practice to leave the repository in a
182 state that allows you to still successfully build your project.
183 In other words, do not commit half of a feature,
184 then add the other half as a separate, later commit.
185 Each commit should take you from one buildable project state
186 to another buildable state.
187 </para></listitem>
188 <listitem><para>
189 <emphasis>Use Branches Liberally:</emphasis>
190 It is very easy to create, use, and delete local branches in
191 your working Git repository.
192 You can name these branches anything you like.
193 It is helpful to give them names associated with the particular
194 feature or change on which you are working.
195 Once you are done with a feature or change and have merged it
196 into your local master branch, simply discard the temporary
197 branch.
198 </para></listitem>
199 <listitem><para>
200 <emphasis>Merge Changes:</emphasis>
201 The <filename>git merge</filename> command allows you to take
202 the changes from one branch and fold them into another branch.
203 This process is especially helpful when more than a single
204 developer might be working on different parts of the same
205 feature.
206 Merging changes also automatically identifies any collisions
207 or "conflicts" that might happen as a result of the same lines
208 of code being altered by two different developers.
209 </para></listitem>
210 <listitem><para>
211 <emphasis>Manage Branches:</emphasis>
212 Because branches are easy to use, you should use a system
213 where branches indicate varying levels of code readiness.
214 For example, you can have a "work" branch to develop in, a
215 "test" branch where the code or change is tested, a "stage"
216 branch where changes are ready to be committed, and so forth.
217 As your project develops, you can merge code across the
218 branches to reflect ever-increasing stable states of the
219 development.
220 </para></listitem>
221 <listitem><para>
222 <emphasis>Use Push and Pull:</emphasis>
223 The push-pull workflow is based on the concept of developers
224 "pushing" local commits to a remote repository, which is
225 usually a contribution repository.
226 This workflow is also based on developers "pulling" known
227 states of the project down into their local development
228 repositories.
229 The workflow easily allows you to pull changes submitted by
230 other developers from the upstream repository into your
231 work area ensuring that you have the most recent software
232 on which to develop.
233 The Yocto Project has two scripts named
234 <filename>create-pull-request</filename> and
235 <filename>send-pull-request</filename> that ship with the
236 release to facilitate this workflow.
237 You can find these scripts in the <filename>scripts</filename>
238 folder of the
239 <link linkend='source-directory'>Source Directory</link>.
240 For information on how to use these scripts, see the
241 "<ulink url='&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</ulink>"
242 section in the Yocto Project Development Tasks Manual.
243 </para></listitem>
244 <listitem><para>
245 <emphasis>Patch Workflow:</emphasis>
246 This workflow allows you to notify the maintainer through an
247 email that you have a change (or patch) you would like
248 considered for the "master" branch of the Git repository.
249 To send this type of change, you format the patch and then
250 send the email using the Git commands
251 <filename>git format-patch</filename> and
252 <filename>git send-email</filename>.
253 For information on how to use these scripts, see the
254 "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>"
255 section in the Yocto Project Development Tasks Manual.
256 </para></listitem>
257 </itemizedlist>
258 </para>
259</section>
260
261<section id='git'>
262 <title>Git</title>
263
264 <para>
265 The Yocto Project makes extensive use of Git, which is a
266 free, open source distributed version control system.
267 Git supports distributed development, non-linear development,
268 and can handle large projects.
269 It is best that you have some fundamental understanding
270 of how Git tracks projects and how to work with Git if
271 you are going to use the Yocto Project for development.
272 This section provides a quick overview of how Git works and
273 provides you with a summary of some essential Git commands.
274 <note><title>Notes</title>
275 <itemizedlist>
276 <listitem><para>
277 For more information on Git, see
278 <ulink url='http://git-scm.com/documentation'></ulink>.
279 </para></listitem>
280 <listitem><para>
281 If you need to download Git, it is recommended that you add
282 Git to your system through your distribution's "software
283 store" (e.g. for Ubuntu, use the Ubuntu Software feature).
284 For the Git download page, see
285 <ulink url='http://git-scm.com/download'></ulink>.
286 </para></listitem>
287 <listitem><para>
288 For examples beyond the limited few in this section on how
289 to use Git with the Yocto Project, see the
290 "<ulink url='&YOCTO_DOCS_DEV_URL;#working-with-yocto-project-source-files'>Working With Yocto Project Source Files</ulink>"
291 section in the Yocto Project Development Tasks Manual.
292 </para></listitem>
293 </itemizedlist>
294 </note>
295 </para>
296
297 <section id='repositories-tags-and-branches'>
298 <title>Repositories, Tags, and Branches</title>
299
300 <para>
301 As mentioned briefly in the previous section and also in the
302 "<link linkend='workflows'>Workflows</link>" section,
303 the Yocto Project maintains source repositories at
304 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.
305 If you look at this web-interface of the repositories, each item
306 is a separate Git repository.
307 </para>
308
309 <para>
310 Git repositories use branching techniques that track content
311 change (not files) within a project (e.g. a new feature or updated
312 documentation).
313 Creating a tree-like structure based on project divergence allows
314 for excellent historical information over the life of a project.
315 This methodology also allows for an environment from which you can
316 do lots of local experimentation on projects as you develop
317 changes or new features.
318 </para>
319
320 <para>
321 A Git repository represents all development efforts for a given
322 project.
323 For example, the Git repository <filename>poky</filename> contains
324 all changes and developments for Poky over the course of its
325 entire life.
326 That means that all changes that make up all releases are captured.
327 The repository maintains a complete history of changes.
328 </para>
329
330 <para>
331 You can create a local copy of any repository by "cloning" it
332 with the <filename>git clone</filename> command.
333 When you clone a Git repository, you end up with an identical
334 copy of the repository on your development system.
335 Once you have a local copy of a repository, you can take steps to
336 develop locally.
337 For examples on how to clone Git repositories, see the
338 "<ulink url='&YOCTO_DOCS_DEV_URL;#working-with-yocto-project-source-files'>Working With Yocto Project Source Files</ulink>"
339 section in the Yocto Project Development Tasks Manual.
340 </para>
341
342 <para>
343 It is important to understand that Git tracks content change and
344 not files.
345 Git uses "branches" to organize different development efforts.
346 For example, the <filename>poky</filename> repository has
347 several branches that include the current "&DISTRO_NAME_NO_CAP;"
348 branch, the "master" branch, and many branches for past
349 Yocto Project releases.
350 You can see all the branches by going to
351 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
352 clicking on the
353 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
354 link beneath the "Branch" heading.
355 </para>
356
357 <para>
358 Each of these branches represents a specific area of development.
359 The "master" branch represents the current or most recent
360 development.
361 All other branches represent offshoots of the "master" branch.
362 </para>
363
364 <para>
365 When you create a local copy of a Git repository, the copy has
366 the same set of branches as the original.
367 This means you can use Git to create a local working area
368 (also called a branch) that tracks a specific development branch
369 from the upstream source Git repository.
370 in other words, you can define your local Git environment to
371 work on any development branch in the repository.
372 To help illustrate, consider the following example Git commands:
373 <literallayout class='monospaced'>
374 $ cd ~
375 $ git clone git://git.yoctoproject.org/poky
376 $ cd poky
377 $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
378 </literallayout>
379 In the previous example after moving to the home directory, the
380 <filename>git clone</filename> command creates a
381 local copy of the upstream <filename>poky</filename> Git repository.
382 By default, Git checks out the "master" branch for your work.
383 After changing the working directory to the new local repository
384 (i.e. <filename>poky</filename>), the
385 <filename>git checkout</filename> command creates
386 and checks out a local branch named "&DISTRO_NAME_NO_CAP;", which
387 tracks the upstream "origin/&DISTRO_NAME_NO_CAP;" branch.
388 Changes you make while in this branch would ultimately affect
389 the upstream "&DISTRO_NAME_NO_CAP;" branch of the
390 <filename>poky</filename> repository.
391 </para>
392
393 <para>
394 It is important to understand that when you create and checkout a
395 local working branch based on a branch name,
396 your local environment matches the "tip" of that particular
397 development branch at the time you created your local branch,
398 which could be different from the files in the "master" branch
399 of the upstream repository.
400 In other words, creating and checking out a local branch based on
401 the "&DISTRO_NAME_NO_CAP;" branch name is not the same as
402 cloning and checking out the "master" branch if the repository.
403 Keep reading to see how you create a local snapshot of a Yocto
404 Project Release.
405 </para>
406
407 <para>
408 Git uses "tags" to mark specific changes in a repository.
409 Typically, a tag is used to mark a special point such as the final
410 change before a project is released.
411 You can see the tags used with the <filename>poky</filename> Git
412 repository by going to
413 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
414 clicking on the
415 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
416 link beneath the "Tag" heading.
417 </para>
418
419 <para>
420 Some key tags for the <filename>poky</filename> are
421 <filename>jethro-14.0.3</filename>,
422 <filename>morty-16.0.1</filename>,
423 <filename>pyro-17.0.0</filename>, and
424 <filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>.
425 These tags represent Yocto Project releases.
426 </para>
427
428 <para>
429 When you create a local copy of the Git repository, you also
430 have access to all the tags in the upstream repository.
431 Similar to branches, you can create and checkout a local working
432 Git branch based on a tag name.
433 When you do this, you get a snapshot of the Git repository that
434 reflects the state of the files when the change was made associated
435 with that tag.
436 The most common use is to checkout a working branch that matches
437 a specific Yocto Project release.
438 Here is an example:
439 <literallayout class='monospaced'>
440 $ cd ~
441 $ git clone git://git.yoctoproject.org/poky
442 $ cd poky
443 $ git fetch --all --tags --prune
444 $ git checkout tags/pyro-17.0.0 -b my-pyro-17.0.0
445 </literallayout>
446 In this example, the name of the top-level directory of your
447 local Yocto Project repository is <filename>poky</filename>.
448 After moving to the <filename>poky</filename> directory, the
449 <filename>git fetch</filename> command makes all the upstream
450 tags available locally in your repository.
451 Finally, the <filename>git checkout</filename> command
452 creates and checks out a branch named "my-pyro-17.0.0" that is
453 based on the specific change upstream in the repository
454 associated with the "pyro-17.0.0" tag.
455 The files in your repository now exactly match that particular
456 Yocto Project release as it is tagged in the upstream Git
457 repository.
458 It is important to understand that when you create and
459 checkout a local working branch based on a tag, your environment
460 matches a specific point in time and not the entire development
461 branch (i.e. the "tip" of the branch).
462 </para>
463 </section>
464
465 <section id='basic-commands'>
466 <title>Basic Commands</title>
467
468 <para>
469 Git has an extensive set of commands that lets you manage changes
470 and perform collaboration over the life of a project.
471 Conveniently though, you can manage with a small set of basic
472 operations and workflows once you understand the basic
473 philosophy behind Git.
474 You do not have to be an expert in Git to be functional.
475 A good place to look for instruction on a minimal set of Git
476 commands is
477 <ulink url='http://git-scm.com/documentation'>here</ulink>.
478 </para>
479
480 <para>
481 If you do not know much about Git, you should educate
482 yourself by visiting the links previously mentioned.
483 </para>
484
485 <para>
486 The following list of Git commands briefly describes some basic
487 Git operations as a way to get started.
488 As with any set of commands, this list (in most cases) simply shows
489 the base command and omits the many arguments they support.
490 See the Git documentation for complete descriptions and strategies
491 on how to use these commands:
492 <itemizedlist>
493 <listitem><para>
494 <emphasis><filename>git init</filename>:</emphasis>
495 Initializes an empty Git repository.
496 You cannot use Git commands unless you have a
497 <filename>.git</filename> repository.
498 </para></listitem>
499 <listitem><para id='git-commands-clone'>
500 <emphasis><filename>git clone</filename>:</emphasis>
501 Creates a local clone of a Git repository that is on
502 equal footing with a fellow developer’s Git repository
503 or an upstream repository.
504 </para></listitem>
505 <listitem><para>
506 <emphasis><filename>git add</filename>:</emphasis>
507 Locally stages updated file contents to the index that
508 Git uses to track changes.
509 You must stage all files that have changed before you
510 can commit them.
511 </para></listitem>
512 <listitem><para>
513 <emphasis><filename>git commit</filename>:</emphasis>
514 Creates a local "commit" that documents the changes you
515 made.
516 Only changes that have been staged can be committed.
517 Commits are used for historical purposes, for determining
518 if a maintainer of a project will allow the change,
519 and for ultimately pushing the change from your local
520 Git repository into the project’s upstream repository.
521 </para></listitem>
522 <listitem><para>
523 <emphasis><filename>git status</filename>:</emphasis>
524 Reports any modified files that possibly need to be
525 staged and gives you a status of where you stand regarding
526 local commits as compared to the upstream repository.
527 </para></listitem>
528 <listitem><para>
529 <emphasis><filename>git checkout</filename> <replaceable>branch-name</replaceable>:</emphasis>
530 Changes your working branch.
531 This command is analogous to "cd".
532 </para></listitem>
533 <listitem><para><emphasis><filename>git checkout –b</filename> <replaceable>working-branch</replaceable>:</emphasis>
534 Creates and checks out a working branch on your local
535 machine that you can use to isolate your work.
536 It is a good idea to use local branches when adding
537 specific features or changes.
538 Using isolated branches facilitates easy removal of
539 changes if they do not work out.
540 </para></listitem>
541 <listitem><para><emphasis><filename>git branch</filename>:</emphasis>
542 Displays the existing local branches associated with your
543 local repository.
544 The branch that you have currently checked out is noted
545 with an asterisk character.
546 </para></listitem>
547 <listitem><para>
548 <emphasis><filename>git branch -D</filename> <replaceable>branch-name</replaceable>:</emphasis>
549 Deletes an existing local branch.
550 You need to be in a local branch other than the one you
551 are deleting in order to delete
552 <replaceable>branch-name</replaceable>.
553 </para></listitem>
554 <listitem><para>
555 <emphasis><filename>git pull</filename>:</emphasis>
556 Retrieves information from an upstream Git repository
557 and places it in your local Git repository.
558 You use this command to make sure you are synchronized with
559 the repository from which you are basing changes
560 (.e.g. the "master" branch).
561 </para></listitem>
562 <listitem><para>
563 <emphasis><filename>git push</filename>:</emphasis>
564 Sends all your committed local changes to the upstream Git
565 repository that your local repository is tracking
566 (e.g. a contribution repository).
567 The maintainer of the project draws from these repositories
568 to merge changes (commits) into the appropriate branch
569 of project's upstream repository.
570 </para></listitem>
571 <listitem><para>
572 <emphasis><filename>git merge</filename>:</emphasis>
573 Combines or adds changes from one
574 local branch of your repository with another branch.
575 When you create a local Git repository, the default branch
576 is named "master".
577 A typical workflow is to create a temporary branch that is
578 based off "master" that you would use for isolated work.
579 You would make your changes in that isolated branch,
580 stage and commit them locally, switch to the "master"
581 branch, and then use the <filename>git merge</filename>
582 command to apply the changes from your isolated branch
583 into the currently checked out branch (e.g. "master").
584 After the merge is complete and if you are done with
585 working in that isolated branch, you can safely delete
586 the isolated branch.
587 </para></listitem>
588 <listitem><para>
589 <emphasis><filename>git cherry-pick</filename>:</emphasis>
590 Choose and apply specific commits from one branch
591 into another branch.
592 There are times when you might not be able to merge
593 all the changes in one branch with
594 another but need to pick out certain ones.
595 </para></listitem>
596 <listitem><para>
597 <emphasis><filename>gitk</filename>:</emphasis>
598 Provides a GUI view of the branches and changes in your
599 local Git repository.
600 This command is a good way to graphically see where things
601 have diverged in your local repository.
602 <note>
603 You need to install the <filename>gitk</filename>
604 package on your development system to use this
605 command.
606 </note>
607 </para></listitem>
608 <listitem><para>
609 <emphasis><filename>git log</filename>:</emphasis>
610 Reports a history of your commits to the repository.
611 This report lists all commits regardless of whether you
612 have pushed them upstream or not.
613 </para></listitem>
614 <listitem><para>
615 <emphasis><filename>git diff</filename>:</emphasis>
616 Displays line-by-line differences between a local
617 working file and the same file as understood by Git.
618 This command is useful to see what you have changed
619 in any given file.
620 </para></listitem>
621 </itemizedlist>
622 </para>
623 </section>
624</section>
625
626<section id='yocto-project-repositories'>
627 <title>Yocto Project Source Repositories</title>
628
629 <para>
630 The Yocto Project team maintains complete source repositories for all
631 Yocto Project files at
632 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>.
633 This web-based source code browser is organized into categories by
634 function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and
635 so forth.
636 From the interface, you can click on any particular item in the "Name"
637 column and see the URL at the bottom of the page that you need to clone
638 a Git repository for that particular item.
639 Having a local Git repository of the
640 <link linkend='source-directory'>Source Directory</link>, which is
641 usually named "poky", allows
642 you to make changes, contribute to the history, and ultimately enhance
643 the Yocto Project's tools, Board Support Packages, and so forth.
644 </para>
645
646 <para>
647 For any supported release of Yocto Project, you can also go to the
648 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
649 select the "Downloads" tab and get a released tarball of the
650 <filename>poky</filename> repository or any supported BSP tarballs.
651 Unpacking these tarballs gives you a snapshot of the released
652 files.
653 <note><title>Notes</title>
654 <itemizedlist>
655 <listitem><para>
656 The recommended method for setting up the Yocto Project
657 <link linkend='source-directory'>Source Directory</link>
658 and the files for supported BSPs
659 (e.g., <filename>meta-intel</filename>) is to use
660 <link linkend='git'>Git</link> to create a local copy of
661 the upstream repositories.
662 </para></listitem>
663 <listitem><para>
664 Be sure to always work in matching branches for both
665 the selected BSP repository and the
666 <link linkend='source-directory'>Source Directory</link>
667 (i.e. <filename>poky</filename>) repository.
668 For example, if you have checked out the "master" branch
669 of <filename>poky</filename> and you are going to use
670 <filename>meta-intel</filename>, be sure to checkout the
671 "master" branch of <filename>meta-intel</filename>.
672 </para></listitem>
673 </itemizedlist>
674 </note>
675 </para>
676
677 <para>
678 In summary, here is where you can get the project files needed for
679 development:
680 <itemizedlist>
681 <listitem><para id='source-repositories'>
682 <emphasis>
683 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink>
684 </emphasis>
685 This area contains IDE Plugins, Matchbox, Poky, Poky Support,
686 Tools, Yocto Linux Kernel, and Yocto Metadata Layers.
687 You can create local copies of Git repositories for each of
688 these areas.</para>
689
690 <para>
691 <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
692 For steps on how to view and access these upstream Git
693 repositories, see the
694 "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-source-repositories'>Accessing Source Repositories</ulink>"
695 Section in the Yocto Project Development Tasks Manual.
696 </para></listitem>
697 <listitem><para><anchor id='index-downloads' />
698 <emphasis>
699 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink>
700 </emphasis>
701 This is an index of releases such as
702 the <trademark class='trade'>Eclipse</trademark>
703 Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers
704 for cross-development toolchains, and all released versions of
705 Yocto Project in the form of images or tarballs.
706 Downloading and extracting these files does not produce a local
707 copy of the Git repository but rather a snapshot of a
708 particular release or image.</para>
709
710 <para>
711 <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" />
712 For steps on how to view and access these files, see the
713 "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-index-of-releases'>Accessing Index of Releases</ulink>"
714 section in the Yocto Project Development Tasks Manual.
715 </para></listitem>
716 <listitem><para id='downloads-page'>
717 <emphasis>"Downloads" page for the
718 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:
719 </emphasis></para>
720
721 <para role="writernotes">This section will change due to
722 reworking of the YP Website.</para>
723
724 <para>The Yocto Project website includes a "Downloads" tab
725 that allows you to download any Yocto Project
726 release and Board Support Package (BSP) in tarball form.
727 The tarballs are similar to those found in the
728 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para>
729
730 <para>
731 <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
732 For steps on how to use the "Downloads" page, see the
733 "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-downloads-page'>Using the Downloads Page</ulink>"
734 section in the Yocto Project Development Tasks Manual.
735 </para></listitem>
736 </itemizedlist>
737 </para>
738</section>
739
740<section id='licensing'>
741 <title>Licensing</title>
742
743 <para>
744 Because open source projects are open to the public, they have
745 different licensing structures in place.
746 License evolution for both Open Source and Free Software has an
747 interesting history.
748 If you are interested in this history, you can find basic information
749 here:
750 <itemizedlist>
751 <listitem><para>
752 <ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink>
753 </para></listitem>
754 <listitem><para>
755 <ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license history</ulink>
756 </para></listitem>
757 </itemizedlist>
758 </para>
759
760 <para>
761 In general, the Yocto Project is broadly licensed under the
762 Massachusetts Institute of Technology (MIT) License.
763 MIT licensing permits the reuse of software within proprietary
764 software as long as the license is distributed with that software.
765 MIT is also compatible with the GNU General Public License (GPL).
766 Patches to the Yocto Project follow the upstream licensing scheme.
767 You can find information on the MIT license
768 <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
769 You can find information on the GNU GPL
770 <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>here</ulink>.
771 </para>
772
773 <para>
774 When you build an image using the Yocto Project, the build process
775 uses a known list of licenses to ensure compliance.
776 You can find this list in the
777 <link linkend='source-directory'>Source Directory</link> at
778 <filename>meta/files/common-licenses</filename>.
779 Once the build completes, the list of all licenses found and used
780 during that build are kept in the
781 <link linkend='build-directory'>Build Directory</link>
782 at <filename>tmp/deploy/licenses</filename>.
783 </para>
784
785 <para>
786 If a module requires a license that is not in the base list, the
787 build process generates a warning during the build.
788 These tools make it easier for a developer to be certain of the
789 licenses with which their shipped products must comply.
790 However, even with these tools it is still up to the developer to
791 resolve potential licensing issues.
792 </para>
793
794 <para>
795 The base list of licenses used by the build process is a combination
796 of the Software Package Data Exchange (SPDX) list and the Open
797 Source Initiative (OSI) projects.
798 <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of
799 the Linux Foundation that maintains a specification for a standard
800 format for communicating the components, licenses, and copyrights
801 associated with a software package.
802 <ulink url='http://opensource.org'>OSI</ulink> is a corporation
803 dedicated to the Open Source Definition and the effort for reviewing
804 and approving licenses that conform to the Open Source Definition
805 (OSD).
806 </para>
807
808 <para>
809 You can find a list of the combined SPDX and OSI licenses that the
810 Yocto Project uses in the
811 <filename>meta/files/common-licenses</filename> directory in your
812 <link linkend='source-directory'>Source Directory</link>.
813 </para>
814
815 <para>
816 For information that can help you maintain compliance with various
817 open source licensing during the lifecycle of a product created using
818 the Yocto Project, see the
819 "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</ulink>"
820 section in the Yocto Project Development Tasks Manual.
821 </para>
822</section>
823
824<section id='recipe-syntax'>
825 <title>Recipe Syntax</title>
826
827 <para>
828 Understanding recipe file syntax is important for
829 writing recipes.
830 The following list overviews the basic items that make up a
831 BitBake recipe file.
832 For more complete BitBake syntax descriptions, see the
833 "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>"
834 chapter of the BitBake User Manual.
835 <itemizedlist>
836 <listitem><para><emphasis>Variable Assignments and Manipulations:</emphasis>
837 Variable assignments allow a value to be assigned to a
838 variable.
839 The assignment can be static text or might include
840 the contents of other variables.
841 In addition to the assignment, appending and prepending
842 operations are also supported.</para>
843 <para>The following example shows some of the ways
844 you can use variables in recipes:
845 <literallayout class='monospaced'>
846 S = "${WORKDIR}/postfix-${PV}"
847 CFLAGS += "-DNO_ASM"
848 SRC_URI_append = " file://fixup.patch"
849 </literallayout>
850 </para></listitem>
851 <listitem><para><emphasis>Functions:</emphasis>
852 Functions provide a series of actions to be performed.
853 You usually use functions to override the default
854 implementation of a task function or to complement
855 a default function (i.e. append or prepend to an
856 existing function).
857 Standard functions use <filename>sh</filename> shell
858 syntax, although access to OpenEmbedded variables and
859 internal methods are also available.</para>
860 <para>The following is an example function from the
861 <filename>sed</filename> recipe:
862 <literallayout class='monospaced'>
863 do_install () {
864 autotools_do_install
865 install -d ${D}${base_bindir}
866 mv ${D}${bindir}/sed ${D}${base_bindir}/sed
867 rmdir ${D}${bindir}/
868 }
869 </literallayout>
870 It is also possible to implement new functions that
871 are called between existing tasks as long as the
872 new functions are not replacing or complementing the
873 default functions.
874 You can implement functions in Python
875 instead of shell.
876 Both of these options are not seen in the majority of
877 recipes.</para></listitem>
878 <listitem><para><emphasis>Keywords:</emphasis>
879 BitBake recipes use only a few keywords.
880 You use keywords to include common
881 functions (<filename>inherit</filename>), load parts
882 of a recipe from other files
883 (<filename>include</filename> and
884 <filename>require</filename>) and export variables
885 to the environment (<filename>export</filename>).</para>
886 <para>The following example shows the use of some of
887 these keywords:
888 <literallayout class='monospaced'>
889 export POSTCONF = "${STAGING_BINDIR}/postconf"
890 inherit autoconf
891 require otherfile.inc
892 </literallayout>
893 </para></listitem>
894 <listitem><para><emphasis>Comments:</emphasis>
895 Any lines that begin with the hash character
896 (<filename>#</filename>) are treated as comment lines
897 and are ignored:
898 <literallayout class='monospaced'>
899 # This is a comment
900 </literallayout>
901 </para></listitem>
902 </itemizedlist>
903 </para>
904
905 <para>
906 This next list summarizes the most important and most commonly
907 used parts of the recipe syntax.
908 For more information on these parts of the syntax, you can
909 reference the
910 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>
911 chapter in the BitBake User Manual.
912 <itemizedlist>
913 <listitem><para><emphasis>Line Continuation: <filename>\</filename></emphasis> -
914 Use the backward slash (<filename>\</filename>)
915 character to split a statement over multiple lines.
916 Place the slash character at the end of the line that
917 is to be continued on the next line:
918 <literallayout class='monospaced'>
919 VAR = "A really long \
920 line"
921 </literallayout>
922 <note>
923 You cannot have any characters including spaces
924 or tabs after the slash character.
925 </note>
926 </para></listitem>
927 <listitem><para>
928 <emphasis>Using Variables: <filename>${...}</filename></emphasis> -
929 Use the <filename>${<replaceable>VARNAME</replaceable>}</filename> syntax to
930 access the contents of a variable:
931 <literallayout class='monospaced'>
932 SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz"
933 </literallayout>
934 <note>
935 It is important to understand that the value of a
936 variable expressed in this form does not get
937 substituted automatically.
938 The expansion of these expressions happens
939 on-demand later (e.g. usually when a function that
940 makes reference to the variable executes).
941 This behavior ensures that the values are most
942 appropriate for the context in which they are
943 finally used.
944 On the rare occasion that you do need the variable
945 expression to be expanded immediately, you can use
946 the <filename>:=</filename> operator instead of
947 <filename>=</filename> when you make the
948 assignment, but this is not generally needed.
949 </note>
950 </para></listitem>
951 <listitem><para><emphasis>Quote All Assignments: <filename>"<replaceable>value</replaceable>"</filename></emphasis> -
952 Use double quotes around the value in all variable
953 assignments.
954 <literallayout class='monospaced'>
955 VAR1 = "${OTHERVAR}"
956 VAR2 = "The version is ${PV}"
957 </literallayout>
958 </para></listitem>
959 <listitem><para><emphasis>Conditional Assignment: <filename>?=</filename></emphasis> -
960 Conditional assignment is used to assign a value to
961 a variable, but only when the variable is currently
962 unset.
963 Use the question mark followed by the equal sign
964 (<filename>?=</filename>) to make a "soft" assignment
965 used for conditional assignment.
966 Typically, "soft" assignments are used in the
967 <filename>local.conf</filename> file for variables
968 that are allowed to come through from the external
969 environment.
970 </para>
971 <para>Here is an example where
972 <filename>VAR1</filename> is set to "New value" if
973 it is currently empty.
974 However, if <filename>VAR1</filename> has already been
975 set, it remains unchanged:
976 <literallayout class='monospaced'>
977 VAR1 ?= "New value"
978 </literallayout>
979 In this next example, <filename>VAR1</filename>
980 is left with the value "Original value":
981 <literallayout class='monospaced'>
982 VAR1 = "Original value"
983 VAR1 ?= "New value"
984 </literallayout>
985 </para></listitem>
986 <listitem><para><emphasis>Appending: <filename>+=</filename></emphasis> -
987 Use the plus character followed by the equals sign
988 (<filename>+=</filename>) to append values to existing
989 variables.
990 <note>
991 This operator adds a space between the existing
992 content of the variable and the new content.
993 </note></para>
994 <para>Here is an example:
995 <literallayout class='monospaced'>
996 SRC_URI += "file://fix-makefile.patch"
997 </literallayout>
998 </para></listitem>
999 <listitem><para><emphasis>Prepending: <filename>=+</filename></emphasis> -
1000 Use the equals sign followed by the plus character
1001 (<filename>=+</filename>) to prepend values to existing
1002 variables.
1003 <note>
1004 This operator adds a space between the new content
1005 and the existing content of the variable.
1006 </note></para>
1007 <para>Here is an example:
1008 <literallayout class='monospaced'>
1009 VAR =+ "Starts"
1010 </literallayout>
1011 </para></listitem>
1012 <listitem><para><emphasis>Appending: <filename>_append</filename></emphasis> -
1013 Use the <filename>_append</filename> operator to
1014 append values to existing variables.
1015 This operator does not add any additional space.
1016 Also, the operator is applied after all the
1017 <filename>+=</filename>, and
1018 <filename>=+</filename> operators have been applied and
1019 after all <filename>=</filename> assignments have
1020 occurred.
1021 </para>
1022 <para>The following example shows the space being
1023 explicitly added to the start to ensure the appended
1024 value is not merged with the existing value:
1025 <literallayout class='monospaced'>
1026 SRC_URI_append = " file://fix-makefile.patch"
1027 </literallayout>
1028 You can also use the <filename>_append</filename>
1029 operator with overrides, which results in the actions
1030 only being performed for the specified target or
1031 machine:
1032 <literallayout class='monospaced'>
1033 SRC_URI_append_sh4 = " file://fix-makefile.patch"
1034 </literallayout>
1035 </para></listitem>
1036 <listitem><para><emphasis>Prepending: <filename>_prepend</filename></emphasis> -
1037 Use the <filename>_prepend</filename> operator to
1038 prepend values to existing variables.
1039 This operator does not add any additional space.
1040 Also, the operator is applied after all the
1041 <filename>+=</filename>, and
1042 <filename>=+</filename> operators have been applied and
1043 after all <filename>=</filename> assignments have
1044 occurred.
1045 </para>
1046 <para>The following example shows the space being
1047 explicitly added to the end to ensure the prepended
1048 value is not merged with the existing value:
1049 <literallayout class='monospaced'>
1050 CFLAGS_prepend = "-I${S}/myincludes "
1051 </literallayout>
1052 You can also use the <filename>_prepend</filename>
1053 operator with overrides, which results in the actions
1054 only being performed for the specified target or
1055 machine:
1056 <literallayout class='monospaced'>
1057 CFLAGS_prepend_sh4 = "-I${S}/myincludes "
1058 </literallayout>
1059 </para></listitem>
1060 <listitem><para><emphasis>Overrides:</emphasis> -
1061 You can use overrides to set a value conditionally,
1062 typically based on how the recipe is being built.
1063 For example, to set the
1064 <link linkend='var-KBRANCH'><filename>KBRANCH</filename></link>
1065 variable's value to "standard/base" for any target
1066 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>,
1067 except for qemuarm where it should be set to
1068 "standard/arm-versatile-926ejs", you would do the
1069 following:
1070 <literallayout class='monospaced'>
1071 KBRANCH = "standard/base"
1072 KBRANCH_qemuarm = "standard/arm-versatile-926ejs"
1073 </literallayout>
1074 Overrides are also used to separate alternate values
1075 of a variable in other situations.
1076 For example, when setting variables such as
1077 <link linkend='var-FILES'><filename>FILES</filename></link>
1078 and
1079 <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>
1080 that are specific to individual packages produced by
1081 a recipe, you should always use an override that
1082 specifies the name of the package.
1083 </para></listitem>
1084 <listitem><para><emphasis>Indentation:</emphasis>
1085 Use spaces for indentation rather than than tabs.
1086 For shell functions, both currently work.
1087 However, it is a policy decision of the Yocto Project
1088 to use tabs in shell functions.
1089 Realize that some layers have a policy to use spaces
1090 for all indentation.
1091 </para></listitem>
1092 <listitem><para><emphasis>Using Python for Complex Operations: <filename>${@<replaceable>python_code</replaceable>}</filename></emphasis> -
1093 For more advanced processing, it is possible to use
1094 Python code during variable assignments (e.g.
1095 search and replacement on a variable).</para>
1096 <para>You indicate Python code using the
1097 <filename>${@<replaceable>python_code</replaceable>}</filename>
1098 syntax for the variable assignment:
1099 <literallayout class='monospaced'>
1100 SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz
1101 </literallayout>
1102 </para></listitem>
1103 <listitem><para><emphasis>Shell Function Syntax:</emphasis>
1104 Write shell functions as if you were writing a shell
1105 script when you describe a list of actions to take.
1106 You should ensure that your script works with a generic
1107 <filename>sh</filename> and that it does not require
1108 any <filename>bash</filename> or other shell-specific
1109 functionality.
1110 The same considerations apply to various system
1111 utilities (e.g. <filename>sed</filename>,
1112 <filename>grep</filename>, <filename>awk</filename>,
1113 and so forth) that you might wish to use.
1114 If in doubt, you should check with multiple
1115 implementations - including those from BusyBox.
1116 </para></listitem>
1117 </itemizedlist>
1118 </para>
1119</section>
1120
1121<section id="development-concepts">
1122 <title>Development Concepts</title>
1123
1124 <para>
1125 This section takes a more detailed look inside the development
1126 process.
1127 The following diagram represents development at a high level.
1128 The remainder of this chapter expands on the fundamental input, output,
1129 process, and
1130 <link linkend='metadata'>Metadata</link>) blocks
1131 that make up development in the Yocto Project environment.
1132 </para>
1133
1134 <para id='general-yocto-environment-figure'>
1135 <imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" />
1136 </para>
1137
1138 <para>
1139 In general, development consists of several functional areas:
1140 <itemizedlist>
1141 <listitem><para><emphasis>User Configuration:</emphasis>
1142 Metadata you can use to control the build process.
1143 </para></listitem>
1144 <listitem><para><emphasis>Metadata Layers:</emphasis>
1145 Various layers that provide software, machine, and
1146 distro Metadata.</para></listitem>
1147 <listitem><para><emphasis>Source Files:</emphasis>
1148 Upstream releases, local projects, and SCMs.</para></listitem>
1149 <listitem><para><emphasis>Build System:</emphasis>
1150 Processes under the control of
1151 <link linkend='bitbake-term'>BitBake</link>.
1152 This block expands on how BitBake fetches source, applies
1153 patches, completes compilation, analyzes output for package
1154 generation, creates and tests packages, generates images, and
1155 generates cross-development tools.</para></listitem>
1156 <listitem><para><emphasis>Package Feeds:</emphasis>
1157 Directories containing output packages (RPM, DEB or IPK),
1158 which are subsequently used in the construction of an image or
1159 SDK, produced by the build system.
1160 These feeds can also be copied and shared using a web server or
1161 other means to facilitate extending or updating existing
1162 images on devices at runtime if runtime package management is
1163 enabled.</para></listitem>
1164 <listitem><para><emphasis>Images:</emphasis>
1165 Images produced by the development process.
1166 </para></listitem>
1167 <listitem><para><emphasis>Application Development SDK:</emphasis>
1168 Cross-development tools that are produced along with an image
1169 or separately with BitBake.</para></listitem>
1170 </itemizedlist>
1171 </para>
1172
1173 <section id="user-configuration">
1174 <title>User Configuration</title>
1175
1176 <para>
1177 User configuration helps define the build.
1178 Through user configuration, you can tell BitBake the
1179 target architecture for which you are building the image,
1180 where to store downloaded source, and other build properties.
1181 </para>
1182
1183 <para>
1184 The following figure shows an expanded representation of the
1185 "User Configuration" box of the
1186 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>:
1187 </para>
1188
1189 <para>
1190 <imagedata fileref="figures/user-configuration.png" align="center" />
1191 </para>
1192
1193 <para>
1194 BitBake needs some basic configuration files in order to complete
1195 a build.
1196 These files are <filename>*.conf</filename> files.
1197 The minimally necessary ones reside as example files in the
1198 <link linkend='source-directory'>Source Directory</link>.
1199 For simplicity, this section refers to the Source Directory as
1200 the "Poky Directory."
1201 </para>
1202
1203 <para>
1204 When you clone the <filename>poky</filename> Git repository or you
1205 download and unpack a Yocto Project release, you can set up the
1206 Source Directory to be named anything you want.
1207 For this discussion, the cloned repository uses the default
1208 name <filename>poky</filename>.
1209 <note>
1210 The Poky repository is primarily an aggregation of existing
1211 repositories.
1212 It is not a canonical upstream source.
1213 </note>
1214 </para>
1215
1216 <para>
1217 The <filename>meta-poky</filename> layer inside Poky contains
1218 a <filename>conf</filename> directory that has example
1219 configuration files.
1220 These example files are used as a basis for creating actual
1221 configuration files when you source the build environment
1222 script
1223 (i.e.
1224 <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>).
1225 </para>
1226
1227 <para>
1228 Sourcing the build environment script creates a
1229 <link linkend='build-directory'>Build Directory</link>
1230 if one does not already exist.
1231 BitBake uses the Build Directory for all its work during builds.
1232 The Build Directory has a <filename>conf</filename> directory that
1233 contains default versions of your <filename>local.conf</filename>
1234 and <filename>bblayers.conf</filename> configuration files.
1235 These default configuration files are created only if versions
1236 do not already exist in the Build Directory at the time you
1237 source the build environment setup script.
1238 </para>
1239
1240 <para>
1241 Because the Poky repository is fundamentally an aggregation of
1242 existing repositories, some users might be familiar with running
1243 the <filename>&OE_INIT_FILE;</filename> script in the context
1244 of separate OpenEmbedded-Core and BitBake repositories rather than a
1245 single Poky repository.
1246 This discussion assumes the script is executed from within a cloned
1247 or unpacked version of Poky.
1248 </para>
1249
1250 <para>
1251 Depending on where the script is sourced, different sub-scripts
1252 are called to set up the Build Directory (Yocto or OpenEmbedded).
1253 Specifically, the script
1254 <filename>scripts/oe-setup-builddir</filename> inside the
1255 poky directory sets up the Build Directory and seeds the directory
1256 (if necessary) with configuration files appropriate for the
1257 Yocto Project development environment.
1258 <note>
1259 The <filename>scripts/oe-setup-builddir</filename> script
1260 uses the <filename>$TEMPLATECONF</filename> variable to
1261 determine which sample configuration files to locate.
1262 </note>
1263 </para>
1264
1265 <para>
1266 The <filename>local.conf</filename> file provides many
1267 basic variables that define a build environment.
1268 Here is a list of a few.
1269 To see the default configurations in a <filename>local.conf</filename>
1270 file created by the build environment script, see the
1271 <filename>local.conf.sample</filename> in the
1272 <filename>meta-poky</filename> layer:
1273 <itemizedlist>
1274 <listitem><para><emphasis>Parallelism Options:</emphasis>
1275 Controlled by the
1276 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>,
1277 <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>,
1278 and
1279 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink>
1280 variables.</para></listitem>
1281 <listitem><para><emphasis>Target Machine Selection:</emphasis>
1282 Controlled by the
1283 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>
1284 variable.</para></listitem>
1285 <listitem><para><emphasis>Download Directory:</emphasis>
1286 Controlled by the
1287 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
1288 variable.</para></listitem>
1289 <listitem><para><emphasis>Shared State Directory:</emphasis>
1290 Controlled by the
1291 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
1292 variable.</para></listitem>
1293 <listitem><para><emphasis>Build Output:</emphasis>
1294 Controlled by the
1295 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
1296 variable.</para></listitem>
1297 </itemizedlist>
1298 <note>
1299 Configurations set in the <filename>conf/local.conf</filename>
1300 file can also be set in the
1301 <filename>conf/site.conf</filename> and
1302 <filename>conf/auto.conf</filename> configuration files.
1303 </note>
1304 </para>
1305
1306 <para>
1307 The <filename>bblayers.conf</filename> file tells BitBake what
1308 layers you want considered during the build.
1309 By default, the layers listed in this file include layers
1310 minimally needed by the build system.
1311 However, you must manually add any custom layers you have created.
1312 You can find more information on working with the
1313 <filename>bblayers.conf</filename> file in the
1314 "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>"
1315 section in the Yocto Project Development Tasks Manual.
1316 </para>
1317
1318 <para>
1319 The files <filename>site.conf</filename> and
1320 <filename>auto.conf</filename> are not created by the environment
1321 initialization script.
1322 If you want the <filename>site.conf</filename> file, you need to
1323 create that yourself.
1324 The <filename>auto.conf</filename> file is typically created by
1325 an autobuilder:
1326 <itemizedlist>
1327 <listitem><para><emphasis><filename>site.conf</filename>:</emphasis>
1328 You can use the <filename>conf/site.conf</filename>
1329 configuration file to configure multiple build directories.
1330 For example, suppose you had several build environments and
1331 they shared some common features.
1332 You can set these default build properties here.
1333 A good example is perhaps the packaging format to use
1334 through the
1335 <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
1336 variable.</para>
1337 <para>One useful scenario for using the
1338 <filename>conf/site.conf</filename> file is to extend your
1339 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
1340 variable to include the path to a
1341 <filename>conf/site.conf</filename>.
1342 Then, when BitBake looks for Metadata using
1343 <filename>BBPATH</filename>, it finds the
1344 <filename>conf/site.conf</filename> file and applies your
1345 common configurations found in the file.
1346 To override configurations in a particular build directory,
1347 alter the similar configurations within that build
1348 directory's <filename>conf/local.conf</filename> file.
1349 </para></listitem>
1350 <listitem><para><emphasis><filename>auto.conf</filename>:</emphasis>
1351 The file is usually created and written to by
1352 an autobuilder.
1353 The settings put into the file are typically the same as
1354 you would find in the <filename>conf/local.conf</filename>
1355 or the <filename>conf/site.conf</filename> files.
1356 </para></listitem>
1357 </itemizedlist>
1358 </para>
1359
1360 <para>
1361 You can edit all configuration files to further define
1362 any particular build environment.
1363 This process is represented by the "User Configuration Edits"
1364 box in the figure.
1365 </para>
1366
1367 <para>
1368 When you launch your build with the
1369 <filename>bitbake <replaceable>target</replaceable></filename>
1370 command, BitBake sorts out the configurations to ultimately
1371 define your build environment.
1372 It is important to understand that the OpenEmbedded build system
1373 reads the configuration files in a specific order:
1374 <filename>site.conf</filename>, <filename>auto.conf</filename>,
1375 and <filename>local.conf</filename>.
1376 And, the build system applies the normal assignment statement
1377 rules.
1378 Because the files are parsed in a specific order, variable
1379 assignments for the same variable could be affected.
1380 For example, if the <filename>auto.conf</filename> file and
1381 the <filename>local.conf</filename> set
1382 <replaceable>variable1</replaceable> to different values, because
1383 the build system parses <filename>local.conf</filename> after
1384 <filename>auto.conf</filename>,
1385 <replaceable>variable1</replaceable> is assigned the value from
1386 the <filename>local.conf</filename> file.
1387 </para>
1388 </section>
1389
1390 <section id="metadata-machine-configuration-and-policy-configuration">
1391 <title>Metadata, Machine Configuration, and Policy Configuration</title>
1392
1393 <para>
1394 The previous section described the user configurations that
1395 define BitBake's global behavior.
1396 This section takes a closer look at the layers the build system
1397 uses to further control the build.
1398 These layers provide Metadata for the software, machine, and
1399 policy.
1400 </para>
1401
1402 <para>
1403 In general, three types of layer input exist:
1404 <itemizedlist>
1405 <listitem><para><emphasis>Policy Configuration:</emphasis>
1406 Distribution Layers provide top-level or general
1407 policies for the image or SDK being built.
1408 For example, this layer would dictate whether BitBake
1409 produces RPM or IPK packages.</para></listitem>
1410 <listitem><para><emphasis>Machine Configuration:</emphasis>
1411 Board Support Package (BSP) layers provide machine
1412 configurations.
1413 This type of information is specific to a particular
1414 target architecture.</para></listitem>
1415 <listitem><para><emphasis>Metadata:</emphasis>
1416 Software layers contain user-supplied recipe files,
1417 patches, and append files.
1418 </para></listitem>
1419 </itemizedlist>
1420 </para>
1421
1422 <para>
1423 The following figure shows an expanded representation of the
1424 Metadata, Machine Configuration, and Policy Configuration input
1425 (layers) boxes of the
1426 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>:
1427 </para>
1428
1429 <para>
1430 <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" />
1431 </para>
1432
1433 <para>
1434 In general, all layers have a similar structure.
1435 They all contain a licensing file
1436 (e.g. <filename>COPYING</filename>) if the layer is to be
1437 distributed, a <filename>README</filename> file as good practice
1438 and especially if the layer is to be distributed, a
1439 configuration directory, and recipe directories.
1440 </para>
1441
1442 <para>
1443 The Yocto Project has many layers that can be used.
1444 You can see a web-interface listing of them on the
1445 <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink>
1446 page.
1447 The layers are shown at the bottom categorized under
1448 "Yocto Metadata Layers."
1449 These layers are fundamentally a subset of the
1450 <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>,
1451 which lists all layers provided by the OpenEmbedded community.
1452 <note>
1453 Layers exist in the Yocto Project Source Repositories that
1454 cannot be found in the OpenEmbedded Metadata Index.
1455 These layers are either deprecated or experimental in nature.
1456 </note>
1457 </para>
1458
1459 <para>
1460 BitBake uses the <filename>conf/bblayers.conf</filename> file,
1461 which is part of the user configuration, to find what layers it
1462 should be using as part of the build.
1463 </para>
1464
1465 <para>
1466 For more information on layers, see the
1467 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
1468 section in the Yocto Project Development Tasks Manual.
1469 </para>
1470
1471 <section id="distro-layer">
1472 <title>Distro Layer</title>
1473
1474 <para>
1475 The distribution layer provides policy configurations for your
1476 distribution.
1477 Best practices dictate that you isolate these types of
1478 configurations into their own layer.
1479 Settings you provide in
1480 <filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override
1481 similar
1482 settings that BitBake finds in your
1483 <filename>conf/local.conf</filename> file in the Build
1484 Directory.
1485 </para>
1486
1487 <para>
1488 The following list provides some explanation and references
1489 for what you typically find in the distribution layer:
1490 <itemizedlist>
1491 <listitem><para><emphasis>classes:</emphasis>
1492 Class files (<filename>.bbclass</filename>) hold
1493 common functionality that can be shared among
1494 recipes in the distribution.
1495 When your recipes inherit a class, they take on the
1496 settings and functions for that class.
1497 You can read more about class files in the
1498 "<link linkend='ref-classes'>Classes</link>" section.
1499 </para></listitem>
1500 <listitem><para><emphasis>conf:</emphasis>
1501 This area holds configuration files for the
1502 layer (<filename>conf/layer.conf</filename>),
1503 the distribution
1504 (<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>),
1505 and any distribution-wide include files.
1506 </para></listitem>
1507 <listitem><para><emphasis>recipes-*:</emphasis>
1508 Recipes and append files that affect common
1509 functionality across the distribution.
1510 This area could include recipes and append files
1511 to add distribution-specific configuration,
1512 initialization scripts, custom image recipes,
1513 and so forth.</para></listitem>
1514 </itemizedlist>
1515 </para>
1516 </section>
1517
1518 <section id="bsp-layer">
1519 <title>BSP Layer</title>
1520
1521 <para>
1522 The BSP Layer provides machine configurations.
1523 Everything in this layer is specific to the machine for which
1524 you are building the image or the SDK.
1525 A common structure or form is defined for BSP layers.
1526 You can learn more about this structure in the
1527 <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.
1528 <note>
1529 In order for a BSP layer to be considered compliant with the
1530 Yocto Project, it must meet some structural requirements.
1531 </note>
1532 </para>
1533
1534 <para>
1535 The BSP Layer's configuration directory contains
1536 configuration files for the machine
1537 (<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>) and,
1538 of course, the layer (<filename>conf/layer.conf</filename>).
1539 </para>
1540
1541 <para>
1542 The remainder of the layer is dedicated to specific recipes
1543 by function: <filename>recipes-bsp</filename>,
1544 <filename>recipes-core</filename>,
1545 <filename>recipes-graphics</filename>, and
1546 <filename>recipes-kernel</filename>.
1547 Metadata can exist for multiple formfactors, graphics
1548 support systems, and so forth.
1549 <note>
1550 While the figure shows several <filename>recipes-*</filename>
1551 directories, not all these directories appear in all
1552 BSP layers.
1553 </note>
1554 </para>
1555 </section>
1556
1557 <section id="software-layer">
1558 <title>Software Layer</title>
1559
1560 <para>
1561 The software layer provides the Metadata for additional
1562 software packages used during the build.
1563 This layer does not include Metadata that is specific to the
1564 distribution or the machine, which are found in their
1565 respective layers.
1566 </para>
1567
1568 <para>
1569 This layer contains any new recipes that your project needs
1570 in the form of recipe files.
1571 </para>
1572 </section>
1573 </section>
1574
1575 <section id="sources-dev-environment">
1576 <title>Sources</title>
1577
1578 <para>
1579 In order for the OpenEmbedded build system to create an image or
1580 any target, it must be able to access source files.
1581 The
1582 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>
1583 represents source files using the "Upstream Project Releases",
1584 "Local Projects", and "SCMs (optional)" boxes.
1585 The figure represents mirrors, which also play a role in locating
1586 source files, with the "Source Mirror(s)" box.
1587 </para>
1588
1589 <para>
1590 The method by which source files are ultimately organized is
1591 a function of the project.
1592 For example, for released software, projects tend to use tarballs
1593 or other archived files that can capture the state of a release
1594 guaranteeing that it is statically represented.
1595 On the other hand, for a project that is more dynamic or
1596 experimental in nature, a project might keep source files in a
1597 repository controlled by a Source Control Manager (SCM) such as
1598 Git.
1599 Pulling source from a repository allows you to control
1600 the point in the repository (the revision) from which you want to
1601 build software.
1602 Finally, a combination of the two might exist, which would give the
1603 consumer a choice when deciding where to get source files.
1604 </para>
1605
1606 <para>
1607 BitBake uses the
1608 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
1609 variable to point to source files regardless of their location.
1610 Each recipe must have a <filename>SRC_URI</filename> variable
1611 that points to the source.
1612 </para>
1613
1614 <para>
1615 Another area that plays a significant role in where source files
1616 come from is pointed to by the
1617 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
1618 variable.
1619 This area is a cache that can hold previously downloaded source.
1620 You can also instruct the OpenEmbedded build system to create
1621 tarballs from Git repositories, which is not the default behavior,
1622 and store them in the <filename>DL_DIR</filename> by using the
1623 <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link>
1624 variable.
1625 </para>
1626
1627 <para>
1628 Judicious use of a <filename>DL_DIR</filename> directory can
1629 save the build system a trip across the Internet when looking
1630 for files.
1631 A good method for using a download directory is to have
1632 <filename>DL_DIR</filename> point to an area outside of your
1633 Build Directory.
1634 Doing so allows you to safely delete the Build Directory
1635 if needed without fear of removing any downloaded source file.
1636 </para>
1637
1638 <para>
1639 The remainder of this section provides a deeper look into the
1640 source files and the mirrors.
1641 Here is a more detailed look at the source file area of the
1642 base figure:
1643 <imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" />
1644 </para>
1645
1646 <section id='upstream-project-releases'>
1647 <title>Upstream Project Releases</title>
1648
1649 <para>
1650 Upstream project releases exist anywhere in the form of an
1651 archived file (e.g. tarball or zip file).
1652 These files correspond to individual recipes.
1653 For example, the figure uses specific releases each for
1654 BusyBox, Qt, and Dbus.
1655 An archive file can be for any released product that can be
1656 built using a recipe.
1657 </para>
1658 </section>
1659
1660 <section id='local-projects'>
1661 <title>Local Projects</title>
1662
1663 <para>
1664 Local projects are custom bits of software the user provides.
1665 These bits reside somewhere local to a project - perhaps
1666 a directory into which the user checks in items (e.g.
1667 a local directory containing a development source tree
1668 used by the group).
1669 </para>
1670
1671 <para>
1672 The canonical method through which to include a local project
1673 is to use the
1674 <link linkend='ref-classes-externalsrc'><filename>externalsrc</filename></link>
1675 class to include that local project.
1676 You use either the <filename>local.conf</filename> or a
1677 recipe's append file to override or set the
1678 recipe to point to the local directory on your disk to pull
1679 in the whole source tree.
1680 </para>
1681
1682 <para>
1683 For information on how to use the
1684 <filename>externalsrc</filename> class, see the
1685 "<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>"
1686 section.
1687 </para>
1688 </section>
1689
1690 <section id='scms'>
1691 <title>Source Control Managers (Optional)</title>
1692
1693 <para>
1694 Another place the build system can get source files from is
1695 through an SCM such as Git or Subversion.
1696 In this case, a repository is cloned or checked out.
1697 The
1698 <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link>
1699 task inside BitBake uses
1700 the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
1701 variable and the argument's prefix to determine the correct
1702 fetcher module.
1703 </para>
1704
1705 <note>
1706 For information on how to have the OpenEmbedded build system
1707 generate tarballs for Git repositories and place them in the
1708 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
1709 directory, see the
1710 <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link>
1711 variable.
1712 </note>
1713
1714 <para>
1715 When fetching a repository, BitBake uses the
1716 <link linkend='var-SRCREV'><filename>SRCREV</filename></link>
1717 variable to determine the specific revision from which to
1718 build.
1719 </para>
1720 </section>
1721
1722 <section id='source-mirrors'>
1723 <title>Source Mirror(s)</title>
1724
1725 <para>
1726 Two kinds of mirrors exist: pre-mirrors and regular mirrors.
1727 The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
1728 and
1729 <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
1730 variables point to these, respectively.
1731 BitBake checks pre-mirrors before looking upstream for any
1732 source files.
1733 Pre-mirrors are appropriate when you have a shared directory
1734 that is not a directory defined by the
1735 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
1736 variable.
1737 A Pre-mirror typically points to a shared directory that is
1738 local to your organization.
1739 </para>
1740
1741 <para>
1742 Regular mirrors can be any site across the Internet that is
1743 used as an alternative location for source code should the
1744 primary site not be functioning for some reason or another.
1745 </para>
1746 </section>
1747 </section>
1748
1749 <section id="package-feeds-dev-environment">
1750 <title>Package Feeds</title>
1751
1752 <para>
1753 When the OpenEmbedded build system generates an image or an SDK,
1754 it gets the packages from a package feed area located in the
1755 <link linkend='build-directory'>Build Directory</link>.
1756 The
1757 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>
1758 shows this package feeds area in the upper-right corner.
1759 </para>
1760
1761 <para>
1762 This section looks a little closer into the package feeds area used
1763 by the build system.
1764 Here is a more detailed look at the area:
1765 <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
1766 </para>
1767
1768 <para>
1769 Package feeds are an intermediary step in the build process.
1770 The OpenEmbedded build system provides classes to generate
1771 different package types, and you specify which classes to enable
1772 through the
1773 <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
1774 variable.
1775 Before placing the packages into package feeds,
1776 the build process validates them with generated output quality
1777 assurance checks through the
1778 <link linkend='ref-classes-insane'><filename>insane</filename></link>
1779 class.
1780 </para>
1781
1782 <para>
1783 The package feed area resides in the Build Directory.
1784 The directory the build system uses to temporarily store packages
1785 is determined by a combination of variables and the particular
1786 package manager in use.
1787 See the "Package Feeds" box in the illustration and note the
1788 information to the right of that area.
1789 In particular, the following defines where package files are
1790 kept:
1791 <itemizedlist>
1792 <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:
1793 Defined as <filename>tmp/deploy</filename> in the Build
1794 Directory.
1795 </para></listitem>
1796 <listitem><para><filename>DEPLOY_DIR_*</filename>:
1797 Depending on the package manager used, the package type
1798 sub-folder.
1799 Given RPM, IPK, or DEB packaging and tarball creation, the
1800 <link linkend='var-DEPLOY_DIR_RPM'><filename>DEPLOY_DIR_RPM</filename></link>,
1801 <link linkend='var-DEPLOY_DIR_IPK'><filename>DEPLOY_DIR_IPK</filename></link>,
1802 <link linkend='var-DEPLOY_DIR_DEB'><filename>DEPLOY_DIR_DEB</filename></link>,
1803 or
1804 <link linkend='var-DEPLOY_DIR_TAR'><filename>DEPLOY_DIR_TAR</filename></link>,
1805 variables are used, respectively.
1806 </para></listitem>
1807 <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>:
1808 Defines architecture-specific sub-folders.
1809 For example, packages could exist for the i586 or qemux86
1810 architectures.
1811 </para></listitem>
1812 </itemizedlist>
1813 </para>
1814
1815 <para>
1816 BitBake uses the <filename>do_package_write_*</filename> tasks to
1817 generate packages and place them into the package holding area (e.g.
1818 <filename>do_package_write_ipk</filename> for IPK packages).
1819 See the
1820 "<link linkend='ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></link>",
1821 "<link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>",
1822 "<link linkend='ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></link>",
1823 and
1824 "<link linkend='ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></link>"
1825 sections for additional information.
1826 As an example, consider a scenario where an IPK packaging manager
1827 is being used and package architecture support for both i586
1828 and qemux86 exist.
1829 Packages for the i586 architecture are placed in
1830 <filename>build/tmp/deploy/ipk/i586</filename>, while packages for
1831 the qemux86 architecture are placed in
1832 <filename>build/tmp/deploy/ipk/qemux86</filename>.
1833 </para>
1834 </section>
1835
1836 <section id='bitbake-dev-environment'>
1837 <title>BitBake</title>
1838
1839 <para>
1840 The OpenEmbedded build system uses
1841 <link linkend='bitbake-term'>BitBake</link>
1842 to produce images.
1843 You can see from the
1844 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>,
1845 the BitBake area consists of several functional areas.
1846 This section takes a closer look at each of those areas.
1847 </para>
1848
1849 <para>
1850 Separate documentation exists for the BitBake tool.
1851 See the
1852 <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>
1853 for reference material on BitBake.
1854 </para>
1855
1856 <section id='source-fetching-dev-environment'>
1857 <title>Source Fetching</title>
1858
1859 <para>
1860 The first stages of building a recipe are to fetch and unpack
1861 the source code:
1862 <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" />
1863 </para>
1864
1865 <para>
1866 The
1867 <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link>
1868 and
1869 <link linkend='ref-tasks-unpack'><filename>do_unpack</filename></link>
1870 tasks fetch the source files and unpack them into the work
1871 directory.
1872 <note>
1873 For every local file (e.g. <filename>file://</filename>)
1874 that is part of a recipe's
1875 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
1876 statement, the OpenEmbedded build system takes a checksum
1877 of the file for the recipe and inserts the checksum into
1878 the signature for the <filename>do_fetch</filename>.
1879 If any local file has been modified, the
1880 <filename>do_fetch</filename> task and all tasks that
1881 depend on it are re-executed.
1882 </note>
1883 By default, everything is accomplished in the
1884 <link linkend='build-directory'>Build Directory</link>,
1885 which has a defined structure.
1886 For additional general information on the Build Directory,
1887 see the
1888 "<link linkend='structure-core-build'><filename>build/</filename></link>"
1889 section.
1890 </para>
1891
1892 <para>
1893 Unpacked source files are pointed to by the
1894 <link linkend='var-S'><filename>S</filename></link> variable.
1895 Each recipe has an area in the Build Directory where the
1896 unpacked source code resides.
1897 The name of that directory for any given recipe is defined from
1898 several different variables.
1899 You can see the variables that define these directories
1900 by looking at the figure:
1901 <itemizedlist>
1902 <listitem><para><link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> -
1903 The base directory where the OpenEmbedded build system
1904 performs all its work during the build.
1905 </para></listitem>
1906 <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link> -
1907 The architecture of the built package or packages.
1908 </para></listitem>
1909 <listitem><para><link linkend='var-TARGET_OS'><filename>TARGET_OS</filename></link> -
1910 The operating system of the target device.
1911 </para></listitem>
1912 <listitem><para><link linkend='var-PN'><filename>PN</filename></link> -
1913 The name of the built package.
1914 </para></listitem>
1915 <listitem><para><link linkend='var-PV'><filename>PV</filename></link> -
1916 The version of the recipe used to build the package.
1917 </para></listitem>
1918 <listitem><para><link linkend='var-PR'><filename>PR</filename></link> -
1919 The revision of the recipe used to build the package.
1920 </para></listitem>
1921 <listitem><para><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> -
1922 The location within <filename>TMPDIR</filename> where
1923 a specific package is built.
1924 </para></listitem>
1925 <listitem><para><link linkend='var-S'><filename>S</filename></link> -
1926 Contains the unpacked source files for a given recipe.
1927 </para></listitem>
1928 </itemizedlist>
1929 </para>
1930 </section>
1931
1932 <section id='patching-dev-environment'>
1933 <title>Patching</title>
1934
1935 <para>
1936 Once source code is fetched and unpacked, BitBake locates
1937 patch files and applies them to the source files:
1938 <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" />
1939 </para>
1940
1941 <para>
1942 The
1943 <link linkend='ref-tasks-patch'><filename>do_patch</filename></link>
1944 task processes recipes by
1945 using the
1946 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
1947 variable to locate applicable patch files, which by default
1948 are <filename>*.patch</filename> or
1949 <filename>*.diff</filename> files, or any file if
1950 "apply=yes" is specified for the file in
1951 <filename>SRC_URI</filename>.
1952 </para>
1953
1954 <para>
1955 BitBake finds and applies multiple patches for a single recipe
1956 in the order in which it finds the patches.
1957 Patches are applied to the recipe's source files located in the
1958 <link linkend='var-S'><filename>S</filename></link> directory.
1959 </para>
1960
1961 <para>
1962 For more information on how the source directories are
1963 created, see the
1964 "<link linkend='source-fetching-dev-environment'>Source Fetching</link>"
1965 section.
1966 </para>
1967 </section>
1968
1969 <section id='configuration-and-compilation-dev-environment'>
1970 <title>Configuration and Compilation</title>
1971
1972 <para>
1973 After source code is patched, BitBake executes tasks that
1974 configure and compile the source code:
1975 <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" />
1976 </para>
1977
1978 <para>
1979 This step in the build process consists of three tasks:
1980 <itemizedlist>
1981 <listitem><para>
1982 <emphasis><link linkend='ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></link>:</emphasis>
1983 This task sets up the two sysroots in
1984 <filename>${</filename><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link><filename>}</filename>
1985 (i.e. <filename>recipe-sysroot</filename> and
1986 <filename>recipe-sysroot-native</filename>) so that
1987 the sysroots contain the contents of the
1988 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>
1989 tasks of the recipes on which the recipe
1990 containing the tasks depends.
1991 A sysroot exists for both the target and for the native
1992 binaries, which run on the host system.
1993 </para></listitem>
1994 <listitem><para><emphasis><filename>do_configure</filename>:</emphasis>
1995 This task configures the source by enabling and
1996 disabling any build-time and configuration options for
1997 the software being built.
1998 Configurations can come from the recipe itself as well
1999 as from an inherited class.
2000 Additionally, the software itself might configure itself
2001 depending on the target for which it is being built.
2002 </para>
2003
2004 <para>The configurations handled by the
2005 <link linkend='ref-tasks-configure'><filename>do_configure</filename></link>
2006 task are specific
2007 to source code configuration for the source code
2008 being built by the recipe.</para>
2009
2010 <para>If you are using the
2011 <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
2012 class,
2013 you can add additional configuration options by using
2014 the <link linkend='var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></link>
2015 or
2016 <link linkend='var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></link>
2017 variables.
2018 For information on how this variable works within
2019 that class, see the
2020 <filename>meta/classes/autotools.bbclass</filename> file.
2021 </para></listitem>
2022 <listitem><para><emphasis><filename>do_compile</filename>:</emphasis>
2023 Once a configuration task has been satisfied, BitBake
2024 compiles the source using the
2025 <link linkend='ref-tasks-compile'><filename>do_compile</filename></link>
2026 task.
2027 Compilation occurs in the directory pointed to by the
2028 <link linkend='var-B'><filename>B</filename></link>
2029 variable.
2030 Realize that the <filename>B</filename> directory is, by
2031 default, the same as the
2032 <link linkend='var-S'><filename>S</filename></link>
2033 directory.</para></listitem>
2034 <listitem><para><emphasis><filename>do_install</filename>:</emphasis>
2035 Once compilation is done, BitBake executes the
2036 <link linkend='ref-tasks-install'><filename>do_install</filename></link>
2037 task.
2038 This task copies files from the <filename>B</filename>
2039 directory and places them in a holding area pointed to
2040 by the
2041 <link linkend='var-D'><filename>D</filename></link>
2042 variable.</para></listitem>
2043 </itemizedlist>
2044 </para>
2045 </section>
2046
2047 <section id='package-splitting-dev-environment'>
2048 <title>Package Splitting</title>
2049
2050 <para>
2051 After source code is configured and compiled, the
2052 OpenEmbedded build system analyzes
2053 the results and splits the output into packages:
2054 <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" />
2055 </para>
2056
2057 <para>
2058 The
2059 <link linkend='ref-tasks-package'><filename>do_package</filename></link>
2060 and
2061 <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>
2062 tasks combine to analyze
2063 the files found in the
2064 <link linkend='var-D'><filename>D</filename></link> directory
2065 and split them into subsets based on available packages and
2066 files.
2067 The analyzing process involves the following as well as other
2068 items: splitting out debugging symbols,
2069 looking at shared library dependencies between packages,
2070 and looking at package relationships.
2071 The <filename>do_packagedata</filename> task creates package
2072 metadata based on the analysis such that the
2073 OpenEmbedded build system can generate the final packages.
2074 Working, staged, and intermediate results of the analysis
2075 and package splitting process use these areas:
2076 <itemizedlist>
2077 <listitem><para><link linkend='var-PKGD'><filename>PKGD</filename></link> -
2078 The destination directory for packages before they are
2079 split.
2080 </para></listitem>
2081 <listitem><para><link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> -
2082 A shared, global-state directory that holds data
2083 generated during the packaging process.
2084 </para></listitem>
2085 <listitem><para><link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link> -
2086 A temporary work area used by the
2087 <filename>do_package</filename> task.
2088 </para></listitem>
2089 <listitem><para><link linkend='var-PKGDEST'><filename>PKGDEST</filename></link> -
2090 The parent directory for packages after they have
2091 been split.
2092 </para></listitem>
2093 </itemizedlist>
2094 The <link linkend='var-FILES'><filename>FILES</filename></link>
2095 variable defines the files that go into each package in
2096 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>.
2097 If you want details on how this is accomplished, you can
2098 look at the
2099 <link linkend='ref-classes-package'><filename>package</filename></link>
2100 class.
2101 </para>
2102
2103 <para>
2104 Depending on the type of packages being created (RPM, DEB, or
2105 IPK), the <filename>do_package_write_*</filename> task
2106 creates the actual packages and places them in the
2107 Package Feed area, which is
2108 <filename>${TMPDIR}/deploy</filename>.
2109 You can see the
2110 "<link linkend='package-feeds-dev-environment'>Package Feeds</link>"
2111 section for more detail on that part of the build process.
2112 <note>
2113 Support for creating feeds directly from the
2114 <filename>deploy/*</filename> directories does not exist.
2115 Creating such feeds usually requires some kind of feed
2116 maintenance mechanism that would upload the new packages
2117 into an official package feed (e.g. the
2118 Ångström distribution).
2119 This functionality is highly distribution-specific
2120 and thus is not provided out of the box.
2121 </note>
2122 </para>
2123 </section>
2124
2125 <section id='image-generation-dev-environment'>
2126 <title>Image Generation</title>
2127
2128 <para>
2129 Once packages are split and stored in the Package Feeds area,
2130 the OpenEmbedded build system uses BitBake to generate the
2131 root filesystem image:
2132 <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" />
2133 </para>
2134
2135 <para>
2136 The image generation process consists of several stages and
2137 depends on several tasks and variables.
2138 The
2139 <link linkend='ref-tasks-rootfs'><filename>do_rootfs</filename></link>
2140 task creates the root filesystem (file and directory structure)
2141 for an image.
2142 This task uses several key variables to help create the list
2143 of packages to actually install:
2144 <itemizedlist>
2145 <listitem><para><link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>:
2146 Lists out the base set of packages to install from
2147 the Package Feeds area.</para></listitem>
2148 <listitem><para><link linkend='var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></link>:
2149 Specifies packages that should not be installed.
2150 </para></listitem>
2151 <listitem><para><link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>:
2152 Specifies features to include in the image.
2153 Most of these features map to additional packages for
2154 installation.</para></listitem>
2155 <listitem><para><link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>:
2156 Specifies the package backend to use and consequently
2157 helps determine where to locate packages within the
2158 Package Feeds area.</para></listitem>
2159 <listitem><para><link linkend='var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></link>:
2160 Determines the language(s) for which additional
2161 language support packages are installed.
2162 </para></listitem>
2163 <listitem><para><link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link>:
2164 The final list of packages passed to the package manager
2165 for installation into the image.
2166 </para></listitem>
2167 </itemizedlist>
2168 </para>
2169
2170 <para>
2171 With
2172 <link linkend='var-IMAGE_ROOTFS'><filename>IMAGE_ROOTFS</filename></link>
2173 pointing to the location of the filesystem under construction and
2174 the <filename>PACKAGE_INSTALL</filename> variable providing the
2175 final list of packages to install, the root file system is
2176 created.
2177 </para>
2178
2179 <para>
2180 Package installation is under control of the package manager
2181 (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of whether or
2182 not package management is enabled for the target.
2183 At the end of the process, if package management is not
2184 enabled for the target, the package manager's data files
2185 are deleted from the root filesystem.
2186 As part of the final stage of package installation, postinstall
2187 scripts that are part of the packages are run.
2188 Any scripts that fail to run
2189 on the build host are run on the target when the target system
2190 is first booted.
2191 If you are using a
2192 <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>,
2193 all the post installation scripts must succeed during the
2194 package installation phase since the root filesystem is
2195 read-only.
2196 </para>
2197
2198 <para>
2199 The final stages of the <filename>do_rootfs</filename> task
2200 handle post processing.
2201 Post processing includes creation of a manifest file and
2202 optimizations.
2203 </para>
2204
2205 <para>
2206 The manifest file (<filename>.manifest</filename>) resides
2207 in the same directory as the root filesystem image.
2208 This file lists out, line-by-line, the installed packages.
2209 The manifest file is useful for the
2210 <link linkend='ref-classes-testimage*'><filename>testimage</filename></link>
2211 class, for example, to determine whether or not to run
2212 specific tests.
2213 See the
2214 <link linkend='var-IMAGE_MANIFEST'><filename>IMAGE_MANIFEST</filename></link>
2215 variable for additional information.
2216 </para>
2217
2218 <para>
2219 Optimizing processes run across the image include
2220 <filename>mklibs</filename>, <filename>prelink</filename>,
2221 and any other post-processing commands as defined by the
2222 <link linkend='var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></link>
2223 variable.
2224 The <filename>mklibs</filename> process optimizes the size
2225 of the libraries, while the
2226 <filename>prelink</filename> process optimizes the dynamic
2227 linking of shared libraries to reduce start up time of
2228 executables.
2229 </para>
2230
2231 <para>
2232 After the root filesystem is built, processing begins on
2233 the image through the
2234 <link linkend='ref-tasks-image'><filename>do_image</filename></link>
2235 task.
2236 The build system runs any pre-processing commands as defined
2237 by the
2238 <link linkend='var-IMAGE_PREPROCESS_COMMAND'><filename>IMAGE_PREPROCESS_COMMAND</filename></link>
2239 variable.
2240 This variable specifies a list of functions to call before
2241 the OpenEmbedded build system creates the final image output
2242 files.
2243 </para>
2244
2245 <para>
2246 The OpenEmbedded build system dynamically creates
2247 <filename>do_image_*</filename> tasks as needed, based
2248 on the image types specified in the
2249 <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
2250 variable.
2251 The process turns everything into an image file or a set of
2252 image files and compresses the root filesystem image to reduce
2253 the overall size of the image.
2254 The formats used for the root filesystem depend on the
2255 <filename>IMAGE_FSTYPES</filename> variable.
2256 </para>
2257
2258 <para>
2259 As an example, a dynamically created task when creating a
2260 particular image <replaceable>type</replaceable> would take the
2261 following form:
2262 <literallayout class='monospaced'>
2263 do_image_<replaceable>type</replaceable>[depends]
2264 </literallayout>
2265 So, if the <replaceable>type</replaceable> as specified by the
2266 <filename>IMAGE_FSTYPES</filename> were
2267 <filename>ext4</filename>, the dynamically generated task
2268 would be as follows:
2269 <literallayout class='monospaced'>
2270 do_image_ext4[depends]
2271 </literallayout>
2272 </para>
2273
2274 <para>
2275 The final task involved in image creation is the
2276 <link linkend='ref-tasks-image-complete'><filename>do_image_complete</filename></link>
2277 task.
2278 This task completes the image by applying any image
2279 post processing as defined through the
2280 <link linkend='var-IMAGE_POSTPROCESS_COMMAND'><filename>IMAGE_POSTPROCESS_COMMAND</filename></link>
2281 variable.
2282 The variable specifies a list of functions to call once the
2283 OpenEmbedded build system has created the final image output
2284 files.
2285 </para>
2286
2287 <note>
2288 The entire image generation process is run under Pseudo.
2289 Running under Pseudo ensures that the files in the root
2290 filesystem have correct ownership.
2291 </note>
2292 </section>
2293
2294 <section id='sdk-generation-dev-environment'>
2295 <title>SDK Generation</title>
2296
2297 <para>
2298 The OpenEmbedded build system uses BitBake to generate the
2299 Software Development Kit (SDK) installer script for both the
2300 standard and extensible SDKs:
2301 <imagedata fileref="figures/sdk-generation.png" align="center" />
2302 </para>
2303
2304 <note>
2305 For more information on the cross-development toolchain
2306 generation, see the
2307 "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
2308 section.
2309 For information on advantages gained when building a
2310 cross-development toolchain using the
2311 <link linkend='ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></link>
2312 task, see the
2313 "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>"
2314 section in the Yocto Project Application Development and the
2315 Extensible Software Development Kit (SDK) manual.
2316 </note>
2317
2318 <para>
2319 Like image generation, the SDK script process consists of
2320 several stages and depends on many variables.
2321 The <filename>do_populate_sdk</filename> and
2322 <filename>do_populate_sdk_ext</filename> tasks use these
2323 key variables to help create the list of packages to actually
2324 install.
2325 For information on the variables listed in the figure, see the
2326 "<link linkend='sdk-dev-environment'>Application Development SDK</link>"
2327 section.
2328 </para>
2329
2330 <para>
2331 The <filename>do_populate_sdk</filename> task helps create
2332 the standard SDK and handles two parts: a target part and a
2333 host part.
2334 The target part is the part built for the target hardware and
2335 includes libraries and headers.
2336 The host part is the part of the SDK that runs on the
2337 <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>.
2338 </para>
2339
2340 <para>
2341 The <filename>do_populate_sdk_ext</filename> task helps create
2342 the extensible SDK and handles host and target parts
2343 differently than its counter part does for the standard SDK.
2344 For the extensible SDK, the task encapsulates the build system,
2345 which includes everything needed (host and target) for the SDK.
2346 </para>
2347
2348 <para>
2349 Regardless of the type of SDK being constructed, the
2350 tasks perform some cleanup after which a cross-development
2351 environment setup script and any needed configuration files
2352 are created.
2353 The final output is the Cross-development
2354 toolchain installation script (<filename>.sh</filename> file),
2355 which includes the environment setup script.
2356 </para>
2357 </section>
2358
2359 <section id='stamp-files-and-the-rerunning-of-tasks'>
2360 <title>Stamp Files and the Rerunning of Tasks</title>
2361
2362 <para>
2363 For each task that completes successfully, BitBake writes a
2364 stamp file into the
2365 <link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link>
2366 directory.
2367 The beginning of the stamp file's filename is determined by the
2368 <link linkend='var-STAMP'><filename>STAMP</filename></link>
2369 variable, and the end of the name consists of the task's name
2370 and current
2371 <ulink url='&YOCTO_DOCS_BB_URL;#checksums'>input checksum</ulink>.
2372 <note>
2373 This naming scheme assumes that
2374 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SIGNATURE_HANDLER'><filename>BB_SIGNATURE_HANDLER</filename></ulink>
2375 is "OEBasicHash", which is almost always the case in
2376 current OpenEmbedded.
2377 </note>
2378 To determine if a task needs to be rerun, BitBake checks if a
2379 stamp file with a matching input checksum exists for the task.
2380 If such a stamp file exists, the task's output is assumed to
2381 exist and still be valid.
2382 If the file does not exist, the task is rerun.
2383 <note>
2384 <para>The stamp mechanism is more general than the shared
2385 state (sstate) cache mechanism described in the
2386 "<link linkend='setscene-tasks-and-shared-state'>Setscene Tasks and Shared State</link>"
2387 section.
2388 BitBake avoids rerunning any task that has a valid
2389 stamp file, not just tasks that can be accelerated through
2390 the sstate cache.</para>
2391 <para>However, you should realize that stamp files only
2392 serve as a marker that some work has been done and that
2393 these files do not record task output.
2394 The actual task output would usually be somewhere in
2395 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
2396 (e.g. in some recipe's
2397 <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.)
2398 What the sstate cache mechanism adds is a way to cache task
2399 output that can then be shared between build machines.
2400 </para>
2401 </note>
2402 Since <filename>STAMPS_DIR</filename> is usually a subdirectory
2403 of <filename>TMPDIR</filename>, removing
2404 <filename>TMPDIR</filename> will also remove
2405 <filename>STAMPS_DIR</filename>, which means tasks will
2406 properly be rerun to repopulate <filename>TMPDIR</filename>.
2407 </para>
2408
2409 <para>
2410 If you want some task to always be considered "out of date",
2411 you can mark it with the
2412 <ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink>
2413 varflag.
2414 If some other task depends on such a task, then that task will
2415 also always be considered out of date, which might not be what
2416 you want.
2417 </para>
2418
2419 <para>
2420 For details on how to view information about a task's
2421 signature, see the
2422 "<link linkend='usingpoky-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>"
2423 section.
2424 </para>
2425 </section>
2426
2427 <section id='setscene-tasks-and-shared-state'>
2428 <title>Setscene Tasks and Shared State</title>
2429
2430 <para>
2431 The description of tasks so far assumes that BitBake needs to
2432 build everything and there are no prebuilt objects available.
2433 BitBake does support skipping tasks if prebuilt objects are
2434 available.
2435 These objects are usually made available in the form of a
2436 shared state (sstate) cache.
2437 <note>
2438 For information on variables affecting sstate, see the
2439 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
2440 and
2441 <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
2442 variables.
2443 </note>
2444 </para>
2445
2446 <para>
2447 The idea of a setscene task (i.e
2448 <filename>do_</filename><replaceable>taskname</replaceable><filename>_setscene</filename>)
2449 is a version of the task where
2450 instead of building something, BitBake can skip to the end
2451 result and simply place a set of files into specific locations
2452 as needed.
2453 In some cases, it makes sense to have a setscene task variant
2454 (e.g. generating package files in the
2455 <filename>do_package_write_*</filename> task).
2456 In other cases, it does not make sense, (e.g. a
2457 <link linkend='ref-tasks-patch'><filename>do_patch</filename></link>
2458 task or
2459 <link linkend='ref-tasks-unpack'><filename>do_unpack</filename></link>
2460 task) since the work involved would be equal to or greater than
2461 the underlying task.
2462 </para>
2463
2464 <para>
2465 In the OpenEmbedded build system, the common tasks that have
2466 setscene variants are <link linkend='ref-tasks-package'><filename>do_package</filename></link>,
2467 <filename>do_package_write_*</filename>,
2468 <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link>,
2469 <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link>,
2470 and
2471 <link linkend='ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></link>.
2472 Notice that these are most of the tasks whose output is an
2473 end result.
2474 </para>
2475
2476 <para>
2477 The OpenEmbedded build system has knowledge of the relationship
2478 between these tasks and other tasks that precede them.
2479 For example, if BitBake runs
2480 <filename>do_populate_sysroot_setscene</filename> for
2481 something, there is little point in running any of the
2482 <filename>do_fetch</filename>, <filename>do_unpack</filename>,
2483 <filename>do_patch</filename>,
2484 <filename>do_configure</filename>,
2485 <filename>do_compile</filename>, and
2486 <filename>do_install</filename> tasks.
2487 However, if <filename>do_package</filename> needs to be run,
2488 BitBake would need to run those other tasks.
2489 </para>
2490
2491 <para>
2492 It becomes more complicated if everything can come from an
2493 sstate cache because some objects are simply not required at
2494 all.
2495 For example, you do not need a compiler or native tools, such
2496 as quilt, if there is nothing to compile or patch.
2497 If the <filename>do_package_write_*</filename> packages are
2498 available from sstate, BitBake does not need the
2499 <filename>do_package</filename> task data.
2500 </para>
2501
2502 <para>
2503 To handle all these complexities, BitBake runs in two phases.
2504 The first is the "setscene" stage.
2505 During this stage, BitBake first checks the sstate cache for
2506 any targets it is planning to build.
2507 BitBake does a fast check to see if the object exists rather
2508 than a complete download.
2509 If nothing exists, the second phase, which is the setscene
2510 stage, completes and the main build proceeds.
2511 </para>
2512
2513 <para>
2514 If objects are found in the sstate cache, the OpenEmbedded
2515 build system works backwards from the end targets specified
2516 by the user.
2517 For example, if an image is being built, the OpenEmbedded build
2518 system first looks for the packages needed for that image and
2519 the tools needed to construct an image.
2520 If those are available, the compiler is not needed.
2521 Thus, the compiler is not even downloaded.
2522 If something was found to be unavailable, or the download or
2523 setscene task fails, the OpenEmbedded build system then tries
2524 to install dependencies, such as the compiler, from the cache.
2525 </para>
2526
2527 <para>
2528 The availability of objects in the sstate cache is handled by
2529 the function specified by the
2530 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></ulink>
2531 variable and returns a list of the objects that are available.
2532 The function specified by the
2533 <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></ulink>
2534 variable is the function that determines whether a given
2535 dependency needs to be followed, and whether for any given
2536 relationship the function needs to be passed.
2537 The function returns a True or False value.
2538 </para>
2539 </section>
2540 </section>
2541
2542 <section id='images-dev-environment'>
2543 <title>Images</title>
2544
2545 <para>
2546 The images produced by the OpenEmbedded build system
2547 are compressed forms of the
2548 root filesystem that are ready to boot on a target device.
2549 You can see from the
2550 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>
2551 that BitBake output, in part, consists of images.
2552 This section is going to look more closely at this output:
2553 <imagedata fileref="figures/images.png" align="center" width="5.5in" depth="5.5in" />
2554 </para>
2555
2556 <para>
2557 For a list of example images that the Yocto Project provides,
2558 see the
2559 "<link linkend='ref-images'>Images</link>" chapter.
2560 </para>
2561
2562 <para>
2563 Images are written out to the
2564 <link linkend='build-directory'>Build Directory</link>
2565 inside the <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename>
2566 folder as shown in the figure.
2567 This folder contains any files expected to be loaded on the
2568 target device.
2569 The
2570 <link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>
2571 variable points to the <filename>deploy</filename> directory,
2572 while the
2573 <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link>
2574 variable points to the appropriate directory containing images for
2575 the current configuration.
2576 <itemizedlist>
2577 <listitem><para><filename><replaceable>kernel-image</replaceable></filename>:
2578 A kernel binary file.
2579 The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link>
2580 variable setting determines the naming scheme for the
2581 kernel image file.
2582 Depending on that variable, the file could begin with
2583 a variety of naming strings.
2584 The <filename>deploy/images/<replaceable>machine</replaceable></filename>
2585 directory can contain multiple image files for the
2586 machine.</para></listitem>
2587 <listitem><para><filename><replaceable>root-filesystem-image</replaceable></filename>:
2588 Root filesystems for the target device (e.g.
2589 <filename>*.ext3</filename> or <filename>*.bz2</filename>
2590 files).
2591 The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
2592 variable setting determines the root filesystem image
2593 type.
2594 The <filename>deploy/images/<replaceable>machine</replaceable></filename>
2595 directory can contain multiple root filesystems for the
2596 machine.</para></listitem>
2597 <listitem><para><filename><replaceable>kernel-modules</replaceable></filename>:
2598 Tarballs that contain all the modules built for the kernel.
2599 Kernel module tarballs exist for legacy purposes and
2600 can be suppressed by setting the
2601 <link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link>
2602 variable to "0".
2603 The <filename>deploy/images/<replaceable>machine</replaceable></filename>
2604 directory can contain multiple kernel module tarballs
2605 for the machine.</para></listitem>
2606 <listitem><para><filename><replaceable>bootloaders</replaceable></filename>:
2607 Bootloaders supporting the image, if applicable to the
2608 target machine.
2609 The <filename>deploy/images/<replaceable>machine</replaceable></filename>
2610 directory can contain multiple bootloaders for the
2611 machine.</para></listitem>
2612 <listitem><para><filename><replaceable>symlinks</replaceable></filename>:
2613 The <filename>deploy/images/<replaceable>machine</replaceable></filename>
2614 folder contains
2615 a symbolic link that points to the most recently built file
2616 for each machine.
2617 These links might be useful for external scripts that
2618 need to obtain the latest version of each file.
2619 </para></listitem>
2620 </itemizedlist>
2621 </para>
2622 </section>
2623
2624 <section id='sdk-dev-environment'>
2625 <title>Application Development SDK</title>
2626
2627 <para>
2628 In the
2629 <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>,
2630 the output labeled "Application Development SDK" represents an
2631 SDK.
2632 The SDK generation process differs depending on whether you build
2633 a standard SDK
2634 (e.g. <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>)
2635 or an extensible SDK
2636 (e.g. <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>).
2637 This section is going to take a closer look at this output:
2638 <imagedata fileref="figures/sdk.png" align="center" width="9in" depth="7.25in" />
2639 </para>
2640
2641 <para>
2642 The specific form of this output is a self-extracting
2643 SDK installer (<filename>*.sh</filename>) that, when run,
2644 installs the SDK, which consists of a cross-development
2645 toolchain, a set of libraries and headers, and an SDK
2646 environment setup script.
2647 Running this installer essentially sets up your
2648 cross-development environment.
2649 You can think of the cross-toolchain as the "host"
2650 part because it runs on the SDK machine.
2651 You can think of the libraries and headers as the "target"
2652 part because they are built for the target hardware.
2653 The environment setup script is added so that you can initialize
2654 the environment before using the tools.
2655 </para>
2656
2657 <note>
2658 <para>
2659 The Yocto Project supports several methods by which you can
2660 set up this cross-development environment.
2661 These methods include downloading pre-built SDK installers
2662 or building and installing your own SDK installer.
2663 </para>
2664
2665 <para>
2666 For background information on cross-development toolchains
2667 in the Yocto Project development environment, see the
2668 "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
2669 section.
2670 For information on setting up a cross-development
2671 environment, see the
2672 <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink>
2673 manual.
2674 </para>
2675 </note>
2676
2677 <para>
2678 Once built, the SDK installers are written out to the
2679 <filename>deploy/sdk</filename> folder inside the
2680 <link linkend='build-directory'>Build Directory</link>
2681 as shown in the figure at the beginning of this section.
2682 Depending on the type of SDK, several variables exist that help
2683 configure these files.
2684 The following list shows the variables associated with a standard
2685 SDK:
2686 <itemizedlist>
2687 <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:
2688 Points to the <filename>deploy</filename>
2689 directory.</para></listitem>
2690 <listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>:
2691 Specifies the architecture of the machine
2692 on which the cross-development tools are run to
2693 create packages for the target hardware.
2694 </para></listitem>
2695 <listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>:
2696 Lists the features to include in the "target" part
2697 of the SDK.
2698 </para></listitem>
2699 <listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>:
2700 Lists packages that make up the host
2701 part of the SDK (i.e. the part that runs on
2702 the <filename>SDKMACHINE</filename>).
2703 When you use
2704 <filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename>
2705 to create the SDK, a set of default packages
2706 apply.
2707 This variable allows you to add more packages.
2708 </para></listitem>
2709 <listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>:
2710 Lists packages that make up the target part
2711 of the SDK (i.e. the part built for the
2712 target hardware).
2713 </para></listitem>
2714 <listitem><para><link linkend='var-SDKPATH'><filename>SDKPATH</filename></link>:
2715 Defines the default SDK installation path offered by the
2716 installation script.
2717 </para></listitem>
2718 </itemizedlist>
2719 This next list, shows the variables associated with an extensible
2720 SDK:
2721 <itemizedlist>
2722 <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:
2723 Points to the <filename>deploy</filename> directory.
2724 </para></listitem>
2725 <listitem><para><link linkend='var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></link>:
2726 Controls whether or not shared state artifacts are copied
2727 into the extensible SDK.
2728 By default, all required shared state artifacts are copied
2729 into the SDK.
2730 </para></listitem>
2731 <listitem><para><link linkend='var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></link>:
2732 Specifies whether or not packagedata will be included in
2733 the extensible SDK for all recipes in the "world" target.
2734 </para></listitem>
2735 <listitem><para><link linkend='var-SDK_INCLUDE_TOOLCHAIN'><filename>SDK_INCLUDE_TOOLCHAIN</filename></link>:
2736 Specifies whether or not the toolchain will be included
2737 when building the extensible SDK.
2738 </para></listitem>
2739 <listitem><para><link linkend='var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></link>:
2740 A list of variables allowed through from the build system
2741 configuration into the extensible SDK configuration.
2742 </para></listitem>
2743 <listitem><para><link linkend='var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></link>:
2744 A list of variables not allowed through from the build
2745 system configuration into the extensible SDK configuration.
2746 </para></listitem>
2747 <listitem><para><link linkend='var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></link>:
2748 A list of classes to remove from the
2749 <link linkend='var-INHERIT'><filename>INHERIT</filename></link>
2750 value globally within the extensible SDK configuration.
2751 </para></listitem>
2752 </itemizedlist>
2753 </para>
2754 </section>
2755</section>
2756
2757</chapter>
2758<!--
2759vim: expandtab tw=80 ts=4
2760-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-02-01 21:32:51 +0000