diff options
Diffstat (limited to 'documentation/overview-manual/overview-development-environment.xml')
-rw-r--r-- | documentation/overview-manual/overview-development-environment.xml | 2773 |
1 files changed, 2773 insertions, 0 deletions
diff --git a/documentation/overview-manual/overview-development-environment.xml b/documentation/overview-manual/overview-development-environment.xml new file mode 100644 index 0000000000..32fe1816ab --- /dev/null +++ b/documentation/overview-manual/overview-development-environment.xml | |||
@@ -0,0 +1,2773 @@ | |||
1 | <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" | ||
2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" | ||
3 | [<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > | ||
4 | |||
5 | <chapter id='overview-development-environment'> | ||
6 | <title>The Yocto Project Development Environment</title> | ||
7 | |||
8 | <para> | ||
9 | This chapter takes a look at the Yocto Project development | ||
10 | environment and also provides a detailed look at what goes on during | ||
11 | development in that environment. | ||
12 | The chapter provides Yocto Project Development environment concepts that | ||
13 | help you understand how work is accomplished in an open source environment, | ||
14 | which is very different as compared to work accomplished in a closed, | ||
15 | proprietary environment. | ||
16 | </para> | ||
17 | |||
18 | <para> | ||
19 | Specifically, this chapter addresses open source philosophy, workflows, | ||
20 | Git, source repositories, licensing, recipe syntax, and development | ||
21 | syntax. | ||
22 | </para> | ||
23 | |||
24 | <section id='open-source-philosophy'> | ||
25 | <title>Open Source Philosophy</title> | ||
26 | |||
27 | <para> | ||
28 | Open source philosophy is characterized by software development | ||
29 | directed by peer production and collaboration through an active | ||
30 | community of developers. | ||
31 | Contrast this to the more standard centralized development models | ||
32 | used by commercial software companies where a finite set of developers | ||
33 | produces a product for sale using a defined set of procedures that | ||
34 | ultimately result in an end product whose architecture and source | ||
35 | material are closed to the public. | ||
36 | </para> | ||
37 | |||
38 | <para> | ||
39 | Open source projects conceptually have differing concurrent agendas, | ||
40 | approaches, and production. | ||
41 | These facets of the development process can come from anyone in the | ||
42 | public (community) that has a stake in the software project. | ||
43 | The open source environment contains new copyright, licensing, domain, | ||
44 | and consumer issues that differ from the more traditional development | ||
45 | environment. | ||
46 | In an open source environment, the end product, source material, | ||
47 | and documentation are all available to the public at no cost. | ||
48 | </para> | ||
49 | |||
50 | <para> | ||
51 | A benchmark example of an open source project is the Linux kernel, | ||
52 | which was initially conceived and created by Finnish computer science | ||
53 | student Linus Torvalds in 1991. | ||
54 | Conversely, a good example of a non-open source project is the | ||
55 | <trademark class='registered'>Windows</trademark> family of operating | ||
56 | systems developed by | ||
57 | <trademark class='registered'>Microsoft</trademark> Corporation. | ||
58 | </para> | ||
59 | |||
60 | <para> | ||
61 | Wikipedia has a good historical description of the Open Source | ||
62 | Philosophy | ||
63 | <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. | ||
64 | You can also find helpful information on how to participate in the | ||
65 | Linux Community | ||
66 | <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>. | ||
67 | </para> | ||
68 | </section> | ||
69 | |||
70 | <section id='workflows'> | ||
71 | <title>Workflows</title> | ||
72 | |||
73 | <para> | ||
74 | This section provides workflow concepts using the Yocto Project and | ||
75 | Git. | ||
76 | In particular, the information covers basic practices that describe | ||
77 | roles and actions in a collaborative development environment. | ||
78 | <note> | ||
79 | If you are familiar with this type of development environment, you | ||
80 | might not want to read this section. | ||
81 | </note> | ||
82 | </para> | ||
83 | |||
84 | <para> | ||
85 | The Yocto Project files are maintained using Git in "master" | ||
86 | branches whose Git histories track every change and whose structures | ||
87 | provides branches for all diverging functionality. | ||
88 | Although there is no need to use Git, many open source projects do so. | ||
89 | <para> | ||
90 | |||
91 | </para> | ||
92 | For the Yocto Project, a key individual called the "maintainer" is | ||
93 | responsible for the "master" branch of a given Git repository. | ||
94 | The "master" branch is the “upstream” repository from which final or | ||
95 | most recent builds of the project occur. | ||
96 | The maintainer is responsible for accepting changes from other | ||
97 | developers and for organizing the underlying branch structure to | ||
98 | reflect release strategies and so forth. | ||
99 | <note>For information on finding out who is responsible for (maintains) | ||
100 | a particular area of code, see the | ||
101 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" | ||
102 | section of the Yocto Project Development Tasks Manual. | ||
103 | </note> | ||
104 | </para> | ||
105 | |||
106 | <para> | ||
107 | The Yocto Project <filename>poky</filename> Git repository also has an | ||
108 | upstream contribution Git repository named | ||
109 | <filename>poky-contrib</filename>. | ||
110 | You can see all the branches in this repository using the web interface | ||
111 | of the | ||
112 | <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized | ||
113 | within the "Poky Support" area. | ||
114 | These branches temporarily hold changes to the project that have been | ||
115 | submitted or committed by the Yocto Project development team and by | ||
116 | community members who contribute to the project. | ||
117 | The maintainer determines if the changes are qualified to be moved | ||
118 | from the "contrib" branches into the "master" branch of the Git | ||
119 | repository. | ||
120 | </para> | ||
121 | |||
122 | <para> | ||
123 | Developers (including contributing community members) create and | ||
124 | maintain cloned repositories of the upstream "master" branch. | ||
125 | The cloned repositories are local to their development platforms and | ||
126 | are used to develop changes. | ||
127 | When a developer is satisfied with a particular feature or change, | ||
128 | they "push" the changes to the appropriate "contrib" repository. | ||
129 | </para> | ||
130 | |||
131 | <para> | ||
132 | Developers are responsible for keeping their local repository | ||
133 | up-to-date with "master". | ||
134 | They are also responsible for straightening out any conflicts that | ||
135 | might arise within files that are being worked on simultaneously by | ||
136 | more than one person. | ||
137 | All this work is done locally on the developer’s machine before | ||
138 | anything is pushed to a "contrib" area and examined at the maintainer’s | ||
139 | level. | ||
140 | </para> | ||
141 | |||
142 | <para> | ||
143 | A somewhat formal method exists by which developers commit changes | ||
144 | and push them into the "contrib" area and subsequently request that | ||
145 | the maintainer include them into "master". | ||
146 | This process is called “submitting a patch” or "submitting a change." | ||
147 | For information on submitting patches and changes, see the | ||
148 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" | ||
149 | section in the Yocto Project Development Tasks Manual. | ||
150 | </para> | ||
151 | |||
152 | <para> | ||
153 | To summarize the development workflow: a single point of entry | ||
154 | exists for changes into the project’s "master" branch of the | ||
155 | Git repository, which is controlled by the project’s maintainer. | ||
156 | And, a set of developers exist who independently develop, test, and | ||
157 | submit changes to "contrib" areas for the maintainer to examine. | ||
158 | The maintainer then chooses which changes are going to become a | ||
159 | permanent part of the project. | ||
160 | </para> | ||
161 | |||
162 | <para> | ||
163 | <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" /> | ||
164 | </para> | ||
165 | |||
166 | <para> | ||
167 | While each development environment is unique, there are some best | ||
168 | practices or methods that help development run smoothly. | ||
169 | The following list describes some of these practices. | ||
170 | For more information about Git workflows, see the workflow topics in | ||
171 | the | ||
172 | <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. | ||
173 | <itemizedlist> | ||
174 | <listitem><para> | ||
175 | <emphasis>Make Small Changes:</emphasis> | ||
176 | It is best to keep the changes you commit small as compared to | ||
177 | bundling many disparate changes into a single commit. | ||
178 | This practice not only keeps things manageable but also allows | ||
179 | the maintainer to more easily include or refuse changes.</para> | ||
180 | |||
181 | <para>It is also good practice to leave the repository in a | ||
182 | state that allows you to still successfully build your project. | ||
183 | In other words, do not commit half of a feature, | ||
184 | then add the other half as a separate, later commit. | ||
185 | Each commit should take you from one buildable project state | ||
186 | to another buildable state. | ||
187 | </para></listitem> | ||
188 | <listitem><para> | ||
189 | <emphasis>Use Branches Liberally:</emphasis> | ||
190 | It is very easy to create, use, and delete local branches in | ||
191 | your working Git repository. | ||
192 | You can name these branches anything you like. | ||
193 | It is helpful to give them names associated with the particular | ||
194 | feature or change on which you are working. | ||
195 | Once you are done with a feature or change and have merged it | ||
196 | into your local master branch, simply discard the temporary | ||
197 | branch. | ||
198 | </para></listitem> | ||
199 | <listitem><para> | ||
200 | <emphasis>Merge Changes:</emphasis> | ||
201 | The <filename>git merge</filename> command allows you to take | ||
202 | the changes from one branch and fold them into another branch. | ||
203 | This process is especially helpful when more than a single | ||
204 | developer might be working on different parts of the same | ||
205 | feature. | ||
206 | Merging changes also automatically identifies any collisions | ||
207 | or "conflicts" that might happen as a result of the same lines | ||
208 | of code being altered by two different developers. | ||
209 | </para></listitem> | ||
210 | <listitem><para> | ||
211 | <emphasis>Manage Branches:</emphasis> | ||
212 | Because branches are easy to use, you should use a system | ||
213 | where branches indicate varying levels of code readiness. | ||
214 | For example, you can have a "work" branch to develop in, a | ||
215 | "test" branch where the code or change is tested, a "stage" | ||
216 | branch where changes are ready to be committed, and so forth. | ||
217 | As your project develops, you can merge code across the | ||
218 | branches to reflect ever-increasing stable states of the | ||
219 | development. | ||
220 | </para></listitem> | ||
221 | <listitem><para> | ||
222 | <emphasis>Use Push and Pull:</emphasis> | ||
223 | The push-pull workflow is based on the concept of developers | ||
224 | "pushing" local commits to a remote repository, which is | ||
225 | usually a contribution repository. | ||
226 | This workflow is also based on developers "pulling" known | ||
227 | states of the project down into their local development | ||
228 | repositories. | ||
229 | The workflow easily allows you to pull changes submitted by | ||
230 | other developers from the upstream repository into your | ||
231 | work area ensuring that you have the most recent software | ||
232 | on which to develop. | ||
233 | The Yocto Project has two scripts named | ||
234 | <filename>create-pull-request</filename> and | ||
235 | <filename>send-pull-request</filename> that ship with the | ||
236 | release to facilitate this workflow. | ||
237 | You can find these scripts in the <filename>scripts</filename> | ||
238 | folder of the | ||
239 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. | ||
240 | For information on how to use these scripts, see the | ||
241 | "<ulink url='&YOCTO_DOCS_DEV_URL;#pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</ulink>" | ||
242 | section in the Yocto Project Development Tasks Manual. | ||
243 | </para></listitem> | ||
244 | <listitem><para> | ||
245 | <emphasis>Patch Workflow:</emphasis> | ||
246 | This workflow allows you to notify the maintainer through an | ||
247 | email that you have a change (or patch) you would like | ||
248 | considered for the "master" branch of the Git repository. | ||
249 | To send this type of change, you format the patch and then | ||
250 | send the email using the Git commands | ||
251 | <filename>git format-patch</filename> and | ||
252 | <filename>git send-email</filename>. | ||
253 | For information on how to use these scripts, see the | ||
254 | "<ulink url='&YOCTO_DOCS_DEV_URL;#how-to-submit-a-change'>Submitting a Change to the Yocto Project</ulink>" | ||
255 | section in the Yocto Project Development Tasks Manual. | ||
256 | </para></listitem> | ||
257 | </itemizedlist> | ||
258 | </para> | ||
259 | </section> | ||
260 | |||
261 | <section id='git'> | ||
262 | <title>Git</title> | ||
263 | |||
264 | <para> | ||
265 | The Yocto Project makes extensive use of Git, which is a | ||
266 | free, open source distributed version control system. | ||
267 | Git supports distributed development, non-linear development, | ||
268 | and can handle large projects. | ||
269 | It is best that you have some fundamental understanding | ||
270 | of how Git tracks projects and how to work with Git if | ||
271 | you are going to use the Yocto Project for development. | ||
272 | This section provides a quick overview of how Git works and | ||
273 | provides you with a summary of some essential Git commands. | ||
274 | <note><title>Notes</title> | ||
275 | <itemizedlist> | ||
276 | <listitem><para> | ||
277 | For more information on Git, see | ||
278 | <ulink url='http://git-scm.com/documentation'></ulink>. | ||
279 | </para></listitem> | ||
280 | <listitem><para> | ||
281 | If you need to download Git, it is recommended that you add | ||
282 | Git to your system through your distribution's "software | ||
283 | store" (e.g. for Ubuntu, use the Ubuntu Software feature). | ||
284 | For the Git download page, see | ||
285 | <ulink url='http://git-scm.com/download'></ulink>. | ||
286 | </para></listitem> | ||
287 | <listitem><para> | ||
288 | For examples beyond the limited few in this section on how | ||
289 | to use Git with the Yocto Project, see the | ||
290 | "<ulink url='&YOCTO_DOCS_DEV_URL;#working-with-yocto-project-source-files'>Working With Yocto Project Source Files</ulink>" | ||
291 | section in the Yocto Project Development Tasks Manual. | ||
292 | </para></listitem> | ||
293 | </itemizedlist> | ||
294 | </note> | ||
295 | </para> | ||
296 | |||
297 | <section id='repositories-tags-and-branches'> | ||
298 | <title>Repositories, Tags, and Branches</title> | ||
299 | |||
300 | <para> | ||
301 | As mentioned briefly in the previous section and also in the | ||
302 | "<link linkend='workflows'>Workflows</link>" section, | ||
303 | the Yocto Project maintains source repositories at | ||
304 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>. | ||
305 | If you look at this web-interface of the repositories, each item | ||
306 | is a separate Git repository. | ||
307 | </para> | ||
308 | |||
309 | <para> | ||
310 | Git repositories use branching techniques that track content | ||
311 | change (not files) within a project (e.g. a new feature or updated | ||
312 | documentation). | ||
313 | Creating a tree-like structure based on project divergence allows | ||
314 | for excellent historical information over the life of a project. | ||
315 | This methodology also allows for an environment from which you can | ||
316 | do lots of local experimentation on projects as you develop | ||
317 | changes or new features. | ||
318 | </para> | ||
319 | |||
320 | <para> | ||
321 | A Git repository represents all development efforts for a given | ||
322 | project. | ||
323 | For example, the Git repository <filename>poky</filename> contains | ||
324 | all changes and developments for Poky over the course of its | ||
325 | entire life. | ||
326 | That means that all changes that make up all releases are captured. | ||
327 | The repository maintains a complete history of changes. | ||
328 | </para> | ||
329 | |||
330 | <para> | ||
331 | You can create a local copy of any repository by "cloning" it | ||
332 | with the <filename>git clone</filename> command. | ||
333 | When you clone a Git repository, you end up with an identical | ||
334 | copy of the repository on your development system. | ||
335 | Once you have a local copy of a repository, you can take steps to | ||
336 | develop locally. | ||
337 | For examples on how to clone Git repositories, see the | ||
338 | "<ulink url='&YOCTO_DOCS_DEV_URL;#working-with-yocto-project-source-files'>Working With Yocto Project Source Files</ulink>" | ||
339 | section in the Yocto Project Development Tasks Manual. | ||
340 | </para> | ||
341 | |||
342 | <para> | ||
343 | It is important to understand that Git tracks content change and | ||
344 | not files. | ||
345 | Git uses "branches" to organize different development efforts. | ||
346 | For example, the <filename>poky</filename> repository has | ||
347 | several branches that include the current "&DISTRO_NAME_NO_CAP;" | ||
348 | branch, the "master" branch, and many branches for past | ||
349 | Yocto Project releases. | ||
350 | You can see all the branches by going to | ||
351 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and | ||
352 | clicking on the | ||
353 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename> | ||
354 | link beneath the "Branch" heading. | ||
355 | </para> | ||
356 | |||
357 | <para> | ||
358 | Each of these branches represents a specific area of development. | ||
359 | The "master" branch represents the current or most recent | ||
360 | development. | ||
361 | All other branches represent offshoots of the "master" branch. | ||
362 | </para> | ||
363 | |||
364 | <para> | ||
365 | When you create a local copy of a Git repository, the copy has | ||
366 | the same set of branches as the original. | ||
367 | This means you can use Git to create a local working area | ||
368 | (also called a branch) that tracks a specific development branch | ||
369 | from the upstream source Git repository. | ||
370 | in other words, you can define your local Git environment to | ||
371 | work on any development branch in the repository. | ||
372 | To help illustrate, consider the following example Git commands: | ||
373 | <literallayout class='monospaced'> | ||
374 | $ cd ~ | ||
375 | $ git clone git://git.yoctoproject.org/poky | ||
376 | $ cd poky | ||
377 | $ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP; | ||
378 | </literallayout> | ||
379 | In the previous example after moving to the home directory, the | ||
380 | <filename>git clone</filename> command creates a | ||
381 | local copy of the upstream <filename>poky</filename> Git repository. | ||
382 | By default, Git checks out the "master" branch for your work. | ||
383 | After changing the working directory to the new local repository | ||
384 | (i.e. <filename>poky</filename>), the | ||
385 | <filename>git checkout</filename> command creates | ||
386 | and checks out a local branch named "&DISTRO_NAME_NO_CAP;", which | ||
387 | tracks the upstream "origin/&DISTRO_NAME_NO_CAP;" branch. | ||
388 | Changes you make while in this branch would ultimately affect | ||
389 | the upstream "&DISTRO_NAME_NO_CAP;" branch of the | ||
390 | <filename>poky</filename> repository. | ||
391 | </para> | ||
392 | |||
393 | <para> | ||
394 | It is important to understand that when you create and checkout a | ||
395 | local working branch based on a branch name, | ||
396 | your local environment matches the "tip" of that particular | ||
397 | development branch at the time you created your local branch, | ||
398 | which could be different from the files in the "master" branch | ||
399 | of the upstream repository. | ||
400 | In other words, creating and checking out a local branch based on | ||
401 | the "&DISTRO_NAME_NO_CAP;" branch name is not the same as | ||
402 | cloning and checking out the "master" branch if the repository. | ||
403 | Keep reading to see how you create a local snapshot of a Yocto | ||
404 | Project Release. | ||
405 | </para> | ||
406 | |||
407 | <para> | ||
408 | Git uses "tags" to mark specific changes in a repository. | ||
409 | Typically, a tag is used to mark a special point such as the final | ||
410 | change before a project is released. | ||
411 | You can see the tags used with the <filename>poky</filename> Git | ||
412 | repository by going to | ||
413 | <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and | ||
414 | clicking on the | ||
415 | <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename> | ||
416 | link beneath the "Tag" heading. | ||
417 | </para> | ||
418 | |||
419 | <para> | ||
420 | Some key tags for the <filename>poky</filename> are | ||
421 | <filename>jethro-14.0.3</filename>, | ||
422 | <filename>morty-16.0.1</filename>, | ||
423 | <filename>pyro-17.0.0</filename>, and | ||
424 | <filename>&DISTRO_NAME_NO_CAP;-&POKYVERSION;</filename>. | ||
425 | These tags represent Yocto Project releases. | ||
426 | </para> | ||
427 | |||
428 | <para> | ||
429 | When you create a local copy of the Git repository, you also | ||
430 | have access to all the tags in the upstream repository. | ||
431 | Similar to branches, you can create and checkout a local working | ||
432 | Git branch based on a tag name. | ||
433 | When you do this, you get a snapshot of the Git repository that | ||
434 | reflects the state of the files when the change was made associated | ||
435 | with that tag. | ||
436 | The most common use is to checkout a working branch that matches | ||
437 | a specific Yocto Project release. | ||
438 | Here is an example: | ||
439 | <literallayout class='monospaced'> | ||
440 | $ cd ~ | ||
441 | $ git clone git://git.yoctoproject.org/poky | ||
442 | $ cd poky | ||
443 | $ git fetch --all --tags --prune | ||
444 | $ git checkout tags/pyro-17.0.0 -b my-pyro-17.0.0 | ||
445 | </literallayout> | ||
446 | In this example, the name of the top-level directory of your | ||
447 | local Yocto Project repository is <filename>poky</filename>. | ||
448 | After moving to the <filename>poky</filename> directory, the | ||
449 | <filename>git fetch</filename> command makes all the upstream | ||
450 | tags available locally in your repository. | ||
451 | Finally, the <filename>git checkout</filename> command | ||
452 | creates and checks out a branch named "my-pyro-17.0.0" that is | ||
453 | based on the specific change upstream in the repository | ||
454 | associated with the "pyro-17.0.0" tag. | ||
455 | The files in your repository now exactly match that particular | ||
456 | Yocto Project release as it is tagged in the upstream Git | ||
457 | repository. | ||
458 | It is important to understand that when you create and | ||
459 | checkout a local working branch based on a tag, your environment | ||
460 | matches a specific point in time and not the entire development | ||
461 | branch (i.e. the "tip" of the branch). | ||
462 | </para> | ||
463 | </section> | ||
464 | |||
465 | <section id='basic-commands'> | ||
466 | <title>Basic Commands</title> | ||
467 | |||
468 | <para> | ||
469 | Git has an extensive set of commands that lets you manage changes | ||
470 | and perform collaboration over the life of a project. | ||
471 | Conveniently though, you can manage with a small set of basic | ||
472 | operations and workflows once you understand the basic | ||
473 | philosophy behind Git. | ||
474 | You do not have to be an expert in Git to be functional. | ||
475 | A good place to look for instruction on a minimal set of Git | ||
476 | commands is | ||
477 | <ulink url='http://git-scm.com/documentation'>here</ulink>. | ||
478 | </para> | ||
479 | |||
480 | <para> | ||
481 | If you do not know much about Git, you should educate | ||
482 | yourself by visiting the links previously mentioned. | ||
483 | </para> | ||
484 | |||
485 | <para> | ||
486 | The following list of Git commands briefly describes some basic | ||
487 | Git operations as a way to get started. | ||
488 | As with any set of commands, this list (in most cases) simply shows | ||
489 | the base command and omits the many arguments they support. | ||
490 | See the Git documentation for complete descriptions and strategies | ||
491 | on how to use these commands: | ||
492 | <itemizedlist> | ||
493 | <listitem><para> | ||
494 | <emphasis><filename>git init</filename>:</emphasis> | ||
495 | Initializes an empty Git repository. | ||
496 | You cannot use Git commands unless you have a | ||
497 | <filename>.git</filename> repository. | ||
498 | </para></listitem> | ||
499 | <listitem><para id='git-commands-clone'> | ||
500 | <emphasis><filename>git clone</filename>:</emphasis> | ||
501 | Creates a local clone of a Git repository that is on | ||
502 | equal footing with a fellow developer’s Git repository | ||
503 | or an upstream repository. | ||
504 | </para></listitem> | ||
505 | <listitem><para> | ||
506 | <emphasis><filename>git add</filename>:</emphasis> | ||
507 | Locally stages updated file contents to the index that | ||
508 | Git uses to track changes. | ||
509 | You must stage all files that have changed before you | ||
510 | can commit them. | ||
511 | </para></listitem> | ||
512 | <listitem><para> | ||
513 | <emphasis><filename>git commit</filename>:</emphasis> | ||
514 | Creates a local "commit" that documents the changes you | ||
515 | made. | ||
516 | Only changes that have been staged can be committed. | ||
517 | Commits are used for historical purposes, for determining | ||
518 | if a maintainer of a project will allow the change, | ||
519 | and for ultimately pushing the change from your local | ||
520 | Git repository into the project’s upstream repository. | ||
521 | </para></listitem> | ||
522 | <listitem><para> | ||
523 | <emphasis><filename>git status</filename>:</emphasis> | ||
524 | Reports any modified files that possibly need to be | ||
525 | staged and gives you a status of where you stand regarding | ||
526 | local commits as compared to the upstream repository. | ||
527 | </para></listitem> | ||
528 | <listitem><para> | ||
529 | <emphasis><filename>git checkout</filename> <replaceable>branch-name</replaceable>:</emphasis> | ||
530 | Changes your working branch. | ||
531 | This command is analogous to "cd". | ||
532 | </para></listitem> | ||
533 | <listitem><para><emphasis><filename>git checkout –b</filename> <replaceable>working-branch</replaceable>:</emphasis> | ||
534 | Creates and checks out a working branch on your local | ||
535 | machine that you can use to isolate your work. | ||
536 | It is a good idea to use local branches when adding | ||
537 | specific features or changes. | ||
538 | Using isolated branches facilitates easy removal of | ||
539 | changes if they do not work out. | ||
540 | </para></listitem> | ||
541 | <listitem><para><emphasis><filename>git branch</filename>:</emphasis> | ||
542 | Displays the existing local branches associated with your | ||
543 | local repository. | ||
544 | The branch that you have currently checked out is noted | ||
545 | with an asterisk character. | ||
546 | </para></listitem> | ||
547 | <listitem><para> | ||
548 | <emphasis><filename>git branch -D</filename> <replaceable>branch-name</replaceable>:</emphasis> | ||
549 | Deletes an existing local branch. | ||
550 | You need to be in a local branch other than the one you | ||
551 | are deleting in order to delete | ||
552 | <replaceable>branch-name</replaceable>. | ||
553 | </para></listitem> | ||
554 | <listitem><para> | ||
555 | <emphasis><filename>git pull</filename>:</emphasis> | ||
556 | Retrieves information from an upstream Git repository | ||
557 | and places it in your local Git repository. | ||
558 | You use this command to make sure you are synchronized with | ||
559 | the repository from which you are basing changes | ||
560 | (.e.g. the "master" branch). | ||
561 | </para></listitem> | ||
562 | <listitem><para> | ||
563 | <emphasis><filename>git push</filename>:</emphasis> | ||
564 | Sends all your committed local changes to the upstream Git | ||
565 | repository that your local repository is tracking | ||
566 | (e.g. a contribution repository). | ||
567 | The maintainer of the project draws from these repositories | ||
568 | to merge changes (commits) into the appropriate branch | ||
569 | of project's upstream repository. | ||
570 | </para></listitem> | ||
571 | <listitem><para> | ||
572 | <emphasis><filename>git merge</filename>:</emphasis> | ||
573 | Combines or adds changes from one | ||
574 | local branch of your repository with another branch. | ||
575 | When you create a local Git repository, the default branch | ||
576 | is named "master". | ||
577 | A typical workflow is to create a temporary branch that is | ||
578 | based off "master" that you would use for isolated work. | ||
579 | You would make your changes in that isolated branch, | ||
580 | stage and commit them locally, switch to the "master" | ||
581 | branch, and then use the <filename>git merge</filename> | ||
582 | command to apply the changes from your isolated branch | ||
583 | into the currently checked out branch (e.g. "master"). | ||
584 | After the merge is complete and if you are done with | ||
585 | working in that isolated branch, you can safely delete | ||
586 | the isolated branch. | ||
587 | </para></listitem> | ||
588 | <listitem><para> | ||
589 | <emphasis><filename>git cherry-pick</filename>:</emphasis> | ||
590 | Choose and apply specific commits from one branch | ||
591 | into another branch. | ||
592 | There are times when you might not be able to merge | ||
593 | all the changes in one branch with | ||
594 | another but need to pick out certain ones. | ||
595 | </para></listitem> | ||
596 | <listitem><para> | ||
597 | <emphasis><filename>gitk</filename>:</emphasis> | ||
598 | Provides a GUI view of the branches and changes in your | ||
599 | local Git repository. | ||
600 | This command is a good way to graphically see where things | ||
601 | have diverged in your local repository. | ||
602 | <note> | ||
603 | You need to install the <filename>gitk</filename> | ||
604 | package on your development system to use this | ||
605 | command. | ||
606 | </note> | ||
607 | </para></listitem> | ||
608 | <listitem><para> | ||
609 | <emphasis><filename>git log</filename>:</emphasis> | ||
610 | Reports a history of your commits to the repository. | ||
611 | This report lists all commits regardless of whether you | ||
612 | have pushed them upstream or not. | ||
613 | </para></listitem> | ||
614 | <listitem><para> | ||
615 | <emphasis><filename>git diff</filename>:</emphasis> | ||
616 | Displays line-by-line differences between a local | ||
617 | working file and the same file as understood by Git. | ||
618 | This command is useful to see what you have changed | ||
619 | in any given file. | ||
620 | </para></listitem> | ||
621 | </itemizedlist> | ||
622 | </para> | ||
623 | </section> | ||
624 | </section> | ||
625 | |||
626 | <section id='yocto-project-repositories'> | ||
627 | <title>Yocto Project Source Repositories</title> | ||
628 | |||
629 | <para> | ||
630 | The Yocto Project team maintains complete source repositories for all | ||
631 | Yocto Project files at | ||
632 | <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>. | ||
633 | This web-based source code browser is organized into categories by | ||
634 | function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and | ||
635 | so forth. | ||
636 | From the interface, you can click on any particular item in the "Name" | ||
637 | column and see the URL at the bottom of the page that you need to clone | ||
638 | a Git repository for that particular item. | ||
639 | Having a local Git repository of the | ||
640 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>, | ||
641 | which is usually named "poky", allows | ||
642 | you to make changes, contribute to the history, and ultimately enhance | ||
643 | the Yocto Project's tools, Board Support Packages, and so forth. | ||
644 | </para> | ||
645 | |||
646 | <para> | ||
647 | For any supported release of Yocto Project, you can also go to the | ||
648 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and | ||
649 | select the "Downloads" tab and get a released tarball of the | ||
650 | <filename>poky</filename> repository or any supported BSP tarballs. | ||
651 | Unpacking these tarballs gives you a snapshot of the released | ||
652 | files. | ||
653 | <note><title>Notes</title> | ||
654 | <itemizedlist> | ||
655 | <listitem><para> | ||
656 | The recommended method for setting up the Yocto Project | ||
657 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> | ||
658 | and the files for supported BSPs | ||
659 | (e.g., <filename>meta-intel</filename>) is to use | ||
660 | <link linkend='git'>Git</link> to create a local copy of | ||
661 | the upstream repositories. | ||
662 | </para></listitem> | ||
663 | <listitem><para> | ||
664 | Be sure to always work in matching branches for both | ||
665 | the selected BSP repository and the | ||
666 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> | ||
667 | (i.e. <filename>poky</filename>) repository. | ||
668 | For example, if you have checked out the "master" branch | ||
669 | of <filename>poky</filename> and you are going to use | ||
670 | <filename>meta-intel</filename>, be sure to checkout the | ||
671 | "master" branch of <filename>meta-intel</filename>. | ||
672 | </para></listitem> | ||
673 | </itemizedlist> | ||
674 | </note> | ||
675 | </para> | ||
676 | |||
677 | <para> | ||
678 | In summary, here is where you can get the project files needed for | ||
679 | development: | ||
680 | <itemizedlist> | ||
681 | <listitem><para id='source-repositories'> | ||
682 | <emphasis> | ||
683 | <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink> | ||
684 | </emphasis> | ||
685 | This area contains IDE Plugins, Matchbox, Poky, Poky Support, | ||
686 | Tools, Yocto Linux Kernel, and Yocto Metadata Layers. | ||
687 | You can create local copies of Git repositories for each of | ||
688 | these areas.</para> | ||
689 | |||
690 | <para> | ||
691 | <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" /> | ||
692 | For steps on how to view and access these upstream Git | ||
693 | repositories, see the | ||
694 | "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-source-repositories'>Accessing Source Repositories</ulink>" | ||
695 | Section in the Yocto Project Development Tasks Manual. | ||
696 | </para></listitem> | ||
697 | <listitem><para><anchor id='index-downloads' /> | ||
698 | <emphasis> | ||
699 | <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> | ||
700 | </emphasis> | ||
701 | This is an index of releases such as | ||
702 | the <trademark class='trade'>Eclipse</trademark> | ||
703 | Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers | ||
704 | for cross-development toolchains, and all released versions of | ||
705 | Yocto Project in the form of images or tarballs. | ||
706 | Downloading and extracting these files does not produce a local | ||
707 | copy of the Git repository but rather a snapshot of a | ||
708 | particular release or image.</para> | ||
709 | |||
710 | <para> | ||
711 | <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" /> | ||
712 | For steps on how to view and access these files, see the | ||
713 | "<ulink url='&YOCTO_DOCS_DEV_URL;#accessing-index-of-releases'>Accessing Index of Releases</ulink>" | ||
714 | section in the Yocto Project Development Tasks Manual. | ||
715 | </para></listitem> | ||
716 | <listitem><para id='downloads-page'> | ||
717 | <emphasis>"Downloads" page for the | ||
718 | <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>: | ||
719 | </emphasis></para> | ||
720 | |||
721 | <para role="writernotes">This section will change due to | ||
722 | reworking of the YP Website.</para> | ||
723 | |||
724 | <para>The Yocto Project website includes a "Downloads" tab | ||
725 | that allows you to download any Yocto Project | ||
726 | release and Board Support Package (BSP) in tarball form. | ||
727 | The tarballs are similar to those found in the | ||
728 | <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para> | ||
729 | |||
730 | <para> | ||
731 | <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" /> | ||
732 | For steps on how to use the "Downloads" page, see the | ||
733 | "<ulink url='&YOCTO_DOCS_DEV_URL;#using-the-downloads-page'>Using the Downloads Page</ulink>" | ||
734 | section in the Yocto Project Development Tasks Manual. | ||
735 | </para></listitem> | ||
736 | </itemizedlist> | ||
737 | </para> | ||
738 | </section> | ||
739 | |||
740 | <section id='licensing'> | ||
741 | <title>Licensing</title> | ||
742 | |||
743 | <para> | ||
744 | Because open source projects are open to the public, they have | ||
745 | different licensing structures in place. | ||
746 | License evolution for both Open Source and Free Software has an | ||
747 | interesting history. | ||
748 | If you are interested in this history, you can find basic information | ||
749 | here: | ||
750 | <itemizedlist> | ||
751 | <listitem><para> | ||
752 | <ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink> | ||
753 | </para></listitem> | ||
754 | <listitem><para> | ||
755 | <ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license history</ulink> | ||
756 | </para></listitem> | ||
757 | </itemizedlist> | ||
758 | </para> | ||
759 | |||
760 | <para> | ||
761 | In general, the Yocto Project is broadly licensed under the | ||
762 | Massachusetts Institute of Technology (MIT) License. | ||
763 | MIT licensing permits the reuse of software within proprietary | ||
764 | software as long as the license is distributed with that software. | ||
765 | MIT is also compatible with the GNU General Public License (GPL). | ||
766 | Patches to the Yocto Project follow the upstream licensing scheme. | ||
767 | You can find information on the MIT license | ||
768 | <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>. | ||
769 | You can find information on the GNU GPL | ||
770 | <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>here</ulink>. | ||
771 | </para> | ||
772 | |||
773 | <para> | ||
774 | When you build an image using the Yocto Project, the build process | ||
775 | uses a known list of licenses to ensure compliance. | ||
776 | You can find this list in the | ||
777 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> | ||
778 | at <filename>meta/files/common-licenses</filename>. | ||
779 | Once the build completes, the list of all licenses found and used | ||
780 | during that build are kept in the | ||
781 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> | ||
782 | at <filename>tmp/deploy/licenses</filename>. | ||
783 | </para> | ||
784 | |||
785 | <para> | ||
786 | If a module requires a license that is not in the base list, the | ||
787 | build process generates a warning during the build. | ||
788 | These tools make it easier for a developer to be certain of the | ||
789 | licenses with which their shipped products must comply. | ||
790 | However, even with these tools it is still up to the developer to | ||
791 | resolve potential licensing issues. | ||
792 | </para> | ||
793 | |||
794 | <para> | ||
795 | The base list of licenses used by the build process is a combination | ||
796 | of the Software Package Data Exchange (SPDX) list and the Open | ||
797 | Source Initiative (OSI) projects. | ||
798 | <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of | ||
799 | the Linux Foundation that maintains a specification for a standard | ||
800 | format for communicating the components, licenses, and copyrights | ||
801 | associated with a software package. | ||
802 | <ulink url='http://opensource.org'>OSI</ulink> is a corporation | ||
803 | dedicated to the Open Source Definition and the effort for reviewing | ||
804 | and approving licenses that conform to the Open Source Definition | ||
805 | (OSD). | ||
806 | </para> | ||
807 | |||
808 | <para> | ||
809 | You can find a list of the combined SPDX and OSI licenses that the | ||
810 | Yocto Project uses in the | ||
811 | <filename>meta/files/common-licenses</filename> directory in your | ||
812 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. | ||
813 | </para> | ||
814 | |||
815 | <para> | ||
816 | For information that can help you maintain compliance with various | ||
817 | open source licensing during the lifecycle of a product created using | ||
818 | the Yocto Project, see the | ||
819 | "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</ulink>" | ||
820 | section in the Yocto Project Development Tasks Manual. | ||
821 | </para> | ||
822 | </section> | ||
823 | |||
824 | <section id='recipe-syntax'> | ||
825 | <title>Recipe Syntax</title> | ||
826 | |||
827 | <para> | ||
828 | Understanding recipe file syntax is important for | ||
829 | writing recipes. | ||
830 | The following list overviews the basic items that make up a | ||
831 | BitBake recipe file. | ||
832 | For more complete BitBake syntax descriptions, see the | ||
833 | "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>" | ||
834 | chapter of the BitBake User Manual. | ||
835 | <itemizedlist> | ||
836 | <listitem><para><emphasis>Variable Assignments and Manipulations:</emphasis> | ||
837 | Variable assignments allow a value to be assigned to a | ||
838 | variable. | ||
839 | The assignment can be static text or might include | ||
840 | the contents of other variables. | ||
841 | In addition to the assignment, appending and prepending | ||
842 | operations are also supported.</para> | ||
843 | <para>The following example shows some of the ways | ||
844 | you can use variables in recipes: | ||
845 | <literallayout class='monospaced'> | ||
846 | S = "${WORKDIR}/postfix-${PV}" | ||
847 | CFLAGS += "-DNO_ASM" | ||
848 | SRC_URI_append = " file://fixup.patch" | ||
849 | </literallayout> | ||
850 | </para></listitem> | ||
851 | <listitem><para><emphasis>Functions:</emphasis> | ||
852 | Functions provide a series of actions to be performed. | ||
853 | You usually use functions to override the default | ||
854 | implementation of a task function or to complement | ||
855 | a default function (i.e. append or prepend to an | ||
856 | existing function). | ||
857 | Standard functions use <filename>sh</filename> shell | ||
858 | syntax, although access to OpenEmbedded variables and | ||
859 | internal methods are also available.</para> | ||
860 | <para>The following is an example function from the | ||
861 | <filename>sed</filename> recipe: | ||
862 | <literallayout class='monospaced'> | ||
863 | do_install () { | ||
864 | autotools_do_install | ||
865 | install -d ${D}${base_bindir} | ||
866 | mv ${D}${bindir}/sed ${D}${base_bindir}/sed | ||
867 | rmdir ${D}${bindir}/ | ||
868 | } | ||
869 | </literallayout> | ||
870 | It is also possible to implement new functions that | ||
871 | are called between existing tasks as long as the | ||
872 | new functions are not replacing or complementing the | ||
873 | default functions. | ||
874 | You can implement functions in Python | ||
875 | instead of shell. | ||
876 | Both of these options are not seen in the majority of | ||
877 | recipes.</para></listitem> | ||
878 | <listitem><para><emphasis>Keywords:</emphasis> | ||
879 | BitBake recipes use only a few keywords. | ||
880 | You use keywords to include common | ||
881 | functions (<filename>inherit</filename>), load parts | ||
882 | of a recipe from other files | ||
883 | (<filename>include</filename> and | ||
884 | <filename>require</filename>) and export variables | ||
885 | to the environment (<filename>export</filename>).</para> | ||
886 | <para>The following example shows the use of some of | ||
887 | these keywords: | ||
888 | <literallayout class='monospaced'> | ||
889 | export POSTCONF = "${STAGING_BINDIR}/postconf" | ||
890 | inherit autoconf | ||
891 | require otherfile.inc | ||
892 | </literallayout> | ||
893 | </para></listitem> | ||
894 | <listitem><para><emphasis>Comments:</emphasis> | ||
895 | Any lines that begin with the hash character | ||
896 | (<filename>#</filename>) are treated as comment lines | ||
897 | and are ignored: | ||
898 | <literallayout class='monospaced'> | ||
899 | # This is a comment | ||
900 | </literallayout> | ||
901 | </para></listitem> | ||
902 | </itemizedlist> | ||
903 | </para> | ||
904 | |||
905 | <para> | ||
906 | This next list summarizes the most important and most commonly | ||
907 | used parts of the recipe syntax. | ||
908 | For more information on these parts of the syntax, you can | ||
909 | reference the | ||
910 | <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink> | ||
911 | chapter in the BitBake User Manual. | ||
912 | <itemizedlist> | ||
913 | <listitem><para><emphasis>Line Continuation: <filename>\</filename></emphasis> - | ||
914 | Use the backward slash (<filename>\</filename>) | ||
915 | character to split a statement over multiple lines. | ||
916 | Place the slash character at the end of the line that | ||
917 | is to be continued on the next line: | ||
918 | <literallayout class='monospaced'> | ||
919 | VAR = "A really long \ | ||
920 | line" | ||
921 | </literallayout> | ||
922 | <note> | ||
923 | You cannot have any characters including spaces | ||
924 | or tabs after the slash character. | ||
925 | </note> | ||
926 | </para></listitem> | ||
927 | <listitem><para> | ||
928 | <emphasis>Using Variables: <filename>${...}</filename></emphasis> - | ||
929 | Use the <filename>${<replaceable>VARNAME</replaceable>}</filename> syntax to | ||
930 | access the contents of a variable: | ||
931 | <literallayout class='monospaced'> | ||
932 | SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz" | ||
933 | </literallayout> | ||
934 | <note> | ||
935 | It is important to understand that the value of a | ||
936 | variable expressed in this form does not get | ||
937 | substituted automatically. | ||
938 | The expansion of these expressions happens | ||
939 | on-demand later (e.g. usually when a function that | ||
940 | makes reference to the variable executes). | ||
941 | This behavior ensures that the values are most | ||
942 | appropriate for the context in which they are | ||
943 | finally used. | ||
944 | On the rare occasion that you do need the variable | ||
945 | expression to be expanded immediately, you can use | ||
946 | the <filename>:=</filename> operator instead of | ||
947 | <filename>=</filename> when you make the | ||
948 | assignment, but this is not generally needed. | ||
949 | </note> | ||
950 | </para></listitem> | ||
951 | <listitem><para><emphasis>Quote All Assignments: <filename>"<replaceable>value</replaceable>"</filename></emphasis> - | ||
952 | Use double quotes around the value in all variable | ||
953 | assignments. | ||
954 | <literallayout class='monospaced'> | ||
955 | VAR1 = "${OTHERVAR}" | ||
956 | VAR2 = "The version is ${PV}" | ||
957 | </literallayout> | ||
958 | </para></listitem> | ||
959 | <listitem><para><emphasis>Conditional Assignment: <filename>?=</filename></emphasis> - | ||
960 | Conditional assignment is used to assign a value to | ||
961 | a variable, but only when the variable is currently | ||
962 | unset. | ||
963 | Use the question mark followed by the equal sign | ||
964 | (<filename>?=</filename>) to make a "soft" assignment | ||
965 | used for conditional assignment. | ||
966 | Typically, "soft" assignments are used in the | ||
967 | <filename>local.conf</filename> file for variables | ||
968 | that are allowed to come through from the external | ||
969 | environment. | ||
970 | </para> | ||
971 | <para>Here is an example where | ||
972 | <filename>VAR1</filename> is set to "New value" if | ||
973 | it is currently empty. | ||
974 | However, if <filename>VAR1</filename> has already been | ||
975 | set, it remains unchanged: | ||
976 | <literallayout class='monospaced'> | ||
977 | VAR1 ?= "New value" | ||
978 | </literallayout> | ||
979 | In this next example, <filename>VAR1</filename> | ||
980 | is left with the value "Original value": | ||
981 | <literallayout class='monospaced'> | ||
982 | VAR1 = "Original value" | ||
983 | VAR1 ?= "New value" | ||
984 | </literallayout> | ||
985 | </para></listitem> | ||
986 | <listitem><para><emphasis>Appending: <filename>+=</filename></emphasis> - | ||
987 | Use the plus character followed by the equals sign | ||
988 | (<filename>+=</filename>) to append values to existing | ||
989 | variables. | ||
990 | <note> | ||
991 | This operator adds a space between the existing | ||
992 | content of the variable and the new content. | ||
993 | </note></para> | ||
994 | <para>Here is an example: | ||
995 | <literallayout class='monospaced'> | ||
996 | SRC_URI += "file://fix-makefile.patch" | ||
997 | </literallayout> | ||
998 | </para></listitem> | ||
999 | <listitem><para><emphasis>Prepending: <filename>=+</filename></emphasis> - | ||
1000 | Use the equals sign followed by the plus character | ||
1001 | (<filename>=+</filename>) to prepend values to existing | ||
1002 | variables. | ||
1003 | <note> | ||
1004 | This operator adds a space between the new content | ||
1005 | and the existing content of the variable. | ||
1006 | </note></para> | ||
1007 | <para>Here is an example: | ||
1008 | <literallayout class='monospaced'> | ||
1009 | VAR =+ "Starts" | ||
1010 | </literallayout> | ||
1011 | </para></listitem> | ||
1012 | <listitem><para><emphasis>Appending: <filename>_append</filename></emphasis> - | ||
1013 | Use the <filename>_append</filename> operator to | ||
1014 | append values to existing variables. | ||
1015 | This operator does not add any additional space. | ||
1016 | Also, the operator is applied after all the | ||
1017 | <filename>+=</filename>, and | ||
1018 | <filename>=+</filename> operators have been applied and | ||
1019 | after all <filename>=</filename> assignments have | ||
1020 | occurred. | ||
1021 | </para> | ||
1022 | <para>The following example shows the space being | ||
1023 | explicitly added to the start to ensure the appended | ||
1024 | value is not merged with the existing value: | ||
1025 | <literallayout class='monospaced'> | ||
1026 | SRC_URI_append = " file://fix-makefile.patch" | ||
1027 | </literallayout> | ||
1028 | You can also use the <filename>_append</filename> | ||
1029 | operator with overrides, which results in the actions | ||
1030 | only being performed for the specified target or | ||
1031 | machine: | ||
1032 | <literallayout class='monospaced'> | ||
1033 | SRC_URI_append_sh4 = " file://fix-makefile.patch" | ||
1034 | </literallayout> | ||
1035 | </para></listitem> | ||
1036 | <listitem><para><emphasis>Prepending: <filename>_prepend</filename></emphasis> - | ||
1037 | Use the <filename>_prepend</filename> operator to | ||
1038 | prepend values to existing variables. | ||
1039 | This operator does not add any additional space. | ||
1040 | Also, the operator is applied after all the | ||
1041 | <filename>+=</filename>, and | ||
1042 | <filename>=+</filename> operators have been applied and | ||
1043 | after all <filename>=</filename> assignments have | ||
1044 | occurred. | ||
1045 | </para> | ||
1046 | <para>The following example shows the space being | ||
1047 | explicitly added to the end to ensure the prepended | ||
1048 | value is not merged with the existing value: | ||
1049 | <literallayout class='monospaced'> | ||
1050 | CFLAGS_prepend = "-I${S}/myincludes " | ||
1051 | </literallayout> | ||
1052 | You can also use the <filename>_prepend</filename> | ||
1053 | operator with overrides, which results in the actions | ||
1054 | only being performed for the specified target or | ||
1055 | machine: | ||
1056 | <literallayout class='monospaced'> | ||
1057 | CFLAGS_prepend_sh4 = "-I${S}/myincludes " | ||
1058 | </literallayout> | ||
1059 | </para></listitem> | ||
1060 | <listitem><para><emphasis>Overrides:</emphasis> - | ||
1061 | You can use overrides to set a value conditionally, | ||
1062 | typically based on how the recipe is being built. | ||
1063 | For example, to set the | ||
1064 | <ulink url='&YOCTO_DOCS_REF_URL;#var-KBRANCH'><filename>KBRANCH</filename></ulink> | ||
1065 | variable's value to "standard/base" for any target | ||
1066 | <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>, | ||
1067 | except for qemuarm where it should be set to | ||
1068 | "standard/arm-versatile-926ejs", you would do the | ||
1069 | following: | ||
1070 | <literallayout class='monospaced'> | ||
1071 | KBRANCH = "standard/base" | ||
1072 | KBRANCH_qemuarm = "standard/arm-versatile-926ejs" | ||
1073 | </literallayout> | ||
1074 | Overrides are also used to separate alternate values | ||
1075 | of a variable in other situations. | ||
1076 | For example, when setting variables such as | ||
1077 | <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink> | ||
1078 | and | ||
1079 | <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> | ||
1080 | that are specific to individual packages produced by | ||
1081 | a recipe, you should always use an override that | ||
1082 | specifies the name of the package. | ||
1083 | </para></listitem> | ||
1084 | <listitem><para><emphasis>Indentation:</emphasis> | ||
1085 | Use spaces for indentation rather than than tabs. | ||
1086 | For shell functions, both currently work. | ||
1087 | However, it is a policy decision of the Yocto Project | ||
1088 | to use tabs in shell functions. | ||
1089 | Realize that some layers have a policy to use spaces | ||
1090 | for all indentation. | ||
1091 | </para></listitem> | ||
1092 | <listitem><para><emphasis>Using Python for Complex Operations: <filename>${@<replaceable>python_code</replaceable>}</filename></emphasis> - | ||
1093 | For more advanced processing, it is possible to use | ||
1094 | Python code during variable assignments (e.g. | ||
1095 | search and replacement on a variable).</para> | ||
1096 | <para>You indicate Python code using the | ||
1097 | <filename>${@<replaceable>python_code</replaceable>}</filename> | ||
1098 | syntax for the variable assignment: | ||
1099 | <literallayout class='monospaced'> | ||
1100 | SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz | ||
1101 | </literallayout> | ||
1102 | </para></listitem> | ||
1103 | <listitem><para><emphasis>Shell Function Syntax:</emphasis> | ||
1104 | Write shell functions as if you were writing a shell | ||
1105 | script when you describe a list of actions to take. | ||
1106 | You should ensure that your script works with a generic | ||
1107 | <filename>sh</filename> and that it does not require | ||
1108 | any <filename>bash</filename> or other shell-specific | ||
1109 | functionality. | ||
1110 | The same considerations apply to various system | ||
1111 | utilities (e.g. <filename>sed</filename>, | ||
1112 | <filename>grep</filename>, <filename>awk</filename>, | ||
1113 | and so forth) that you might wish to use. | ||
1114 | If in doubt, you should check with multiple | ||
1115 | implementations - including those from BusyBox. | ||
1116 | </para></listitem> | ||
1117 | </itemizedlist> | ||
1118 | </para> | ||
1119 | </section> | ||
1120 | |||
1121 | <section id="development-concepts"> | ||
1122 | <title>Development Concepts</title> | ||
1123 | |||
1124 | <para> | ||
1125 | This section takes a more detailed look inside the development | ||
1126 | process. | ||
1127 | The following diagram represents development at a high level. | ||
1128 | The remainder of this chapter expands on the fundamental input, output, | ||
1129 | process, and | ||
1130 | <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>) blocks | ||
1131 | that make up development in the Yocto Project environment. | ||
1132 | </para> | ||
1133 | |||
1134 | <para id='general-yocto-environment-figure'> | ||
1135 | <imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" /> | ||
1136 | </para> | ||
1137 | |||
1138 | <para> | ||
1139 | In general, development consists of several functional areas: | ||
1140 | <itemizedlist> | ||
1141 | <listitem><para><emphasis>User Configuration:</emphasis> | ||
1142 | Metadata you can use to control the build process. | ||
1143 | </para></listitem> | ||
1144 | <listitem><para><emphasis>Metadata Layers:</emphasis> | ||
1145 | Various layers that provide software, machine, and | ||
1146 | distro Metadata.</para></listitem> | ||
1147 | <listitem><para><emphasis>Source Files:</emphasis> | ||
1148 | Upstream releases, local projects, and SCMs.</para></listitem> | ||
1149 | <listitem><para><emphasis>Build System:</emphasis> | ||
1150 | Processes under the control of | ||
1151 | <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink>. | ||
1152 | This block expands on how BitBake fetches source, applies | ||
1153 | patches, completes compilation, analyzes output for package | ||
1154 | generation, creates and tests packages, generates images, and | ||
1155 | generates cross-development tools.</para></listitem> | ||
1156 | <listitem><para><emphasis>Package Feeds:</emphasis> | ||
1157 | Directories containing output packages (RPM, DEB or IPK), | ||
1158 | which are subsequently used in the construction of an image or | ||
1159 | SDK, produced by the build system. | ||
1160 | These feeds can also be copied and shared using a web server or | ||
1161 | other means to facilitate extending or updating existing | ||
1162 | images on devices at runtime if runtime package management is | ||
1163 | enabled.</para></listitem> | ||
1164 | <listitem><para><emphasis>Images:</emphasis> | ||
1165 | Images produced by the development process. | ||
1166 | </para></listitem> | ||
1167 | <listitem><para><emphasis>Application Development SDK:</emphasis> | ||
1168 | Cross-development tools that are produced along with an image | ||
1169 | or separately with BitBake.</para></listitem> | ||
1170 | </itemizedlist> | ||
1171 | </para> | ||
1172 | |||
1173 | <section id="user-configuration"> | ||
1174 | <title>User Configuration</title> | ||
1175 | |||
1176 | <para> | ||
1177 | User configuration helps define the build. | ||
1178 | Through user configuration, you can tell BitBake the | ||
1179 | target architecture for which you are building the image, | ||
1180 | where to store downloaded source, and other build properties. | ||
1181 | </para> | ||
1182 | |||
1183 | <para> | ||
1184 | The following figure shows an expanded representation of the | ||
1185 | "User Configuration" box of the | ||
1186 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: | ||
1187 | </para> | ||
1188 | |||
1189 | <para> | ||
1190 | <imagedata fileref="figures/user-configuration.png" align="center" width="8in" depth="4.5in" /> | ||
1191 | </para> | ||
1192 | |||
1193 | <para> | ||
1194 | BitBake needs some basic configuration files in order to complete | ||
1195 | a build. | ||
1196 | These files are <filename>*.conf</filename> files. | ||
1197 | The minimally necessary ones reside as example files in the | ||
1198 | <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. | ||
1199 | For simplicity, this section refers to the Source Directory as | ||
1200 | the "Poky Directory." | ||
1201 | </para> | ||
1202 | |||
1203 | <para> | ||
1204 | When you clone the <filename>poky</filename> Git repository or you | ||
1205 | download and unpack a Yocto Project release, you can set up the | ||
1206 | Source Directory to be named anything you want. | ||
1207 | For this discussion, the cloned repository uses the default | ||
1208 | name <filename>poky</filename>. | ||
1209 | <note> | ||
1210 | The Poky repository is primarily an aggregation of existing | ||
1211 | repositories. | ||
1212 | It is not a canonical upstream source. | ||
1213 | </note> | ||
1214 | </para> | ||
1215 | |||
1216 | <para> | ||
1217 | The <filename>meta-poky</filename> layer inside Poky contains | ||
1218 | a <filename>conf</filename> directory that has example | ||
1219 | configuration files. | ||
1220 | These example files are used as a basis for creating actual | ||
1221 | configuration files when you source the build environment | ||
1222 | script | ||
1223 | (i.e. | ||
1224 | <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>). | ||
1225 | </para> | ||
1226 | |||
1227 | <para> | ||
1228 | Sourcing the build environment script creates a | ||
1229 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> | ||
1230 | if one does not already exist. | ||
1231 | BitBake uses the Build Directory for all its work during builds. | ||
1232 | The Build Directory has a <filename>conf</filename> directory that | ||
1233 | contains default versions of your <filename>local.conf</filename> | ||
1234 | and <filename>bblayers.conf</filename> configuration files. | ||
1235 | These default configuration files are created only if versions | ||
1236 | do not already exist in the Build Directory at the time you | ||
1237 | source the build environment setup script. | ||
1238 | </para> | ||
1239 | |||
1240 | <para> | ||
1241 | Because the Poky repository is fundamentally an aggregation of | ||
1242 | existing repositories, some users might be familiar with running | ||
1243 | the <filename>&OE_INIT_FILE;</filename> script in the context | ||
1244 | of separate OpenEmbedded-Core and BitBake repositories rather than a | ||
1245 | single Poky repository. | ||
1246 | This discussion assumes the script is executed from within a cloned | ||
1247 | or unpacked version of Poky. | ||
1248 | </para> | ||
1249 | |||
1250 | <para> | ||
1251 | Depending on where the script is sourced, different sub-scripts | ||
1252 | are called to set up the Build Directory (Yocto or OpenEmbedded). | ||
1253 | Specifically, the script | ||
1254 | <filename>scripts/oe-setup-builddir</filename> inside the | ||
1255 | poky directory sets up the Build Directory and seeds the directory | ||
1256 | (if necessary) with configuration files appropriate for the | ||
1257 | Yocto Project development environment. | ||
1258 | <note> | ||
1259 | The <filename>scripts/oe-setup-builddir</filename> script | ||
1260 | uses the <filename>$TEMPLATECONF</filename> variable to | ||
1261 | determine which sample configuration files to locate. | ||
1262 | </note> | ||
1263 | </para> | ||
1264 | |||
1265 | <para> | ||
1266 | The <filename>local.conf</filename> file provides many | ||
1267 | basic variables that define a build environment. | ||
1268 | Here is a list of a few. | ||
1269 | To see the default configurations in a <filename>local.conf</filename> | ||
1270 | file created by the build environment script, see the | ||
1271 | <filename>local.conf.sample</filename> in the | ||
1272 | <filename>meta-poky</filename> layer: | ||
1273 | <itemizedlist> | ||
1274 | <listitem><para><emphasis>Parallelism Options:</emphasis> | ||
1275 | Controlled by the | ||
1276 | <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></ulink>, | ||
1277 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink>, | ||
1278 | and | ||
1279 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink> | ||
1280 | variables.</para></listitem> | ||
1281 | <listitem><para><emphasis>Target Machine Selection:</emphasis> | ||
1282 | Controlled by the | ||
1283 | <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> | ||
1284 | variable.</para></listitem> | ||
1285 | <listitem><para><emphasis>Download Directory:</emphasis> | ||
1286 | Controlled by the | ||
1287 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> | ||
1288 | variable.</para></listitem> | ||
1289 | <listitem><para><emphasis>Shared State Directory:</emphasis> | ||
1290 | Controlled by the | ||
1291 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink> | ||
1292 | variable.</para></listitem> | ||
1293 | <listitem><para><emphasis>Build Output:</emphasis> | ||
1294 | Controlled by the | ||
1295 | <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> | ||
1296 | variable.</para></listitem> | ||
1297 | </itemizedlist> | ||
1298 | <note> | ||
1299 | Configurations set in the <filename>conf/local.conf</filename> | ||
1300 | file can also be set in the | ||
1301 | <filename>conf/site.conf</filename> and | ||
1302 | <filename>conf/auto.conf</filename> configuration files. | ||
1303 | </note> | ||
1304 | </para> | ||
1305 | |||
1306 | <para> | ||
1307 | The <filename>bblayers.conf</filename> file tells BitBake what | ||
1308 | layers you want considered during the build. | ||
1309 | By default, the layers listed in this file include layers | ||
1310 | minimally needed by the build system. | ||
1311 | However, you must manually add any custom layers you have created. | ||
1312 | You can find more information on working with the | ||
1313 | <filename>bblayers.conf</filename> file in the | ||
1314 | "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>" | ||
1315 | section in the Yocto Project Development Tasks Manual. | ||
1316 | </para> | ||
1317 | |||
1318 | <para> | ||
1319 | The files <filename>site.conf</filename> and | ||
1320 | <filename>auto.conf</filename> are not created by the environment | ||
1321 | initialization script. | ||
1322 | If you want the <filename>site.conf</filename> file, you need to | ||
1323 | create that yourself. | ||
1324 | The <filename>auto.conf</filename> file is typically created by | ||
1325 | an autobuilder: | ||
1326 | <itemizedlist> | ||
1327 | <listitem><para><emphasis><filename>site.conf</filename>:</emphasis> | ||
1328 | You can use the <filename>conf/site.conf</filename> | ||
1329 | configuration file to configure multiple build directories. | ||
1330 | For example, suppose you had several build environments and | ||
1331 | they shared some common features. | ||
1332 | You can set these default build properties here. | ||
1333 | A good example is perhaps the packaging format to use | ||
1334 | through the | ||
1335 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink> | ||
1336 | variable.</para> | ||
1337 | <para>One useful scenario for using the | ||
1338 | <filename>conf/site.conf</filename> file is to extend your | ||
1339 | <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> | ||
1340 | variable to include the path to a | ||
1341 | <filename>conf/site.conf</filename>. | ||
1342 | Then, when BitBake looks for Metadata using | ||
1343 | <filename>BBPATH</filename>, it finds the | ||
1344 | <filename>conf/site.conf</filename> file and applies your | ||
1345 | common configurations found in the file. | ||
1346 | To override configurations in a particular build directory, | ||
1347 | alter the similar configurations within that build | ||
1348 | directory's <filename>conf/local.conf</filename> file. | ||
1349 | </para></listitem> | ||
1350 | <listitem><para><emphasis><filename>auto.conf</filename>:</emphasis> | ||
1351 | The file is usually created and written to by | ||
1352 | an autobuilder. | ||
1353 | The settings put into the file are typically the same as | ||
1354 | you would find in the <filename>conf/local.conf</filename> | ||
1355 | or the <filename>conf/site.conf</filename> files. | ||
1356 | </para></listitem> | ||
1357 | </itemizedlist> | ||
1358 | </para> | ||
1359 | |||
1360 | <para> | ||
1361 | You can edit all configuration files to further define | ||
1362 | any particular build environment. | ||
1363 | This process is represented by the "User Configuration Edits" | ||
1364 | box in the figure. | ||
1365 | </para> | ||
1366 | |||
1367 | <para> | ||
1368 | When you launch your build with the | ||
1369 | <filename>bitbake <replaceable>target</replaceable></filename> | ||
1370 | command, BitBake sorts out the configurations to ultimately | ||
1371 | define your build environment. | ||
1372 | It is important to understand that the OpenEmbedded build system | ||
1373 | reads the configuration files in a specific order: | ||
1374 | <filename>site.conf</filename>, <filename>auto.conf</filename>, | ||
1375 | and <filename>local.conf</filename>. | ||
1376 | And, the build system applies the normal assignment statement | ||
1377 | rules. | ||
1378 | Because the files are parsed in a specific order, variable | ||
1379 | assignments for the same variable could be affected. | ||
1380 | For example, if the <filename>auto.conf</filename> file and | ||
1381 | the <filename>local.conf</filename> set | ||
1382 | <replaceable>variable1</replaceable> to different values, because | ||
1383 | the build system parses <filename>local.conf</filename> after | ||
1384 | <filename>auto.conf</filename>, | ||
1385 | <replaceable>variable1</replaceable> is assigned the value from | ||
1386 | the <filename>local.conf</filename> file. | ||
1387 | </para> | ||
1388 | </section> | ||
1389 | |||
1390 | <section id="metadata-machine-configuration-and-policy-configuration"> | ||
1391 | <title>Metadata, Machine Configuration, and Policy Configuration</title> | ||
1392 | |||
1393 | <para> | ||
1394 | The previous section described the user configurations that | ||
1395 | define BitBake's global behavior. | ||
1396 | This section takes a closer look at the layers the build system | ||
1397 | uses to further control the build. | ||
1398 | These layers provide Metadata for the software, machine, and | ||
1399 | policy. | ||
1400 | </para> | ||
1401 | |||
1402 | <para> | ||
1403 | In general, three types of layer input exist: | ||
1404 | <itemizedlist> | ||
1405 | <listitem><para><emphasis>Policy Configuration:</emphasis> | ||
1406 | Distribution Layers provide top-level or general | ||
1407 | policies for the image or SDK being built. | ||
1408 | For example, this layer would dictate whether BitBake | ||
1409 | produces RPM or IPK packages.</para></listitem> | ||
1410 | <listitem><para><emphasis>Machine Configuration:</emphasis> | ||
1411 | Board Support Package (BSP) layers provide machine | ||
1412 | configurations. | ||
1413 | This type of information is specific to a particular | ||
1414 | target architecture.</para></listitem> | ||
1415 | <listitem><para><emphasis>Metadata:</emphasis> | ||
1416 | Software layers contain user-supplied recipe files, | ||
1417 | patches, and append files. | ||
1418 | </para></listitem> | ||
1419 | </itemizedlist> | ||
1420 | </para> | ||
1421 | |||
1422 | <para> | ||
1423 | The following figure shows an expanded representation of the | ||
1424 | Metadata, Machine Configuration, and Policy Configuration input | ||
1425 | (layers) boxes of the | ||
1426 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: | ||
1427 | </para> | ||
1428 | |||
1429 | <para> | ||
1430 | <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" /> | ||
1431 | </para> | ||
1432 | |||
1433 | <para> | ||
1434 | In general, all layers have a similar structure. | ||
1435 | They all contain a licensing file | ||
1436 | (e.g. <filename>COPYING</filename>) if the layer is to be | ||
1437 | distributed, a <filename>README</filename> file as good practice | ||
1438 | and especially if the layer is to be distributed, a | ||
1439 | configuration directory, and recipe directories. | ||
1440 | </para> | ||
1441 | |||
1442 | <para> | ||
1443 | The Yocto Project has many layers that can be used. | ||
1444 | You can see a web-interface listing of them on the | ||
1445 | <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink> | ||
1446 | page. | ||
1447 | The layers are shown at the bottom categorized under | ||
1448 | "Yocto Metadata Layers." | ||
1449 | These layers are fundamentally a subset of the | ||
1450 | <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>, | ||
1451 | which lists all layers provided by the OpenEmbedded community. | ||
1452 | <note> | ||
1453 | Layers exist in the Yocto Project Source Repositories that | ||
1454 | cannot be found in the OpenEmbedded Metadata Index. | ||
1455 | These layers are either deprecated or experimental in nature. | ||
1456 | </note> | ||
1457 | </para> | ||
1458 | |||
1459 | <para> | ||
1460 | BitBake uses the <filename>conf/bblayers.conf</filename> file, | ||
1461 | which is part of the user configuration, to find what layers it | ||
1462 | should be using as part of the build. | ||
1463 | </para> | ||
1464 | |||
1465 | <para> | ||
1466 | For more information on layers, see the | ||
1467 | "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>" | ||
1468 | section in the Yocto Project Development Tasks Manual. | ||
1469 | </para> | ||
1470 | |||
1471 | <section id="distro-layer"> | ||
1472 | <title>Distro Layer</title> | ||
1473 | |||
1474 | <para> | ||
1475 | The distribution layer provides policy configurations for your | ||
1476 | distribution. | ||
1477 | Best practices dictate that you isolate these types of | ||
1478 | configurations into their own layer. | ||
1479 | Settings you provide in | ||
1480 | <filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override | ||
1481 | similar | ||
1482 | settings that BitBake finds in your | ||
1483 | <filename>conf/local.conf</filename> file in the Build | ||
1484 | Directory. | ||
1485 | </para> | ||
1486 | |||
1487 | <para> | ||
1488 | The following list provides some explanation and references | ||
1489 | for what you typically find in the distribution layer: | ||
1490 | <itemizedlist> | ||
1491 | <listitem><para><emphasis>classes:</emphasis> | ||
1492 | Class files (<filename>.bbclass</filename>) hold | ||
1493 | common functionality that can be shared among | ||
1494 | recipes in the distribution. | ||
1495 | When your recipes inherit a class, they take on the | ||
1496 | settings and functions for that class. | ||
1497 | You can read more about class files in the | ||
1498 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>" | ||
1499 | section of the Yocto Reference Manual. | ||
1500 | </para></listitem> | ||
1501 | <listitem><para><emphasis>conf:</emphasis> | ||
1502 | This area holds configuration files for the | ||
1503 | layer (<filename>conf/layer.conf</filename>), | ||
1504 | the distribution | ||
1505 | (<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>), | ||
1506 | and any distribution-wide include files. | ||
1507 | </para></listitem> | ||
1508 | <listitem><para><emphasis>recipes-*:</emphasis> | ||
1509 | Recipes and append files that affect common | ||
1510 | functionality across the distribution. | ||
1511 | This area could include recipes and append files | ||
1512 | to add distribution-specific configuration, | ||
1513 | initialization scripts, custom image recipes, | ||
1514 | and so forth.</para></listitem> | ||
1515 | </itemizedlist> | ||
1516 | </para> | ||
1517 | </section> | ||
1518 | |||
1519 | <section id="bsp-layer"> | ||
1520 | <title>BSP Layer</title> | ||
1521 | |||
1522 | <para> | ||
1523 | The BSP Layer provides machine configurations. | ||
1524 | Everything in this layer is specific to the machine for which | ||
1525 | you are building the image or the SDK. | ||
1526 | A common structure or form is defined for BSP layers. | ||
1527 | You can learn more about this structure in the | ||
1528 | <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>. | ||
1529 | <note> | ||
1530 | In order for a BSP layer to be considered compliant with the | ||
1531 | Yocto Project, it must meet some structural requirements. | ||
1532 | </note> | ||
1533 | </para> | ||
1534 | |||
1535 | <para> | ||
1536 | The BSP Layer's configuration directory contains | ||
1537 | configuration files for the machine | ||
1538 | (<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>) and, | ||
1539 | of course, the layer (<filename>conf/layer.conf</filename>). | ||
1540 | </para> | ||
1541 | |||
1542 | <para> | ||
1543 | The remainder of the layer is dedicated to specific recipes | ||
1544 | by function: <filename>recipes-bsp</filename>, | ||
1545 | <filename>recipes-core</filename>, | ||
1546 | <filename>recipes-graphics</filename>, and | ||
1547 | <filename>recipes-kernel</filename>. | ||
1548 | Metadata can exist for multiple formfactors, graphics | ||
1549 | support systems, and so forth. | ||
1550 | <note> | ||
1551 | While the figure shows several <filename>recipes-*</filename> | ||
1552 | directories, not all these directories appear in all | ||
1553 | BSP layers. | ||
1554 | </note> | ||
1555 | </para> | ||
1556 | </section> | ||
1557 | |||
1558 | <section id="software-layer"> | ||
1559 | <title>Software Layer</title> | ||
1560 | |||
1561 | <para> | ||
1562 | The software layer provides the Metadata for additional | ||
1563 | software packages used during the build. | ||
1564 | This layer does not include Metadata that is specific to the | ||
1565 | distribution or the machine, which are found in their | ||
1566 | respective layers. | ||
1567 | </para> | ||
1568 | |||
1569 | <para> | ||
1570 | This layer contains any new recipes that your project needs | ||
1571 | in the form of recipe files. | ||
1572 | </para> | ||
1573 | </section> | ||
1574 | </section> | ||
1575 | |||
1576 | <section id="sources-dev-environment"> | ||
1577 | <title>Sources</title> | ||
1578 | |||
1579 | <para> | ||
1580 | In order for the OpenEmbedded build system to create an image or | ||
1581 | any target, it must be able to access source files. | ||
1582 | The | ||
1583 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> | ||
1584 | represents source files using the "Upstream Project Releases", | ||
1585 | "Local Projects", and "SCMs (optional)" boxes. | ||
1586 | The figure represents mirrors, which also play a role in locating | ||
1587 | source files, with the "Source Mirror(s)" box. | ||
1588 | </para> | ||
1589 | |||
1590 | <para> | ||
1591 | The method by which source files are ultimately organized is | ||
1592 | a function of the project. | ||
1593 | For example, for released software, projects tend to use tarballs | ||
1594 | or other archived files that can capture the state of a release | ||
1595 | guaranteeing that it is statically represented. | ||
1596 | On the other hand, for a project that is more dynamic or | ||
1597 | experimental in nature, a project might keep source files in a | ||
1598 | repository controlled by a Source Control Manager (SCM) such as | ||
1599 | Git. | ||
1600 | Pulling source from a repository allows you to control | ||
1601 | the point in the repository (the revision) from which you want to | ||
1602 | build software. | ||
1603 | Finally, a combination of the two might exist, which would give the | ||
1604 | consumer a choice when deciding where to get source files. | ||
1605 | </para> | ||
1606 | |||
1607 | <para> | ||
1608 | BitBake uses the | ||
1609 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> | ||
1610 | variable to point to source files regardless of their location. | ||
1611 | Each recipe must have a <filename>SRC_URI</filename> variable | ||
1612 | that points to the source. | ||
1613 | </para> | ||
1614 | |||
1615 | <para> | ||
1616 | Another area that plays a significant role in where source files | ||
1617 | come from is pointed to by the | ||
1618 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> | ||
1619 | variable. | ||
1620 | This area is a cache that can hold previously downloaded source. | ||
1621 | You can also instruct the OpenEmbedded build system to create | ||
1622 | tarballs from Git repositories, which is not the default behavior, | ||
1623 | and store them in the <filename>DL_DIR</filename> by using the | ||
1624 | <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink> | ||
1625 | variable. | ||
1626 | </para> | ||
1627 | |||
1628 | <para> | ||
1629 | Judicious use of a <filename>DL_DIR</filename> directory can | ||
1630 | save the build system a trip across the Internet when looking | ||
1631 | for files. | ||
1632 | A good method for using a download directory is to have | ||
1633 | <filename>DL_DIR</filename> point to an area outside of your | ||
1634 | Build Directory. | ||
1635 | Doing so allows you to safely delete the Build Directory | ||
1636 | if needed without fear of removing any downloaded source file. | ||
1637 | </para> | ||
1638 | |||
1639 | <para> | ||
1640 | The remainder of this section provides a deeper look into the | ||
1641 | source files and the mirrors. | ||
1642 | Here is a more detailed look at the source file area of the | ||
1643 | base figure: | ||
1644 | <imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" /> | ||
1645 | </para> | ||
1646 | |||
1647 | <section id='upstream-project-releases'> | ||
1648 | <title>Upstream Project Releases</title> | ||
1649 | |||
1650 | <para> | ||
1651 | Upstream project releases exist anywhere in the form of an | ||
1652 | archived file (e.g. tarball or zip file). | ||
1653 | These files correspond to individual recipes. | ||
1654 | For example, the figure uses specific releases each for | ||
1655 | BusyBox, Qt, and Dbus. | ||
1656 | An archive file can be for any released product that can be | ||
1657 | built using a recipe. | ||
1658 | </para> | ||
1659 | </section> | ||
1660 | |||
1661 | <section id='local-projects'> | ||
1662 | <title>Local Projects</title> | ||
1663 | |||
1664 | <para> | ||
1665 | Local projects are custom bits of software the user provides. | ||
1666 | These bits reside somewhere local to a project - perhaps | ||
1667 | a directory into which the user checks in items (e.g. | ||
1668 | a local directory containing a development source tree | ||
1669 | used by the group). | ||
1670 | </para> | ||
1671 | |||
1672 | <para> | ||
1673 | The canonical method through which to include a local project | ||
1674 | is to use the | ||
1675 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink> | ||
1676 | class to include that local project. | ||
1677 | You use either the <filename>local.conf</filename> or a | ||
1678 | recipe's append file to override or set the | ||
1679 | recipe to point to the local directory on your disk to pull | ||
1680 | in the whole source tree. | ||
1681 | </para> | ||
1682 | |||
1683 | <para> | ||
1684 | For information on how to use the | ||
1685 | <filename>externalsrc</filename> class, see the | ||
1686 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></ulink>" | ||
1687 | section. | ||
1688 | </para> | ||
1689 | </section> | ||
1690 | |||
1691 | <section id='scms'> | ||
1692 | <title>Source Control Managers (Optional)</title> | ||
1693 | |||
1694 | <para> | ||
1695 | Another place the build system can get source files from is | ||
1696 | through an SCM such as Git or Subversion. | ||
1697 | In this case, a repository is cloned or checked out. | ||
1698 | The | ||
1699 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink> | ||
1700 | task inside BitBake uses | ||
1701 | the <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> | ||
1702 | variable and the argument's prefix to determine the correct | ||
1703 | fetcher module. | ||
1704 | </para> | ||
1705 | |||
1706 | <note> | ||
1707 | For information on how to have the OpenEmbedded build system | ||
1708 | generate tarballs for Git repositories and place them in the | ||
1709 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> | ||
1710 | directory, see the | ||
1711 | <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink> | ||
1712 | variable. | ||
1713 | </note> | ||
1714 | |||
1715 | <para> | ||
1716 | When fetching a repository, BitBake uses the | ||
1717 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> | ||
1718 | variable to determine the specific revision from which to | ||
1719 | build. | ||
1720 | </para> | ||
1721 | </section> | ||
1722 | |||
1723 | <section id='source-mirrors'> | ||
1724 | <title>Source Mirror(s)</title> | ||
1725 | |||
1726 | <para> | ||
1727 | Two kinds of mirrors exist: pre-mirrors and regular mirrors. | ||
1728 | The | ||
1729 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink> | ||
1730 | and | ||
1731 | <ulink url='&YOCTO_DOCS_REF_URL;#var-MIRRORS'><filename>MIRRORS</filename></ulink> | ||
1732 | variables point to these, respectively. | ||
1733 | BitBake checks pre-mirrors before looking upstream for any | ||
1734 | source files. | ||
1735 | Pre-mirrors are appropriate when you have a shared directory | ||
1736 | that is not a directory defined by the | ||
1737 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> | ||
1738 | variable. | ||
1739 | A Pre-mirror typically points to a shared directory that is | ||
1740 | local to your organization. | ||
1741 | </para> | ||
1742 | |||
1743 | <para> | ||
1744 | Regular mirrors can be any site across the Internet that is | ||
1745 | used as an alternative location for source code should the | ||
1746 | primary site not be functioning for some reason or another. | ||
1747 | </para> | ||
1748 | </section> | ||
1749 | </section> | ||
1750 | |||
1751 | <section id="package-feeds-dev-environment"> | ||
1752 | <title>Package Feeds</title> | ||
1753 | |||
1754 | <para> | ||
1755 | When the OpenEmbedded build system generates an image or an SDK, | ||
1756 | it gets the packages from a package feed area located in the | ||
1757 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. | ||
1758 | The | ||
1759 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> | ||
1760 | shows this package feeds area in the upper-right corner. | ||
1761 | </para> | ||
1762 | |||
1763 | <para> | ||
1764 | This section looks a little closer into the package feeds area used | ||
1765 | by the build system. | ||
1766 | Here is a more detailed look at the area: | ||
1767 | <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" /> | ||
1768 | </para> | ||
1769 | |||
1770 | <para> | ||
1771 | Package feeds are an intermediary step in the build process. | ||
1772 | The OpenEmbedded build system provides classes to generate | ||
1773 | different package types, and you specify which classes to enable | ||
1774 | through the | ||
1775 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink> | ||
1776 | variable. | ||
1777 | Before placing the packages into package feeds, | ||
1778 | the build process validates them with generated output quality | ||
1779 | assurance checks through the | ||
1780 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink> | ||
1781 | class. | ||
1782 | </para> | ||
1783 | |||
1784 | <para> | ||
1785 | The package feed area resides in the Build Directory. | ||
1786 | The directory the build system uses to temporarily store packages | ||
1787 | is determined by a combination of variables and the particular | ||
1788 | package manager in use. | ||
1789 | See the "Package Feeds" box in the illustration and note the | ||
1790 | information to the right of that area. | ||
1791 | In particular, the following defines where package files are | ||
1792 | kept: | ||
1793 | <itemizedlist> | ||
1794 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>: | ||
1795 | Defined as <filename>tmp/deploy</filename> in the Build | ||
1796 | Directory. | ||
1797 | </para></listitem> | ||
1798 | <listitem><para><filename>DEPLOY_DIR_*</filename>: | ||
1799 | Depending on the package manager used, the package type | ||
1800 | sub-folder. | ||
1801 | Given RPM, IPK, or DEB packaging and tarball creation, the | ||
1802 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_RPM'><filename>DEPLOY_DIR_RPM</filename></ulink>, | ||
1803 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IPK'><filename>DEPLOY_DIR_IPK</filename></ulink>, | ||
1804 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_DEB'><filename>DEPLOY_DIR_DEB</filename></ulink>, | ||
1805 | or | ||
1806 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_TAR'><filename>DEPLOY_DIR_TAR</filename></ulink>, | ||
1807 | variables are used, respectively. | ||
1808 | </para></listitem> | ||
1809 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>: | ||
1810 | Defines architecture-specific sub-folders. | ||
1811 | For example, packages could exist for the i586 or qemux86 | ||
1812 | architectures. | ||
1813 | </para></listitem> | ||
1814 | </itemizedlist> | ||
1815 | </para> | ||
1816 | |||
1817 | <para> | ||
1818 | BitBake uses the <filename>do_package_write_*</filename> tasks to | ||
1819 | generate packages and place them into the package holding area (e.g. | ||
1820 | <filename>do_package_write_ipk</filename> for IPK packages). | ||
1821 | See the | ||
1822 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></ulink>", | ||
1823 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></ulink>", | ||
1824 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></ulink>", | ||
1825 | and | ||
1826 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></ulink>" | ||
1827 | sections for additional information. | ||
1828 | As an example, consider a scenario where an IPK packaging manager | ||
1829 | is being used and package architecture support for both i586 | ||
1830 | and qemux86 exist. | ||
1831 | Packages for the i586 architecture are placed in | ||
1832 | <filename>build/tmp/deploy/ipk/i586</filename>, while packages for | ||
1833 | the qemux86 architecture are placed in | ||
1834 | <filename>build/tmp/deploy/ipk/qemux86</filename>. | ||
1835 | </para> | ||
1836 | </section> | ||
1837 | |||
1838 | <section id='bitbake-dev-environment'> | ||
1839 | <title>BitBake</title> | ||
1840 | |||
1841 | <para> | ||
1842 | The OpenEmbedded build system uses | ||
1843 | <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink> | ||
1844 | to produce images. | ||
1845 | You can see from the | ||
1846 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, | ||
1847 | the BitBake area consists of several functional areas. | ||
1848 | This section takes a closer look at each of those areas. | ||
1849 | </para> | ||
1850 | |||
1851 | <para> | ||
1852 | Separate documentation exists for the BitBake tool. | ||
1853 | See the | ||
1854 | <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink> | ||
1855 | for reference material on BitBake. | ||
1856 | </para> | ||
1857 | |||
1858 | <section id='source-fetching-dev-environment'> | ||
1859 | <title>Source Fetching</title> | ||
1860 | |||
1861 | <para> | ||
1862 | The first stages of building a recipe are to fetch and unpack | ||
1863 | the source code: | ||
1864 | <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" /> | ||
1865 | </para> | ||
1866 | |||
1867 | <para> | ||
1868 | The | ||
1869 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink> | ||
1870 | and | ||
1871 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> | ||
1872 | tasks fetch the source files and unpack them into the work | ||
1873 | directory. | ||
1874 | <note> | ||
1875 | For every local file (e.g. <filename>file://</filename>) | ||
1876 | that is part of a recipe's | ||
1877 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> | ||
1878 | statement, the OpenEmbedded build system takes a checksum | ||
1879 | of the file for the recipe and inserts the checksum into | ||
1880 | the signature for the <filename>do_fetch</filename>. | ||
1881 | If any local file has been modified, the | ||
1882 | <filename>do_fetch</filename> task and all tasks that | ||
1883 | depend on it are re-executed. | ||
1884 | </note> | ||
1885 | By default, everything is accomplished in the | ||
1886 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>, | ||
1887 | which has a defined structure. | ||
1888 | For additional general information on the Build Directory, | ||
1889 | see the | ||
1890 | "<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-build'><filename>build/</filename></ulink>" | ||
1891 | section in the Yocto Project Reference Manual. | ||
1892 | </para> | ||
1893 | |||
1894 | <para> | ||
1895 | Unpacked source files are pointed to by the | ||
1896 | <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> | ||
1897 | variable. | ||
1898 | Each recipe has an area in the Build Directory where the | ||
1899 | unpacked source code resides. | ||
1900 | The name of that directory for any given recipe is defined from | ||
1901 | several different variables. | ||
1902 | You can see the variables that define these directories | ||
1903 | by looking at the figure: | ||
1904 | <itemizedlist> | ||
1905 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> - | ||
1906 | The base directory where the OpenEmbedded build system | ||
1907 | performs all its work during the build. | ||
1908 | </para></listitem> | ||
1909 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink> - | ||
1910 | The architecture of the built package or packages. | ||
1911 | </para></listitem> | ||
1912 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_OS'><filename>TARGET_OS</filename></ulink> - | ||
1913 | The operating system of the target device. | ||
1914 | </para></listitem> | ||
1915 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> - | ||
1916 | The name of the built package. | ||
1917 | </para></listitem> | ||
1918 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - | ||
1919 | The version of the recipe used to build the package. | ||
1920 | </para></listitem> | ||
1921 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> - | ||
1922 | The revision of the recipe used to build the package. | ||
1923 | </para></listitem> | ||
1924 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink> - | ||
1925 | The location within <filename>TMPDIR</filename> where | ||
1926 | a specific package is built. | ||
1927 | </para></listitem> | ||
1928 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> - | ||
1929 | Contains the unpacked source files for a given recipe. | ||
1930 | </para></listitem> | ||
1931 | </itemizedlist> | ||
1932 | </para> | ||
1933 | </section> | ||
1934 | |||
1935 | <section id='patching-dev-environment'> | ||
1936 | <title>Patching</title> | ||
1937 | |||
1938 | <para> | ||
1939 | Once source code is fetched and unpacked, BitBake locates | ||
1940 | patch files and applies them to the source files: | ||
1941 | <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" /> | ||
1942 | </para> | ||
1943 | |||
1944 | <para> | ||
1945 | The | ||
1946 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> | ||
1947 | task processes recipes by | ||
1948 | using the | ||
1949 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> | ||
1950 | variable to locate applicable patch files, which by default | ||
1951 | are <filename>*.patch</filename> or | ||
1952 | <filename>*.diff</filename> files, or any file if | ||
1953 | "apply=yes" is specified for the file in | ||
1954 | <filename>SRC_URI</filename>. | ||
1955 | </para> | ||
1956 | |||
1957 | <para> | ||
1958 | BitBake finds and applies multiple patches for a single recipe | ||
1959 | in the order in which it finds the patches. | ||
1960 | Patches are applied to the recipe's source files located in the | ||
1961 | <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> | ||
1962 | directory. | ||
1963 | </para> | ||
1964 | |||
1965 | <para> | ||
1966 | For more information on how the source directories are | ||
1967 | created, see the | ||
1968 | "<link linkend='source-fetching-dev-environment'>Source Fetching</link>" | ||
1969 | section. | ||
1970 | </para> | ||
1971 | </section> | ||
1972 | |||
1973 | <section id='configuration-and-compilation-dev-environment'> | ||
1974 | <title>Configuration and Compilation</title> | ||
1975 | |||
1976 | <para> | ||
1977 | After source code is patched, BitBake executes tasks that | ||
1978 | configure and compile the source code: | ||
1979 | <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" /> | ||
1980 | </para> | ||
1981 | |||
1982 | <para> | ||
1983 | This step in the build process consists of three tasks: | ||
1984 | <itemizedlist> | ||
1985 | <listitem><para> | ||
1986 | <emphasis><ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-prepare_recipe_sysroot'><filename>do_prepare_recipe_sysroot</filename></ulink>:</emphasis> | ||
1987 | This task sets up the two sysroots in | ||
1988 | <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename> | ||
1989 | (i.e. <filename>recipe-sysroot</filename> and | ||
1990 | <filename>recipe-sysroot-native</filename>) so that | ||
1991 | the sysroots contain the contents of the | ||
1992 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink> | ||
1993 | tasks of the recipes on which the recipe | ||
1994 | containing the tasks depends. | ||
1995 | A sysroot exists for both the target and for the native | ||
1996 | binaries, which run on the host system. | ||
1997 | </para></listitem> | ||
1998 | <listitem><para><emphasis><filename>do_configure</filename>:</emphasis> | ||
1999 | This task configures the source by enabling and | ||
2000 | disabling any build-time and configuration options for | ||
2001 | the software being built. | ||
2002 | Configurations can come from the recipe itself as well | ||
2003 | as from an inherited class. | ||
2004 | Additionally, the software itself might configure itself | ||
2005 | depending on the target for which it is being built. | ||
2006 | </para> | ||
2007 | |||
2008 | <para>The configurations handled by the | ||
2009 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> | ||
2010 | task are specific | ||
2011 | to source code configuration for the source code | ||
2012 | being built by the recipe.</para> | ||
2013 | |||
2014 | <para>If you are using the | ||
2015 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> | ||
2016 | class, | ||
2017 | you can add additional configuration options by using | ||
2018 | the | ||
2019 | <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink> | ||
2020 | or | ||
2021 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink> | ||
2022 | variables. | ||
2023 | For information on how this variable works within | ||
2024 | that class, see the | ||
2025 | <filename>meta/classes/autotools.bbclass</filename> file. | ||
2026 | </para></listitem> | ||
2027 | <listitem><para><emphasis><filename>do_compile</filename>:</emphasis> | ||
2028 | Once a configuration task has been satisfied, BitBake | ||
2029 | compiles the source using the | ||
2030 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> | ||
2031 | task. | ||
2032 | Compilation occurs in the directory pointed to by the | ||
2033 | <ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink> | ||
2034 | variable. | ||
2035 | Realize that the <filename>B</filename> directory is, by | ||
2036 | default, the same as the | ||
2037 | <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> | ||
2038 | directory.</para></listitem> | ||
2039 | <listitem><para><emphasis><filename>do_install</filename>:</emphasis> | ||
2040 | Once compilation is done, BitBake executes the | ||
2041 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> | ||
2042 | task. | ||
2043 | This task copies files from the <filename>B</filename> | ||
2044 | directory and places them in a holding area pointed to | ||
2045 | by the | ||
2046 | <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> | ||
2047 | variable.</para></listitem> | ||
2048 | </itemizedlist> | ||
2049 | </para> | ||
2050 | </section> | ||
2051 | |||
2052 | <section id='package-splitting-dev-environment'> | ||
2053 | <title>Package Splitting</title> | ||
2054 | |||
2055 | <para> | ||
2056 | After source code is configured and compiled, the | ||
2057 | OpenEmbedded build system analyzes | ||
2058 | the results and splits the output into packages: | ||
2059 | <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" /> | ||
2060 | </para> | ||
2061 | |||
2062 | <para> | ||
2063 | The | ||
2064 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink> | ||
2065 | and | ||
2066 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata'><filename>do_packagedata</filename></ulink> | ||
2067 | tasks combine to analyze | ||
2068 | the files found in the | ||
2069 | <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> directory | ||
2070 | and split them into subsets based on available packages and | ||
2071 | files. | ||
2072 | The analyzing process involves the following as well as other | ||
2073 | items: splitting out debugging symbols, | ||
2074 | looking at shared library dependencies between packages, | ||
2075 | and looking at package relationships. | ||
2076 | The <filename>do_packagedata</filename> task creates package | ||
2077 | metadata based on the analysis such that the | ||
2078 | OpenEmbedded build system can generate the final packages. | ||
2079 | Working, staged, and intermediate results of the analysis | ||
2080 | and package splitting process use these areas: | ||
2081 | <itemizedlist> | ||
2082 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PKGD'><filename>PKGD</filename></ulink> - | ||
2083 | The destination directory for packages before they are | ||
2084 | split. | ||
2085 | </para></listitem> | ||
2086 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink> - | ||
2087 | A shared, global-state directory that holds data | ||
2088 | generated during the packaging process. | ||
2089 | </para></listitem> | ||
2090 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDESTWORK'><filename>PKGDESTWORK</filename></ulink> - | ||
2091 | A temporary work area used by the | ||
2092 | <filename>do_package</filename> task. | ||
2093 | </para></listitem> | ||
2094 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDEST'><filename>PKGDEST</filename></ulink> - | ||
2095 | The parent directory for packages after they have | ||
2096 | been split. | ||
2097 | </para></listitem> | ||
2098 | </itemizedlist> | ||
2099 | The <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink> | ||
2100 | variable defines the files that go into each package in | ||
2101 | <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>. | ||
2102 | If you want details on how this is accomplished, you can | ||
2103 | look at the | ||
2104 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-package'><filename>package</filename></ulink> | ||
2105 | class. | ||
2106 | </para> | ||
2107 | |||
2108 | <para> | ||
2109 | Depending on the type of packages being created (RPM, DEB, or | ||
2110 | IPK), the <filename>do_package_write_*</filename> task | ||
2111 | creates the actual packages and places them in the | ||
2112 | Package Feed area, which is | ||
2113 | <filename>${TMPDIR}/deploy</filename>. | ||
2114 | You can see the | ||
2115 | "<link linkend='package-feeds-dev-environment'>Package Feeds</link>" | ||
2116 | section for more detail on that part of the build process. | ||
2117 | <note> | ||
2118 | Support for creating feeds directly from the | ||
2119 | <filename>deploy/*</filename> directories does not exist. | ||
2120 | Creating such feeds usually requires some kind of feed | ||
2121 | maintenance mechanism that would upload the new packages | ||
2122 | into an official package feed (e.g. the | ||
2123 | Ångström distribution). | ||
2124 | This functionality is highly distribution-specific | ||
2125 | and thus is not provided out of the box. | ||
2126 | </note> | ||
2127 | </para> | ||
2128 | </section> | ||
2129 | |||
2130 | <section id='image-generation-dev-environment'> | ||
2131 | <title>Image Generation</title> | ||
2132 | |||
2133 | <para> | ||
2134 | Once packages are split and stored in the Package Feeds area, | ||
2135 | the OpenEmbedded build system uses BitBake to generate the | ||
2136 | root filesystem image: | ||
2137 | <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" /> | ||
2138 | </para> | ||
2139 | |||
2140 | <para> | ||
2141 | The image generation process consists of several stages and | ||
2142 | depends on several tasks and variables. | ||
2143 | The | ||
2144 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink> | ||
2145 | task creates the root filesystem (file and directory structure) | ||
2146 | for an image. | ||
2147 | This task uses several key variables to help create the list | ||
2148 | of packages to actually install: | ||
2149 | <itemizedlist> | ||
2150 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>: | ||
2151 | Lists out the base set of packages to install from | ||
2152 | the Package Feeds area.</para></listitem> | ||
2153 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>: | ||
2154 | Specifies packages that should not be installed. | ||
2155 | </para></listitem> | ||
2156 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>: | ||
2157 | Specifies features to include in the image. | ||
2158 | Most of these features map to additional packages for | ||
2159 | installation.</para></listitem> | ||
2160 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>: | ||
2161 | Specifies the package backend to use and consequently | ||
2162 | helps determine where to locate packages within the | ||
2163 | Package Feeds area.</para></listitem> | ||
2164 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></ulink>: | ||
2165 | Determines the language(s) for which additional | ||
2166 | language support packages are installed. | ||
2167 | </para></listitem> | ||
2168 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></ulink>: | ||
2169 | The final list of packages passed to the package manager | ||
2170 | for installation into the image. | ||
2171 | </para></listitem> | ||
2172 | </itemizedlist> | ||
2173 | </para> | ||
2174 | |||
2175 | <para> | ||
2176 | With | ||
2177 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_ROOTFS'><filename>IMAGE_ROOTFS</filename></ulink> | ||
2178 | pointing to the location of the filesystem under construction and | ||
2179 | the <filename>PACKAGE_INSTALL</filename> variable providing the | ||
2180 | final list of packages to install, the root file system is | ||
2181 | created. | ||
2182 | </para> | ||
2183 | |||
2184 | <para> | ||
2185 | Package installation is under control of the package manager | ||
2186 | (e.g. dnf/rpm, opkg, or apt/dpkg) regardless of whether or | ||
2187 | not package management is enabled for the target. | ||
2188 | At the end of the process, if package management is not | ||
2189 | enabled for the target, the package manager's data files | ||
2190 | are deleted from the root filesystem. | ||
2191 | As part of the final stage of package installation, postinstall | ||
2192 | scripts that are part of the packages are run. | ||
2193 | Any scripts that fail to run | ||
2194 | on the build host are run on the target when the target system | ||
2195 | is first booted. | ||
2196 | If you are using a | ||
2197 | <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>, | ||
2198 | all the post installation scripts must succeed during the | ||
2199 | package installation phase since the root filesystem is | ||
2200 | read-only. | ||
2201 | </para> | ||
2202 | |||
2203 | <para> | ||
2204 | The final stages of the <filename>do_rootfs</filename> task | ||
2205 | handle post processing. | ||
2206 | Post processing includes creation of a manifest file and | ||
2207 | optimizations. | ||
2208 | </para> | ||
2209 | |||
2210 | <para> | ||
2211 | The manifest file (<filename>.manifest</filename>) resides | ||
2212 | in the same directory as the root filesystem image. | ||
2213 | This file lists out, line-by-line, the installed packages. | ||
2214 | The manifest file is useful for the | ||
2215 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink> | ||
2216 | class, for example, to determine whether or not to run | ||
2217 | specific tests. | ||
2218 | See the | ||
2219 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_MANIFEST'><filename>IMAGE_MANIFEST</filename></ulink> | ||
2220 | variable for additional information. | ||
2221 | </para> | ||
2222 | |||
2223 | <para> | ||
2224 | Optimizing processes run across the image include | ||
2225 | <filename>mklibs</filename>, <filename>prelink</filename>, | ||
2226 | and any other post-processing commands as defined by the | ||
2227 | <ulink url='&YOCTO_DOCS_REF_URL;#var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></ulink> | ||
2228 | variable. | ||
2229 | The <filename>mklibs</filename> process optimizes the size | ||
2230 | of the libraries, while the | ||
2231 | <filename>prelink</filename> process optimizes the dynamic | ||
2232 | linking of shared libraries to reduce start up time of | ||
2233 | executables. | ||
2234 | </para> | ||
2235 | |||
2236 | <para> | ||
2237 | After the root filesystem is built, processing begins on | ||
2238 | the image through the | ||
2239 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image'><filename>do_image</filename></ulink> | ||
2240 | task. | ||
2241 | The build system runs any pre-processing commands as defined | ||
2242 | by the | ||
2243 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_PREPROCESS_COMMAND'><filename>IMAGE_PREPROCESS_COMMAND</filename></ulink> | ||
2244 | variable. | ||
2245 | This variable specifies a list of functions to call before | ||
2246 | the OpenEmbedded build system creates the final image output | ||
2247 | files. | ||
2248 | </para> | ||
2249 | |||
2250 | <para> | ||
2251 | The OpenEmbedded build system dynamically creates | ||
2252 | <filename>do_image_*</filename> tasks as needed, based | ||
2253 | on the image types specified in the | ||
2254 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink> | ||
2255 | variable. | ||
2256 | The process turns everything into an image file or a set of | ||
2257 | image files and compresses the root filesystem image to reduce | ||
2258 | the overall size of the image. | ||
2259 | The formats used for the root filesystem depend on the | ||
2260 | <filename>IMAGE_FSTYPES</filename> variable. | ||
2261 | </para> | ||
2262 | |||
2263 | <para> | ||
2264 | As an example, a dynamically created task when creating a | ||
2265 | particular image <replaceable>type</replaceable> would take the | ||
2266 | following form: | ||
2267 | <literallayout class='monospaced'> | ||
2268 | do_image_<replaceable>type</replaceable>[depends] | ||
2269 | </literallayout> | ||
2270 | So, if the <replaceable>type</replaceable> as specified by the | ||
2271 | <filename>IMAGE_FSTYPES</filename> were | ||
2272 | <filename>ext4</filename>, the dynamically generated task | ||
2273 | would be as follows: | ||
2274 | <literallayout class='monospaced'> | ||
2275 | do_image_ext4[depends] | ||
2276 | </literallayout> | ||
2277 | </para> | ||
2278 | |||
2279 | <para> | ||
2280 | The final task involved in image creation is the | ||
2281 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image-complete'><filename>do_image_complete</filename></ulink> | ||
2282 | task. | ||
2283 | This task completes the image by applying any image | ||
2284 | post processing as defined through the | ||
2285 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_POSTPROCESS_COMMAND'><filename>IMAGE_POSTPROCESS_COMMAND</filename></ulink> | ||
2286 | variable. | ||
2287 | The variable specifies a list of functions to call once the | ||
2288 | OpenEmbedded build system has created the final image output | ||
2289 | files. | ||
2290 | </para> | ||
2291 | |||
2292 | <note> | ||
2293 | The entire image generation process is run under Pseudo. | ||
2294 | Running under Pseudo ensures that the files in the root | ||
2295 | filesystem have correct ownership. | ||
2296 | </note> | ||
2297 | </section> | ||
2298 | |||
2299 | <section id='sdk-generation-dev-environment'> | ||
2300 | <title>SDK Generation</title> | ||
2301 | |||
2302 | <para> | ||
2303 | The OpenEmbedded build system uses BitBake to generate the | ||
2304 | Software Development Kit (SDK) installer script for both the | ||
2305 | standard and extensible SDKs: | ||
2306 | <imagedata fileref="figures/sdk-generation.png" align="center" /> | ||
2307 | </para> | ||
2308 | |||
2309 | <note> | ||
2310 | For more information on the cross-development toolchain | ||
2311 | generation, see the | ||
2312 | "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>" | ||
2313 | section in the Yocto Project Reference Manual. | ||
2314 | For information on advantages gained when building a | ||
2315 | cross-development toolchain using the | ||
2316 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></ulink> | ||
2317 | task, see the | ||
2318 | "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>" | ||
2319 | section in the Yocto Project Application Development and the | ||
2320 | Extensible Software Development Kit (SDK) manual. | ||
2321 | </note> | ||
2322 | |||
2323 | <para> | ||
2324 | Like image generation, the SDK script process consists of | ||
2325 | several stages and depends on many variables. | ||
2326 | The <filename>do_populate_sdk</filename> and | ||
2327 | <filename>do_populate_sdk_ext</filename> tasks use these | ||
2328 | key variables to help create the list of packages to actually | ||
2329 | install. | ||
2330 | For information on the variables listed in the figure, see the | ||
2331 | "<link linkend='sdk-dev-environment'>Application Development SDK</link>" | ||
2332 | section. | ||
2333 | </para> | ||
2334 | |||
2335 | <para> | ||
2336 | The <filename>do_populate_sdk</filename> task helps create | ||
2337 | the standard SDK and handles two parts: a target part and a | ||
2338 | host part. | ||
2339 | The target part is the part built for the target hardware and | ||
2340 | includes libraries and headers. | ||
2341 | The host part is the part of the SDK that runs on the | ||
2342 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>. | ||
2343 | </para> | ||
2344 | |||
2345 | <para> | ||
2346 | The <filename>do_populate_sdk_ext</filename> task helps create | ||
2347 | the extensible SDK and handles host and target parts | ||
2348 | differently than its counter part does for the standard SDK. | ||
2349 | For the extensible SDK, the task encapsulates the build system, | ||
2350 | which includes everything needed (host and target) for the SDK. | ||
2351 | </para> | ||
2352 | |||
2353 | <para> | ||
2354 | Regardless of the type of SDK being constructed, the | ||
2355 | tasks perform some cleanup after which a cross-development | ||
2356 | environment setup script and any needed configuration files | ||
2357 | are created. | ||
2358 | The final output is the Cross-development | ||
2359 | toolchain installation script (<filename>.sh</filename> file), | ||
2360 | which includes the environment setup script. | ||
2361 | </para> | ||
2362 | </section> | ||
2363 | |||
2364 | <section id='stamp-files-and-the-rerunning-of-tasks'> | ||
2365 | <title>Stamp Files and the Rerunning of Tasks</title> | ||
2366 | |||
2367 | <para> | ||
2368 | For each task that completes successfully, BitBake writes a | ||
2369 | stamp file into the | ||
2370 | <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR'><filename>STAMPS_DIR</filename></ulink> | ||
2371 | directory. | ||
2372 | The beginning of the stamp file's filename is determined by the | ||
2373 | <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMP'><filename>STAMP</filename></ulink> | ||
2374 | variable, and the end of the name consists of the task's name | ||
2375 | and current | ||
2376 | <ulink url='&YOCTO_DOCS_BB_URL;#checksums'>input checksum</ulink>. | ||
2377 | <note> | ||
2378 | This naming scheme assumes that | ||
2379 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SIGNATURE_HANDLER'><filename>BB_SIGNATURE_HANDLER</filename></ulink> | ||
2380 | is "OEBasicHash", which is almost always the case in | ||
2381 | current OpenEmbedded. | ||
2382 | </note> | ||
2383 | To determine if a task needs to be rerun, BitBake checks if a | ||
2384 | stamp file with a matching input checksum exists for the task. | ||
2385 | If such a stamp file exists, the task's output is assumed to | ||
2386 | exist and still be valid. | ||
2387 | If the file does not exist, the task is rerun. | ||
2388 | <note> | ||
2389 | <para>The stamp mechanism is more general than the shared | ||
2390 | state (sstate) cache mechanism described in the | ||
2391 | "<link linkend='setscene-tasks-and-shared-state'>Setscene Tasks and Shared State</link>" | ||
2392 | section. | ||
2393 | BitBake avoids rerunning any task that has a valid | ||
2394 | stamp file, not just tasks that can be accelerated through | ||
2395 | the sstate cache.</para> | ||
2396 | <para>However, you should realize that stamp files only | ||
2397 | serve as a marker that some work has been done and that | ||
2398 | these files do not record task output. | ||
2399 | The actual task output would usually be somewhere in | ||
2400 | <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> | ||
2401 | (e.g. in some recipe's | ||
2402 | <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>.) | ||
2403 | What the sstate cache mechanism adds is a way to cache task | ||
2404 | output that can then be shared between build machines. | ||
2405 | </para> | ||
2406 | </note> | ||
2407 | Since <filename>STAMPS_DIR</filename> is usually a subdirectory | ||
2408 | of <filename>TMPDIR</filename>, removing | ||
2409 | <filename>TMPDIR</filename> will also remove | ||
2410 | <filename>STAMPS_DIR</filename>, which means tasks will | ||
2411 | properly be rerun to repopulate <filename>TMPDIR</filename>. | ||
2412 | </para> | ||
2413 | |||
2414 | <para> | ||
2415 | If you want some task to always be considered "out of date", | ||
2416 | you can mark it with the | ||
2417 | <ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink> | ||
2418 | varflag. | ||
2419 | If some other task depends on such a task, then that task will | ||
2420 | also always be considered out of date, which might not be what | ||
2421 | you want. | ||
2422 | </para> | ||
2423 | |||
2424 | <para> | ||
2425 | For details on how to view information about a task's | ||
2426 | signature, see the | ||
2427 | "<ulink url='&YOCTO_DOCS_REF_URL;#usingpoky-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</ulink>" | ||
2428 | section in the Yocto Project Reference Manual. | ||
2429 | </para> | ||
2430 | </section> | ||
2431 | |||
2432 | <section id='setscene-tasks-and-shared-state'> | ||
2433 | <title>Setscene Tasks and Shared State</title> | ||
2434 | |||
2435 | <para> | ||
2436 | The description of tasks so far assumes that BitBake needs to | ||
2437 | build everything and there are no prebuilt objects available. | ||
2438 | BitBake does support skipping tasks if prebuilt objects are | ||
2439 | available. | ||
2440 | These objects are usually made available in the form of a | ||
2441 | shared state (sstate) cache. | ||
2442 | <note> | ||
2443 | For information on variables affecting sstate, see the | ||
2444 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink> | ||
2445 | and | ||
2446 | <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></ulink> | ||
2447 | variables. | ||
2448 | </note> | ||
2449 | </para> | ||
2450 | |||
2451 | <para> | ||
2452 | The idea of a setscene task (i.e | ||
2453 | <filename>do_</filename><replaceable>taskname</replaceable><filename>_setscene</filename>) | ||
2454 | is a version of the task where | ||
2455 | instead of building something, BitBake can skip to the end | ||
2456 | result and simply place a set of files into specific locations | ||
2457 | as needed. | ||
2458 | In some cases, it makes sense to have a setscene task variant | ||
2459 | (e.g. generating package files in the | ||
2460 | <filename>do_package_write_*</filename> task). | ||
2461 | In other cases, it does not make sense, (e.g. a | ||
2462 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> | ||
2463 | task or | ||
2464 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> | ||
2465 | task) since the work involved would be equal to or greater than | ||
2466 | the underlying task. | ||
2467 | </para> | ||
2468 | |||
2469 | <para> | ||
2470 | In the OpenEmbedded build system, the common tasks that have | ||
2471 | setscene variants are | ||
2472 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>, | ||
2473 | <filename>do_package_write_*</filename>, | ||
2474 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-deploy'><filename>do_deploy</filename></ulink>, | ||
2475 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-packagedata'><filename>do_packagedata</filename></ulink>, | ||
2476 | and | ||
2477 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink>. | ||
2478 | Notice that these are most of the tasks whose output is an | ||
2479 | end result. | ||
2480 | </para> | ||
2481 | |||
2482 | <para> | ||
2483 | The OpenEmbedded build system has knowledge of the relationship | ||
2484 | between these tasks and other tasks that precede them. | ||
2485 | For example, if BitBake runs | ||
2486 | <filename>do_populate_sysroot_setscene</filename> for | ||
2487 | something, there is little point in running any of the | ||
2488 | <filename>do_fetch</filename>, <filename>do_unpack</filename>, | ||
2489 | <filename>do_patch</filename>, | ||
2490 | <filename>do_configure</filename>, | ||
2491 | <filename>do_compile</filename>, and | ||
2492 | <filename>do_install</filename> tasks. | ||
2493 | However, if <filename>do_package</filename> needs to be run, | ||
2494 | BitBake would need to run those other tasks. | ||
2495 | </para> | ||
2496 | |||
2497 | <para> | ||
2498 | It becomes more complicated if everything can come from an | ||
2499 | sstate cache because some objects are simply not required at | ||
2500 | all. | ||
2501 | For example, you do not need a compiler or native tools, such | ||
2502 | as quilt, if there is nothing to compile or patch. | ||
2503 | If the <filename>do_package_write_*</filename> packages are | ||
2504 | available from sstate, BitBake does not need the | ||
2505 | <filename>do_package</filename> task data. | ||
2506 | </para> | ||
2507 | |||
2508 | <para> | ||
2509 | To handle all these complexities, BitBake runs in two phases. | ||
2510 | The first is the "setscene" stage. | ||
2511 | During this stage, BitBake first checks the sstate cache for | ||
2512 | any targets it is planning to build. | ||
2513 | BitBake does a fast check to see if the object exists rather | ||
2514 | than a complete download. | ||
2515 | If nothing exists, the second phase, which is the setscene | ||
2516 | stage, completes and the main build proceeds. | ||
2517 | </para> | ||
2518 | |||
2519 | <para> | ||
2520 | If objects are found in the sstate cache, the OpenEmbedded | ||
2521 | build system works backwards from the end targets specified | ||
2522 | by the user. | ||
2523 | For example, if an image is being built, the OpenEmbedded build | ||
2524 | system first looks for the packages needed for that image and | ||
2525 | the tools needed to construct an image. | ||
2526 | If those are available, the compiler is not needed. | ||
2527 | Thus, the compiler is not even downloaded. | ||
2528 | If something was found to be unavailable, or the download or | ||
2529 | setscene task fails, the OpenEmbedded build system then tries | ||
2530 | to install dependencies, such as the compiler, from the cache. | ||
2531 | </para> | ||
2532 | |||
2533 | <para> | ||
2534 | The availability of objects in the sstate cache is handled by | ||
2535 | the function specified by the | ||
2536 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHCHECK_FUNCTION'><filename>BB_HASHCHECK_FUNCTION</filename></ulink> | ||
2537 | variable and returns a list of the objects that are available. | ||
2538 | The function specified by the | ||
2539 | <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_SETSCENE_DEPVALID'><filename>BB_SETSCENE_DEPVALID</filename></ulink> | ||
2540 | variable is the function that determines whether a given | ||
2541 | dependency needs to be followed, and whether for any given | ||
2542 | relationship the function needs to be passed. | ||
2543 | The function returns a True or False value. | ||
2544 | </para> | ||
2545 | </section> | ||
2546 | </section> | ||
2547 | |||
2548 | <section id='images-dev-environment'> | ||
2549 | <title>Images</title> | ||
2550 | |||
2551 | <para> | ||
2552 | The images produced by the OpenEmbedded build system | ||
2553 | are compressed forms of the | ||
2554 | root filesystem that are ready to boot on a target device. | ||
2555 | You can see from the | ||
2556 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> | ||
2557 | that BitBake output, in part, consists of images. | ||
2558 | This section is going to look more closely at this output: | ||
2559 | <imagedata fileref="figures/images.png" align="center" width="5.5in" depth="5.5in" /> | ||
2560 | </para> | ||
2561 | |||
2562 | <para> | ||
2563 | For a list of example images that the Yocto Project provides, | ||
2564 | see the | ||
2565 | "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" | ||
2566 | chapter in the Yocto Project Reference Manual. | ||
2567 | </para> | ||
2568 | |||
2569 | <para> | ||
2570 | Images are written out to the | ||
2571 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> | ||
2572 | inside the | ||
2573 | <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename> | ||
2574 | folder as shown in the figure. | ||
2575 | This folder contains any files expected to be loaded on the | ||
2576 | target device. | ||
2577 | The | ||
2578 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink> | ||
2579 | variable points to the <filename>deploy</filename> directory, | ||
2580 | while the | ||
2581 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></ulink> | ||
2582 | variable points to the appropriate directory containing images for | ||
2583 | the current configuration. | ||
2584 | <itemizedlist> | ||
2585 | <listitem><para><filename><replaceable>kernel-image</replaceable></filename>: | ||
2586 | A kernel binary file. | ||
2587 | The | ||
2588 | <ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></ulink> | ||
2589 | variable setting determines the naming scheme for the | ||
2590 | kernel image file. | ||
2591 | Depending on that variable, the file could begin with | ||
2592 | a variety of naming strings. | ||
2593 | The <filename>deploy/images/<replaceable>machine</replaceable></filename> | ||
2594 | directory can contain multiple image files for the | ||
2595 | machine.</para></listitem> | ||
2596 | <listitem><para><filename><replaceable>root-filesystem-image</replaceable></filename>: | ||
2597 | Root filesystems for the target device (e.g. | ||
2598 | <filename>*.ext3</filename> or <filename>*.bz2</filename> | ||
2599 | files). | ||
2600 | The | ||
2601 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink> | ||
2602 | variable setting determines the root filesystem image | ||
2603 | type. | ||
2604 | The <filename>deploy/images/<replaceable>machine</replaceable></filename> | ||
2605 | directory can contain multiple root filesystems for the | ||
2606 | machine.</para></listitem> | ||
2607 | <listitem><para><filename><replaceable>kernel-modules</replaceable></filename>: | ||
2608 | Tarballs that contain all the modules built for the kernel. | ||
2609 | Kernel module tarballs exist for legacy purposes and | ||
2610 | can be suppressed by setting the | ||
2611 | <ulink url='&YOCTO_DOCS_REF_URL;#var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></ulink> | ||
2612 | variable to "0". | ||
2613 | The <filename>deploy/images/<replaceable>machine</replaceable></filename> | ||
2614 | directory can contain multiple kernel module tarballs | ||
2615 | for the machine.</para></listitem> | ||
2616 | <listitem><para><filename><replaceable>bootloaders</replaceable></filename>: | ||
2617 | Bootloaders supporting the image, if applicable to the | ||
2618 | target machine. | ||
2619 | The <filename>deploy/images/<replaceable>machine</replaceable></filename> | ||
2620 | directory can contain multiple bootloaders for the | ||
2621 | machine.</para></listitem> | ||
2622 | <listitem><para><filename><replaceable>symlinks</replaceable></filename>: | ||
2623 | The <filename>deploy/images/<replaceable>machine</replaceable></filename> | ||
2624 | folder contains | ||
2625 | a symbolic link that points to the most recently built file | ||
2626 | for each machine. | ||
2627 | These links might be useful for external scripts that | ||
2628 | need to obtain the latest version of each file. | ||
2629 | </para></listitem> | ||
2630 | </itemizedlist> | ||
2631 | </para> | ||
2632 | </section> | ||
2633 | |||
2634 | <section id='sdk-dev-environment'> | ||
2635 | <title>Application Development SDK</title> | ||
2636 | |||
2637 | <para> | ||
2638 | In the | ||
2639 | <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, | ||
2640 | the output labeled "Application Development SDK" represents an | ||
2641 | SDK. | ||
2642 | The SDK generation process differs depending on whether you build | ||
2643 | a standard SDK | ||
2644 | (e.g. <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>) | ||
2645 | or an extensible SDK | ||
2646 | (e.g. <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>). | ||
2647 | This section is going to take a closer look at this output: | ||
2648 | <imagedata fileref="figures/sdk.png" align="center" width="9in" depth="7.25in" /> | ||
2649 | </para> | ||
2650 | |||
2651 | <para> | ||
2652 | The specific form of this output is a self-extracting | ||
2653 | SDK installer (<filename>*.sh</filename>) that, when run, | ||
2654 | installs the SDK, which consists of a cross-development | ||
2655 | toolchain, a set of libraries and headers, and an SDK | ||
2656 | environment setup script. | ||
2657 | Running this installer essentially sets up your | ||
2658 | cross-development environment. | ||
2659 | You can think of the cross-toolchain as the "host" | ||
2660 | part because it runs on the SDK machine. | ||
2661 | You can think of the libraries and headers as the "target" | ||
2662 | part because they are built for the target hardware. | ||
2663 | The environment setup script is added so that you can initialize | ||
2664 | the environment before using the tools. | ||
2665 | </para> | ||
2666 | |||
2667 | <note><title>Notes</title> | ||
2668 | <itemizedlist> | ||
2669 | <listitem><para> | ||
2670 | The Yocto Project supports several methods by which you can | ||
2671 | set up this cross-development environment. | ||
2672 | These methods include downloading pre-built SDK installers | ||
2673 | or building and installing your own SDK installer. | ||
2674 | </para></listitem> | ||
2675 | <listitem><para> | ||
2676 | For background information on cross-development toolchains | ||
2677 | in the Yocto Project development environment, see the | ||
2678 | "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>" | ||
2679 | section in the Yocto Project Reference Manual. | ||
2680 | </para></listitem> | ||
2681 | <listitem><para> | ||
2682 | For information on setting up a cross-development | ||
2683 | environment, see the | ||
2684 | <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink> | ||
2685 | manual. | ||
2686 | </para></listitem> | ||
2687 | </itemizedlist> | ||
2688 | </note> | ||
2689 | |||
2690 | <para> | ||
2691 | Once built, the SDK installers are written out to the | ||
2692 | <filename>deploy/sdk</filename> folder inside the | ||
2693 | <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> | ||
2694 | as shown in the figure at the beginning of this section. | ||
2695 | Depending on the type of SDK, several variables exist that help | ||
2696 | configure these files. | ||
2697 | The following list shows the variables associated with a standard | ||
2698 | SDK: | ||
2699 | <itemizedlist> | ||
2700 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>: | ||
2701 | Points to the <filename>deploy</filename> | ||
2702 | directory.</para></listitem> | ||
2703 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDKMACHINE'><filename>SDKMACHINE</filename></ulink>: | ||
2704 | Specifies the architecture of the machine | ||
2705 | on which the cross-development tools are run to | ||
2706 | create packages for the target hardware. | ||
2707 | </para></listitem> | ||
2708 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></ulink>: | ||
2709 | Lists the features to include in the "target" part | ||
2710 | of the SDK. | ||
2711 | </para></listitem> | ||
2712 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></ulink>: | ||
2713 | Lists packages that make up the host | ||
2714 | part of the SDK (i.e. the part that runs on | ||
2715 | the <filename>SDKMACHINE</filename>). | ||
2716 | When you use | ||
2717 | <filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename> | ||
2718 | to create the SDK, a set of default packages | ||
2719 | apply. | ||
2720 | This variable allows you to add more packages. | ||
2721 | </para></listitem> | ||
2722 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></ulink>: | ||
2723 | Lists packages that make up the target part | ||
2724 | of the SDK (i.e. the part built for the | ||
2725 | target hardware). | ||
2726 | </para></listitem> | ||
2727 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDKPATH'><filename>SDKPATH</filename></ulink>: | ||
2728 | Defines the default SDK installation path offered by the | ||
2729 | installation script. | ||
2730 | </para></listitem> | ||
2731 | </itemizedlist> | ||
2732 | This next list, shows the variables associated with an extensible | ||
2733 | SDK: | ||
2734 | <itemizedlist> | ||
2735 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></ulink>: | ||
2736 | Points to the <filename>deploy</filename> directory. | ||
2737 | </para></listitem> | ||
2738 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></ulink>: | ||
2739 | Controls whether or not shared state artifacts are copied | ||
2740 | into the extensible SDK. | ||
2741 | By default, all required shared state artifacts are copied | ||
2742 | into the SDK. | ||
2743 | </para></listitem> | ||
2744 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></ulink>: | ||
2745 | Specifies whether or not packagedata will be included in | ||
2746 | the extensible SDK for all recipes in the "world" target. | ||
2747 | </para></listitem> | ||
2748 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INCLUDE_TOOLCHAIN'><filename>SDK_INCLUDE_TOOLCHAIN</filename></ulink>: | ||
2749 | Specifies whether or not the toolchain will be included | ||
2750 | when building the extensible SDK. | ||
2751 | </para></listitem> | ||
2752 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></ulink>: | ||
2753 | A list of variables allowed through from the build system | ||
2754 | configuration into the extensible SDK configuration. | ||
2755 | </para></listitem> | ||
2756 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></ulink>: | ||
2757 | A list of variables not allowed through from the build | ||
2758 | system configuration into the extensible SDK configuration. | ||
2759 | </para></listitem> | ||
2760 | <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></ulink>: | ||
2761 | A list of classes to remove from the | ||
2762 | <ulink url='&YOCTO_DOCS_REF_URL;#var-INHERIT'><filename>INHERIT</filename></ulink> | ||
2763 | value globally within the extensible SDK configuration. | ||
2764 | </para></listitem> | ||
2765 | </itemizedlist> | ||
2766 | </para> | ||
2767 | </section> | ||
2768 | </section> | ||
2769 | |||
2770 | </chapter> | ||
2771 | <!-- | ||
2772 | vim: expandtab tw=80 ts=4 | ||
2773 | --> | ||