path: root/documentation/dev-manual/dev-manual-newbie.xml
diff options
Diffstat (limited to 'documentation/dev-manual/dev-manual-newbie.xml')
1 files changed, 1687 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml
new file mode 100644
index 0000000..37fa5af
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-newbie.xml
@@ -0,0 +1,1687 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
5<chapter id='dev-manual-newbie'>
7<title>The Yocto Project Open Source Development Environment</title>
10 This chapter helps you understand the Yocto Project as an open source development project.
11 In general, working in an open source environment is very different from working in a
12 closed, proprietary environment.
13 Additionally, the Yocto Project uses specific tools and constructs as part of its development
14 environment.
15 This chapter specifically addresses open source philosophy, using the
16 Yocto Project in a team environment, source repositories, Yocto Project
17 terms, licensing, the open source distributed version control system Git,
18 workflows, bug tracking, and how to submit changes.
21<section id='open-source-philosophy'>
22 <title>Open Source Philosophy</title>
24 <para>
25 Open source philosophy is characterized by software development directed by peer production
26 and collaboration through an active community of developers.
27 Contrast this to the more standard centralized development models used by commercial software
28 companies where a finite set of developers produces a product for sale using a defined set
29 of procedures that ultimately result in an end product whose architecture and source material
30 are closed to the public.
31 </para>
33 <para>
34 Open source projects conceptually have differing concurrent agendas, approaches, and production.
35 These facets of the development process can come from anyone in the public (community) that has a
36 stake in the software project.
37 The open source environment contains new copyright, licensing, domain, and consumer issues
38 that differ from the more traditional development environment.
39 In an open source environment, the end product, source material, and documentation are
40 all available to the public at no cost.
41 </para>
43 <para>
44 A benchmark example of an open source project is the Linux Kernel, which was initially conceived
45 and created by Finnish computer science student Linus Torvalds in 1991.
46 Conversely, a good example of a non-open source project is the
47 <trademark class='registered'>Windows</trademark> family of operating
48 systems developed by <trademark class='registered'>Microsoft</trademark> Corporation.
49 </para>
51 <para>
52 Wikipedia has a good historical description of the Open Source Philosophy
53 <ulink url=''>here</ulink>.
54 You can also find helpful information on how to participate in the Linux Community
55 <ulink url=''>here</ulink>.
56 </para>
59<section id="usingpoky-changes-collaborate">
60 <title>Using the Yocto Project in a Team Environment</title>
62 <para>
63 It might not be immediately clear how you can use the Yocto
64 Project in a team environment, or scale it for a large team of
65 developers.
66 One of the strengths of the Yocto Project is that it is extremely
67 flexible.
68 Thus, you can adapt it to many different use cases and scenarios.
69 However, these characteristics can cause a struggle if you are trying
70 to create a working setup that scales across a large team.
71 </para>
73 <para>
74 To help with these types of situations, this section presents
75 some of the project's most successful experiences,
76 practices, solutions, and available technologies that work well.
77 Keep in mind, the information here is a starting point.
78 You can build off it and customize it to fit any
79 particular working environment and set of practices.
80 </para>
82 <section id='best-practices-system-configurations'>
83 <title>System Configurations</title>
85 <para>
86 Systems across a large team should meet the needs of
87 two types of developers: those working on the contents of the
88 operating system image itself and those developing applications.
89 Regardless of the type of developer, their workstations must
90 be both reasonably powerful and run Linux.
91 </para>
93 <section id='best-practices-application-development'>
94 <title>Application Development</title>
96 <para>
97 For developers who mainly do application level work
98 on top of an existing software stack,
99 here are some practices that work best:
100 <itemizedlist>
101 <listitem><para>Use a pre-built toolchain that
102 contains the software stack itself.
103 Then, develop the application code on top of the
104 stack.
105 This method works well for small numbers of relatively
106 isolated applications.</para></listitem>
107 <listitem><para>When possible, use the Yocto Project
108 plug-in for the <trademark class='trade'>Eclipse</trademark> IDE
109 and other pieces of Application Development
110 Technology (ADT).
111 For more information, see the
112 "<link linkend='application-development-workflow'>Application
113 Development Workflow</link>" section as well as the
114 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
115 </para></listitem>
116 <listitem><para>Keep your cross-development toolchains
117 updated.
118 You can do this through provisioning either as new
119 toolchain downloads or as updates through a package
120 update mechanism using <filename>opkg</filename>
121 to provide updates to an existing toolchain.
122 The exact mechanics of how and when to do this are a
123 question for local policy.</para></listitem>
124 <listitem><para>Use multiple toolchains installed locally
125 into different locations to allow development across
126 versions.</para></listitem>
127 </itemizedlist>
128 </para>
129 </section>
131 <section id='best-practices-core-system-development'>
132 <title>Core System Development</title>
134 <para>
135 For core system development, it is often best to have the
136 build system itself available on the developer workstations
137 so developers can run their own builds and directly
138 rebuild the software stack.
139 You should keep the core system unchanged as much as
140 possible and do your work in layers on top of the core system.
141 Doing so gives you a greater level of portability when
142 upgrading to new versions of the core system or Board
143 Support Packages (BSPs).
144 You can share layers amongst the developers of a particular
145 project and contain the policy configuration that defines
146 the project.
147 </para>
149 <para>
150 Aside from the previous best practices, there exists a number
151 of tips and tricks that can help speed up core development
152 projects:
153 <itemizedlist>
154 <listitem><para>Use a
155 <ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>
156 (sstate) among groups of developers who are on a
157 fast network.
158 The best way to share sstate is through a
159 Network File System (NFS) share.
160 The first user to build a given component for the
161 first time contributes that object to the sstate,
162 while subsequent builds from other developers then
163 reuse the object rather than rebuild it themselves.
164 </para>
165 <para>Although it is possible to use other protocols for the
166 sstate such as HTTP and FTP, you should avoid these.
167 Using HTTP limits the sstate to read-only and
168 FTP provides poor performance.
169 </para></listitem>
170 <listitem><para>Have autobuilders contribute to the sstate
171 pool similarly to how the developer workstations
172 contribute.
173 For information, see the
174 "<link linkend='best-practices-autobuilders'>Autobuilders</link>"
175 section.</para></listitem>
176 <listitem><para>Build stand-alone tarballs that contain
177 "missing" system requirements if for some reason
178 developer workstations do not meet minimum system
179 requirements such as latest Python versions,
180 <filename>chrpath</filename>, or other tools.
181 You can install and relocate the tarball exactly as you
182 would the usual cross-development toolchain so that
183 all developers can meet minimum version requirements
184 on most distributions.</para></listitem>
185 <listitem><para>Use a small number of shared,
186 high performance systems for testing purposes
187 (e.g. dual, six-core Xeons with 24 Gbytes of RAM
188 and plenty of disk space).
189 Developers can use these systems for wider, more
190 extensive testing while they continue to develop
191 locally using their primary development system.
192 </para></listitem>
193 <listitem><para>Enable the PR Service when package feeds
194 need to be incremental with continually increasing
195 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'>PR</ulink>
196 values.
197 Typically, this situation occurs when you use or
198 publish package feeds and use a shared state.
199 You should enable the PR Service for all users who
200 use the shared state pool.
201 For more information on the PR Service, see the
202 "<link linkend='working-with-a-pr-service'>Working With a PR Service</link>".
203 </para></listitem>
204 </itemizedlist>
205 </para>
206 </section>
207 </section>
209 <section id='best-practices-source-control-management'>
210 <title>Source Control Management (SCM)</title>
212 <para>
213 Keeping your
214 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
215 and any software you are developing under the
216 control of an SCM system that is compatible
217 with the OpenEmbedded build system is advisable.
218 Of the SCMs BitBake supports, the
219 Yocto Project team strongly recommends using
220 <link linkend='git'>Git</link>.
221 Git is a distributed system that is easy to backup,
222 allows you to work remotely, and then connects back to the
223 infrastructure.
224 <note>
225 For information about BitBake, see the
226 <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
227 </note>
228 </para>
230 <para>
231 It is relatively easy to set up Git services and create
232 infrastructure like
233 <ulink url='&YOCTO_GIT_URL;'></ulink>,
234 which is based on server software called
235 <filename>gitolite</filename> with <filename>cgit</filename>
236 being used to generate the web interface that lets you view the
237 repositories.
238 The <filename>gitolite</filename> software identifies users
239 using SSH keys and allows branch-based
240 access controls to repositories that you can control as little
241 or as much as necessary.
242 </para>
244 <note>
245 The setup of these services is beyond the scope of this manual.
246 However, sites such as these exist that describe how to perform
247 setup:
248 <itemizedlist>
249 <listitem><para><ulink url=''>Git documentation</ulink>:
250 Describes how to install <filename>gitolite</filename>
251 on the server.</para></listitem>
252 <listitem><para><ulink url=''>The <filename>gitolite</filename> master index</ulink>:
253 All topics for <filename>gitolite</filename>.
254 </para></listitem>
255 <listitem><para><ulink url=',_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
256 Documentation on how to create interfaces and frontends
257 for Git.</para></listitem>
258 </itemizedlist>
259 </note>
260 </section>
262 <section id='best-practices-autobuilders'>
263 <title>Autobuilders</title>
265 <para>
266 Autobuilders are often the core of a development project.
267 It is here that changes from individual developers are brought
268 together and centrally tested and subsequent decisions about
269 releases can be made.
270 Autobuilders also allow for "continuous integration" style
271 testing of software components and regression identification
272 and tracking.
273 </para>
275 <para>
276 See "<ulink url=''>Yocto Project Autobuilder</ulink>"
277 for more information and links to buildbot.
278 The Yocto Project team has found this implementation
279 works well in this role.
280 A public example of this is the Yocto Project
281 Autobuilders, which we use to test the overall health of the
282 project.
283 </para>
285 <para>
286 The features of this system are:
287 <itemizedlist>
288 <listitem><para>Highlights when commits break the build.
289 </para></listitem>
290 <listitem><para>Populates an sstate cache from which
291 developers can pull rather than requiring local
292 builds.</para></listitem>
293 <listitem><para>Allows commit hook triggers,
294 which trigger builds when commits are made.
295 </para></listitem>
296 <listitem><para>Allows triggering of automated image booting
297 and testing under the QuickEMUlator (QEMU).
298 </para></listitem>
299 <listitem><para>Supports incremental build testing and
300 from-scratch builds.</para></listitem>
301 <listitem><para>Shares output that allows developer
302 testing and historical regression investigation.
303 </para></listitem>
304 <listitem><para>Creates output that can be used for releases.
305 </para></listitem>
306 <listitem><para>Allows scheduling of builds so that resources
307 can be used efficiently.</para></listitem>
308 </itemizedlist>
309 </para>
310 </section>
312 <section id='best-practices-policies-and-change-flow'>
313 <title>Policies and Change Flow</title>
315 <para>
316 The Yocto Project itself uses a hierarchical structure and a
317 pull model.
318 Scripts exist to create and send pull requests
319 (i.e. <filename>create-pull-request</filename> and
320 <filename>send-pull-request</filename>).
321 This model is in line with other open source projects where
322 maintainers are responsible for specific areas of the project
323 and a single maintainer handles the final "top-of-tree" merges.
324 </para>
326 <note>
327 You can also use a more collective push model.
328 The <filename>gitolite</filename> software supports both the
329 push and pull models quite easily.
330 </note>
332 <para>
333 As with any development environment, it is important
334 to document the policy used as well as any main project
335 guidelines so they are understood by everyone.
336 It is also a good idea to have well structured
337 commit messages, which are usually a part of a project's
338 guidelines.
339 Good commit messages are essential when looking back in time and
340 trying to understand why changes were made.
341 </para>
343 <para>
344 If you discover that changes are needed to the core layer of the
345 project, it is worth sharing those with the community as soon
346 as possible.
347 Chances are if you have discovered the need for changes, someone
348 else in the community needs them also.
349 </para>
350 </section>
352 <section id='best-practices-summary'>
353 <title>Summary</title>
355 <para>
356 This section summarizes the key recommendations described in the
357 previous sections:
358 <itemizedlist>
359 <listitem><para>Use <link linkend='git'>Git</link>
360 as the source control system.</para></listitem>
361 <listitem><para>Maintain your Metadata in layers that make sense
362 for your situation.
363 See the "<link linkend='understanding-and-creating-layers'>Understanding
364 and Creating Layers</link>" section for more information on
365 layers.</para></listitem>
366 <listitem><para>
367 Separate the project's Metadata and code by using
368 separate Git repositories.
369 See the
370 "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>"
371 section for information on these repositories.
372 See the
373 "<link linkend='getting-setup'>Getting Set Up</link>"
374 section for information on how to set up local Git
375 repositories for related upstream Yocto Project
376 Git repositories.
377 </para></listitem>
378 <listitem><para>Set up the directory for the shared state cache
379 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
380 where it makes sense.
381 For example, set up the sstate cache on a system used
382 by developers in the same organization and share the
383 same source directories on their machines.
384 </para></listitem>
385 <listitem><para>Set up an Autobuilder and have it populate the
386 sstate cache and source directories.</para></listitem>
387 <listitem><para>The Yocto Project community encourages you
388 to send patches to the project to fix bugs or add features.
389 If you do submit patches, follow the project commit
390 guidelines for writing good commit messages.
391 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
392 section.</para></listitem>
393 <listitem><para>Send changes to the core sooner than later
394 as others are likely to run into the same issues.
395 For some guidance on mailing lists to use, see the list in the
396 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
397 section.
398 For a description of the available mailing lists, see the
399 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
400 section in the Yocto Project Reference Manual.
401 </para></listitem>
402 </itemizedlist>
403 </para>
404 </section>
407<section id='yocto-project-repositories'>
408 <title>Yocto Project Source Repositories</title>
410 <para>
411 The Yocto Project team maintains complete source repositories for all
412 Yocto Project files at
413 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>.
414 This web-based source code browser is organized into categories by
415 function such as IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and
416 so forth.
417 From the interface, you can click on any particular item in the "Name"
418 column and see the URL at the bottom of the page that you need to clone
419 a Git repository for that particular item.
420 Having a local Git repository of the
421 <link linkend='source-directory'>Source Directory</link>, which is
422 usually named "poky", allows
423 you to make changes, contribute to the history, and ultimately enhance
424 the Yocto Project's tools, Board Support Packages, and so forth.
425 </para>
427 <para>
428 For any supported release of Yocto Project, you can also go to the
429 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
430 select the "Downloads" tab and get a released tarball of the
431 <filename>poky</filename> repository or any supported BSP tarballs.
432 Unpacking these tarballs gives you a snapshot of the released
433 files.
434 <note><title>Notes</title>
435 <itemizedlist>
436 <listitem><para>
437 The recommended method for setting up the Yocto Project
438 <link linkend='source-directory'>Source Directory</link>
439 and the files for supported BSPs
440 (e.g., <filename>meta-intel</filename>) is to use
441 <link linkend='git'>Git</link> to create a local copy of
442 the upstream repositories.
443 </para></listitem>
444 <listitem><para>
445 Be sure to always work in matching branches for both
446 the <filename>meta-intel</filename> repository and the
447 <link linkend='source-directory'>Source Directory</link>
448 (i.e. <filename>poky</filename>) repository.
449 For example, if you have checked out the "master" branch
450 of <filename>poky</filename> and you are going to use
451 <filename>meta-intel</filename>, be sure to checkout the
452 "master" branch of <filename>meta-intel</filename>.
453 </para></listitem>
454 </itemizedlist>
455 </note>
456 </para>
458 <para>
459 In summary, here is where you can get the project files needed for development:
460 <itemizedlist>
461 <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis>
462 This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto
463 Metadata Layers.
464 You can create local copies of Git repositories for each of these areas.</para>
465 <para>
466 <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
467 </para></listitem>
468 <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis>
469 This is an index of releases such as
470 the <trademark class='trade'>Eclipse</trademark>
471 Yocto Plug-in, miscellaneous support, Poky, Pseudo, installers for cross-development toolchains,
472 and all released versions of Yocto Project in the form of images or tarballs.
473 Downloading and extracting these files does not produce a local copy of the
474 Git repository but rather a snapshot of a particular release or image.</para>
475 <para>
476 <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" />
477 </para></listitem>
478 <listitem><para><emphasis>"Downloads" page for the
479 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:</emphasis>
480 Access this page by going to the website and then selecting
481 the "Downloads" tab.
482 This page allows you to download any Yocto Project
483 release or Board Support Package (BSP) in tarball form.
484 The tarballs are similar to those found in the
485 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para>
486 <para>
487 <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
488 </para></listitem>
489 </itemizedlist>
490 </para>
493<section id='yocto-project-terms'>
494 <title>Yocto Project Terms</title>
496 <para>
497 Following is a list of terms and definitions users new to the Yocto Project development
498 environment might find helpful.
499 While some of these terms are universal, the list includes them just in case:
500 <itemizedlist>
501 <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to
502 a recipe file.
503 Append files are known as BitBake append files and <filename>.bbappend</filename> files.
504 The OpenEmbedded build system expects every append file to have a corresponding
505 recipe (<filename>.bb</filename>) file.
506 Furthermore, the append file and corresponding recipe file
507 must use the same root filename.
508 The filenames can differ only in the file type suffix used (e.g.
509 <filename></filename> and <filename>formfactor_0.0.bbappend</filename>).
510 </para>
511 <para>Information in append files overrides the information in the similarly-named recipe file.
512 For an example of an append file in use, see the
513 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section.
514 </para></listitem>
515 <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis>
516 The task executor and scheduler used by the OpenEmbedded build
517 system to build images.
518 For more information on BitBake, see the
519 <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>.
520 </para></listitem>
521 <listitem>
522 <para id='build-directory'><emphasis>Build Directory:</emphasis>
523 This term refers to the area used by the OpenEmbedded build
524 system for builds.
525 The area is created when you <filename>source</filename> the
526 setup environment script that is found in the Source Directory
527 (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
528 or
529 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
530 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
531 variable points to the Build Directory.</para>
533 <para>
534 You have a lot of flexibility when creating the Build
535 Directory.
536 Following are some examples that show how to create the
537 directory.
538 The examples assume your
539 <link linkend='source-directory'>Source Directory</link> is
540 named <filename>poky</filename>:
541 <itemizedlist>
542 <listitem><para>Create the Build Directory inside your
543 Source Directory and let the name of the Build
544 Directory default to <filename>build</filename>:
545 <literallayout class='monospaced'>
546 $ cd $HOME/poky
547 $ source &OE_INIT_FILE;
548 </literallayout></para></listitem>
549 <listitem><para>Create the Build Directory inside your
550 home directory and specifically name it
551 <filename>test-builds</filename>:
552 <literallayout class='monospaced'>
553 $ cd $HOME
554 $ source poky/&OE_INIT_FILE; test-builds
555 </literallayout></para></listitem>
556 <listitem><para>
557 Provide a directory path and
558 specifically name the Build Directory.
559 Any intermediate folders in the pathname must
560 exist.
561 This next example creates a Build Directory named
562 <filename>YP-&POKYVERSION;</filename>
563 in your home directory within the existing
564 directory <filename>mybuilds</filename>:
565 <literallayout class='monospaced'>
566 $cd $HOME
567 $ source $HOME/poky/&OE_INIT_FILE; $HOME/mybuilds/YP-&POKYVERSION;
568 </literallayout></para></listitem>
569 </itemizedlist>
570 <note>
571 By default, the Build Directory contains
572 <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>,
573 which is a temporary directory the build system uses for
574 its work.
575 <filename>TMPDIR</filename> cannot be under NFS.
576 Thus, by default, the Build Directory cannot be under NFS.
577 However, if you need the Build Directory to be under NFS,
578 you can set this up by setting <filename>TMPDIR</filename>
579 in your <filename>local.conf</filename> file
580 to use a local drive.
581 Doing so effectively separates <filename>TMPDIR</filename>
582 from <filename>TOPDIR</filename>, which is the Build
583 Directory.
584 </note>
585 </para></listitem>
586 <listitem><para id='build-system-term'><emphasis>Build System:</emphasis>
587 In the context of the Yocto Project,
588 this term refers to the OpenEmbedded build system used by the project.
589 This build system is based on the project known as "Poky."
590 For some historical information about Poky, see the
591 <link linkend='poky'>Poky</link> term.
592 </para></listitem>
593 <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation
594 and inheritance so that commonly used patterns can be defined once and then easily used
595 in multiple recipes.
596 For reference information on the Yocto Project classes, see the
597 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes'>Classes</ulink>" chapter of the
598 Yocto Project Reference Manual.
599 Class files end with the <filename>.bbclass</filename> filename extension.
600 </para></listitem>
601 <listitem><para><emphasis>Configuration File:</emphasis>
602 Configuration information in various <filename>.conf</filename>
603 files provides global definitions of variables.
604 The <filename>conf/local.conf</filename> configuration file in
605 the
606 <link linkend='build-directory'>Build Directory</link>
607 contains user-defined variables that affect every build.
608 The <filename>meta-yocto/conf/distro/poky.conf</filename>
609 configuration file defines Yocto "distro" configuration
610 variables used only when building with this policy.
611 Machine configuration files, which
612 are located throughout the
613 <link linkend='source-directory'>Source Directory</link>, define
614 variables for specific hardware and are only used when building
615 for that target (e.g. the
616 <filename>machine/beaglebone.conf</filename> configuration
617 file defines variables for the Texas Instruments ARM Cortex-A8
618 development board).
619 Configuration files end with a <filename>.conf</filename>
620 filename extension.
621 </para></listitem>
622 <listitem><para id='cross-development-toolchain'>
623 <emphasis>Cross-Development Toolchain:</emphasis>
624 In general, a cross-development toolchain is a collection of
625 software development tools and utilities that run on one
626 architecture and allow you to develop software for a
627 different, or targeted, architecture.
628 These toolchains contain cross-compilers, linkers, and
629 debuggers that are specific to the target architecture.
630 </para>
632 <para>The Yocto Project supports two different cross-development
633 toolchains:
634 <itemizedlist>
635 <listitem><para>A toolchain only used by and within
636 BitBake when building an image for a target
637 architecture.</para></listitem>
638 <listitem><para>A relocatable toolchain used outside of
639 BitBake by developers when developing applications
640 that will run on a targeted device.
641 Sometimes this relocatable cross-development
642 toolchain is referred to as the meta-toolchain.
643 </para></listitem>
644 </itemizedlist>
645 </para>
647 <para>
648 Creation of these toolchains is simple and automated.
649 For information on toolchain concepts as they apply to the
650 Yocto Project, see the
651 "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>"
652 section in the Yocto Project Reference Manual.
653 You can also find more information on using the
654 relocatable toolchain in the
655 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project
656 Application Developer's Guide</ulink>.
657 </para></listitem>
658 <listitem><para><emphasis>Image:</emphasis>
659 An image is the result produced when BitBake processes a given
660 collection of recipes and related Metadata.
661 Images are the binary output that run on specific hardware or
662 QEMU and are used for specific use-cases.
663 For a list of the supported image types that the Yocto Project provides, see the
664 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
665 chapter in the Yocto Project Reference Manual.</para></listitem>
666 <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core,
667 a BSP, or an application stack.
668 For a discussion on BSP Layers, see the
669 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
670 section in the Yocto Project Board Support Packages (BSP)
671 Developer's Guide.</para></listitem>
672 <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
673 A term sometimes used for
674 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
675 </para></listitem>
676 <listitem><para id='metadata'><emphasis>Metadata:</emphasis>
677 The files that BitBake parses when building an image.
678 In general, Metadata includes recipes, classes, and
679 configuration files.
680 In the context of the kernel ("kernel Metadata"),
681 it refers to Metadata in the <filename>meta</filename>
682 branches of the kernel source Git repositories.
683 </para></listitem>
684 <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of Metadata originating
685 with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
686 This Metadata is found in the <filename>meta</filename> directory of the
687 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
688 <listitem><para><emphasis>Package:</emphasis>
689 In the context of the Yocto Project, this term refers a
690 recipe's packaged output produced by BitBake (i.e. a
691 "baked recipe").
692 A package is generally the compiled binaries produced from the
693 recipe's sources.
694 You "bake" something by running it through BitBake.</para>
695 <para>It is worth noting that the term "package" can, in general, have subtle
696 meanings. For example, the packages referred to in the
697 "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Packages</ulink>" section are
698 compiled binaries that when installed add functionality to your Linux
699 distribution.</para>
700 <para>Another point worth noting is that historically within the Yocto Project,
701 recipes were referred to as packages - thus, the existence of several BitBake
702 variables that are seemingly mis-named,
703 (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>,
704 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
705 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
706 </para></listitem>
707 <listitem><para><emphasis>Package Groups:</emphasis>
708 Arbitrary groups of software Recipes.
709 You use package groups to hold recipes that, when built,
710 usually accomplish a single task.
711 For example, a package group could contain the recipes for a
712 company’s proprietary or value-add software.
713 Or, the package group could contain the recipes that enable
714 graphics.
715 A package group is really just another recipe.
716 Because package group files are recipes, they end with the
717 <filename>.bb</filename> filename extension.</para></listitem>
718 <listitem><para id='poky'><emphasis>Poky:</emphasis> The term "poky" can mean several things.
719 In its most general sense, it is an open-source project that was initially developed
720 by OpenedHand. With OpenedHand, poky was developed off of the existing OpenEmbedded
721 build system becoming a build system for embedded images.
722 After Intel Corporation acquired OpenedHand, the project poky became the basis for
723 the Yocto Project's build system.</para>
724 <para>
725 Within the Yocto Project source repositories, <filename>poky</filename>
726 exists as a separate Git repository
727 that can be cloned to yield a local copy on the host system.
728 Thus, "poky" can refer to the local copy of the Source Directory used to develop within
729 the Yocto Project.</para></listitem>
730 <listitem><para><emphasis>Recipe:</emphasis>
731 A set of instructions for building packages.
732 A recipe describes where you get source code and which patches
733 to apply.
734 Recipes describe dependencies for libraries or for other
735 recipes, and they also contain configuration and compilation
736 options.
737 Recipes contain the logical unit of execution, the software
738 to build, the images to build, and use the
739 <filename>.bb</filename> file extension.
740 </para></listitem>
741 <listitem>
742 <para id='source-directory'><emphasis>Source Directory:</emphasis>
743 This term refers to the directory structure created as a result
744 of creating a local copy of the <filename>poky</filename> Git
745 repository <filename>git://</filename>
746 or expanding a released <filename>poky</filename> tarball.
747 <note>
748 Creating a local copy of the <filename>poky</filename>
749 Git repository is the recommended method for setting up
750 your Source Directory.
751 </note>
752 Sometimes you might hear the term "poky directory" used to refer
753 to this directory structure.
754 <note>
755 The OpenEmbedded build system does not support file or
756 directory names that contain spaces.
757 Be sure that the Source Directory you use does not contain
758 these types of names.
759 </note></para>
761 <para>The Source Directory contains BitBake, Documentation,
762 Metadata and other files that all support the Yocto Project.
763 Consequently, you must have the Source Directory in place on
764 your development system in order to do any development using
765 the Yocto Project.</para>
767 <para>When you create a local copy of the Git repository, you
768 can name the repository anything you like.
769 Throughout much of the documentation, "poky"
770 is used as the name of the top-level folder of the local copy of
771 the poky Git repository.
772 So, for example, cloning the <filename>poky</filename> Git
773 repository results in a local Git repository whose top-level
774 folder is also named "poky".</para>
776 <para>While it is not recommended that you use tarball expansion
777 to setup the Source Directory, if you do, the top-level
778 directory name of the Source Directory is derived from the
779 Yocto Project release tarball.
780 For example, downloading and unpacking
781 <filename>&YOCTO_POKY_TARBALL;</filename> results in a
782 Source Directory whose root folder is named
783 <filename>&YOCTO_POKY;</filename>.</para>
785 <para>It is important to understand the differences between the
786 Source Directory created by unpacking a released tarball as
787 compared to cloning
788 <filename>git://</filename>.
789 When you unpack a tarball, you have an exact copy of the files
790 based on the time of release - a fixed release point.
791 Any changes you make to your local files in the Source Directory
792 are on top of the release and will remain local only.
793 On the other hand, when you clone the <filename>poky</filename>
794 Git repository, you have an active development repository with
795 access to the upstream repository's branches and tags.
796 In this case, any local changes you make to the local
797 Source Directory can be later applied to active development
798 branches of the upstream <filename>poky</filename> Git
799 repository.</para>
801 <para>For more information on concepts related to Git
802 repositories, branches, and tags, see the
803 "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>"
804 section.</para></listitem>
805 <listitem><para><emphasis>Task:</emphasis>
806 A unit of execution for BitBake (e.g.
807 <filename>do_compile</filename>,
808 <filename>do_fetch</filename>, <filename>do_patch</filename>,
809 and so forth).
810 </para></listitem>
811 <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
812 that are not local to the development system but located in a master area that is controlled
813 by the maintainer of the source code.
814 For example, in order for a developer to work on a particular piece of code, they need to
815 first get a copy of it from an "upstream" source.</para></listitem>
816 </itemizedlist>
817 </para>
820<section id='licensing'>
821 <title>Licensing</title>
823 <para>
824 Because open source projects are open to the public, they have different licensing structures in place.
825 License evolution for both Open Source and Free Software has an interesting history.
826 If you are interested in this history, you can find basic information here:
827 <itemizedlist>
828 <listitem><para><ulink url=''>Open source license history</ulink>
829 </para></listitem>
830 <listitem><para><ulink url=''>Free software license
831 history</ulink></para></listitem>
832 </itemizedlist>
833 </para>
835 <para>
836 In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology
837 (MIT) License.
838 MIT licensing permits the reuse of software within proprietary software as long as the
839 license is distributed with that software.
840 MIT is also compatible with the GNU General Public License (GPL).
841 Patches to the Yocto Project follow the upstream licensing scheme.
842 You can find information on the MIT license at
843 <ulink url=''>here</ulink>.
844 You can find information on the GNU GPL <ulink url=''>
845 here</ulink>.
846 </para>
848 <para>
849 When you build an image using the Yocto Project, the build process uses a
850 known list of licenses to ensure compliance.
851 You can find this list in the
852 <link linkend='source-directory'>Source Directory</link> at
853 <filename>meta/files/common-licenses</filename>.
854 Once the build completes, the list of all licenses found and used during that build are
855 kept in the
856 <link linkend='build-directory'>Build Directory</link> at
857 <filename>tmp/deploy/licenses</filename>.
858 </para>
860 <para>
861 If a module requires a license that is not in the base list, the build process
862 generates a warning during the build.
863 These tools make it easier for a developer to be certain of the licenses with which
864 their shipped products must comply.
865 However, even with these tools it is still up to the developer to resolve potential licensing issues.
866 </para>
868 <para>
869 The base list of licenses used by the build process is a combination of the Software Package
870 Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects.
871 <ulink url=''>SPDX Group</ulink> is a working group of the Linux Foundation
872 that maintains a specification
873 for a standard format for communicating the components, licenses, and copyrights
874 associated with a software package.
875 <ulink url=''>OSI</ulink> is a corporation dedicated to the Open Source
876 Definition and the effort for reviewing and approving licenses that
877 conform to the Open Source Definition (OSD).
878 </para>
880 <para>
881 You can find a list of the combined SPDX and OSI licenses that the
882 Yocto Project uses in the
883 <filename>meta/files/common-licenses</filename> directory in your
884 <link linkend='source-directory'>Source Directory</link>.
885 </para>
887 <para>
888 For information that can help you maintain compliance with various
889 open source licensing during the lifecycle of a product created using
890 the Yocto Project, see the
891 "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>"
892 section.
893 </para>
896<section id='git'>
897 <title>Git</title>
899 <para>
900 The Yocto Project makes extensive use of Git,
901 which is a free, open source distributed version control system.
902 Git supports distributed development, non-linear development, and can handle large projects.
903 It is best that you have some fundamental understanding of how Git tracks projects and
904 how to work with Git if you are going to use the Yocto Project for development.
905 This section provides a quick overview of how Git works and provides you with a summary
906 of some essential Git commands.
907 </para>
909 <para>
910 For more information on Git, see
911 <ulink url=''></ulink>.
912 If you need to download Git, go to <ulink url=''></ulink>.
913 </para>
915 <section id='repositories-tags-and-branches'>
916 <title>Repositories, Tags, and Branches</title>
918 <para>
919 As mentioned earlier in the section
920 "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>",
921 the Yocto Project maintains source repositories at
922 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.
923 If you look at this web-interface of the repositories, each item is a separate
924 Git repository.
925 </para>
927 <para>
928 Git repositories use branching techniques that track content change (not files)
929 within a project (e.g. a new feature or updated documentation).
930 Creating a tree-like structure based on project divergence allows for excellent historical
931 information over the life of a project.
932 This methodology also allows for an environment from which you can do lots of
933 local experimentation on projects as you develop changes or new features.
934 </para>
936 <para>
937 A Git repository represents all development efforts for a given project.
938 For example, the Git repository <filename>poky</filename> contains all changes
939 and developments for Poky over the course of its entire life.
940 That means that all changes that make up all releases are captured.
941 The repository maintains a complete history of changes.
942 </para>
944 <para>
945 You can create a local copy of any repository by "cloning" it with the Git
946 <filename>clone</filename> command.
947 When you clone a Git repository, you end up with an identical copy of the
948 repository on your development system.
949 Once you have a local copy of a repository, you can take steps to develop locally.
950 For examples on how to clone Git repositories, see the
951 "<link linkend='getting-setup'>Getting Set Up</link>" section.
952 </para>
954 <para>
955 It is important to understand that Git tracks content change and
956 not files.
957 Git uses "branches" to organize different development efforts.
958 For example, the <filename>poky</filename> repository has
959 <filename>denzil</filename>, <filename>danny</filename>,
960 <filename>dylan</filename>, <filename>dora</filename>,
961 <filename>daisy</filename>, and <filename>master</filename> branches
962 among others.
963 You can see all the branches by going to
964 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
965 clicking on the
966 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
967 link beneath the "Branch" heading.
968 </para>
970 <para>
971 Each of these branches represents a specific area of development.
972 The <filename>master</filename> branch represents the current or most recent
973 development.
974 All other branches represent off-shoots of the <filename>master</filename>
975 branch.
976 </para>
978 <para>
979 When you create a local copy of a Git repository, the copy has the same set
980 of branches as the original.
981 This means you can use Git to create a local working area (also called a branch)
982 that tracks a specific development branch from the source Git repository.
983 in other words, you can define your local Git environment to work on any development
984 branch in the repository.
985 To help illustrate, here is a set of commands that creates a local copy of the
986 <filename>poky</filename> Git repository and then creates and checks out a local
987 Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development:
988 <literallayout class='monospaced'>
989 $ cd ~
990 $ git clone git://
991 $ cd poky
992 $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
993 </literallayout>
994 In this example, the name of the top-level directory of your local
995 <link linkend='source-directory'>Source Directory</link>
996 is "poky" and the name of that local working area (local branch)
997 you just created and checked out is "&DISTRO_NAME;".
998 The files in your local repository now reflect the same files that
999 are in the "&DISTRO_NAME;" development branch of the
1000 Yocto Project's "poky" upstream repository.
1001 It is important to understand that when you create and checkout a
1002 local working branch based on a branch name,
1003 your local environment matches the "tip" of that development branch
1004 at the time you created your local branch, which could be
1005 different from the files at the time of a similarly named release.
1006 In other words, creating and checking out a local branch based on
1007 the "&DISTRO_NAME;" branch name is not the same as
1008 cloning and checking out the "master" branch.
1009 Keep reading to see how you create a local snapshot of a Yocto
1010 Project Release.
1011 </para>
1013 <para>
1014 Git uses "tags" to mark specific changes in a repository.
1015 Typically, a tag is used to mark a special point such as the final
1016 change before a project is released.
1017 You can see the tags used with the <filename>poky</filename> Git
1018 repository by going to
1019 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
1020 clicking on the
1021 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
1022 link beneath the "Tag" heading.
1023 </para>
1025 <para>
1026 Some key tags are <filename>dylan-9.0.0</filename>,
1027 <filename>dora-10.0.0</filename>,
1028 and <filename>&DISTRO_NAME;-&POKYVERSION;</filename>.
1029 These tags represent Yocto Project releases.
1030 </para>
1032 <para>
1033 When you create a local copy of the Git repository, you also have access to all the
1034 tags.
1035 Similar to branches, you can create and checkout a local working Git branch based
1036 on a tag name.
1037 When you do this, you get a snapshot of the Git repository that reflects
1038 the state of the files when the change was made associated with that tag.
1039 The most common use is to checkout a working branch that matches a specific
1040 Yocto Project release.
1041 Here is an example:
1042 <literallayout class='monospaced'>
1043 $ cd ~
1044 $ git clone git://
1045 $ cd poky
1047 </literallayout>
1048 In this example, the name of the top-level directory of your local Yocto Project
1049 Files Git repository is <filename>poky</filename>.
1050 And, the name of the local branch you have created and checked out is
1051 <filename>my-&DISTRO_NAME;-&POKYVERSION;</filename>.
1052 The files in your repository now exactly match the Yocto Project &DISTRO;
1053 Release tag (<filename>&DISTRO_NAME;-&POKYVERSION;</filename>).
1054 It is important to understand that when you create and checkout a local
1055 working branch based on a tag, your environment matches a specific point
1056 in time and not the entire development branch.
1057 </para>
1058 </section>
1060 <section id='basic-commands'>
1061 <title>Basic Commands</title>
1063 <para>
1064 Git has an extensive set of commands that lets you manage changes and perform
1065 collaboration over the life of a project.
1066 Conveniently though, you can manage with a small set of basic operations and workflows
1067 once you understand the basic philosophy behind Git.
1068 You do not have to be an expert in Git to be functional.
1069 A good place to look for instruction on a minimal set of Git commands is
1070 <ulink url=''>here</ulink>.
1071 If you need to download Git, you can do so
1072 <ulink url=''>here</ulink>.
1073 </para>
1075 <para>
1076 If you do not know much about Git, you should educate
1077 yourself by visiting the links previously mentioned.
1078 </para>
1080 <para>
1081 The following list briefly describes some basic Git operations as a way to get started.
1082 As with any set of commands, this list (in most cases) simply shows the base command and
1083 omits the many arguments they support.
1084 See the Git documentation for complete descriptions and strategies on how to use these commands:
1085 <itemizedlist>
1086 <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository.
1087 You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem>
1088 <listitem><para><emphasis><filename>git clone</filename>:</emphasis>
1089 Creates a local clone of a Git repository.
1090 During collaboration, this command allows you to create a
1091 local Git repository that is on equal footing with a fellow
1092 developer’s Git repository.
1093 </para></listitem>
1094 <listitem><para><emphasis><filename>git add</filename>:</emphasis> Stages updated file contents
1095 to the index that
1096 Git uses to track changes.
1097 You must stage all files that have changed before you can commit them.</para></listitem>
1098 <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a "commit" that documents
1099 the changes you made.
1100 Commits are used for historical purposes, for determining if a maintainer of a project
1101 will allow the change, and for ultimately pushing the change from your local Git repository
1102 into the project’s upstream (or master) repository.</para></listitem>
1103 <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that
1104 possibly need to be staged and committed.</para></listitem>
1105 <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes
1106 your working branch.
1107 This command is analogous to "cd".</para></listitem>
1108 <listitem><para><emphasis><filename>git checkout –b &lt;working-branch&gt;</filename>:</emphasis> Creates
1109 a working branch on your local machine where you can isolate work.
1110 It is a good idea to use local branches when adding specific features or changes.
1111 This way if you do not like what you have done you can easily get rid of the work.</para></listitem>
1112 <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports
1113 existing local branches and
1114 tells you the branch in which you are currently working.</para></listitem>
1115 <listitem><para><emphasis><filename>git branch -D &lt;branch-name&gt;</filename>:</emphasis>
1116 Deletes an existing local branch.
1117 You need to be in a local branch other than the one you are deleting
1118 in order to delete <filename>&lt;branch-name&gt;</filename>.</para></listitem>
1119 <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information
1120 from an upstream Git
1121 repository and places it in your local Git repository.
1122 You use this command to make sure you are synchronized with the repository
1123 from which you are basing changes (.e.g. the master branch).</para></listitem>
1124 <listitem><para><emphasis><filename>git push</filename>:</emphasis>
1125 Sends all your committed local changes to an upstream Git
1126 repository (e.g. a contribution repository).
1127 The maintainer of the project draws from these repositories
1128 when adding changes to the project’s master repository or
1129 other development branch.
1130 </para></listitem>
1131 <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one
1132 local branch of your repository with another branch.
1133 When you create a local Git repository, the default branch is named "master".
1134 A typical workflow is to create a temporary branch for isolated work, make and commit your
1135 changes, switch to your local master branch, merge the changes from the temporary branch into the
1136 local master branch, and then delete the temporary branch.</para></listitem>
1137 <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific
1138 commits from one branch into another branch.
1139 There are times when you might not be able to merge all the changes in one branch with
1140 another but need to pick out certain ones.</para></listitem>
1141 <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches
1142 and changes in your local Git repository.
1143 This command is a good way to graphically see where things have diverged in your
1144 local repository.</para></listitem>
1145 <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the
1146 repository.</para></listitem>
1147 <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences
1148 between your local working files and the same files in the upstream Git repository that your
1149 branch currently tracks.</para></listitem>
1150 </itemizedlist>
1151 </para>
1152 </section>
1155<section id='workflows'>
1156 <title>Workflows</title>
1158 <para>
1159 This section provides some overview on workflows using Git.
1160 In particular, the information covers basic practices that describe roles and actions in a
1161 collaborative development environment.
1162 Again, if you are familiar with this type of development environment, you might want to just
1163 skip this section.
1164 </para>
1166 <para>
1167 The Yocto Project files are maintained using Git in a "master" branch whose Git history
1168 tracks every change and whose structure provides branches for all diverging functionality.
1169 Although there is no need to use Git, many open source projects do so.
1170 For the Yocto Project, a key individual called the "maintainer" is responsible for the "master"
1171 branch of a given Git repository.
1172 The "master" branch is the “upstream” repository where the final builds of the project occur.
1173 The maintainer is responsible for allowing changes in from other developers and for
1174 organizing the underlying branch structure to reflect release strategies and so forth.
1175 <note>For information on finding out who is responsible (maintains)
1176 for a particular area of code, see the
1177 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1178 section.
1179 </note>
1180 </para>
1182 <para>
1183 The project also has an upstream contribution Git repository named
1184 <filename>poky-contrib</filename>.
1185 You can see all the branches in this repository using the web interface
1186 of the
1187 <ulink url='&YOCTO_GIT_URL;'>Source Repositories</ulink> organized
1188 within the "Poky Support" area.
1189 These branches temporarily hold changes to the project that have been
1190 submitted or committed by the Yocto Project development team and by
1191 community members who contribute to the project.
1192 The maintainer determines if the changes are qualified to be moved
1193 from the "contrib" branches into the "master" branch of the Git
1194 repository.
1195 </para>
1197 <para>
1198 Developers (including contributing community members) create and maintain cloned repositories
1199 of the upstream "master" branch.
1200 These repositories are local to their development platforms and are used to develop changes.
1201 When a developer is satisfied with a particular feature or change, they "push" the changes
1202 to the appropriate "contrib" repository.
1203 </para>
1205 <para>
1206 Developers are responsible for keeping their local repository up-to-date with "master".
1207 They are also responsible for straightening out any conflicts that might arise within files
1208 that are being worked on simultaneously by more than one person.
1209 All this work is done locally on the developer’s machines before anything is pushed to a
1210 "contrib" area and examined at the maintainer’s level.
1211 </para>
1213 <para>
1214 A somewhat formal method exists by which developers commit changes and push them into the
1215 "contrib" area and subsequently request that the maintainer include them into "master"
1216 This process is called “submitting a patch” or "submitting a change."
1217 For information on submitting patches and changes, see the
1218 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section.
1219 </para>
1221 <para>
1222 To summarize the environment: a single point of entry exists for
1223 changes into the project’s "master" branch of the Git repository,
1224 which is controlled by the project’s maintainer.
1225 And, a set of developers exist who independently develop, test, and
1226 submit changes to "contrib" areas for the maintainer to examine.
1227 The maintainer then chooses which changes are going to become a
1228 permanent part of the project.
1229 </para>
1231 <para>
1232 <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
1233 </para>
1235 <para>
1236 While each development environment is unique, there are some best practices or methods
1237 that help development run smoothly.
1238 The following list describes some of these practices.
1239 For more information about Git workflows, see the workflow topics in the
1240 <ulink url=''>Git Community Book</ulink>.
1241 <itemizedlist>
1242 <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit
1243 small as compared to bundling many disparate changes into a single commit.
1244 This practice not only keeps things manageable but also allows the maintainer
1245 to more easily include or refuse changes.</para>
1246 <para>It is also good practice to leave the repository in a state that allows you to
1247 still successfully build your project. In other words, do not commit half of a feature,
1248 then add the other half as a separate, later commit.
1249 Each commit should take you from one buildable project state to another
1250 buildable state.</para></listitem>
1251 <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and
1252 delete local branches in your working Git repository.
1253 You can name these branches anything you like.
1254 It is helpful to give them names associated with the particular feature or change
1255 on which you are working.
1256 Once you are done with a feature or change and have merged it
1257 into your local master branch, simply discard the temporary
1258 branch.</para></listitem>
1259 <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename>
1260 command allows you to take the
1261 changes from one branch and fold them into another branch.
1262 This process is especially helpful when more than a single developer might be working
1263 on different parts of the same feature.
1264 Merging changes also automatically identifies any collisions or "conflicts"
1265 that might happen as a result of the same lines of code being altered by two different
1266 developers.</para></listitem>
1267 <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should
1268 use a system where branches indicate varying levels of code readiness.
1269 For example, you can have a "work" branch to develop in, a "test" branch where the code or
1270 change is tested, a "stage" branch where changes are ready to be committed, and so forth.
1271 As your project develops, you can merge code across the branches to reflect ever-increasing
1272 stable states of the development.</para></listitem>
1273 <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the
1274 concept of developers "pushing" local commits to a remote repository, which is
1275 usually a contribution repository.
1276 This workflow is also based on developers "pulling" known states of the project down into their
1277 local development repositories.
1278 The workflow easily allows you to pull changes submitted by other developers from the
1279 upstream repository into your work area ensuring that you have the most recent software
1280 on which to develop.
1281 The Yocto Project has two scripts named <filename>create-pull-request</filename> and
1282 <filename>send-pull-request</filename> that ship with the release to facilitate this
1283 workflow.
1284 You can find these scripts in the <filename>scripts</filename>
1285 folder of the
1286 <link linkend='source-directory'>Source Directory</link>.
1287 For information on how to use these scripts, see the
1288 "<link linkend='pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</link>" section.
1289 </para></listitem>
1290 <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the
1291 maintainer through an email that you have a change (or patch) you would like considered
1292 for the "master" branch of the Git repository.
1293 To send this type of change, you format the patch and then send the email using the Git commands
1294 <filename>git format-patch</filename> and <filename>git send-email</filename>.
1295 For information on how to use these scripts, see the
1296 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1297 section.
1298 </para></listitem>
1299 </itemizedlist>
1300 </para>
1303<section id='tracking-bugs'>
1304 <title>Tracking Bugs</title>
1306 <para>
1307 The Yocto Project uses its own implementation of
1308 <ulink url=''>Bugzilla</ulink> to track bugs.
1309 Implementations of Bugzilla work well for group development because they track bugs and code
1310 changes, can be used to communicate changes and problems with developers, can be used to
1311 submit and review patches, and can be used to manage quality assurance.
1312 The home page for the Yocto Project implementation of Bugzilla is
1313 <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>.
1314 </para>
1316 <para>
1317 Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself
1318 such as when discovering an issue with some component of the build system that acts contrary
1319 to the documentation or your expectations.
1320 Following is the general procedure for submitting a new bug using the Yocto Project
1321 Bugzilla.
1322 You can find more information on defect management, bug tracking, and feature request
1323 processes all accomplished through the Yocto Project Bugzilla on the wiki page
1324 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>here</ulink>.
1325 <orderedlist>
1326 <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit
1327 a bug.</para></listitem>
1328 <listitem><para>When submitting a new bug, be sure to choose the appropriate
1329 Classification, Product, and Component for which the issue was found.
1330 Defects for the Yocto Project fall into one of six classifications: Yocto Project
1331 Components, Infrastructure, Build System &amp; Metadata, Documentation,
1332 QA/Testing, and Runtime.
1333 Each of these Classifications break down into multiple Products and, in some
1334 cases, multiple Components.</para></listitem>
1335 <listitem><para>Use the bug form to choose the correct Hardware and Architecture
1336 for which the bug applies.</para></listitem>
1337 <listitem><para>Indicate the Yocto Project version you were using when the issue
1338 occurred.</para></listitem>
1339 <listitem><para>Be sure to indicate the Severity of the bug.
1340 Severity communicates how the bug impacted your work.</para></listitem>
1341 <listitem><para>Select the appropriate "Documentation change" item
1342 for the bug.
1343 Fixing a bug may or may not affect the Yocto Project
1344 documentation.</para></listitem>
1345 <listitem><para>Provide a brief summary of the issue.
1346 Try to limit your summary to just a line or two and be sure to capture the
1347 essence of the issue.</para></listitem>
1348 <listitem><para>Provide a detailed description of the issue.
1349 You should provide as much detail as you can about the context, behavior, output,
1350 and so forth that surrounds the issue.
1351 You can even attach supporting files for output from logs by
1352 using the "Add an attachment" button.</para></listitem>
1353 <listitem><para>Be sure to copy the appropriate people in the
1354 "CC List" for the bug.
1355 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1356 section for information about finding out who is responsible
1357 for code.</para></listitem>
1358 <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem>
1359 </orderedlist>
1360 </para>
1363<section id='how-to-submit-a-change'>
1364 <title>How to Submit a Change</title>
1366 <para>
1367 Contributions to the Yocto Project and OpenEmbedded are very welcome.
1368 Because the system is extremely configurable and flexible, we recognize that developers
1369 will want to extend, configure or optimize it for their specific uses.
1370 You should send patches to the appropriate mailing list so that they
1371 can be reviewed and merged by the appropriate maintainer.
1372 </para>
1374 <para>
1375 Before submitting any change, be sure to find out who you should be
1376 notifying.
1377 Several methods exist through which you find out who you should be copying
1378 or notifying:
1379 <itemizedlist>
1380 <listitem><para><emphasis>Maintenance File:</emphasis>
1381 Examine the <filename></filename> file, which is
1382 located in the
1383 <link linkend='source-directory'>Source Directory</link>
1384 at <filename>meta-yocto/conf/distro/include</filename>, to
1385 see who is responsible for code.
1386 </para></listitem>
1387 <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis>
1388 For BSP maintainers of supported BSPs, you can examine
1389 individual BSP <filename>README</filename> files.
1390 In addition, some layers (such as the <filename>meta-intel</filename> layer),
1391 include a <filename>MAINTAINERS</filename> file which contains
1392 a list of all supported BSP maintainers for that layer.
1393 </para></listitem>
1394 <listitem><para><emphasis>Search by File:</emphasis>
1395 Using <link linkend='git'>Git</link>, you can enter the
1396 following command to bring up a short list of all commits
1397 against a specific file:
1398 <literallayout class='monospaced'>
1399 git shortlog -- &lt;filename&gt;
1400 </literallayout>
1401 Just provide the name of the file for which you are interested.
1402 The information returned is not ordered by history but does
1403 include a list of all committers grouped by name.
1404 From the list, you can see who is responsible for the bulk of
1405 the changes against the file.
1406 </para></listitem>
1407 </itemizedlist>
1408 </para>
1410 <para>
1411 For a list of the Yocto Project and related mailing lists, see the
1412 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in
1413 the Yocto Project Reference Manual.
1414 </para>
1416 <para>
1417 Here is some guidance on which mailing list to use for what type of change:
1418 <itemizedlist>
1419 <listitem><para>For changes to the core
1420 <link linkend='metadata'>Metadata</link>, send your patch to the
1421 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list.
1422 For example, a change to anything under the <filename>meta</filename> or
1423 <filename>scripts</filename> directories
1424 should be sent to this mailing list.</para></listitem>
1425 <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename>
1426 directory), send your patch to the
1427 <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem>
1428 <listitem><para>For changes to <filename>meta-yocto</filename>, send your patch to the
1429 <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem>
1430 <listitem><para>For changes to other layers hosted on
1431 <filename></filename> (unless the
1432 layer's documentation specifies otherwise), tools, and Yocto Project
1433 documentation, use the
1434 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem>
1435 <listitem><para>For additional recipes that do not fit into the core Metadata,
1436 you should determine which layer the recipe should go into and submit the
1437 change in the manner recommended by the documentation (e.g. README) supplied
1438 with the layer. If in doubt, please ask on the
1439 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or
1440 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink>
1441 mailing lists.</para></listitem>
1442 </itemizedlist>
1443 </para>
1445 <para>
1446 When you send a patch, be sure to include a "Signed-off-by:"
1447 line in the same style as required by the Linux kernel.
1448 Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1
1449 as follows:
1450 <literallayout class='monospaced'>
1451 Developer's Certificate of Origin 1.1
1453 By making a contribution to this project, I certify that:
1455 (a) The contribution was created in whole or in part by me and I
1456 have the right to submit it under the open source license
1457 indicated in the file; or
1459 (b) The contribution is based upon previous work that, to the best
1460 of my knowledge, is covered under an appropriate open source
1461 license and I have the right under that license to submit that
1462 work with modifications, whether created in whole or in part
1463 by me, under the same open source license (unless I am
1464 permitted to submit under a different license), as indicated
1465 in the file; or
1467 (c) The contribution was provided directly to me by some other
1468 person who certified (a), (b) or (c) and I have not modified
1469 it.
1471 (d) I understand and agree that this project and the contribution
1472 are public and that a record of the contribution (including all
1473 personal information I submit with it, including my sign-off) is
1474 maintained indefinitely and may be redistributed consistent with
1475 this project or the open source license(s) involved.
1476 </literallayout>
1477 </para>
1479 <para>
1480 In a collaborative environment, it is necessary to have some sort of standard
1481 or method through which you submit changes.
1482 Otherwise, things could get quite chaotic.
1483 One general practice to follow is to make small, controlled changes.
1484 Keeping changes small and isolated aids review, makes merging/rebasing easier
1485 and keeps the change history clean when anyone needs to refer to it in future.
1486 </para>
1488 <para>
1489 When you make a commit, you must follow certain standards established by the
1490 OpenEmbedded and Yocto Project development teams.
1491 For each commit, you must provide a single-line summary of the change and you
1492 should almost always provide a more detailed description of what you did (i.e.
1493 the body of the commit message).
1494 The only exceptions for not providing a detailed description would be if your
1495 change is a simple, self-explanatory change that needs no further description
1496 beyond the summary.
1497 Here are the guidelines for composing a commit message:
1498 <itemizedlist>
1499 <listitem><para>Provide a single-line, short summary of the change.
1500 This summary is typically viewable in the "shortlist" of changes.
1501 Thus, providing something short and descriptive that gives the reader
1502 a summary of the change is useful when viewing a list of many commits.
1503 This short description should be prefixed by the recipe name (if changing a recipe), or
1504 else the short form path to the file being changed.
1505 </para></listitem>
1506 <listitem><para>For the body of the commit message, provide detailed information
1507 that describes what you changed, why you made the change, and the approach
1508 you used. It may also be helpful if you mention how you tested the change.
1509 Provide as much detail as you can in the body of the commit message.
1510 </para></listitem>
1511 <listitem><para>
1512 If the change addresses a specific bug or issue that is
1513 associated with a bug-tracking ID, include a reference to that
1514 ID in your detailed description.
1515 For example, the Yocto Project uses a specific convention for
1516 bug references - any commit that addresses a specific bug should
1517 use the following form for the detailed description:
1518 <literallayout class='monospaced'>
1519 Fixes [YOCTO #&lt;bug-id&gt;]
1521 &lt;detailed description of change&gt;
1522 </literallayout></para></listitem>
1523 Where &lt;bug-id&gt; is replaced with the specific bug ID from
1524 the Yocto Project Bugzilla instance.
1525 </itemizedlist>
1526 </para>
1528 <para>
1529 You can find more guidance on creating well-formed commit messages at this OpenEmbedded
1530 wiki page:
1531 <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>.
1532 </para>
1534 <para>
1535 The next two sections describe general instructions for both pushing
1536 changes upstream and for submitting changes as patches.
1537 </para>
1539 <section id='pushing-a-change-upstream'>
1540 <title>Using Scripts to Push a Change Upstream and Request a Pull</title>
1542 <para>
1543 The basic flow for pushing a change to an upstream "contrib" Git repository is as follows:
1544 <itemizedlist>
1545 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1546 <listitem><para>Stage your changes by using the <filename>git add</filename>
1547 command on each file you changed.</para></listitem>
1548 <listitem><para>
1549 Commit the change by using the
1550 <filename>git commit</filename> command.
1551 Be sure to provide a commit message that follows the
1552 project’s commit message standards as described earlier.
1553 </para></listitem>
1554 <listitem><para>
1555 Push the change to the upstream "contrib" repository by
1556 using the <filename>git push</filename> command.
1557 </para></listitem>
1558 <listitem><para>Notify the maintainer that you have pushed a change by making a pull
1559 request.
1560 The Yocto Project provides two scripts that conveniently let you generate and send
1561 pull requests to the Yocto Project.
1562 These scripts are <filename>create-pull-request</filename> and
1563 <filename>send-pull-request</filename>.
1564 You can find these scripts in the <filename>scripts</filename> directory
1565 within the <link linkend='source-directory'>Source Directory</link>.</para>
1566 <para>Using these scripts correctly formats the requests without introducing any
1567 whitespace or HTML formatting.
1568 The maintainer that receives your patches needs to be able to save and apply them
1569 directly from your emails.
1570 Using these scripts is the preferred method for sending patches.</para>
1571 <para>For help on using these scripts, simply provide the
1572 <filename>-h</filename> argument as follows:
1573 <literallayout class='monospaced'>
1574 $ poky/scripts/create-pull-request -h
1575 $ poky/scripts/send-pull-request -h
1576 </literallayout></para></listitem>
1577 </itemizedlist>
1578 </para>
1580 <para>
1581 You can find general Git information on how to push a change upstream in the
1582 <ulink url=''>Git Community Book</ulink>.
1583 </para>
1584 </section>
1586 <section id='submitting-a-patch'>
1587 <title>Using Email to Submit a Patch</title>
1589 <para>
1590 You can submit patches without using the <filename>create-pull-request</filename> and
1591 <filename>send-pull-request</filename> scripts described in the previous section.
1592 However, keep in mind, the preferred method is to use the scripts.
1593 </para>
1595 <para>
1596 Depending on the components changed, you need to submit the email to a specific
1597 mailing list.
1598 For some guidance on which mailing list to use, see the list in the
1599 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1600 section.
1601 For a description of the available mailing lists, see the
1602 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
1603 section in the Yocto Project Reference Manual.
1604 </para>
1606 <para>
1607 Here is the general procedure on how to submit a patch through email without using the
1608 scripts:
1609 <itemizedlist>
1610 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1611 <listitem><para>Stage your changes by using the <filename>git add</filename>
1612 command on each file you changed.</para></listitem>
1613 <listitem><para>Commit the change by using the
1614 <filename>git commit --signoff</filename> command.
1615 Using the <filename>--signoff</filename> option identifies you as the person
1616 making the change and also satisfies the Developer's Certificate of
1617 Origin (DCO) shown earlier.</para>
1618 <para>When you form a commit, you must follow certain standards established by the
1619 Yocto Project development team.
1620 See the earlier section
1621 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1622 for Yocto Project commit message standards.</para></listitem>
1623 <listitem><para>Format the commit into an email message.
1624 To format commits, use the <filename>git format-patch</filename> command.
1625 When you provide the command, you must include a revision list or a number of patches
1626 as part of the command.
1627 For example, either of these two commands takes your most
1628 recent single commit and formats it as an email message in
1629 the current directory:
1630 <literallayout class='monospaced'>
1631 $ git format-patch -1
1632 </literallayout>
1633 or
1634 <literallayout class='monospaced'>
1635 $ git format-patch HEAD~
1636 </literallayout></para>
1637 <para>After the command is run, the current directory contains a
1638 numbered <filename>.patch</filename> file for the commit.</para>
1639 <para>If you provide several commits as part of the command,
1640 the <filename>git format-patch</filename> command produces a
1641 series of numbered files in the current directory – one for each commit.
1642 If you have more than one patch, you should also use the
1643 <filename>--cover</filename> option with the command, which generates a
1644 cover letter as the first "patch" in the series.
1645 You can then edit the cover letter to provide a description for
1646 the series of patches.
1647 For information on the <filename>git format-patch</filename> command,
1648 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the
1649 <filename>man git-format-patch</filename> command.</para>
1650 <note>If you are or will be a frequent contributor to the Yocto Project
1651 or to OpenEmbedded, you might consider requesting a contrib area and the
1652 necessary associated rights.</note></listitem>
1653 <listitem><para>Import the files into your mail client by using the
1654 <filename>git send-email</filename> command.
1655 <note>In order to use <filename>git send-email</filename>, you must have the
1656 the proper Git packages installed.
1657 For Ubuntu, Debian, and Fedora the package is <filename>git-email</filename>.</note></para>
1658 <para>The <filename>git send-email</filename> command sends email by using a local
1659 or remote Mail Transport Agent (MTA) such as
1660 <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct
1661 <filename>smtp</filename> configuration in your Git <filename>config</filename>
1662 file.
1663 If you are submitting patches through email only, it is very important
1664 that you submit them without any whitespace or HTML formatting that
1665 either you or your mailer introduces.
1666 The maintainer that receives your patches needs to be able to save and
1667 apply them directly from your emails.
1668 A good way to verify that what you are sending will be applicable by the
1669 maintainer is to do a dry run and send them to yourself and then
1670 save and apply them as the maintainer would.</para>
1671 <para>The <filename>git send-email</filename> command is the preferred method
1672 for sending your patches since there is no risk of compromising whitespace
1673 in the body of the message, which can occur when you use your own mail client.
1674 The command also has several options that let you
1675 specify recipients and perform further editing of the email message.
1676 For information on how to use the <filename>git send-email</filename> command,
1677 see <filename>GIT-SEND-EMAIL(1)</filename> displayed using
1678 the <filename>man git-send-email</filename> command.
1679 </para></listitem>
1680 </itemizedlist>
1681 </para>
1682 </section>
1686vim: expandtab tw=80 ts=4