summaryrefslogtreecommitdiffstats
path: root/documentation/dev-manual/dev-manual-newbie.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/dev-manual/dev-manual-newbie.xml')
-rw-r--r--documentation/dev-manual/dev-manual-newbie.xml1575
1 files changed, 1575 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml
new file mode 100644
index 0000000000..34326d10b6
--- /dev/null
+++ b/documentation/dev-manual/dev-manual-newbie.xml
@@ -0,0 +1,1575 @@
1<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
3[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
4
5<chapter id='dev-manual-newbie'>
6
7<title>The Yocto Project Open Source Development Environment</title>
8
9<para>
10 This chapter helps you understand the Yocto Project as an open source development project.
11 In general, working in an open source environment is very different from working in a
12 closed, proprietary environment.
13 Additionally, the Yocto Project uses specific tools and constructs as part of its development
14 environment.
15 This chapter specifically addresses open source philosophy, using the
16 Yocto Project in a team environment, source repositories, Yocto Project
17 terms, licensing, the open source distributed version control system Git,
18 workflows, bug tracking, and how to submit changes.
19</para>
20
21<section id='open-source-philosophy'>
22 <title>Open Source Philosophy</title>
23
24 <para>
25 Open source philosophy is characterized by software development directed by peer production
26 and collaboration through an active community of developers.
27 Contrast this to the more standard centralized development models used by commercial software
28 companies where a finite set of developers produces a product for sale using a defined set
29 of procedures that ultimately result in an end product whose architecture and source material
30 are closed to the public.
31 </para>
32
33 <para>
34 Open source projects conceptually have differing concurrent agendas, approaches, and production.
35 These facets of the development process can come from anyone in the public (community) that has a
36 stake in the software project.
37 The open source environment contains new copyright, licensing, domain, and consumer issues
38 that differ from the more traditional development environment.
39 In an open source environment, the end product, source material, and documentation are
40 all available to the public at no cost.
41 </para>
42
43 <para>
44 A benchmark example of an open source project is the Linux Kernel, which was initially conceived
45 and created by Finnish computer science student Linus Torvalds in 1991.
46 Conversely, a good example of a non-open source project is the
47 <trademark class='registered'>Windows</trademark> family of operating
48 systems developed by <trademark class='registered'>Microsoft</trademark> Corporation.
49 </para>
50
51 <para>
52 Wikipedia has a good historical description of the Open Source Philosophy
53 <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>.
54 You can also find helpful information on how to participate in the Linux Community
55 <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>.
56 </para>
57</section>
58
59<section id="usingpoky-changes-collaborate">
60 <title>Using the Yocto Project in a Team Environment</title>
61
62 <para>
63 It might not be immediately clear how you can use the Yocto
64 Project in a team environment, or scale it for a large team of
65 developers.
66 One of the strengths of the Yocto Project is that it is extremely
67 flexible.
68 Thus, you can adapt it to many different use cases and scenarios.
69 However, these characteristics can cause a struggle if you are trying
70 to create a working setup that scales across a large team.
71 </para>
72
73 <para>
74 To help with these types of situations, this section presents
75 some of the project's most successful experiences,
76 practices, solutions, and available technologies that work well.
77 Keep in mind, the information here is a starting point.
78 You can build off it and customize it to fit any
79 particular working environment and set of practices.
80 </para>
81
82 <section id='best-practices-system-configurations'>
83 <title>System Configurations</title>
84
85 <para>
86 Systems across a large team should meet the needs of
87 two types of developers: those working on the contents of the
88 operating system image itself and those developing applications.
89 Regardless of the type of developer, their workstations must
90 be both reasonably powerful and run Linux.
91 </para>
92
93 <section id='best-practices-application-development'>
94 <title>Application Development</title>
95
96 <para>
97 For developers who mainly do application level work
98 on top of an existing software stack,
99 here are some practices that work best:
100 <itemizedlist>
101 <listitem><para>Use a pre-built toolchain that
102 contains the software stack itself.
103 Then, develop the application code on top of the
104 stack.
105 This method works well for small numbers of relatively
106 isolated applications.</para></listitem>
107 <listitem><para>When possible, use the Yocto Project
108 plug-in for the <trademark class='trade'>Eclipse</trademark> IDE
109 and other pieces of Application Development
110 Technology (ADT).
111 For more information, see the
112 "<link linkend='application-development-workflow'>Application
113 Development Workflow</link>" section as well as the
114 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>.
115 </para></listitem>
116 <listitem><para>Keep your cross-development toolchains
117 updated.
118 You can do this through provisioning either as new
119 toolchain downloads or as updates through a package
120 update mechanism using <filename>opkg</filename>
121 to provide updates to an existing toolchain.
122 The exact mechanics of how and when to do this are a
123 question for local policy.</para></listitem>
124 <listitem><para>Use multiple toolchains installed locally
125 into different locations to allow development across
126 versions.</para></listitem>
127 </itemizedlist>
128 </para>
129 </section>
130
131 <section id='best-practices-core-system-development'>
132 <title>Core System Development</title>
133
134 <para>
135 For core system development, it is often best to have the
136 build system itself available on the developer workstations
137 so developers can run their own builds and directly
138 rebuild the software stack.
139 You should keep the core system unchanged as much as
140 possible and do your work in layers on top of the core system.
141 Doing so gives you a greater level of portability when
142 upgrading to new versions of the core system or Board
143 Support Packages (BSPs).
144 You can share layers amongst the developers of a particular
145 project and contain the policy configuration that defines
146 the project.
147 </para>
148
149 <para>
150 Aside from the previous best practices, there exists a number
151 of tips and tricks that can help speed up core development
152 projects:
153 <itemizedlist>
154 <listitem><para>Use a
155 <ulink url='&YOCTO_DOCS_REF_URL;#shared-state-cache'>Shared State Cache</ulink>
156 (sstate) among groups of developers who are on a
157 fast network.
158 The best way to share sstate is through a
159 Network File System (NFS) share.
160 The first user to build a given component for the
161 first time contributes that object to the sstate,
162 while subsequent builds from other developers then
163 reuse the object rather than rebuild it themselves.
164 </para>
165 <para>Although it is possible to use other protocols for the
166 sstate such as HTTP and FTP, you should avoid these.
167 Using HTTP limits the sstate to read-only and
168 FTP provides poor performance.
169 </para></listitem>
170 <listitem><para>Have autobuilders contribute to the sstate
171 pool similarly to how the developer workstations
172 contribute.
173 For information, see the
174 <link linkend='best-practices-autobuilders'>Autobuilders</link>
175 section.</para></listitem>
176 <listitem><para>Build stand-alone tarballs that contain
177 "missing" system requirements if for some reason
178 developer workstations do not meet minimum system
179 requirements such as latest Python versions,
180 <filename>chrpath</filename>, or other tools.
181 You can install and relocate the tarball exactly as you
182 would the usual cross-development toolchain so that
183 all developers can meet minimum version requirements
184 on most distributions.</para></listitem>
185 <listitem><para>Use a small number of shared,
186 high performance systems for testing purposes
187 (e.g. dual six core Xeons with 24GB RAM and plenty of
188 disk space).
189 Developers can use these systems for wider, more
190 extensive testing while they continue to develop
191 locally using their primary development system.
192 </para></listitem>
193 <listitem><para>Enable the PR Service when package feeds
194 need to be incremental with continually increasing
195 <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'>PR</ulink>
196 values.
197 Typically, this situation occurs when you use or
198 publish package feeds and use a shared state.
199 You should enable the PR Service for all users who
200 use the shared state pool.
201 For more information on the PR Service, see the
202 "<link linkend='working-with-a-pr-service'>Working With a PR Service</link>".
203 </para></listitem>
204 </itemizedlist>
205 </para>
206 </section>
207 </section>
208
209 <section id='best-practices-source-control-management'>
210 <title>Source Control Management (SCM)</title>
211
212 <para>
213 Keeping your
214 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
215 and any software you are developing under the
216 control of an SCM system that is compatible
217 with the OpenEmbedded build system is advisable.
218 Of the SCMs BitBake supports, the
219 Yocto Project team strongly recommends using
220 <link linkend='git'>Git</link>.
221 Git is a distributed system that is easy to backup,
222 allows you to work remotely, and then connects back to the
223 infrastructure.
224 <note>
225 For information about BitBake and SCMs, see the
226 BitBake manual located in the
227 <filename>bitbake/doc/manual</filename> directory of the
228 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
229 </note>
230 </para>
231
232 <para>
233 It is relatively easy to set up Git services and create
234 infrastructure like
235 <ulink url='&YOCTO_GIT_URL;'>http://git.yoctoproject.org</ulink>,
236 which is based on server software called
237 <filename>gitolite</filename> with <filename>cgit</filename>
238 being used to generate the web interface that lets you view the
239 repositories.
240 The <filename>gitolite</filename> software identifies users
241 using <filename>ssh</filename> keys and allows branch-based
242 access controls to repositories that you can control as little
243 or as much as necessary.
244 </para>
245
246 <note>
247 The setup of these services is beyond the scope of this manual.
248 However, sites such as these exist that describe how to perform
249 setup:
250 <itemizedlist>
251 <listitem><para><ulink url='http://git-scm.com/book/ch4-8.html'>Git documentation</ulink>:
252 Describes how to install <filename>gitolite</filename>
253 on the server.</para></listitem>
254 <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
255 All topics for <filename>gitolite</filename>.
256 </para></listitem>
257 <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
258 Documentation on how to create interfaces and frontends
259 for Git.</para></listitem>
260 </itemizedlist>
261 </note>
262 </section>
263
264 <section id='best-practices-autobuilders'>
265 <title>Autobuilders</title>
266
267 <para>
268 Autobuilders are often the core of a development project.
269 It is here that changes from individual developers are brought
270 together and centrally tested and subsequent decisions about
271 releases can be made.
272 Autobuilders also allow for "continuous integration" style
273 testing of software components and regression identification
274 and tracking.
275 </para>
276
277 <para>
278 See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
279 for more information and links to buildbot.
280 The Yocto Project team has found this implementation
281 works well in this role.
282 A public example of this is the Yocto Project
283 Autobuilders, which we use to test the overall health of the
284 project.
285 </para>
286
287 <para>
288 The features of this system are:
289 <itemizedlist>
290 <listitem><para>Highlights when commits break the build.
291 </para></listitem>
292 <listitem><para>Populates an sstate cache from which
293 developers can pull rather than requiring local
294 builds.</para></listitem>
295 <listitem><para>Allows commit hook triggers,
296 which trigger builds when commits are made.
297 </para></listitem>
298 <listitem><para>Allows triggering of automated image booting
299 and testing under the QuickEMUlator (QEMU).
300 </para></listitem>
301 <listitem><para>Supports incremental build testing and from
302 scratch builds.</para></listitem>
303 <listitem><para>Shares output that allows developer
304 testing and historical regression investigation.
305 </para></listitem>
306 <listitem><para>Creates output that can be used for releases.
307 </para></listitem>
308 <listitem><para>Allows scheduling of builds so that resources
309 can be used efficiently.</para></listitem>
310 </itemizedlist>
311 </para>
312 </section>
313
314 <section id='best-practices-policies-and-change-flow'>
315 <title>Policies and Change Flow</title>
316
317 <para>
318 The Yocto Project itself uses a hierarchical structure and a
319 pull model.
320 Scripts exist to create and send pull requests
321 (i.e. <filename>create-pull-request</filename> and
322 <filename>send-pull-request</filename>).
323 This model is in line with other open source projects where
324 maintainers are responsible for specific areas of the project
325 and a single maintainer handles the final "top-of-tree" merges.
326 </para>
327
328 <note>
329 You can also use a more collective push model.
330 The <filename>gitolite</filename> software supports both the
331 push and pull models quite easily.
332 </note>
333
334 <para>
335 As with any development environment, it is important
336 to document the policy used as well as any main project
337 guidelines so they are understood by everyone.
338 It is also a good idea to have well structured
339 commit messages, which are usually a part of a project's
340 guidelines.
341 Good commit messages are essential when looking back in time and
342 trying to understand why changes were made.
343 </para>
344
345 <para>
346 If you discover that changes are needed to the core layer of the
347 project, it is worth sharing those with the community as soon
348 as possible.
349 Chances are if you have discovered the need for changes, someone
350 else in the community needs them also.
351 </para>
352 </section>
353
354 <section id='best-practices-summary'>
355 <title>Summary</title>
356
357 <para>
358 This section summarizes the key recommendations described in the
359 previous sections:
360 <itemizedlist>
361 <listitem><para>Use <link linkend='git'>Git</link>
362 as the source control system.</para></listitem>
363 <listitem><para>Maintain your Metadata in layers that make sense
364 for your situation.
365 See the "<link linkend='understanding-and-creating-layers'>Understanding
366 and Creating Layers</link>" section for more information on
367 layers.</para></listitem>
368 <listitem><para>Separate the project's Metadata and code by using
369 separate Git repositories.
370 See the "<link linkend='yocto-project-repositories'>Yocto Project
371 Source Repositories</link>" section for information on these
372 repositories.
373 See the "<link linkend='getting-setup'>Getting Set Up</link>" section
374 for information on how to set up various Yocto Project related
375 Git repositories.</para></listitem>
376 <listitem><para>Set up the directory for the shared state cache
377 (<ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>)
378 where it makes sense.
379 For example, set up the sstate cache on a system used
380 by developers in the same organization and share the
381 same source directories on their machines.
382 </para></listitem>
383 <listitem><para>Set up an Autobuilder and have it populate the
384 sstate cache and source directories.</para></listitem>
385 <listitem><para>The Yocto Project community encourages you
386 to send patches to the project to fix bugs or add features.
387 If you do submit patches, follow the project commit
388 guidelines for writing good commit messages.
389 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
390 section.</para></listitem>
391 <listitem><para>Send changes to the core sooner than later
392 as others likely run into the same issues.
393 For some guidance on mailing lists to use, see the list in the
394 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
395 section.
396 For a description of the available mailing lists, see the
397 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
398 section in the Yocto Project Reference Manual.
399 </para></listitem>
400 </itemizedlist>
401 </para>
402 </section>
403</section>
404
405<section id='yocto-project-repositories'>
406 <title>Yocto Project Source Repositories</title>
407
408 <para>
409 The Yocto Project team maintains complete source repositories for all Yocto Project files
410 at <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'></ulink>.
411 This web-based source code browser is organized into categories by function such as
412 IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth.
413 From the interface, you can click on any particular item in the "Name" column and
414 see the URL at the bottom of the page that you need to clone a Git repository for
415 that particular item.
416 Having a local Git repository of the Source Directory (poky) allows you to
417 make changes, contribute to the history, and ultimately enhance the Yocto Project's
418 tools, Board Support Packages, and so forth.
419 </para>
420
421 <para>
422 Conversely, if you are a developer that is not interested in contributing back to the
423 Yocto Project, you have the ability to simply download and extract release tarballs
424 and use them within the Yocto Project environment.
425 All that is required is a particular release of the Yocto Project and
426 your application source code.
427 </para>
428
429 <para>
430 For any supported release of Yocto Project, you can go to the
431 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
432 select the "Downloads" tab and get a tarball of the release.
433 You can also go to this site to download any supported BSP tarballs.
434 Unpacking the tarball gives you a hierarchical Source Directory that lets you develop
435 using the Yocto Project.
436 </para>
437
438 <para>
439 Once you are set up through either tarball extraction or a checkout of Git repositories,
440 you are ready to develop.
441 </para>
442
443 <para>
444 In summary, here is where you can get the project files needed for development:
445 <itemizedlist>
446 <listitem><para id='source-repositories'><emphasis><ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis>
447 This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto
448 Metadata Layers.
449 You can create local copies of Git repositories for each of these areas.</para>
450 <para>
451 <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
452 </para></listitem>
453 <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink></emphasis>
454 This area contains index releases such as
455 the <trademark class='trade'>Eclipse</trademark>
456 Yocto Plug-in, miscellaneous support, poky, pseudo, installers for cross-development toolchains,
457 and all released versions of Yocto Project in the form of images or tarballs.
458 Downloading and extracting these files does not produce a local copy of the
459 Git repository but rather a snapshot of a particular release or image.</para>
460 <para>
461 <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="3.5in" />
462 </para></listitem>
463 <listitem><para><emphasis>"Downloads" page for the
464 <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>:</emphasis>
465 Access this page by going to the website and then selecting
466 the "Downloads" tab.
467 This page allows you to download any Yocto Project
468 release or Board Support Package (BSP) in tarball form.
469 The tarballs are similar to those found in the
470 <ulink url='&YOCTO_DL_URL;/releases/'>Index of /releases:</ulink> area.</para>
471 <para>
472 <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
473 </para></listitem>
474 </itemizedlist>
475 </para>
476</section>
477
478<section id='yocto-project-terms'>
479 <title>Yocto Project Terms</title>
480
481 <para>
482 Following is a list of terms and definitions users new to the Yocto Project development
483 environment might find helpful.
484 While some of these terms are universal, the list includes them just in case:
485 <itemizedlist>
486 <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to
487 a recipe file.
488 Append files are known as BitBake append files and <filename>.bbappend</filename> files.
489 The OpenEmbedded build system expects every append file to have a corresponding
490 recipe (<filename>.bb</filename>) file.
491 Furthermore, the append file and corresponding recipe file
492 must use the same root filename.
493 The filenames can differ only in the file type suffix used (e.g.
494 <filename>formfactor_0.0.bb</filename> and <filename>formfactor_0.0.bbappend</filename>).
495 </para>
496 <para>Information in append files overrides the information in the similarly-named recipe file.
497 For an example of an append file in use, see the
498 "<link linkend='using-bbappend-files'>Using .bbappend Files</link>" section.
499 </para></listitem>
500 <listitem><para id='bitbake-term'><emphasis>BitBake:</emphasis>
501 The task executor and scheduler used by
502 the OpenEmbedded build system to build images.
503 For more information on BitBake, see the BitBake documentation
504 in the <filename>bitbake/doc/manual</filename> directory of the
505 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
506 <listitem>
507 <para id='build-directory'><emphasis>Build Directory:</emphasis>
508 This term refers to the area used by the OpenEmbedded build system for builds.
509 The area is created when you <filename>source</filename> the setup
510 environment script that is found in the Source Directory
511 (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
512 or
513 <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
514 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
515 variable points to the Build Directory.</para>
516
517 <para>You have a lot of flexibility when creating the Build Directory.
518 Following are some examples that show how to create the directory:
519 <itemizedlist>
520 <listitem><para>Create the Build Directory in your current working directory
521 and name it <filename>build</filename>.
522 This is the default behavior.
523 <literallayout class='monospaced'>
524 $ source &OE_INIT_PATH;
525 </literallayout></para></listitem>
526 <listitem><para>Provide a directory path and specifically name the build
527 directory.
528 This next example creates a Build Directory named <filename>YP-&POKYVERSION;</filename>
529 in your home directory within the directory <filename>mybuilds</filename>.
530 If <filename>mybuilds</filename> does not exist, the directory is created for you:
531 <literallayout class='monospaced'>
532 $ source &OE_INIT_PATH; $HOME/mybuilds/YP-&POKYVERSION;
533 </literallayout></para></listitem>
534 <listitem><para>Provide an existing directory to use as the Build Directory
535 and use the default <filename>build</filename> name.
536 <literallayout class='monospaced'>
537 $ source &OE_INIT_PATH; $HOME/mybuilds/
538 </literallayout></para></listitem>
539 </itemizedlist>
540 </para></listitem>
541 <listitem><para><emphasis>Build System:</emphasis> In the context of the Yocto Project,
542 this term refers to the OpenEmbedded build system used by the project.
543 This build system is based on the project known as "Poky."
544 For some historical information about Poky, see the
545 <link linkend='poky'>Poky</link> term.
546 </para></listitem>
547 <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation
548 and inheritance so that commonly used patterns can be defined once and then easily used
549 in multiple recipes.
550 Class files end with the <filename>.bbclass</filename> filename extension.
551 </para></listitem>
552 <listitem><para><emphasis>Configuration File:</emphasis> Configuration information in various
553 <filename>.conf</filename> files provides global definitions of variables.
554 The <filename>conf/local.conf</filename> configuration file in the
555 <link linkend='build-directory'>Build Directory</link>
556 contains user-defined variables that affect each build.
557 The <filename>meta-yocto/conf/distro/poky.conf</filename> configuration file
558 defines Yocto "distro" configuration
559 variables used only when building with this policy.
560 Machine configuration files, which
561 are located throughout the
562 <link linkend='source-directory'>Source Directory</link>, define
563 variables for specific hardware and are only used when building for that target
564 (e.g. the <filename>machine/beagleboard.conf</filename> configuration file defines
565 variables for the Texas Instruments ARM Cortex-A8 development board).
566 Configuration files end with a <filename>.conf</filename> filename extension.
567 </para></listitem>
568 <listitem><para id='cross-development-toolchain'>
569 <emphasis>Cross-Development Toolchain:</emphasis>
570 In general, a cross-development toolchain is a collection of
571 software development tools and utilities that run on one
572 architecture and allow you to develop software for a
573 different, or targeted, architecture.
574 These toolchains contain cross-compilers, linkers, and
575 debuggers that are specific to the target architecture.
576 </para>
577
578 <para>The Yocto Project supports two different cross-development
579 toolchains:
580 <itemizedlist>
581 <listitem><para>A toolchain only used by and within
582 BitBake when building an image for a target
583 architecture.</para></listitem>
584 <listitem><para>A relocatable toolchain used outside of
585 BitBake by developers when developing applications
586 that will run on a targeted device.
587 Sometimes this relocatable cross-development
588 toolchain is referred to as the meta-toolchain.
589 </para></listitem>
590 </itemizedlist>
591 </para>
592
593 <para>
594 Creation of these toolchains is simple and automated.
595 For information on toolchain concepts as they apply to the
596 Yocto Project, see the
597 "<ulink url='&YOCTO_DOCS_REF_URL;#cross-development-toolchain-generation'>Cross-Development Toolchain Generation</ulink>"
598 section in the Yocto Project Reference Manual.
599 You can also find more information on using the
600 relocatable toolchain in the
601 <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project
602 Application Developer's Guide</ulink>.
603 </para></listitem>
604 <listitem><para><emphasis>Image:</emphasis> An image is the result produced when
605 BitBake processes a given collection of recipes and related Metadata.
606 Images are the binary output that run on specific hardware or QEMU
607 and for specific use cases.
608 For a list of the supported image types that the Yocto Project provides, see the
609 "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>"
610 chapter in the Yocto Project Reference Manual.</para></listitem>
611 <listitem><para id='layer'><emphasis>Layer:</emphasis> A collection of recipes representing the core,
612 a BSP, or an application stack.
613 For a discussion on BSP Layers, see the
614 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
615 section in the Yocto Project Board Support Packages (BSP)
616 Developer's Guide.</para></listitem>
617 <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
618 A term sometimes used for
619 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
620 </para></listitem>
621 <listitem><para id='metadata'><emphasis>Metadata:</emphasis>
622 The files that BitBake parses when building an image.
623 In general, Metadata includes recipes, classes, and
624 configuration files.
625 In the context of the kernel ("kernel Metadata"),
626 it refers to Metadata in the <filename>meta</filename>
627 branches of the kernel source Git repositories.
628 </para></listitem>
629 <listitem><para id='oe-core'><emphasis>OE-Core:</emphasis> A core set of Metadata originating
630 with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
631 This Metadata is found in the <filename>meta</filename> directory of the
632 <link linkend='source-directory'>Source Directory</link>.</para></listitem>
633 <listitem><para><emphasis>Package:</emphasis> In the context of the Yocto Project,
634 this term refers to the packaged output from a baked recipe.
635 A package is generally the compiled binaries produced from the recipe's sources.
636 You "bake" something by running it through BitBake.</para>
637 <para>It is worth noting that the term "package" can, in general, have subtle
638 meanings. For example, the packages referred to in the
639 "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Packages</ulink>" section are
640 compiled binaries that when installed add functionality to your Linux
641 distribution.</para>
642 <para>Another point worth noting is that historically within the Yocto Project,
643 recipes were referred to as packages - thus, the existence of several BitBake
644 variables that are seemingly mis-named,
645 (e.g. <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>,
646 <ulink url='&YOCTO_DOCS_REF_URL;#var-PRINC'><filename>PRINC</filename></ulink>,
647 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
648 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
649 </para></listitem>
650 <listitem><para id='poky'><emphasis>Poky:</emphasis> The term "poky" can mean several things.
651 In its most general sense, it is an open-source project that was initially developed
652 by OpenedHand. With OpenedHand, poky was developed off of the existing OpenEmbedded
653 build system becoming a build system for embedded images.
654 After Intel Corporation acquired OpenedHand, the project poky became the basis for
655 the Yocto Project's build system.
656 Within the Yocto Project source repositories, <filename>poky</filename>
657 exists as a separate Git repository
658 that can be cloned to yield a local copy on the host system.
659 Thus, "poky" can refer to the local copy of the Source Directory used to develop within
660 the Yocto Project.</para></listitem>
661 <listitem><para><emphasis>Recipe:</emphasis> A set of instructions for building packages.
662 A recipe describes where you get source code and which patches to apply.
663 Recipes describe dependencies for libraries or for other recipes, and they
664 also contain configuration and compilation options.
665 Recipes contain the logical unit of execution, the software/images to build, and
666 use the <filename>.bb</filename> file extension.</para></listitem>
667 <listitem>
668 <para id='source-directory'><emphasis>Source Directory:</emphasis>
669 This term refers to the directory structure created as a result of either downloading
670 and unpacking a Yocto Project release tarball or creating a local copy of
671 the <filename>poky</filename> Git repository
672 <filename>git://git.yoctoproject.org/poky</filename>.
673 Sometimes you might hear the term "poky directory" used to refer to this
674 directory structure.
675 <note>
676 The OpenEmbedded build system does not support file or directory names that
677 contain spaces.
678 Be sure that the Source Directory you use does not contain these types
679 of names.
680 </note></para>
681 <para>The Source Directory contains BitBake, Documentation, Metadata and
682 other files that all support the Yocto Project.
683 Consequently, you must have the Source Directory in place on your development
684 system in order to do any development using the Yocto Project.</para>
685
686 <para>For tarball expansion, the name of the top-level directory of the Source Directory
687 is derived from the Yocto Project release tarball.
688 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
689 results in a Source Directory whose top-level folder is named
690 <filename>&YOCTO_POKY;</filename>.
691 If you create a local copy of the Git repository, you can name the repository
692 anything you like.
693 Throughout much of the documentation, <filename>poky</filename> is used as the name of
694 the top-level folder of the local copy of the poky Git repository.
695 So, for example, cloning the <filename>poky</filename> Git repository results in a
696 local Git repository whose top-level folder is also named <filename>poky</filename>.</para>
697
698 <para>It is important to understand the differences between the Source Directory created
699 by unpacking a released tarball as compared to cloning
700 <filename>git://git.yoctoproject.org/poky</filename>.
701 When you unpack a tarball, you have an exact copy of the files based on the time of
702 release - a fixed release point.
703 Any changes you make to your local files in the Source Directory are on top of the release.
704 On the other hand, when you clone the <filename>poky</filename> Git repository, you have an
705 active development repository.
706 In this case, any local changes you make to the Source Directory can be later applied
707 to active development branches of the upstream <filename>poky</filename> Git
708 repository.</para>
709
710 <para>Finally, if you want to track a set of local changes while starting from the same point
711 as a release tarball, you can create a local Git branch that
712 reflects the exact copy of the files at the time of their release.
713 You do this by using Git tags that are part of the repository.</para>
714
715 <para>For more information on concepts related to Git repositories, branches, and tags,
716 see the
717 "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>"
718 section.</para></listitem>
719 <listitem><para><emphasis>Tasks:</emphasis> Arbitrary groups of software Recipes.
720 You use tasks to hold recipes that, when built, usually accomplish a single task.
721 For example, a task could contain the recipes for a company’s proprietary or value-add software.
722 Or, the task could contain the recipes that enable graphics.
723 A task is really just another recipe.
724 Because task files are recipes, they end with the <filename>.bb</filename> filename
725 extension.</para></listitem>
726 <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
727 that are not local to the development system but located in a master area that is controlled
728 by the maintainer of the source code.
729 For example, in order for a developer to work on a particular piece of code, they need to
730 first get a copy of it from an "upstream" source.</para></listitem>
731 </itemizedlist>
732 </para>
733</section>
734
735<section id='licensing'>
736 <title>Licensing</title>
737
738 <para>
739 Because open source projects are open to the public, they have different licensing structures in place.
740 License evolution for both Open Source and Free Software has an interesting history.
741 If you are interested in this history, you can find basic information here:
742 <itemizedlist>
743 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink>
744 </para></listitem>
745 <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license
746 history</ulink></para></listitem>
747 </itemizedlist>
748 </para>
749
750 <para>
751 In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology
752 (MIT) License.
753 MIT licensing permits the reuse of software within proprietary software as long as the
754 license is distributed with that software.
755 MIT is also compatible with the GNU General Public License (GPL).
756 Patches to the Yocto Project follow the upstream licensing scheme.
757 You can find information on the MIT license at
758 <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
759 You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>
760 here</ulink>.
761 </para>
762
763 <para>
764 When you build an image using the Yocto Project, the build process uses a
765 known list of licenses to ensure compliance.
766 You can find this list in the
767 <link linkend='source-directory'>Source Directory</link> at
768 <filename>meta/files/common-licenses</filename>.
769 Once the build completes, the list of all licenses found and used during that build are
770 kept in the
771 <link linkend='build-directory'>Build Directory</link> at
772 <filename>tmp/deploy/licenses</filename>.
773 </para>
774
775 <para>
776 If a module requires a license that is not in the base list, the build process
777 generates a warning during the build.
778 These tools make it easier for a developer to be certain of the licenses with which
779 their shipped products must comply.
780 However, even with these tools it is still up to the developer to resolve potential licensing issues.
781 </para>
782
783 <para>
784 The base list of licenses used by the build process is a combination of the Software Package
785 Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects.
786 <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation
787 that maintains a specification
788 for a standard format for communicating the components, licenses, and copyrights
789 associated with a software package.
790 <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source
791 Definition and the effort for reviewing and approving licenses that are OSD-conformant.
792 </para>
793
794 <para>
795 You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses
796 <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/files/common-licenses'>here</ulink>.
797 </para>
798
799 <para>
800 For information that can help you to maintain compliance with various open source licensing
801 during the lifecycle of a product created using the Yocto Project, see the
802 "<link linkend='maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Product's Lifecycle</link>" section.
803 </para>
804</section>
805
806<section id='git'>
807 <title>Git</title>
808
809 <para>
810 The Yocto Project uses Git, which is a free, open source distributed version control system.
811 Git supports distributed development, non-linear development, and can handle large projects.
812 It is best that you have some fundamental understanding of how Git tracks projects and
813 how to work with Git if you are going to use the Yocto Project for development.
814 This section provides a quick overview of how Git works and provides you with a summary
815 of some essential Git commands.
816 </para>
817
818 <para>
819 For more information on Git, see
820 <ulink url='http://git-scm.com/documentation'></ulink>.
821 If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>.
822 </para>
823
824 <section id='repositories-tags-and-branches'>
825 <title>Repositories, Tags, and Branches</title>
826
827 <para>
828 As mentioned earlier in the section
829 "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>",
830 the Yocto Project maintains source repositories at
831 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.
832 If you look at this web-interface of the repositories, each item is a separate
833 Git repository.
834 </para>
835
836 <para>
837 Git repositories use branching techniques that track content change (not files)
838 within a project (e.g. a new feature or updated documentation).
839 Creating a tree-like structure based on project divergence allows for excellent historical
840 information over the life of a project.
841 This methodology also allows for an environment from which you can do lots of
842 local experimentation on projects as you develop changes or new features.
843 </para>
844
845 <para>
846 A Git repository represents all development efforts for a given project.
847 For example, the Git repository <filename>poky</filename> contains all changes
848 and developments for Poky over the course of its entire life.
849 That means that all changes that make up all releases are captured.
850 The repository maintains a complete history of changes.
851 </para>
852
853 <para>
854 You can create a local copy of any repository by "cloning" it with the Git
855 <filename>clone</filename> command.
856 When you clone a Git repository, you end up with an identical copy of the
857 repository on your development system.
858 Once you have a local copy of a repository, you can take steps to develop locally.
859 For examples on how to clone Git repositories, see the
860 "<link linkend='getting-setup'>Getting Set Up</link>" section.
861 </para>
862
863 <para>
864 It is important to understand that Git tracks content change and not files.
865 Git uses "branches" to organize different development efforts.
866 For example, the <filename>poky</filename> repository has
867 <filename>denzil</filename>, <filename>danny</filename>,
868 <filename>dylan</filename>, <filename>dora</filename>,
869 and <filename>master</filename> branches among others.
870 You can see all the branches by going to
871 <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
872 clicking on the
873 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/heads'>[...]</ulink></filename>
874 link beneath the "Branch" heading.
875 </para>
876
877 <para>
878 Each of these branches represents a specific area of development.
879 The <filename>master</filename> branch represents the current or most recent
880 development.
881 All other branches represent off-shoots of the <filename>master</filename>
882 branch.
883 </para>
884
885 <para>
886 When you create a local copy of a Git repository, the copy has the same set
887 of branches as the original.
888 This means you can use Git to create a local working area (also called a branch)
889 that tracks a specific development branch from the source Git repository.
890 in other words, you can define your local Git environment to work on any development
891 branch in the repository.
892 To help illustrate, here is a set of commands that creates a local copy of the
893 <filename>poky</filename> Git repository and then creates and checks out a local
894 Git branch that tracks the Yocto Project &DISTRO; Release (&DISTRO_NAME;) development:
895 <literallayout class='monospaced'>
896 $ cd ~
897 $ git clone git://git.yoctoproject.org/poky
898 $ cd poky
899 $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
900 </literallayout>
901 In this example, the name of the top-level directory of your local
902 <link linkend='source-directory'>Source Directory</link>
903 is <filename>poky</filename>,
904 and the name of that local working area (local branch) you just
905 created and checked out is <filename>&DISTRO_NAME;</filename>.
906 The files in your local repository now reflect the same files that
907 are in the <filename>&DISTRO_NAME;</filename> development
908 branch of the Yocto Project's <filename>poky</filename>
909 upstream repository.
910 It is important to understand that when you create and checkout a
911 local working branch based on a branch name,
912 your local environment matches the "tip" of that development branch
913 at the time you created your local branch, which could be
914 different than the files at the time of a similarly named release.
915 In other words, creating and checking out a local branch based on the
916 <filename>&DISTRO_NAME;</filename> branch name is not the same as
917 cloning and checking out the <filename>master</filename> branch.
918 Keep reading to see how you create a local snapshot of a Yocto Project Release.
919 </para>
920
921 <para>
922 Git uses "tags" to mark specific changes in a repository.
923 Typically, a tag is used to mark a special point such as the final change
924 before a project is released.
925 You can see the tags used with the <filename>poky</filename> Git repository
926 by going to <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
927 clicking on the
928 <filename><ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/refs/tags'>[...]</ulink></filename>
929 link beneath the "Tag" heading.
930 </para>
931
932 <para>
933 Some key tags are <filename>bernard-5.0</filename>, <filename>denzil-7.0</filename>,
934 and <filename>&DISTRO_NAME;-&POKYVERSION;</filename>.
935 These tags represent Yocto Project releases.
936 </para>
937
938 <para>
939 When you create a local copy of the Git repository, you also have access to all the
940 tags.
941 Similar to branches, you can create and checkout a local working Git branch based
942 on a tag name.
943 When you do this, you get a snapshot of the Git repository that reflects
944 the state of the files when the change was made associated with that tag.
945 The most common use is to checkout a working branch that matches a specific
946 Yocto Project release.
947 Here is an example:
948 <literallayout class='monospaced'>
949 $ cd ~
950 $ git clone git://git.yoctoproject.org/poky
951 $ cd poky
952 $ git checkout -b my-&DISTRO_NAME;-&POKYVERSION; &DISTRO_NAME;-&POKYVERSION;
953 </literallayout>
954 In this example, the name of the top-level directory of your local Yocto Project
955 Files Git repository is <filename>poky</filename>.
956 And, the name of the local branch you have created and checked out is
957 <filename>my-&DISTRO_NAME;-&POKYVERSION;</filename>.
958 The files in your repository now exactly match the Yocto Project &DISTRO;
959 Release tag (<filename>&DISTRO_NAME;-&POKYVERSION;</filename>).
960 It is important to understand that when you create and checkout a local
961 working branch based on a tag, your environment matches a specific point
962 in time and not the entire development branch.
963 </para>
964 </section>
965
966 <section id='basic-commands'>
967 <title>Basic Commands</title>
968
969 <para>
970 Git has an extensive set of commands that lets you manage changes and perform
971 collaboration over the life of a project.
972 Conveniently though, you can manage with a small set of basic operations and workflows
973 once you understand the basic philosophy behind Git.
974 You do not have to be an expert in Git to be functional.
975 A good place to look for instruction on a minimal set of Git commands is
976 <ulink url='http://git-scm.com/documentation'>here</ulink>.
977 If you need to download Git, you can do so
978 <ulink url='http://git-scm.com/download'>here</ulink>.
979 </para>
980
981 <para>
982 If you don’t know much about Git, you should educate
983 yourself by visiting the links previously mentioned.
984 </para>
985
986 <para>
987 The following list briefly describes some basic Git operations as a way to get started.
988 As with any set of commands, this list (in most cases) simply shows the base command and
989 omits the many arguments they support.
990 See the Git documentation for complete descriptions and strategies on how to use these commands:
991 <itemizedlist>
992 <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository.
993 You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem>
994 <listitem><para><emphasis><filename>git clone</filename>:</emphasis> Creates a clone of a repository.
995 During collaboration, this command allows you to create a local repository that is on
996 equal footing with a fellow developer’s repository.</para></listitem>
997 <listitem><para><emphasis><filename>git add</filename>:</emphasis> Stages updated file contents
998 to the index that
999 Git uses to track changes.
1000 You must stage all files that have changed before you can commit them.</para></listitem>
1001 <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a "commit" that documents
1002 the changes you made.
1003 Commits are used for historical purposes, for determining if a maintainer of a project
1004 will allow the change, and for ultimately pushing the change from your local Git repository
1005 into the project’s upstream (or master) repository.</para></listitem>
1006 <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that
1007 possibly need staged and committed.</para></listitem>
1008 <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes
1009 your working branch.
1010 This command is analogous to "cd".</para></listitem>
1011 <listitem><para><emphasis><filename>git checkout –b &lt;working-branch&gt;</filename>:</emphasis> Creates
1012 a working branch on your local machine where you can isolate work.
1013 It is a good idea to use local branches when adding specific features or changes.
1014 This way if you do not like what you have done you can easily get rid of the work.</para></listitem>
1015 <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports
1016 existing local branches and
1017 tells you the branch in which you are currently working.</para></listitem>
1018 <listitem><para><emphasis><filename>git branch -D &lt;branch-name&gt;</filename>:</emphasis>
1019 Deletes an existing local branch.
1020 You need to be in a local branch other than the one you are deleting
1021 in order to delete <filename>&lt;branch-name&gt;</filename>.</para></listitem>
1022 <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information
1023 from an upstream Git
1024 repository and places it in your local Git repository.
1025 You use this command to make sure you are synchronized with the repository
1026 from which you are basing changes (.e.g. the master branch).</para></listitem>
1027 <listitem><para><emphasis><filename>git push</filename>:</emphasis>
1028 Sends all your committed local changes to an upstream Git
1029 repository (e.g. a contribution repository).
1030 The maintainer of the project draws from these repositories
1031 when adding changes to the project’s master repository or
1032 other development branch.
1033 </para></listitem>
1034 <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one
1035 local branch of your repository with another branch.
1036 When you create a local Git repository, the default branch is named "master".
1037 A typical workflow is to create a temporary branch for isolated work, make and commit your
1038 changes, switch to your local master branch, merge the changes from the temporary branch into the
1039 local master branch, and then delete the temporary branch.</para></listitem>
1040 <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific
1041 commits from one branch into another branch.
1042 There are times when you might not be able to merge all the changes in one branch with
1043 another but need to pick out certain ones.</para></listitem>
1044 <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches
1045 and changes in your local Git repository.
1046 This command is a good way to graphically see where things have diverged in your
1047 local repository.</para></listitem>
1048 <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the
1049 repository.</para></listitem>
1050 <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences
1051 between your local working files and the same files in the upstream Git repository that your
1052 branch currently tracks.</para></listitem>
1053 </itemizedlist>
1054 </para>
1055 </section>
1056</section>
1057
1058<section id='workflows'>
1059 <title>Workflows</title>
1060
1061 <para>
1062 This section provides some overview on workflows using Git.
1063 In particular, the information covers basic practices that describe roles and actions in a
1064 collaborative development environment.
1065 Again, if you are familiar with this type of development environment, you might want to just
1066 skip this section.
1067 </para>
1068
1069 <para>
1070 The Yocto Project files are maintained using Git in a "master" branch whose Git history
1071 tracks every change and whose structure provides branches for all diverging functionality.
1072 Although there is no need to use Git, many open source projects do so.
1073 For the Yocto Project, a key individual called the "maintainer" is responsible for the "master"
1074 branch of a given Git repository.
1075 The "master" branch is the “upstream” repository where the final builds of the project occur.
1076 The maintainer is responsible for allowing changes in from other developers and for
1077 organizing the underlying branch structure to reflect release strategies and so forth.
1078 <note>For information on finding out who is responsible (maintains)
1079 for a particular area of code, see the
1080 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1081 section.
1082 </note>
1083 </para>
1084
1085 <para>
1086 The project also has contribution repositories known as "contrib" areas.
1087 These areas temporarily hold changes to the project that have been submitted or committed
1088 by the Yocto Project development team and by community members that contribute to the project.
1089 The maintainer determines if the changes are qualified to be moved from the "contrib" areas
1090 into the "master" branch of the Git repository.
1091 </para>
1092
1093 <para>
1094 Developers (including contributing community members) create and maintain cloned repositories
1095 of the upstream "master" branch.
1096 These repositories are local to their development platforms and are used to develop changes.
1097 When a developer is satisfied with a particular feature or change, they "push" the changes
1098 to the appropriate "contrib" repository.
1099 </para>
1100
1101 <para>
1102 Developers are responsible for keeping their local repository up-to-date with "master".
1103 They are also responsible for straightening out any conflicts that might arise within files
1104 that are being worked on simultaneously by more than one person.
1105 All this work is done locally on the developer’s machines before anything is pushed to a
1106 "contrib" area and examined at the maintainer’s level.
1107 </para>
1108
1109 <para>
1110 A somewhat formal method exists by which developers commit changes and push them into the
1111 "contrib" area and subsequently request that the maintainer include them into "master"
1112 This process is called “submitting a patch” or "submitting a change."
1113 For information on submitting patches and changes, see the
1114 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" section.
1115 </para>
1116
1117 <para>
1118 To summarize the environment: we have a single point of entry for changes into the project’s
1119 "master" branch of the Git repository, which is controlled by the project’s maintainer.
1120 And, we have a set of developers who independently develop, test, and submit changes
1121 to "contrib" areas for the maintainer to examine.
1122 The maintainer then chooses which changes are going to become a permanent part of the project.
1123 </para>
1124
1125 <para>
1126 <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
1127 </para>
1128
1129 <para>
1130 While each development environment is unique, there are some best practices or methods
1131 that help development run smoothly.
1132 The following list describes some of these practices.
1133 For more information about Git workflows, see the workflow topics in the
1134 <ulink url='http://book.git-scm.com'>Git Community Book</ulink>.
1135 <itemizedlist>
1136 <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit
1137 small as compared to bundling many disparate changes into a single commit.
1138 This practice not only keeps things manageable but also allows the maintainer
1139 to more easily include or refuse changes.</para>
1140 <para>It is also good practice to leave the repository in a state that allows you to
1141 still successfully build your project. In other words, do not commit half of a feature,
1142 then add the other half as a separate, later commit.
1143 Each commit should take you from one buildable project state to another
1144 buildable state.</para></listitem>
1145 <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and
1146 delete local branches in your working Git repository.
1147 You can name these branches anything you like.
1148 It is helpful to give them names associated with the particular feature or change
1149 on which you are working.
1150 Once you are done with a feature or change and have merged it
1151 into your local master branch, simply discard the temporary
1152 branch.</para></listitem>
1153 <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename>
1154 command allows you to take the
1155 changes from one branch and fold them into another branch.
1156 This process is especially helpful when more than a single developer might be working
1157 on different parts of the same feature.
1158 Merging changes also automatically identifies any collisions or "conflicts"
1159 that might happen as a result of the same lines of code being altered by two different
1160 developers.</para></listitem>
1161 <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should
1162 use a system where branches indicate varying levels of code readiness.
1163 For example, you can have a "work" branch to develop in, a "test" branch where the code or
1164 change is tested, a "stage" branch where changes are ready to be committed, and so forth.
1165 As your project develops, you can merge code across the branches to reflect ever-increasing
1166 stable states of the development.</para></listitem>
1167 <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the
1168 concept of developers "pushing" local commits to a remote repository, which is
1169 usually a contribution repository.
1170 This workflow is also based on developers "pulling" known states of the project down into their
1171 local development repositories.
1172 The workflow easily allows you to pull changes submitted by other developers from the
1173 upstream repository into your work area ensuring that you have the most recent software
1174 on which to develop.
1175 The Yocto Project has two scripts named <filename>create-pull-request</filename> and
1176 <filename>send-pull-request</filename> that ship with the release to facilitate this
1177 workflow.
1178 You can find these scripts in the <filename>scripts</filename>
1179 folder of the
1180 <link linkend='source-directory'>Source Directory</link>.
1181 For information on how to use these scripts, see the
1182 "<link linkend='pushing-a-change-upstream'>Using Scripts to Push a Change Upstream and Request a Pull</link>" section.
1183 </para></listitem>
1184 <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the
1185 maintainer through an email that you have a change (or patch) you would like considered
1186 for the "master" branch of the Git repository.
1187 To send this type of change, you format the patch and then send the email using the Git commands
1188 <filename>git format-patch</filename> and <filename>git send-email</filename>.
1189 For information on how to use these scripts, see the
1190 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1191 section.
1192 </para></listitem>
1193 </itemizedlist>
1194 </para>
1195</section>
1196
1197<section id='tracking-bugs'>
1198 <title>Tracking Bugs</title>
1199
1200 <para>
1201 The Yocto Project uses its own implementation of
1202 <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs.
1203 Implementations of Bugzilla work well for group development because they track bugs and code
1204 changes, can be used to communicate changes and problems with developers, can be used to
1205 submit and review patches, and can be used to manage quality assurance.
1206 The home page for the Yocto Project implementation of Bugzilla is
1207 <ulink url='&YOCTO_BUGZILLA_URL;'>&YOCTO_BUGZILLA_URL;</ulink>.
1208 </para>
1209
1210 <para>
1211 Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself
1212 such as when discovering an issue with some component of the build system that acts contrary
1213 to the documentation or your expectations.
1214 Following is the general procedure for submitting a new bug using the Yocto Project
1215 Bugzilla.
1216 You can find more information on defect management, bug tracking, and feature request
1217 processes all accomplished through the Yocto Project Bugzilla on the wiki page
1218 <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>here</ulink>.
1219 <orderedlist>
1220 <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit
1221 a bug.</para></listitem>
1222 <listitem><para>When submitting a new bug, be sure to choose the appropriate
1223 Classification, Product, and Component for which the issue was found.
1224 Defects for the Yocto Project fall into one of six classifications: Yocto Project
1225 Components, Infrastructure, Build System &amp; Metadata, Documentation,
1226 QA/Testing, and Runtime.
1227 Each of these Classifications break down into multiple Products and, in some
1228 cases, multiple Components.</para></listitem>
1229 <listitem><para>Use the bug form to choose the correct Hardware and Architecture
1230 for which the bug applies.</para></listitem>
1231 <listitem><para>Indicate the Yocto Project version you were using when the issue
1232 occurred.</para></listitem>
1233 <listitem><para>Be sure to indicate the Severity of the bug.
1234 Severity communicates how the bug impacted your work.</para></listitem>
1235 <listitem><para>Select the appropriate "Documentation change" item
1236 for the bug.
1237 Fixing a bug may or may not affect the Yocto Project
1238 documentation.</para></listitem>
1239 <listitem><para>Provide a brief summary of the issue.
1240 Try to limit your summary to just a line or two and be sure to capture the
1241 essence of the issue.</para></listitem>
1242 <listitem><para>Provide a detailed description of the issue.
1243 You should provide as much detail as you can about the context, behavior, output,
1244 and so forth that surrounds the issue.
1245 You can even attach supporting files for output from logs by
1246 using the "Add an attachment" button.</para></listitem>
1247 <listitem><para>Be sure to copy the appropriate people in the
1248 "CC List" for the bug.
1249 See the "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1250 section for information about finding out who is responsible
1251 for code.</para></listitem>
1252 <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem>
1253 </orderedlist>
1254 </para>
1255</section>
1256
1257<section id='how-to-submit-a-change'>
1258 <title>How to Submit a Change</title>
1259
1260 <para>
1261 Contributions to the Yocto Project and OpenEmbedded are very welcome.
1262 Because the system is extremely configurable and flexible, we recognize that developers
1263 will want to extend, configure or optimize it for their specific uses.
1264 You should send patches to the appropriate mailing list so that they
1265 can be reviewed and merged by the appropriate maintainer.
1266 </para>
1267
1268 <para>
1269 Before submitting any change, be sure to find out who you should be
1270 notifying.
1271 Several methods exist through which you find out who you should be copying
1272 or notifying:
1273 <itemizedlist>
1274 <listitem><para><emphasis>Maintenance File:</emphasis>
1275 Examine the <filename>maintainers.inc</filename> file, which is
1276 located in the
1277 <link linkend='source-directory'>Source Directory</link>
1278 at <filename>meta-yocto/conf/distro/include</filename>, to
1279 see who is responsible for code.
1280 </para></listitem>
1281 <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis>
1282 For BSP maintainers of supported BSPs, you can examine
1283 individual BSP <filename>README</filename> files.
1284 Alternatively, you can examine the
1285 <filename>MAINTAINERS</filename> file, which is found in the
1286 <filename>meta-intel</filename>, for a list of all supported
1287 BSP maintainers.
1288 </para></listitem>
1289 <listitem><para><emphasis>Search by File:</emphasis>
1290 Using <link linkend='git'>Git</link>, you can enter the
1291 following command to bring up a short list of all commits
1292 against a specific file:
1293 <literallayout class='monospaced'>
1294 git shortlog -- &lt;filename&gt;
1295 </literallayout>
1296 Just provide the name of the file for which you are interested.
1297 The information returned is not ordered by history but does
1298 include a list of all committers grouped by name.
1299 From the list, you can see who is responsible for the bulk of
1300 the changes against the file.
1301 </para></listitem>
1302 </itemizedlist>
1303 </para>
1304
1305 <para>
1306 For a list of the Yocto Project and related mailing lists, see the
1307 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" section in
1308 the Yocto Project Reference Manual.
1309 </para>
1310
1311 <para>
1312 Here is some guidance on which mailing list to use for what type of change:
1313 <itemizedlist>
1314 <listitem><para>For changes to the core
1315 <link linkend='metadata'>Metadata</link>, send your patch to the
1316 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-core'>openembedded-core</ulink> mailing list.
1317 For example, a change to anything under the <filename>meta</filename> or
1318 <filename>scripts</filename> directories
1319 should be sent to this mailing list.</para></listitem>
1320 <listitem><para>For changes to BitBake (anything under the <filename>bitbake</filename>
1321 directory), send your patch to the
1322 <ulink url='&OE_LISTS_URL;/listinfo/bitbake-devel'>bitbake-devel</ulink> mailing list.</para></listitem>
1323 <listitem><para>For changes to <filename>meta-yocto</filename>, send your patch to the
1324 <ulink url='&YOCTO_LISTS_URL;/listinfo/poky'>poky</ulink> mailing list.</para></listitem>
1325 <listitem><para>For changes to other layers hosted on
1326 <filename>yoctoproject.org</filename> (unless the
1327 layer's documentation specifies otherwise), tools, and Yocto Project
1328 documentation, use the
1329 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> mailing list.</para></listitem>
1330 <listitem><para>For additional recipes that do not fit into the core Metadata,
1331 you should determine which layer the recipe should go into and submit the
1332 change in the manner recommended by the documentation (e.g. README) supplied
1333 with the layer. If in doubt, please ask on the
1334 <ulink url='&YOCTO_LISTS_URL;/listinfo/yocto'>yocto</ulink> or
1335 <ulink url='&OE_LISTS_URL;/listinfo/openembedded-devel'>openembedded-devel</ulink>
1336 mailing lists.</para></listitem>
1337 </itemizedlist>
1338 </para>
1339
1340 <para>
1341 When you send a patch, be sure to include a "Signed-off-by:"
1342 line in the same style as required by the Linux kernel.
1343 Adding this line signifies that you, the submitter, have agreed to the Developer's Certificate of Origin 1.1
1344 as follows:
1345 <literallayout class='monospaced'>
1346 Developer's Certificate of Origin 1.1
1347
1348 By making a contribution to this project, I certify that:
1349
1350 (a) The contribution was created in whole or in part by me and I
1351 have the right to submit it under the open source license
1352 indicated in the file; or
1353
1354 (b) The contribution is based upon previous work that, to the best
1355 of my knowledge, is covered under an appropriate open source
1356 license and I have the right under that license to submit that
1357 work with modifications, whether created in whole or in part
1358 by me, under the same open source license (unless I am
1359 permitted to submit under a different license), as indicated
1360 in the file; or
1361
1362 (c) The contribution was provided directly to me by some other
1363 person who certified (a), (b) or (c) and I have not modified
1364 it.
1365
1366 (d) I understand and agree that this project and the contribution
1367 are public and that a record of the contribution (including all
1368 personal information I submit with it, including my sign-off) is
1369 maintained indefinitely and may be redistributed consistent with
1370 this project or the open source license(s) involved.
1371 </literallayout>
1372 </para>
1373
1374 <para>
1375 In a collaborative environment, it is necessary to have some sort of standard
1376 or method through which you submit changes.
1377 Otherwise, things could get quite chaotic.
1378 One general practice to follow is to make small, controlled changes.
1379 Keeping changes small and isolated aids review, makes merging/rebasing easier
1380 and keeps the change history clean when anyone needs to refer to it in future.
1381 </para>
1382
1383 <para>
1384 When you make a commit, you must follow certain standards established by the
1385 OpenEmbedded and Yocto Project development teams.
1386 For each commit, you must provide a single-line summary of the change and you
1387 should almost always provide a more detailed description of what you did (i.e.
1388 the body of the commit message).
1389 The only exceptions for not providing a detailed description would be if your
1390 change is a simple, self-explanatory change that needs no further description
1391 beyond the summary.
1392 Here are the guidelines for composing a commit message:
1393 <itemizedlist>
1394 <listitem><para>Provide a single-line, short summary of the change.
1395 This summary is typically viewable in the "shortlist" of changes.
1396 Thus, providing something short and descriptive that gives the reader
1397 a summary of the change is useful when viewing a list of many commits.
1398 This short description should be prefixed by the recipe name (if changing a recipe), or
1399 else the short form path to the file being changed.
1400 </para></listitem>
1401 <listitem><para>For the body of the commit message, provide detailed information
1402 that describes what you changed, why you made the change, and the approach
1403 you used. It may also be helpful if you mention how you tested the change.
1404 Provide as much detail as you can in the body of the commit message.
1405 </para></listitem>
1406 <listitem><para>If the change addresses a specific bug or issue that is
1407 associated with a bug-tracking ID, include a reference to that ID in
1408 your detailed description.
1409 For example, the Yocto Project uses a specific convention for bug
1410 references - any commit that addresses a specific bug should include the
1411 bug ID in the description (typically at the beginning) as follows:
1412 <literallayout class='monospaced'>
1413 [YOCTO #&lt;bug-id&gt;]
1414
1415 &lt;detailed description of change&gt;
1416 </literallayout></para></listitem>
1417 Where &lt;bug-id&gt; is replaced with the specific bug ID from the
1418 Yocto Project Bugzilla instance.
1419 </itemizedlist>
1420 </para>
1421
1422 <para>
1423 You can find more guidance on creating well-formed commit messages at this OpenEmbedded
1424 wiki page:
1425 <ulink url='&OE_HOME_URL;/wiki/Commit_Patch_Message_Guidelines'></ulink>.
1426 </para>
1427
1428 <para>
1429 The next two sections describe general instructions for both pushing
1430 changes upstream and for submitting changes as patches.
1431 </para>
1432
1433 <section id='pushing-a-change-upstream'>
1434 <title>Using Scripts to Push a Change Upstream and Request a Pull</title>
1435
1436 <para>
1437 The basic flow for pushing a change to an upstream "contrib" Git repository is as follows:
1438 <itemizedlist>
1439 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1440 <listitem><para>Stage your changes by using the <filename>git add</filename>
1441 command on each file you changed.</para></listitem>
1442 <listitem><para>Commit the change by using the <filename>git commit</filename>
1443 command and push it to the "contrib" repository.
1444 Be sure to provide a commit message that follows the project’s commit message standards
1445 as described earlier.</para></listitem>
1446 <listitem><para>Notify the maintainer that you have pushed a change by making a pull
1447 request.
1448 The Yocto Project provides two scripts that conveniently let you generate and send
1449 pull requests to the Yocto Project.
1450 These scripts are <filename>create-pull-request</filename> and
1451 <filename>send-pull-request</filename>.
1452 You can find these scripts in the <filename>scripts</filename> directory
1453 within the <link linkend='source-directory'>Source Directory</link>.</para>
1454 <para>Using these scripts correctly formats the requests without introducing any
1455 whitespace or HTML formatting.
1456 The maintainer that receives your patches needs to be able to save and apply them
1457 directly from your emails.
1458 Using these scripts is the preferred method for sending patches.</para>
1459 <para>For help on using these scripts, simply provide the
1460 <filename>-h</filename> argument as follows:
1461 <literallayout class='monospaced'>
1462 $ ~/poky/scripts/create-pull-request -h
1463 $ ~/poky/scripts/send-pull-request -h
1464 </literallayout></para></listitem>
1465 </itemizedlist>
1466 </para>
1467
1468 <para>
1469 You can find general Git information on how to push a change upstream in the
1470 <ulink url='http://book.git-scm.com/3_distributed_workflows.html'>Git Community Book</ulink>.
1471 </para>
1472 </section>
1473
1474 <section id='submitting-a-patch'>
1475 <title>Using Email to Submit a Patch</title>
1476
1477 <para>
1478 You can submit patches without using the <filename>create-pull-request</filename> and
1479 <filename>send-pull-request</filename> scripts described in the previous section.
1480 However, keep in mind, the preferred method is to use the scripts.
1481 </para>
1482
1483 <para>
1484 Depending on the components changed, you need to submit the email to a specific
1485 mailing list.
1486 For some guidance on which mailing list to use, see the list in the
1487 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1488 section.
1489 For a description of the available mailing lists, see the
1490 "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
1491 section in the Yocto Project Reference Manual.
1492 </para>
1493
1494 <para>
1495 Here is the general procedure on how to submit a patch through email without using the
1496 scripts:
1497 <itemizedlist>
1498 <listitem><para>Make your changes in your local Git repository.</para></listitem>
1499 <listitem><para>Stage your changes by using the <filename>git add</filename>
1500 command on each file you changed.</para></listitem>
1501 <listitem><para>Commit the change by using the
1502 <filename>git commit --signoff</filename> command.
1503 Using the <filename>--signoff</filename> option identifies you as the person
1504 making the change and also satisfies the Developer's Certificate of
1505 Origin (DCO) shown earlier.</para>
1506 <para>When you form a commit, you must follow certain standards established by the
1507 Yocto Project development team.
1508 See the earlier section
1509 "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
1510 for Yocto Project commit message standards.</para></listitem>
1511 <listitem><para>Format the commit into an email message.
1512 To format commits, use the <filename>git format-patch</filename> command.
1513 When you provide the command, you must include a revision list or a number of patches
1514 as part of the command.
1515 For example, either of these two commands takes your most
1516 recent single commit and formats it as an email message in
1517 the current directory:
1518 <literallayout class='monospaced'>
1519 $ git format-patch -1
1520 </literallayout>
1521 or
1522 <literallayout class='monospaced'>
1523 $ git format-patch HEAD~
1524 </literallayout></para>
1525 <para>After the command is run, the current directory contains a
1526 numbered <filename>.patch</filename> file for the commit.</para>
1527 <para>If you provide several commits as part of the command,
1528 the <filename>git format-patch</filename> command produces a
1529 series of numbered files in the current directory – one for each commit.
1530 If you have more than one patch, you should also use the
1531 <filename>--cover</filename> option with the command, which generates a
1532 cover letter as the first "patch" in the series.
1533 You can then edit the cover letter to provide a description for
1534 the series of patches.
1535 For information on the <filename>git format-patch</filename> command,
1536 see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the
1537 <filename>man git-format-patch</filename> command.</para>
1538 <note>If you are or will be a frequent contributor to the Yocto Project
1539 or to OpenEmbedded, you might consider requesting a contrib area and the
1540 necessary associated rights.</note></listitem>
1541 <listitem><para>Import the files into your mail client by using the
1542 <filename>git send-email</filename> command.
1543 <note>In order to use <filename>git send-email</filename>, you must have the
1544 the proper Git packages installed.
1545 For Ubuntu, Debian, and Fedora the package is <filename>git-email</filename>.</note></para>
1546 <para>The <filename>git send-email</filename> command sends email by using a local
1547 or remote Mail Transport Agent (MTA) such as
1548 <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct
1549 <filename>smtp</filename> configuration in your Git <filename>config</filename>
1550 file.
1551 If you are submitting patches through email only, it is very important
1552 that you submit them without any whitespace or HTML formatting that
1553 either you or your mailer introduces.
1554 The maintainer that receives your patches needs to be able to save and
1555 apply them directly from your emails.
1556 A good way to verify that what you are sending will be applicable by the
1557 maintainer is to do a dry run and send them to yourself and then
1558 save and apply them as the maintainer would.</para>
1559 <para>The <filename>git send-email</filename> command is the preferred method
1560 for sending your patches since there is no risk of compromising whitespace
1561 in the body of the message, which can occur when you use your own mail client.
1562 The command also has several options that let you
1563 specify recipients and perform further editing of the email message.
1564 For information on how to use the <filename>git send-email</filename> command,
1565 see <filename>GIT-SEND-EMAIL(1)</filename> displayed using
1566 the <filename>man git-send-email</filename> command.
1567 </para></listitem>
1568 </itemizedlist>
1569 </para>
1570 </section>
1571</section>
1572</chapter>
1573<!--
1574vim: expandtab tw=80 ts=4
1575-->
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-04-25 09:28:44 +0000