path: root/documentation/dev-manual/dev-manual-newbie.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">

<chapter id='dev-manual-newbie'>

<title>The Yocto Project Open Source Development Environment</title>

<para>
    This chapter helps you understand the Yocto Project as an open source development project.
    In general, working in an open source environment is very different from working in a 
    closed, proprietary environment.  
    Additionally, the Yocto Project uses specific tools and constructs as part of its development 
    environment.  
    This chapter specifically addresses open source philosophy, licensing issues, code repositories, 
    the open source distributed version control system Git, and best practices using the Yocto Project.
</para>

<section id='open-source-philosophy'>
    <title>Open Source Philosophy</title>

    <para>
        Open source philosophy is characterized by software development directed by peer production 
        and collaboration through an active community of developers.
        Contrast this to the more standard centralized development models used by commercial software 
        companies where a finite set of developers produce a product for sale using a defined set 
        of procedures that ultimately result in an end product whose architecture and source material
        are closed to the public.
    </para>

    <para>
        Open source projects conceptually have differing concurrent agendas, approaches, and production.   
        These facets of the development process can come from anyone in the public (community) that has a 
        stake in the software project.  
        The open source environment contains new copyright, licensing, domain, and consumer issues 
        that differ from the more traditional development environment.  
        In an open source environment, the end product, source material, and documentation are
        all available to the public at no cost.
    </para>

    <para>
        A benchmark example of an open source project is the Linux Kernel, which was initially conceived 
        and created by Finnish computer science student Linus Torvalds in 1991.  
        Conversely, a good example of a non-open source project is the 
        <trademark class='registered'>Windows</trademark> family of operating 
        systems developed by <trademark class='registered'>Microsoft</trademark> Corporation.
    </para>

    <para>
        Wikipedia has a good historical description of the Open Source Philosophy  
        <ulink url='http://en.wikipedia.org/wiki/Open_source'>here</ulink>. 
        You can also find helpful information on how to participate in the Linux Community 
        <ulink url='http://ldn.linuxfoundation.org/book/how-participate-linux-community'>here</ulink>.
    </para>
</section>

<section id='yocto-project-repositories'>
    <title>Yocto Project Source Repositories</title>

    <para>
        The Yocto Project team maintains complete source repositories for all Yocto Project files 
        at <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'></ulink>.
        This web-based source code browser is organized into categories by function such as
        IDE Plugins, Matchbox, Poky, Yocto Linux Kernel, and so forth.
        From the interface, you can click on any particular item in the "Name" column and 
        see the URL at the bottom of the page that you need to set up a Git repository for 
        that particular item.
        Having a local Git repository of the Yocto Project files allows you to 
        make changes, contribute to the history, and ultimately enhance the Yocto Project's 
        tools, Board Support Packages, and so forth.
    </para>

    <para>  
        Conversely, if you are a developer that is not interested in contributing back to the 
        Yocto Project, you have the ability to simply download and extract release tarballs
        and use them within the Yocto Project environment.
        All that is required is a particular release of the Yocto Project and 
        your application source code.  
    </para>

    <para>
        For any supported release of Yocto Project, you can go to the Yocto Project website’s 
        <ulink url='http://www.yoctoproject.org/download'>download page</ulink> and get a 
        tarball of the release.  
        You can also go to this site to download any supported BSP tarballs.
        Unpacking the tarball gives you a hierarchical directory structure of Yocto Project 
        files that lets you develop using the Yocto Project.
    </para>

    <para>
        Once you are set up through either tarball extraction or creation of Git repositories, 
        you are ready to develop.
    </para>

    <para>
        In summary, here is where you can get the Yocto Project files needed for development:
        <itemizedlist>
            <listitem><para><emphasis><ulink url='http://git.yoctoproject.org/cgit/cgit.cgi'>Source Repositories:</ulink></emphasis>
                This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto 
                Metadata Layers.
                You can create Git repositories for each of these areas.</para>
                <para>
                <imagedata fileref="figures/source-repos.png" align="center" width="6in" depth="4in" />
                </para></listitem>
            <listitem><para><anchor id='index-downloads' /><emphasis><ulink url='http://downloads.yoctoproject.org/releases/'>Index of /releases:</ulink></emphasis>
                This area contains index releases such as 
                the <trademark class='trade'>Eclipse</trademark>
                Yocto Plug-in, miscellaneous support, Poky, pseudo, cross-development toolchains,
                and all released versions of Yocto Project in the form of images or tarballs.
                Downloading and extracting these files does not produce a Git repository but rather 
                a snapshot of a particular release or image.</para>
                <para>
                <imagedata fileref="figures/index-downloads.png" align="center" width="6in" depth="4in" />
                </para></listitem>
            <listitem><para><emphasis><ulink url='http://www.yoctoproject.org/download'>Yocto Project Download Page</ulink></emphasis>
                This page on the Yocto Project website allows you to download any Yocto Project
                release or Board Support Package (BSP) in tarball form.
                The tarballs are similar to those found in the 
                <ulink url='http://downloads.yoctoproject.org/releases/'>Index of /releases:</ulink> area.</para>
                <para>
                <imagedata fileref="figures/yp-download.png" align="center" width="6in" depth="4in" />
            </para></listitem>
        </itemizedlist>
    </para>
</section>

<section id='yocto-project-terms'>
    <title>Yocto Project Terms</title>

    <para>
        Following is a list of terms and definitions users new to the Yocto Project development 
        environment might find helpful.
        While some of these terms are universal, the list includes them just in case:
        <itemizedlist>
            <listitem><para><emphasis>Append Files:</emphasis> Files that append build information to 
                a recipe file.  
                Information in append files overrides the information in the similarly-named recipe file.
                Append files use the <filename>.bbappend</filename> filename suffix.
                </para></listitem>
            <listitem><para><emphasis>BitBake:</emphasis> The task executor and scheduler used by 
                the Yocto Project to build images. 
                For more information on BitBake, see the <ulink url='http://bitbake.berlios.de/manual/'>
                BitBake documentation</ulink>.</para></listitem>
            <listitem><para><emphasis>Classes:</emphasis> Files that provide for logic encapsulation
                and inheritance allowing commonly used patterns to be defined once and easily used 
                in multiple recipes.  
                Class files end with the <filename>.bbclass</filename> filename extension.
                </para></listitem>
            <listitem><para><emphasis>Configuration File:</emphasis>  Configuration information in various
                <filename>.conf</filename> files provides global definitions of variables.
                The <filename>conf/local.conf</filename> configuration file in the 
                <link linkend='yocto-project-build-directory'>Yocto Project Build Directory</link>
                contains user-defined variables that affect each build.
                The <filename>meta-yocto/conf/distro/poky.conf</filename> configuration file
                defines Yocto ‘distro’ configuration
                variables used only when building with this policy.  
                Machine configuration files, which 
                are located throughout the Yocto Project file structure, define
                variables for specific hardware and are only used when building for that target 
                (e.g. the <filename>machine/beagleboard.conf</filename> configuration file defines 
                variables for the Texas Instruments ARM Cortex-A8 development board).  
                Configuration files end with a <filename>.conf</filename> filename extension.
                </para></listitem>
            <listitem><para><emphasis>Cross-Development Toolchain:</emphasis> 
                A collection of software development
                tools and utilities that allow you to develop software for targeted architectures.
                This toolchain contains cross-compilers, linkers, and debuggers that are specific to 
                an architecture.
                You can use the Yocto Project to build cross-development toolchains in tarball form that when 
                unpacked contain the development tools you need to cross-compile and test your software.
                The Yocto Project ships with images that contain toolchains for supported architectures
                as well.
                Sometimes this toolchain is referred to as the meta-toolchain.</para></listitem>
            <listitem><para><emphasis>Image:</emphasis> An image is the result produced when 
                BitBake processes a given collection of recipes and related metadata.
                Images are the binary output that runs on specific hardware and for specific
                use cases.
                For a list of the supported image types that the Yocto Project provides, see the
                "<ulink url='http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html#ref-images'>Reference: Images</ulink>"
                appendix in The Yocto Project Reference Manual.</para></listitem>
            <listitem><para><emphasis>Layer:</emphasis> A collection of recipes representing the core, 
                a BSP, or an application stack.
                For a discussion on BSP Layers, see the 
                "<ulink url='http://www.yoctoproject.org/docs/latest/bsp-guide/bsp-guide.html#bsp-layers'>BSP Layers</ulink>"
                section in the Yocto Project Board Support Packages (BSP) Developer's Guide.</para></listitem>
            <listitem><para><emphasis>Metadata:</emphasis> The files that BitBake parses when building an image. 
                Metadata includes recipes, classes, and configuration files.</para></listitem>
            <listitem><para><emphasis>OE-Core:</emphasis> A core set of metadata originating 
                with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
                This metadata is found in the <filename>meta</filename> directory of the Yocto Project
                files.</para></listitem>
            <listitem><para><emphasis>Package:</emphasis> The packaged output from a baked recipe.
                A package is generally the compiled binaries produced from the recipe's sources.  
                You ‘bake’ something by running it through BitBake.</para></listitem>
            <listitem><para><emphasis>Poky:</emphasis> The build tool that the Yocto Project 
                uses to create images.</para></listitem>
            <listitem><para><emphasis>Recipe:</emphasis> A set of instructions for building packages.  
                A recipe describes where you get source code and which patches to apply.
                Recipes describe dependencies for libraries or for other recipes, and they 
                also contain configuration and compilation options.  
                Recipes contain the logical unit of execution, the software/images to build, and 
                use the <filename>.bb</filename> file extension.</para></listitem>
            <listitem><para><emphasis>Tasks:</emphasis> Arbitrary groups of software Recipes.  
                You simply use Tasks to hold recipes that, when built, usually accomplish a single task.  
                For example, a task could contain the recipes for a company’s proprietary or value-add software.  
                Or, the task could contain the recipes that enable graphics. 
                A task is really just another recipe.  
                Because task files are recipes, they end with the <filename>.bb</filename> filename 
                extension.</para></listitem>
            <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
                that are not local to the development system but located in a master area that is controlled
                by the maintainer of the source code.
                For example, in order for a developer to work on a particular piece of code, they need to 
                first get a copy of it from an "upstream" source.</para></listitem>
            <listitem>
                <para id='yocto-project-files'><emphasis>Yocto Project Files:</emphasis>
                This term refers to the directory structure created as a result of either downloading 
                and unpacking a Yocto Project release tarball or setting up a Git repository
                by cloning <filename>git://git.yoctoproject.org/poky</filename>.
                Sometimes the term "the Yocto Project Files structure" is used as well.</para>

                <para>The Yocto Project files contain BitBake, Documentation, metadata and 
                other files that all support the development environment. 
                Consequently, you must have the Yocto Project files in place on your development 
                system in order to do any development using the Yocto Project.</para>

                <para>The name of the top-level directory of the Yocto Project file structure  
                is derived from the Yocto Project release tarball.
                For example, downloading and unpacking <filename>poky-edison-6.0.tar.bz2</filename>
                results in a Yocto Project file structure whose Yocto Project source directory is named 
                <filename>poky-edison-6.0</filename>.
                If you create a Git repository, then you can name the repository anything you like.
                By default, your local Git repository is named <filename>poky</filename>.</para>

                <para>It is important to understand the differences between Yocto Project Files created
                by unpacking a release tarball as compared to cloning 
                <filename>git://git.yoctoproject.org/poky</filename>.
                When you unpack a tarball, you have an exact copy of the files based on the time of 
                release - a fixed release point.
                Any changes you make to your local Yocto Project files are on top of the release.
                On the other hand, when you clone the Yocto Project Git repository, you have an
                active development repository.
       creates a build directory named <filename>YP-6.0</filename>
                            in your home directory within the directory <filename>mybuilds</filename>.
                            If <filename>mybuilds</filename> does not exist it is created for you.         In this case, any local changes you make to the Yocto Project can be later applied to active
                development branches of the upstream Yocto Project Git repository.
                If you want to track a set of local changes while starting from the same point
                as a release tarball, you can create a local Git branch that 
                reflects the exact copy of the files at the time of their release. 
                You do this using Git tags that are part of the repository.
                For more information on concepts, see the section 
                "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>".
                </para>

                <para>You can find instruction on how to set up the Yocto Project files on your 
                host development system by reading 
                the 
                "<ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#getting-setup'>Getting
                Setup</ulink>" section.</para></listitem>
            <listitem>
                <para id='yocto-project-build-directory'><emphasis>Yocto Project Build Directory:</emphasis>
                This term refers to the area used by the Yocto Project for builds.  
                The area is created when you <filename>source</filename> the Yocto Project setup 
                environment script that is found in the Yocto Project files area
                (i.e. <filename>oe-init-build-env</filename>).
                The <filename>TOPDIR</filename> variable points to the build directory.</para>
           
                <para>You have a lot of flexibility when creating the Yocto Project Build Directory.
                Following are some examples that show how to create the directory:
                   <itemizedlist>
                        <listitem><para>Create the build directory in your current working directory
                            and name it <filename>build</filename>.
                            This is the default behavior.
                            <literallayout class='monospaced'>
     $ cd ~/poky
     $ source oe-init-build-env
                            </literallayout></para></listitem>
                        <listitem><para>Provide a directory path and specifically name the build 
                            directory. 
                            This next example creates a build directory named <filename>YP-6.0</filename>
                            in your home directory within the directory <filename>mybuilds</filename>.
                            If <filename>mybuilds</filename> does not exist, the directory is created for you:
                            <literallayout class='monospaced'> 
     $ source poky-edison-6.0/oe-init-build-env $HOME/mybuilds/YP-6.0
                            </literallayout></para></listitem>
                        <listitem><para>Provide an existing directory to use as the build directory.
                            This example uses the existing <filename>mybuilds</filename> directory 
                            as the build directory.
                            <literallayout class='monospaced'> 
     $ source poky-edison-6.0/oe-init-build-env $HOME/mybuilds/
                            </literallayout></para></listitem>
                    </itemizedlist> 
                </para></listitem> 
        </itemizedlist>
    </para>
</section>

<section id='licensing'>
    <title>Licensing</title>

    <para>
        Because open source projects are open to the public, they have different licensing structures in place.  
        License evolution for both Open Source and Free Software has an interesting history.  
        If you are interested in this history, you can find basic information here:
    <itemizedlist>
        <listitem><para><ulink url='http://en.wikipedia.org/wiki/Open-source_license'>Open source license history</ulink>
            </para></listitem>
        <listitem><para><ulink url='http://en.wikipedia.org/wiki/Free_software_license'>Free software license
            history</ulink></para></listitem>
    </itemizedlist>
    </para>

    <para>
        In general, the Yocto Project is broadly licensed under the Massachusetts Institute of Technology 
        (MIT) License.  
        MIT licensing permits the reuse of software within proprietary software as long as the 
        license is distributed with that software.  
        MIT is also compatible with the GNU General Public License (GPL).  
        Patches to the Yocto Project follow the upstream licensing scheme.
        You can find information on the MIT license at  
        <ulink url='http://www.opensource.org/licenses/mit-license.php'>here</ulink>.
        You can find information on the GNU GPL <ulink url='http://www.opensource.org/licenses/LGPL-3.0'>
        here</ulink>. 
    </para>

    <para>
        When you build an image using Yocto Project, the build process uses a known list of licenses to 
        ensure compliance.
        You can find this list in the Yocto Project files directory at 
        <filename>meta/files/common-licenses</filename>.  
        Once the build completes, the list of all licenses found and used during that build are 
        kept in the 
        <link linkend='yocto-project-build-directory'>Yocto Project Build Directory</link> at 
        <filename>tmp/deploy/images/licenses</filename>.
    </para>

    <para>  
        If a module requires a license that is not in the base list, the build process 
        generates a warning during the build.  
        These tools make it easier for a developer to be certain of the licenses with which
        their shipped products must comply.
        However, even with these tools it is still up to the developer to resolve potential licensing issues.
    </para>

    <para>
        The base list of licenses used by the build process is a combination of the Software Package 
        Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects.  
        <ulink url='http://spdx.org'>SPDX Group</ulink> is a working group of the Linux Foundation 
        that maintains a specification 
        for a standard format for communicating the components, licenses, and copyrights 
        associated with a software package.  
        <ulink url='http://opensource.org'>OSI</ulink> is a corporation dedicated to the Open Source 
        Definition and the effort for reviewing and approving licenses that are OSD-conformant.  
    </para>

    <para>
        You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses 
        <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/files/common-licenses'>here</ulink>.
        This wiki page discusses the license infrastructure used by the Yocto Project.
    </para>
</section>

<section id='git'>
    <title>Git</title>

    <para>
        The Yocto Project uses Git, which is a free, open source distributed version control system.  
        Git supports distributed development, non-linear development, and can handle large projects.  
        It is best that you have some fundamental understanding of how Git tracks projects and 
        how to work with Git if you are going to use Yocto Project for development. 
        This section provides a quick overview of how Git works and provides you with a summary
        of some essential Git commands.
    </para>

    <para>
        For more information on Git, see
        <ulink url='http://git-scm.com/documentation'></ulink>.   
        If you need to download Git, go to <ulink url='http://git-scm.com/download'></ulink>. 
    </para>

    <section id='repositories-tags-and-branches'>
        <title>Repositories, Tags, and Branches</title>

        <para>
            As mentioned earlier in section 
            "<link linkend='yocto-project-repositories'>Yocto Project Source Repositories</link>",
            the Yocto Project maintains source repositories at 
            <ulink url='http://git.yoctoproject.org/cgit.cgi'></ulink>.
            If you look at this web-interface of the repositories, each item is a separate
            Git repository.
        </para>

        <para>
            Git repositories use branching techniques that track content change (not files) 
            within a project (e.g. a new feature or updated documentation).  
            Creating a tree-like structure based on project divergence allows for excellent historical 
            information over the life of a project.  
            This methodology also allows for an environment in which you can do lots of 
            local experimentation on a project as you develop changes or new features.  
        </para>

        <para>
            A Git repository represents all development efforts for a given project.
            For example, the Git repository <filename>poky</filename> contains all changes
            and developments for Poky over the course of its entire life. 
            That means that all changes that make up all releases are captured.
            The repository maintains a complete history of changes.
        </para>

        <para>
            You can create a local copy of any repository by "cloning" it with the Git
            <filename>clone</filename> command.
            When you clone a Git repository, you end up with an identical copy of the 
            repository on your development system.  
            Once you have a local copy of a repository, you can take steps to develop locally.
            For examples on how to clone Git repositories, see the section
            "<link linkend='getting-setup'>Getting Set Up</link>" earlier in this manual.
        </para>

        <para>
            It is important to understand that Git tracks content change and not files.
            Git uses "branches" to organize different development efforts. 
            For example, the <filename>poky</filename> repository has 
            <filename>laverne</filename>, <filename>bernard</filename>, 
            <filename>edison</filename>, and <filename>master</filename> branches among 
            others.
            You can see all the branches by going to  
            <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/'></ulink> and 
            clicking on the 
            <filename><ulink url='http://git.yoctoproject.org/cgit.cgi/poky/refs/heads'>[...]</ulink></filename> 
            link beneath the "Branch" heading.
        </para>

        <para>
            Each of these branches represents a specific area of development.
            The <filename>master</filename> branch represents the current or most recent 
            development.
            All other branches represent off-shoots of the <filename>master</filename>
            branch. 
        </para>

        <para>
            When you create a local copy of a Git repository, the copy has the same set 
            of branches as the original.
            This means you can use Git to create a local working area (also called a branch)
            that tracks a specific development branch from the source Git repository.  
            in other words, you can define your local Git environment to work on any development
            branch in the repository.
            To help illustrate, here is a set of commands that creates a local copy of the 
            <filename>poky</filename> Git repository and then creates and checks out a local
            Git branch that tracks the Yocto Project 1.1 Release (edison) development:
            <literallayout class='monospaced'>
     $ cd ~
     $ git clone git://git.yoctoproject.org/poky
     $ cd poky
     $ git checkout edison -b edison
            </literallayout>
            In this example, the name of the top-level directory of your local Yocto Project 
            Files Git repository is <filename>poky</filename>.
            And, the name of the local working area (or local branch) you have created and checked 
            out is named <filename>edison</filename>.
            The files in your repository now reflect the same files that are in the 
            <filename>edison</filename> development branch of the Yocto Project's 
            <filename>poky</filename> repository.
            It is important to understand that when you create and checkout a 
            local working branch based on a branch name, 
            your local environment matches the "tip" of that development branch
            at the time you created your local branch, which could be
            different than the files at the time of a similarly named release.
            In other words, creating and checking out a local branch based on the 
            <filename>edison</filename> branch name is not the same as creating and 
            checking out a local branch based on the <filename>edison-6.0</filename>
            release.
            Keep reading to see how you create a local snapshot of a Yocto Project Release.
        </para>

        <para>
            Git uses "tags" to mark specific changes in a repository.
            Typically, a tag is used to mark a special point such as the final change
            before a project is released.  
            You can see the tags used with the <filename>poky</filename> Git repository
            by going to <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/'></ulink> and 
            clicking on the 
            <filename><ulink url='http://git.yoctoproject.org/cgit.cgi/poky/refs/tags'>[...]</ulink></filename> 
            link beneath the "Tag" heading.
        </para>

        <para>
            Some key tags are <filename>laverne-4.0</filename>, <filename>bernard-5.0</filename>,
            and <filename>edison-6.0</filename>.
            These tags represent Yocto Project releases.
        </para>

        <para>
            When you create a local copy of the Git repository, you also have access to all the 
            tags.
            Similar to branches, you can create and checkout a local working Git branch based 
            on a tag name. 
            When you do this, you get a snapshot of the Git repository that reflects 
            the state of the files when the change was made associated with that tag.
            The most common use is to checkout a working branch that matches a specific 
            Yocto Project release. 
            Here is an example:
            <literallayout class='monospaced'>
     $ cd ~
     $ git clone git://git.yoctoproject.org/poky
     $ cd poky
     $ git checkout edison-6.0 -b edison-6.0
            </literallayout>
            In this example, the name of the top-level directory of your local Yocto Project 
            Files Git repository is <filename>poky</filename>.
            And, the name of the local branch you have created and checked out is
            <filename>edison-6.0</filename>.
            The files in your repository now exactly match the Yocto Project 1.1
            Release tag (edison).
            It is important to understand that when you create and checkout a local 
            working branch based on a tag, your environment matches a specific point 
            in time and not a development branch.
        </para>
    </section> 

    <section id='basic-commands'>
        <title>Basic Commands</title>

        <para>
            Git has an extensive set of commands that lets you manage changes and perform 
            collaboration over the life of a project.  
            Conveniently though, you can manage with a small set of basic operations and workflows 
            once you understand the basic philosophy behind Git.  
            You do not have to be an expert in Git to be functional.  
            A good place to look for instruction on a minimal set of Git commands is 
            <ulink url='http://git-scm.com/documentation'>here</ulink>.   
            If you need to download Git, you can do so 
            <ulink url='http://git-scm.com/download'>here</ulink>. 
        </para>

        <para>
            If you don’t know much about Git, we suggest you educate            
            yourself by visiting the links previously mentioned.
        </para>

        <para>
            The following list briefly describes some basic Git operations as a way to get started.  
            As with any set of commands, this list (in most cases) simply shows the base command and 
            omits the many arguments they support.  
            See the Git documentation for complete descriptions and strategies on how to use these commands:
            <itemizedlist>
                <listitem><para><emphasis><filename>git init</filename>:</emphasis> Initializes an empty Git repository.  
                    You cannot use Git commands unless you have a <filename>.git</filename> repository.</para></listitem>
                <listitem><para><emphasis><filename>git clone</filename>:</emphasis> Creates a clone of a repository.  
                    During collaboration, this command allows you to create a local repository that is on 
                    equal footing with a fellow developer’s repository.</para></listitem>
                <listitem><para><emphasis><filename>git add</filename>:</emphasis> Adds updated file contents 
                    to the index that 
                    Git uses to track changes.  
                    You must add all files that have changed before you can commit them.</para></listitem>
                <listitem><para><emphasis><filename>git commit</filename>:</emphasis> Creates a “commit” that documents 
                    the changes you made.  
                    Commits are used for historical purposes, for determining if a maintainer of a project 
                    will allow the change, and for ultimately pushing the change from your local Git repository 
                    into the project’s upstream (or master) repository.</para></listitem>
                <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that 
                    possibly need to be added and committed.</para></listitem>
                <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes 
                    your working branch.  
                    This command is analogous to “cd”.</para></listitem>
                <listitem><para><emphasis><filename>git checkout –b &lt;working-branch&gt;</filename>:</emphasis> Creates 
                    a working branch on your local machine where you can isolate work.  
                    It is a good idea to use local branches when adding specific features or changes.  
                    This way if you don’t like what you have done you can easily get rid of the work.</para></listitem>
                <listitem><para><emphasis><filename>git branch</filename>:</emphasis> Reports existing branches and 
                    tells you which branch in which you are currently working.</para></listitem>
                <listitem><para><emphasis><filename>git branch -D &lt;branch-name&gt;</filename>:</emphasis> 
                    Deletes an existing branch.  
                    You need to be in a branch other than the one you are deleting 
                    in order to delete &lt;branch-name&gt;.</para></listitem>
                <listitem><para><emphasis><filename>git pull</filename>:</emphasis> Retrieves information 
                    from an upstream Git 
                    repository and places it in your local Git repository.  
                    You use this command to make sure you are synchronized with the repository 
                    from which you are basing changes (.e.g. the master repository).</para></listitem>
                <listitem><para><emphasis><filename>git push</filename>:</emphasis> Sends all your local changes you 
                    have committed to an upstream Git repository (e.g. a contribution repository).  
                    The maintainer of the project draws from these repositories when adding your changes to the 
                    project’s master repository.</para></listitem>
                <listitem><para><emphasis><filename>git merge</filename>:</emphasis> Combines or adds changes from one 
                    local branch of your repository with another branch.  
                    When you create a local Git repository, the default branch is named “master”.  
                    A typical workflow is to create a temporary branch for isolated work, make and commit your 
                    changes, switch to your local master branch, merge the changes from the temporary branch into the 
                    local master branch, and then delete the temporary branch.</para></listitem>
                <listitem><para><emphasis><filename>git cherry-pick</filename>:</emphasis> Choose and apply specific 
                    commits from one branch into another branch.  
                    There are times when you might not be able to merge all the changes in one branch with 
                    another but need to pick out certain ones.</para></listitem>
                <listitem><para><emphasis><filename>gitk</filename>:</emphasis> Provides a GUI view of the branches 
                    and changes in your local Git repository.  
                    This command is a good way to graphically see where things have diverged in your 
                    local repository.</para></listitem>
                <listitem><para><emphasis><filename>git log</filename>:</emphasis> Reports a history of your changes to the 
                    repository.</para></listitem>
                <listitem><para><emphasis><filename>git diff</filename>:</emphasis> Displays line-by-line differences
                    between your local working files and the same files in the upstream Git repository that your 
                    branch currently tracks.</para></listitem>
            </itemizedlist>
        </para>
    </section>
</section>

<section id='workflows'>
    <title>Workflows</title>

    <para>
        This section provides some overview on workflows using Git.  
        In particular, the information covers basic practices that describe roles and actions in a 
        collaborative development environment.  
        Again, if you are familiar with this type of development environment, you might want to just 
        skip this section.
    </para>

    <para>
        The Yocto Project files are maintained using Git in a "master" branch whose Git history 
        tracks every change and whose structure provides branches for all diverging functionality.
        Although there is no need to use Git, many open source projects do so.
        For the Yocto Project, a key individual called the "maintainer" is responsible for the "master"
        branch of the Git repository.
        The "master" branch is the “upstream” repository where the final builds of the project occur.  
        The maintainer is responsible for allowing changes in from other developers and for 
        organizing the underlying branch structure to reflect release strategies and so forth.  
        <note>You can see who is the maintainer for Yocto Project files by examining the 
        <filename>distro_tracking_fields.inc</filename> file in the Yocto Project
        <filename>meta/conf/distro/include</filename> directory.</note>  
    </para>

    <para>
        The project also has contribution repositories known as “contrib” areas.  
        These areas temporarily hold changes to the project that have been submitted or committed 
        by the Yocto Project development team and by community members that contribute to the project.  
        The maintainer determines if the changes are qualified to be moved from the "contrib" areas
        into the "master" branch of the Git repository.
    </para>

    <para>
        Developers (including contributing community members) create and maintain cloned repositories 
        of the upstream "master" branch.  
        These repositories are local to their development platforms and are used to develop changes.  
        When a developer is satisfied with a particular feature or change, they “push” the changes 
        to the appropriate "contrib" repository.
    </para>

    <para>  
        Developers are responsible for keeping their local repository up-to-date with "master".  
        They are also responsible for straightening out any conflicts that might arise within files 
        that are being worked on simultaneously by more than one person.  
        All this work is done locally on the developer’s machine before anything is pushed to a 
        "contrib" area and examined at the maintainer’s level.
    </para>

    <para>
        A somewhat formal method exists by which developers commit changes and push them into the 
        "contrib" area and subsequently request that the maintainer include them into "master"  
        This process is called “submitting a patch” or “submitting a change.”  
    </para>

    <para>
        To summarize the environment:  we have a single point of entry for changes into the project’s 
        "master" branch of the Git repository, which is controlled by the project’s maintainer.  
        And, we have a set of developers who independently develop, test, and submit changes 
        to "contrib" areas for the maintainer to examine.  
        The maintainer then chooses which changes are going to become a permanent part of the project.
    </para>

    <para>
        <imagedata fileref="figures/git-workflow.png" width="6in" depth="3in" align="left" scalefit="1" />
    </para> 

    <para>
        While each development environment is unique, there are some best practices or methods 
        that help development run smoothly.  
        The following list describes some of these practices.  
        For more information about Git workflows, see the workflow topics in the 
        <ulink url='http://book.git-scm.com'>Git Community Book</ulink>. 
        <itemizedlist>
            <listitem><para><emphasis>Make Small Changes:</emphasis> It is best to keep the changes you commit
                small as compared to bundling many disparate changes into a single commit.  
                This practice not only keeps things manageable but also allows the maintainer 
                to more easily include or refuse changes.</para>
                <para>It is also good practice to leave the repository in a state that allows you to 
                still successfully build your project.  In other words, do not commit half of a feature,
                then add the other half in a separate, later commit.  
                Each commit should take you from one buildable project state to another 
                buildable state.</para></listitem>
            <listitem><para><emphasis>Use Branches Liberally:</emphasis> It is very easy to create, use, and 
                delete local branches in your working Git repository.  
                You can name these branches anything you like.  
                It is helpful to give them names associated with the particular feature or change 
                on which you are working.  
                Once you are done with a feature or change, simply discard the branch.</para></listitem>
            <listitem><para><emphasis>Merge Changes:</emphasis> The <filename>git merge</filename>
                command allows you to take the 
                changes from one branch and fold them into another branch.  
                This process is especially helpful when more than a single developer might be working 
                on different parts of the same feature.  
                Merging changes also automatically identifies any collisions or “conflicts” 
                that might happen as a result of the same lines of code being altered by two different 
                developers.</para></listitem>
            <listitem><para><emphasis>Manage Branches:</emphasis> Because branches are easy to use, you should 
                use a system where branches indicate varying levels of code readiness.  
                For example, you can have a “work” branch to develop in, a “test” branch where the code or 
                change is tested, a “stage” branch where changes are ready to be committed, and so forth.  
                As your project develops, you can merge code across the branches to reflect ever-increasing 
                stable states of the development.</para></listitem>
            <listitem><para><emphasis>Use Push and Pull:</emphasis> The push-pull workflow is based on the 
                concept of developers “pushing” local commits to a remote repository, which is 
                usually a contribution repository.  
                This workflow is also based on developers “pulling” known states of the project down into their 
                local development repositories.  
                The workflow easily allows you to pull changes submitted by other developers from the 
                upstream repository into your work area ensuring that you have the most recent software 
                on which to develop.
                The Yocto Project has two scripts named <filename>create-pull-request</filename> and 
                <filename>send-pull-request</filename> that ship with the release to facilitate this 
                workflow.
                You can find these scripts in the local Yocto Project files Git repository in
                the <filename>scripts</filename> directory.</para></listitem>
            <listitem><para><emphasis>Patch Workflow:</emphasis> This workflow allows you to notify the 
                maintainer through an email that you have a change (or patch) you would like considered 
                for the "master" branch of the Git repository.  
                To send this type of change you format the patch and then send the email using the Git commands 
                <filename>git format-patch</filename> and <filename>git send-email</filename>. 
                You can find information on how to submit later in this chapter.</para></listitem>
        </itemizedlist>
    </para>
</section>

<section id='tracking-bugs'>
    <title>Tracking Bugs</title>

    <para>
        The Yocto Project uses its own implementation of 
        <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> to track bugs.  
        Implementations of Bugzilla work well for group development because they track bugs and code 
        changes, can be used to communicate changes and problems with developers, can be used to 
        submit and review patches, and can be used to manage quality assurance. 
        The home page for the Yocto Project implementation of Bugzilla is   
        <ulink url='http://bugzilla.yoctoproject.org'>http://bugzilla.yoctoproject.org</ulink>.
    </para>

    <para>
        Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself
        such as when discovering an issue with some component of the build system that acts contrary 
        to the documentation or your expectations.  
        Following is the general procedure for submitting a new bug using the Yocto Project
        Bugzilla.
        You can find more information on defect management, bug tracking, and feature request
        processes all accomplished through the Yocto Project Bugzilla on the wiki page
        <ulink url='https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking'>here</ulink>.     
        <orderedlist>
            <listitem><para>Always use the Yocto Project implementation of Bugzilla to submit
                a bug.</para></listitem>
            <listitem><para>When submitting a new bug, be sure to choose the appropriate
                Classification, Product, and Component for which the issue was found.
                Defects for Yocto Project fall into one of four classifications:  Yocto Projects, 
                Infrastructure, Poky, and Yocto Metadata Layers.
                Each of these Classifications break down into multiple Products and, in some 
                cases, multiple Components.</para></listitem>
            <listitem><para>Use the bug form to choose the correct Hardware and Architecture
                for which the bug applies.</para></listitem>
            <listitem><para>Indicate the Yocto Project version you were using when the issue
                occurred.</para></listitem>
            <listitem><para>Be sure to indicate the Severity of the bug.  
                Severity communicates how the bug impacted your work.</para></listitem>
            <listitem><para>Provide a brief summary of the issue. 
                Try to limit your summary to just a line or two and be sure to capture the 
                essence of the issue.</para></listitem>
            <listitem><para>Provide a detailed description of the issue.  
                You should provide as much detail as you can about the context, behavior, output, 
                and so forth that surround the issue.  
                You can even attach supporting files for output or log by using the "Add an attachment"
                button.</para></listitem>
            <listitem><para>Submit the bug by clicking the "Submit Bug" button.</para></listitem>
        </orderedlist>
    </para>
 
    <note>
        Bugs in the Yocto Project Bugzilla follow naming convention: 
        <filename>[YOCTO #&lt;number&gt;]</filename>, where <filename>&lt;number&gt;</filename> is the 
        assigned defect ID used in Bugzilla.  
        So, for example, a valid way to refer to a defect would be <filename>[YOCTO #1011]</filename>.   
        This convention becomes important if you are submitting patches against the Yocto Project 
        code itself.
    </note>
</section>

<section id='how-to-submit-a-change'>
    <title>How to Submit a Change</title>

    <para>
        Contributions to the Yocto Project are very welcome. 
        You should send patches to the appropriate Yocto Project mailing list to get them 
        in front of the Yocto Project Maintainer.
        For a list of the Yocto Project mailing lists, see the
        "<ulink url='http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html#resources-mailinglist'>Mailing lists</ulink>" section in 
        <ulink url='http://www.yoctoproject.org/docs/latest/poky-ref-manual/poky-ref-manual.html'> The 
        Yocto Project Reference Manual</ulink>.
    </para>

    <para>
        The following is some guidance on which mailing list to use for what type of defect:
        <itemizedlist>
            <listitem><para>For defects against the Yocto Project build system Poky, send 
                your patch to the  
                <ulink url='http://lists.yoctoproject.org/listinfo/poky'></ulink> mailing list. 
                This mailing list corresponds to issues that are not specific to the Yocto Project but
                are part of the OE-core.
                For example, a defect against anything in the <filename>meta</filename> layer 
                or the BitBake Manual could be sent to this mailing list.</para></listitem>
            <listitem><para>For defects against Yocto-specific layers, tools, and Yocto Project 
                documentation use the 
                <ulink url='http://lists.yoctoproject.org/listinfo/yocto'></ulink> mailing list.
                This mailing list corresponds to Yocto-specific areas such as 
                <filename>meta-yocto</filename>, <filename>meta-intel</filename>, 
                <filename>linux-yocto</filename>, and <filename>documentation</filename>.</para></listitem>
        </itemizedlist>      
    </para>

    <para>  
        When you send a patch, be sure to include a "Signed-off-by:"
        line in the same style as required by the Linux kernel. 
        Adding this line signifies the developer has agreed to the Developer's Certificate of Origin 1.1
        as follows:
        <literallayout class='monospaced'>
     Developer's Certificate of Origin 1.1

     By making a contribution to this project, I certify that:

     (a) The contribution was created in whole or in part by me and I
         have the right to submit it under the open source license
         indicated in the file; or

     (b) The contribution is based upon previous work that, to the best
         of my knowledge, is covered under an appropriate open source
         license and I have the right under that license to submit that
         work with modifications, whether created in whole or in part
         by me, under the same open source license (unless I am
         permitted to submit under a different license), as indicated
         in the file; or

     (c) The contribution was provided directly to me by some other
         person who certified (a), (b) or (c) and I have not modified
         it.

     (d) I understand and agree that this project and the contribution
         are public and that a record of the contribution (including all
         personal information I submit with it, including my sign-off) is
         maintained indefinitely and may be redistributed consistent with
         this project or the open source license(s) involved.
        </literallayout>
        A Poky contributions tree (<filename>poky-contrib</filename>, 
        <filename>git://git.yoctoproject.org/poky-contrib.git</filename>)
        exists for contributors to stage contributions.
        If people desire such access, please ask on the mailing list. 
        Usually, the Yocto Project team will grant access to anyone with a proven track 
        record of good patches.
    </para>

    <para>
        In a collaborative environment, it is necessary to have some sort of standard 
        or method through which you submit changes.  
        Otherwise, things could get quite chaotic.  
    </para>

    <para>
        When you create a commit, you must follow certain standards established by the
        Yocto Project development team. 
        For each commit, you must provide a single-line summary of the change and you 
        almost always provide a more detailed description of what you did (i.e. the body 
        of the commit).
        The only exceptions for not providing a detailed description would be if your 
        change is a simple, self-explanatory change that needs no description.
        Here are the Yocto Project commit message guidelines:
        <itemizedlist>
            <listitem><para>Provide a single-line, short summary of the change.
                This summary is typically viewable by source control systems.
                Thus, providing something short and descriptive that gives the reader 
                a summary of the change is useful when viewing a list of many commits.
                </para></listitem>
            <listitem><para>For the body of the commit message, provide detailed information
                that describes what you changed, why you made the change, and the approach
                you used.  
                Provide as much detail as you can in the body of the commit message.
                </para></listitem>
            <listitem><para>If the change addresses a specific bug or issue that is 
                associated with a bug-tracking ID, prefix your detailed description
                with the bug or issue ID.  
                For example, the Yocto Project tracks bugs using a bug-naming convention.
                Any commits that address a bug must start with the bug ID in the description
                as follows:
                <literallayout class='monospaced'>
     YOCTO #&lt;bug-id&gt;:  &lt;Detailed description of commit&gt;
                </literallayout></para></listitem>
        </itemizedlist>
    </para>

    <para>
        You can find more guidance on creating well-formed commit messages at this OpenEmbedded 
        wiki page:
        <ulink url='http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines'></ulink>.
    </para>

    <para>
        Following are general instructions for both pushing changes upstream and for submitting changes as patches.
    </para> 

    <section id='pushing-a-change-upstream'>
        <title>Pushing a Change Upstream and Requesting a Pull</title>

        <para>
            The basic flow for pushing a change to an upstream "contrib" Git repository is as follows:
            <itemizedlist>
                <listitem><para>Make your changes in your local Git repository.</para></listitem>
                <listitem><para>Stage your commit (or change) by using the <filename>git add</filename>
                    command.</para></listitem>
                <listitem><para>Commit the change by using the <filename>git commit</filename>
                    command and push it to the "contrib" repository.  
                    Be sure to provide a commit message that follows the project’s commit standards
                    as described earlier.</para></listitem>
                <listitem><para>Notify the maintainer that you have pushed a change by making a pull 
                    request.
                    The Yocto Project provides two scripts that conveniently let you generate and send
                    pull requests to the Yocto Project.
                    These scripts are <filename>create-pull-request</filename> and 
                    <filename>send-pull-request</filename>.
                    You can find these scripts in the <filename>scripts</filename> directory of the 
                    Yocto Project file structure.</para>
                    <para>For help on using these scripts, simply provide the 
                    <filename>--help</filename> argument as follows:
                    <literallayout class='monospaced'>
     $ ~/poky/scripts/create-pull-request --help
     $ ~/poky/scripts/send-pull-request --help
                    </literallayout></para></listitem>
            </itemizedlist>
        </para>

        <para>            
            You can find general Git information on how to push a change upstream in the 
            <ulink url='http://book.git-scm.com/3_distributed_workflows.html'>Git Community Book</ulink>.
        </para>
    </section>

    <section id='submitting-a-patch'>
        <title>Submitting a Patch Through Email</title>

        <para>
            If you have a just a few changes, you can commit them and then submit them as an 
            email to the maintainer.
            Here is a general procedure:
            <itemizedlist>
                <listitem><para>Make your changes in your local Git repository.</para></listitem>
                <listitem><para>Stage your commit (or change) by using the <filename>git add</filename>
                    command.</para></listitem>
                <listitem><para>Commit the change by using the 
                    <filename>git commit --signoff</filename> command.
                    Using the <filename>--signoff</filename> option identifies you as the person 
                    making the change and also satisfies the Developer's Certificate of 
                    Origin (DCO) shown earlier.</para>
                    <para>When you form a commit you must follow certain standards established by the 
                    Yocto Project development team. 
                    See the earlier section
                    "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>" 
                    for Yocto Project commit message standards.</para></listitem>
                <listitem><para>Format the commit into an email messsage.
                    To format commits, use the <filename>git format-patch</filename> command.
                    When you provide the command, you must include a revision list or a number of patches
                    as part of the command.
                    For example, these two commands each take the most recent single commit and 
                    format it as an email message in the current directory:  
                    <literallayout class='monospaced'>
     $ git format-patch -1
     $ git format-patch HEAD~
                    </literallayout></para>
                    <para>After the command is run, the current directory contains a 
                    numbered <filename>.patch</filename> file for the commit.</para>
                    <para>If you provide several commits as part of the command, 
                    the <filename>git format-patch</filename> command produces a numbered 
                    series of files in the current directory – one for each commit.
                    If you have more than one patch, you should also use the 
                    <filename>--cover</filename> option with the command, which generates a 
                    cover letter as the first "patch" in the series.  
                    You can then edit the cover letter to provide a description for 
                    the series of patches.
                    For information on the <filename>git format-patch</filename> command, 
                    see <filename>GIT_FORMAT_PATCH(1)</filename> displayed using the  
                    <filename>man git-format-patch</filename> command.</para></listitem>
                <listitem><para>Import the files into your mail client by using the 
                    <filename>git send-email</filename> command.
                    <note>In order to use <filename>git send-email</filename>, you must have the 
                    the proper Git packages installed.
                    For Ubuntu and Fedora the package is <filename>git-email</filename>.</note></para>
                    <para>The <filename>git send-email</filename> command sends email by using a local
                    or remote Mail Transport Agent (MTA) such as 
                    <filename>msmtp</filename>, <filename>sendmail</filename>, or through a direct
                    <filename>smtp</filename> configuration in your Git <filename>config</filename>
                    file.
                    If you are submitting patches through email only, it is very important
                    that you submit them without any whitespace or HTML formatting that 
                    either you or your mailer introduces.
                    The maintainer that receives your patches needs to be able to save and 
                    apply them directly from your emails.
                    A good way to verify that what you are sending will be applicable by the 
                    maintainer is to do a dry run and send them to yourself and then 
                    save and apply them as the maintainer would.</para>
                    <para>The <filename>git send-email</filename> command is the preferred method
                    for sending your patches since there is no risk of compromising whitespace
                    in the body of the message, which can occur when you use your own mail client.
                    The command also has several options that let you 
                    specify recipients and perform further editing of the email message.
                    For information on how to use the <filename>git send-email</filename> command,
                    use the <filename>man git-send-email</filename> command.</para></listitem>
            </itemizedlist>
        </para>
    </section>
</section>
















</chapter>
<!--
vim: expandtab tw=80 ts=4
-->