From 4b6d42f629ce8a503fc49386325b4e6018b28c01 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Fri, 15 Jul 2011 12:41:50 -0700 Subject: documentation/dev-manual/dev-manual-newbie.xml: Initial content added. This content is the first cut for the chapter. The original was a copy of some adt-manual chapter. (From yocto-docs rev: 4a563c3ed1e82837410ed114a07e39a0ccaad6f7) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/dev-manual/dev-manual-newbie.xml | 541 +++++++++++++++++++++++-- 1 file changed, 511 insertions(+), 30 deletions(-) (limited to 'documentation') diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml index 5ca2c25b52..68a2f4a211 100644 --- a/documentation/dev-manual/dev-manual-newbie.xml +++ b/documentation/dev-manual/dev-manual-newbie.xml @@ -5,44 +5,516 @@ Working with Open Source Code -
- Introduction + + This chapter presents information for users new or unfamiliar with working in an open source environment. + Working in this type of environment is different than working in a closed, proprietary environment. + The chapter also presents information specific to the Yocto Project environment. + It specifically addresses licensing issues, code repositories, the open-source distributed version control + system Git, and best practices within Yocto Project. + + + + If you are a seasoned open source developer and are familiar with Git, you might just be interested + in the Yocto Project specific information in this chapter. + + +
+ Open Source Philosophy + - The Yocto Project presents the kernel as a fully patched, history-clean Git - repository. - The Git tree represents the selected features, board support, - and configurations extensively tested by Yocto Project. - The Yocto Project kernel allows the end user to leverage community - best practices to seamlessly manage the development, build and debug cycles. + Open source philosophy is characterized by software development directed by peer production, + bartering, and collaboration through a concerned community of developers. + Contrast this to the more standard centralized development models used by commercial software + companies where a finite set of developers produce a product for sale using a defined set + of procedures that ultimately result in an end-product whose architecture and source material + are closed to the public. + - This manual describes the Yocto Project kernel by providing information - on its history, organization, benefits, and use. - The manual consists of two sections: - - Concepts - Describes concepts behind the kernel. - You will understand how the kernel is organized and why it is organized in - the way it is. You will understand the benefits of the kernel's organization - and the mechanisms used to work with the kernel and how to apply it in your - design process. - Using the Kernel - Describes best practices and "how-to" information - that lets you put the kernel to practical use. Some examples are "How to Build a - Project Specific Tree", "How to Examine Changes in a Branch", and "Saving Kernel - Modifications." - + Open source projects conceptually have differing concurrent agendas, approaches, and production. + These facets of the development process can come from anyone in the public (community) that has a + stake in the software project. + The open source environment contains new copyright, licensing, domain, and consumer issues + that differ from the more traditional development environment. + In an open source environment the end-product, source material, and documentation are + all available to the public at no cost. + + + + A benchmark example of an open source project is the Linux Kernel, which was initially conceived + and created by Finnish computer science student Linus Torvalds in 1991. + Conversely, a good example of a non-open source project is the Windows family of operating + systems developed by Microsoft Corporation. + + + + Wikipedia has a good historical description of the Open Source Philosophy + here. + + + + You can also find helpful information on how to participate in the Linux Community + here. + +
+ +
+ Yocto Project Repositories + + + The Yocto Project team maintains complete source repositories that allow further development + of Yocto Project, its tools, Board Support Packages, and so forth. + As a developer who uses Yocto Project, however, you need only to be able to access your + kernel or application source code and any layers (modifications) on which you might be working. + + + + For any supported release of Yocto Project you can go to the Yocto Project website’s + download page and get a + .bz2 tarball of the release. + You can also go to this site to download any supported BSP tarballs. + + + + After obtaining the code, you can unpack the tarballs and have a working Git repository + from which you can develop. + Or, you can take steps to create local repositories of Yocto Project source code and metadata on + your development system. + See the information + here + for information on how to set up these local Git repositories. + + + + Should you be interested in locations of complete Yocto Project development code, there are + two areas where this code is maintained: + + Source Repositories: + This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto + Metadata Layers. + Index of /downloads: + This area contains an index of the Eclipse-plugin, miscellaneous support, poky, pseudo, and + all released versions of Yocto Project. + [WRITER NOTE: link will be http://downloads.yoctoproject.org.] + + +
+ +
+ Licensing + + + Because open source projects are open to the public they have different licensing structures in place. + License evolution for both Open Source and Free Software has an interesting history. + If you are interested in the history you can find basic information here: + + Open source license history + + Free software license + history + + + + + In general, Yocto Project is broadly licensed under the Massachusetts Institute of Technology + (MIT) License. + MIT licensing permits the reuse of software within proprietary software as long as the + license is distributed with that software. + MIT is also compatible with the GNU General Public License (GPL). + Patches to the Yocto Project follow the up-stream licensing scheme. + + + + You can find information on the MIT License here. + You can find information on the GNU GPL here. + + + + When you build an image using Yocto Project the build process uses a known list of licenses to + ensure compliance. + Once the build completes the list of all licenses found and used during the build are + kept in the resulting build directory at + <build_directory>/tmp/deploy/images/licenses. + If a module requires a license that is not in the base list then the build process + generates a warning during the build. + It is up to the developer to resolve potential licensing issues. + + + + The base list of licenses used by the build process is a combination of the Software Package + Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects. + SPDX Group is a working group of the Linux Foundation + that maintains a specification + for a standard format for communicating the components, licenses, and copyrights + associated with a software package. + OSI is a corporation dedicated to the Open Source + Definition and the effort for reviewing + and approving licenses that are OSD-conformant. + + + + You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses + here. + The wiki page discusses the license infrastructure used by the Yocto Project. + +
+ +
+ Git + + + The Yocto Project uses Git, which is a free, open source distributed version control. + Git supports distributed development, non-linear development, can handle large projects, + cryptographic authentication of history, and toolkit design. + It is best that you know how to work with Git if you are going to use Yocto Project for development. + + + + Git has an extensive set of commands that lets you manage and collaborate changes over the life + of a project. + Conveniently though, you can manage with a small set of basic operations and workflows + once you understand the basic philosophy behind Git. + You do not have to be an expert in Git to be functional. + A good place to look for instruction on a minimal set of Git commands is + here. + If you need to download Git you can do so + here. + + + + Git works by using branching techniques that track content change (not files) + within a project (e.g. a new feature or updated documentation). + Creating a tree-like structure based on project divergence allows for excellent historical + information over the life of a project. + This methodology also allows for an environment in which you can do lots of + experimentation on your project as you develop changes or new features. + For example, you can create a “branch”, experiment with some feature, and then + if you like the feature you incorporate the branch into the tree. + If you don’t, you cut the branch off by deleting it. + + + + If you don’t know much about Git it is strongly suggested that you educate + yourself by visiting the links previously mentioned. + + + + The following list briefly describes some basic Git operations as a way to get started. + As with any set of commands, this list (in most cases) simply shows the base command and + omits the many arguments they support. + See the Git documentation for complete descriptions and strategies on how to use these commands: + + git init – Initializes an empty Git repository. + You cannot use Git commands unless you have a .git repository. + git clone – Creates a clone of a repository. + During collaboration this command allows you to create a local repository that is on + equal footing with a fellow developer’s repository. + git add – Adds updated file contents to the index that + Git uses to track changes. + All files that have changed must be added before they can be committed. + git commit – Creates a “commit” that documents the changes you made. + Commits are used for historical purposes, for determining if a maintainer of a project + will allow the change, and for ultimately pushing the change from your local Git repository + into the project’s upstream (or master) repository. + git status – Reports any modified files that possibly need + added and committed. + git checkout <branch-name> - Changes + your working branch. This command is analogous to “cd”. + git checkout –b <working-branch> - Creates + a working branch on your local machine where you can isolate work. + It is a good idea to use local branches when adding specific features or changes. + This way if you don’t like what you have done you can easily get rid of the work. + git branch – Reports existing branches and tells you which + branch in which you are currently working. + git pull – Retrieves information from an upstream Git + repository and places it in your local Git repository. + You use this command to make sure you are synchronized with the upstream repository + from which the project’s maintainer uses to pull changes into the master repository. + git push – Sends all your local changes you have committed + to an upstream Git repository. + The maintainer of the project draws from these repositories when adding your changes to the + project’s master repository. + git merge – Combines or adds changes from one local branch of + your repository with another branch. + When you create a local Git repository the default branch is named “master”. + A typical workflow is to create a temporary branch for isolated work, make and commit your + changes, switch to the master branch, merge the changes in the temporary branch with the + master branch, and then delete the temporary branch + git cherry-pick – Choose and apply specific commits from + one branch into another branch. + There are times when you might not be able to merge all the changes in one branch with + another but need to pick out certain ones. + gitk – Provides a GUI view of the branches and changes in + your local Git repository. + This command is a good way to see where things have diverged in your local repository. + git log – Reports a history of your changes to the + repository. + + +
+ +
+ Workflows + + + This section provides some overview on workflows using Git. + In particular, the information covers basic practices that describe roles and actions in a + collaborative development environment. + Again, if you are familiar with this type of development environment you might want to just skip the section. + + + + Following are the definitions for some terms used in the Yocto Project. + [WRITER NOTE: I need to move this list of definitions somewhere useful.] + + Image - An image is a collection of recipes created with + Bitbake (baked). Images run on specific hardware and use cases. + Recipe - A set of instructions for building packages. + A recipe describes where you get the source and which patches to apply. + Recipes describe dependencies for libraries or for other recipes and they + also contain configuration and compilation options. + Recipes also let you ‘install’ customizations. + Recipes contain the logical unit of execution, the software/images to build and + use the .bb file extension. + BitBake - The task executor and scheduler used by Yocto Project + to build images. + For more information on BitBake, see the + BitBake documentation. + Package - A collection of ‘baked’ recipes. + You ‘bake’ something by running it through Bitbake. + Layer - A logical collection of recipes representing the core, + a BSP, or an application stack. + Metadata - Information for a build that is generally + architecture-independent. + This information includes Task definitions in recipes, classes, and configuration + information. + Configuration File: Configuration information in the + .conf files provides global definition of variables. + The build/conf/local.conf configuration file defines local user-defined variables. + The distro/poky.conf configuration file defines Yocto ‘distro’ configuration + variables. + The machine/beagleboard.conf configuration file defines machine-specific variables. + Configuration files end with a .conf filename extension. + Classes - Files that encapsulate and inherit logic. + Class files end with the .bbclass filename extension. + Tasks - Arbitrary groups of software used to contain Recipes. + You simply use Tasks to hold recipes that when build usually accomplished a single task. + For example, a task could contain the recipes for a company’s proprietary or value-add software. + Or the task could contain the recipes that enable graphics. + A task is really just another recipe. + Because task files are recipes, they end with the .bb filename + extension. + Common OE-Core - A core set of metadata originating + with OpenEmbedded (OE) that is shared between OE and the Yocto Project. + + + + + A master Git repository exists that contains the project. + Usually a key individual is responsible for this repository. + It is the “upstream” repository where the final builds of the project occur. + The maintainer is responsible for allowing changes in from other developers and for + organizing the branch structure of the repository to reflect release strategies and so forth. + + + + The maintainer of the project also owns a contribution repository usually known as a “contrib” area. + The contrib area temporarily holds changes to the project that have been submitted or committed + by the development team. + The maintainer determines if the changes are qualified to be moved into the master repository. + + + + Developers create and maintain cloned repositories of the upstream master repository. + These repositories are local to their development platforms and are used to develop changes. + When a developer is satisfied with a particular feature or change they “push” the changes + up to the contrib repository. + Developers are responsible for keeping their local repository up-to-date with the master + repository. + They are also responsible for straightening out any conflicts that might arise within files + that are being worked on simultaneously by more than one person. + All this work is done locally on the developer’s machine before anything is pushed upstream + and examined at the maintainer’s level. + + + + A somewhat formal method exists by which developers commit changes and push them into the + contrib area and subsequently request that the maintainer include them into the master repository. + This process is called “submitting a patch” or “submitting a change.” + - For more information on the kernel, see the following links: + To summarize the environment: we have a single point of entry for changes into the project’s + master repository, which is controlled by the project’s maintainer. + And, we have a set of developers who independently develop, test, and submit changes + upstream for the maintainer to examine. + The maintainer then chooses which changes are going to become permanently a part of the project. + + + + [WRITER NOTE: Would like a figure here for Git workflow] + + + + While each development environment is unique, there are some best practices or methods + that help development run smoothly. + The following list describes some of these practices. + For more detailed information about these strategies see + Git Workflows. + + Small Changes - It is best to keep your changes you commit + small as compared to bundling many disparate changes into a single commit. + This practice not only keeps things manageable but also allows the maintainer + to more easily include or refuse changes. + Use Branches Liberally - It is very easy to create, use, and + delete local branches in your working Git repository. + You can name these branches anything you like. + It is helpful to give them names associated with the particular feature or change + on which you are working. + Once you are done with a feature or change you simply discard the branch. + Merge Changes - The Git merge command allows you to take the + changes from one branch and fold them into another branch. + This process is especially helpful when more than a single developer might be working + on different parts of the same feature. + Merging changes also automatically identifies any collisions or “conflicts” + that might happen resulting from the same lines of code be altered by two different + developers. + Manage Branches - Because branches are easy to use, you should + use a system where branches indicate varying levels of code readiness. + For example, you can have a “work” branch to develop in, a “test” branch where the code or + change is tested, a “stage” branch where changes are ready to be committed, and so forth. + As your project develops, you can merge code across the branches to reflect ever-increasing + stable states of the development. + Use Push and Pull - The push-pull workflow is based on the + concept of developers “pushing” local commits upstream to the remote repository, which is + usually a contribution repository. + It is also based on the developers “pulling” known states of the project down into their + local development repositories. + This workflow easily allows you to pull changes submitted by other developers from the + upstream repository into your work area ensuring that you have the most recent software + on which to develop. + Patch Workflow - This workflow allows you to notify the + maintainer through an email that you have a change (or patch) you would like considered + for the master repository. + To send this type of change you format the patch and then send the email using the Git commands + git format-patch and git send-email. + You can find information on how to submit later in this chapter. + + +
+ +
+ Tracking Bugs + + + The Yocto Project uses Bugzilla to track bugs. + This bug-tracking application works well for group development because it tracks bugs and code + changes, can be used to communicate changes and problems with developers, can be used to + submit and review patches, and can be used to manage quality assurance. + You can find a good overview of Bugzilla here. + + + + Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself. + While normally this is a process relevant only to Yocto Project developers, you can find information + for Bugzilla configuration and bug tracking procedures specific to the Yocto Project + here. + + + + The Yocto Project uses its own version of the Bugzilla application. + You can find the home page here. + You need to use this implementation of Bugzilla when logging a defect against anything released + by the Yocto Project team. + + + + Here are some things to remember when dealing with bugs against the Yocto Project: + + The Yocto Project follows a naming bug-naming convention: + [YOCTO <number>], where <number> is the + assigned defect ID used in Bugzilla. + So, for example, a valid way to refer to a defect when creating a commit comment + would be [YOCTO 1011]. + This convention becomes important if you are submitting patches against the Yocto Project + code itself (see the next section “How to Submit a Change”). + Defects for Yocto Project fall into one of four classifications: Yocto Projects, + Infrastructure, Poky, and Yocto Metadata Layers. + + +
+ +
+ How to Submit a Change + + + During the development process it is necessary to submit your changes to the maintainer + of the project. + Furthermore, in a collaborative environment it is necessary to have some sort of standard + or method through which you submit changes. + Otherwise, things would get quite chaotic. + + + + Sometimes you might find it necessary to submit a change or patch to the Yocto Project. + If so, you must follow certain procedures. + In particular, the headers in patches and the commit messages must follow a certain standard. + The general process is the same as described earlier in this section. + For complete details on how to create proper commit messages and patch headers see + [WRITER NOTE: I need the link to Mark's wiki page here that describes the process.] + + + + Following are general instructions for both pushing changes upstream and for submitting changes as patches. + + +
+ Pushing a Change Upstream + + + The basic flow for pushing a change to an upstream contrib repository is as follows: - - - + Make your changes in your local repository. + Stage your commit (or change) by using the git add + command. + Commit the change by using the git commit + command and push it to an upstream contrib repository. + Be sure to provide a commit message that follows the project’s commit standards. + Notify the maintainer that you have pushed a change. - - You can find more information on Yocto Project by visiting the website at - . + You can find detailed information on how to push a change upstream + + here. - +
+ +
+ Submitting a Patch + + + If you have a just a few changes you can commit them and then submit them as an email to the maintainer. + Here is the general procedure: + + Make your changes in your local repository. + Stage your commit (or change) by using the git add + command. + Commit the change by using the git commit command. + Be sure to provide a commit message that follows the project’s commit standards. + Format the commit by using the git-format-patch + command. + This step produces a numbered series of files in the current directory – one for + each commit. + Import the files into your mail client by using the + git-send-email command. + Send the email by hand to the maintainer. + + Be aware that there could be protocols and standards that you need to follow for your particular + project. + You can find detailed information on the general process + + here. + +
@@ -51,6 +523,15 @@ + + + + + + + + +