From cd9279ab5cb572d804806283c13857346dc54de8 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Tue, 20 Jun 2017 06:37:18 -0700 Subject: dev-manual: Updated the section on the YP development environment Fixes [YOCTO #11630] Extensively updated the development environment section by creating a task-oriented section. (From yocto-docs rev: d440f0b5ceb1aca57a73d8bced515ae0bd3188f7) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- documentation/dev-manual/dev-manual-newbie.xml | 631 +++++++++++++------------ 1 file changed, 322 insertions(+), 309 deletions(-) (limited to 'documentation/dev-manual/dev-manual-newbie.xml') diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml index 1bb82b711d..1d9926c6cd 100644 --- a/documentation/dev-manual/dev-manual-newbie.xml +++ b/documentation/dev-manual/dev-manual-newbie.xml @@ -7,12 +7,12 @@ The Yocto Project Open Source Development Environment
- Using the Yocto Project in a Team Environment + Setting Up a Team Yocto Project Development Environment It might not be immediately clear how you can use the Yocto - Project in a team environment, or scale it for a large team of - developers. + Project in a team development environment, or scale it for a large + team of developers. One of the strengths of the Yocto Project is that it is extremely flexible. Thus, you can adapt it to many different use cases and scenarios. @@ -21,337 +21,350 @@ - To help with these types of situations, this section presents - some of the project's most successful experiences, - practices, solutions, and available technologies that work well. - Keep in mind, the information here is a starting point. + To help you understand how to set up this type of environment, + this section presents a procedure that gives you the information + to learn how to get the results you want. + The procedure is high-level and presents some of the project's most + successful experiences, practices, solutions, and available + technologies that work well. + Keep in mind, the procedure here is a starting point. You can build off it and customize it to fit any particular working environment and set of practices. - - -
- System Configurations - - - Systems across a large team should meet the needs of - two types of developers: those working on the contents of the - operating system image itself and those developing applications. - Regardless of the type of developer, their workstations must - be both reasonably powerful and run Linux. - + + + Determine Who is Going to be Developing: + You need to understand who is going to be doing anything + related to the Yocto Project and what their roles would be. + Making this determination is essential to completing the + steps two and three, which are to get your equipment together + and set up your development environment's hardware topology. + + + The following roles exist: + + + Application Development: + These types of developers do application level work + on top of an existing software stack. + + + Core System Development: + These types of developers work on the contents of the + operating system image itself. + + + Build Engineer: + This type of developer manages Autobuilders and + releases. + Not all environments need a Build Engineer. + + + Test Engineer: + This type of developer creates and manages automated + tests needed to ensure all application and core + system development meets desired quality standards. + + + + + Gather the Hardware: + Based on the size and make-up of the team, get the hardware + together. + Any development, build, or test engineer should be using + a system that is running a supported Linux distribution. + Systems, in general, should be high performance (e.g. dual, + six-core Xeons with 24 Gbytes of RAM and plenty of disk space). + You can help ensure efficiency by having any machines used + for testing or that run Autobuilders be as high performance + as possible. + + + Understand the Hardware Topology of the Environment: + Now that you know how many developers and support engineers + are required, you can understand the topology of the + hardware environment. + The following figure shows a moderately sized Yocto Project + development environment. -
- Application Development + + Need figure. - - For developers who mainly do application level work - on top of an existing software stack, - the following list shows practices that work best. - For information on using a Software Development Kit (SDK), see - the - Yocto Project Software Development Kit (SDK) Developer's Guide: + + + Use Git as Your Source Control Manager (SCM): + Keeping your + Metadata + and any software you are developing under the + control of an SCM system that is compatible + with the OpenEmbedded build system is advisable. + Of the SCMs BitBake supports, the + Yocto Project team strongly recommends using + Git. + Git is a distributed system that is easy to backup, + allows you to work remotely, and then connects back to the + infrastructure. + + For information about BitBake, see the + BitBake User Manual. + + + It is relatively easy to set up Git services and create + infrastructure like + http://git.yoctoproject.org, + which is based on server software called + gitolite with cgit + being used to generate the web interface that lets you view the + repositories. + The gitolite software identifies users + using SSH keys and allows branch-based + access controls to repositories that you can control as little + or as much as necessary. + + + The setup of these services is beyond the scope of this + manual. + However, sites such as these exist that describe how to + perform setup: + + + Git documentation: + Describes how to install gitolite + on the server. + + + The gitolite master index: + All topics for gitolite. + + + Interfaces, frontends, and tools: + Documentation on how to create interfaces and frontends + for Git. + + + + + + Set up the Application Development Machines: + As mentioned earlier, application developers are creating + applications on top of existing software stacks. + Following are some best practices for setting up machines + that do application development: - Use a pre-built toolchain that + + Use a pre-built toolchain that contains the software stack itself. Then, develop the application code on top of the stack. This method works well for small numbers of relatively - isolated applications. - When possible, use the Yocto Project - plug-in for the Eclipse IDE + isolated applications. + + + When possible, use the Yocto Project + plug-in for the + Eclipse IDE and SDK development practices. For more information, see the "Yocto Project Software Development Kit (SDK) Developer's Guide". - Keep your cross-development toolchains - updated. + + Keep your cross-development toolchains updated. You can do this through provisioning either as new toolchain downloads or as updates through a package update mechanism using opkg to provide updates to an existing toolchain. The exact mechanics of how and when to do this are a - question for local policy. - Use multiple toolchains installed locally + question for local policy. + + + Use multiple toolchains installed locally into different locations to allow development across - versions. + versions. + - -
- -
- Core System Development - - - For core system development, it is often best to have the - build system itself available on the developer workstations - so developers can run their own builds and directly - rebuild the software stack. - You should keep the core system unchanged as much as - possible and do your work in layers on top of the core system. - Doing so gives you a greater level of portability when - upgrading to new versions of the core system or Board - Support Packages (BSPs). - You can share layers amongst the developers of a particular - project and contain the policy configuration that defines - the project. - - - - Aside from the previous best practices, there exists a number - of tips and tricks that can help speed up core development - projects: + + + Set up the Core Development Machines: + As mentioned earlier, these types of developers work on the + contents of the operating system itself. + Following are some best practices for setting up machines + used for developing images: - Use a - Shared State Cache - (sstate) among groups of developers who are on a - fast network. - The best way to share sstate is through a - Network File System (NFS) share. - The first user to build a given component for the - first time contributes that object to the sstate, - while subsequent builds from other developers then - reuse the object rather than rebuild it themselves. - - Although it is possible to use other protocols for the - sstate such as HTTP and FTP, you should avoid these. - Using HTTP limits the sstate to read-only and - FTP provides poor performance. + + Have the Yocto Project build system itself available on + the developer workstations so developers can run their own + builds and directly rebuild the software stack. - Have autobuilders contribute to the sstate - pool similarly to how the developer workstations - contribute. - For information, see the - "Autobuilders" - section. - Build stand-alone tarballs that contain - "missing" system requirements if for some reason - developer workstations do not meet minimum system - requirements such as latest Python versions, - chrpath, or other tools. - You can install and relocate the tarball exactly as you - would the usual cross-development toolchain so that - all developers can meet minimum version requirements - on most distributions. - Use a small number of shared, - high performance systems for testing purposes - (e.g. dual, six-core Xeons with 24 Gbytes of RAM - and plenty of disk space). - Developers can use these systems for wider, more - extensive testing while they continue to develop - locally using their primary development system. + + Keep the core system unchanged as much as + possible and do your work in layers on top of the + core system. + Doing so gives you a greater level of portability when + upgrading to new versions of the core system or Board + Support Packages (BSPs). - Enable the PR Service when package feeds - need to be incremental with continually increasing - PR - values. - Typically, this situation occurs when you use or - publish package feeds and use a shared state. - You should enable the PR Service for all users who - use the shared state pool. - For more information on the PR Service, see the - "Working With a PR Service". + + Share layers amongst the developers of a + particular project and contain the policy configuration + that defines the project. - -
-
- -
- Source Control Management (SCM) - - - Keeping your - Metadata - and any software you are developing under the - control of an SCM system that is compatible - with the OpenEmbedded build system is advisable. - Of the SCMs BitBake supports, the - Yocto Project team strongly recommends using - Git. - Git is a distributed system that is easy to backup, - allows you to work remotely, and then connects back to the - infrastructure. - - For information about BitBake, see the - BitBake User Manual. - - - - - It is relatively easy to set up Git services and create - infrastructure like - http://git.yoctoproject.org, - which is based on server software called - gitolite with cgit - being used to generate the web interface that lets you view the - repositories. - The gitolite software identifies users - using SSH keys and allows branch-based - access controls to repositories that you can control as little - or as much as necessary. - - - - The setup of these services is beyond the scope of this manual. - However, sites such as these exist that describe how to perform - setup: - - Git documentation: - Describes how to install gitolite - on the server. - The gitolite master index: - All topics for gitolite. - - Interfaces, frontends, and tools: - Documentation on how to create interfaces and frontends - for Git. - - -
- -
- Autobuilders - - - Autobuilders are often the core of a development project. - It is here that changes from individual developers are brought - together and centrally tested and subsequent decisions about - releases can be made. - Autobuilders also allow for "continuous integration" style - testing of software components and regression identification - and tracking. - - - - See "Yocto Project Autobuilder" - for more information and links to buildbot. - The Yocto Project team has found this implementation - works well in this role. - A public example of this is the Yocto Project - Autobuilders, which we use to test the overall health of the - project. - - - - The features of this system are: - - Highlights when commits break the build. - - Populates an sstate cache from which - developers can pull rather than requiring local - builds. - Allows commit hook triggers, - which trigger builds when commits are made. - - Allows triggering of automated image booting - and testing under the QuickEMUlator (QEMU). - - Supports incremental build testing and - from-scratch builds. - Shares output that allows developer - testing and historical regression investigation. - - Creates output that can be used for releases. - - Allows scheduling of builds so that resources - can be used efficiently. - - -
- -
- Policies and Change Flow - - - The Yocto Project itself uses a hierarchical structure and a - pull model. - Scripts exist to create and send pull requests - (i.e. create-pull-request and - send-pull-request). - This model is in line with other open source projects where - maintainers are responsible for specific areas of the project - and a single maintainer handles the final "top-of-tree" merges. - - - - You can also use a more collective push model. - The gitolite software supports both the - push and pull models quite easily. - - - - As with any development environment, it is important - to document the policy used as well as any main project - guidelines so they are understood by everyone. - It is also a good idea to have well structured - commit messages, which are usually a part of a project's - guidelines. - Good commit messages are essential when looking back in time and - trying to understand why changes were made. - - - - If you discover that changes are needed to the core layer of the - project, it is worth sharing those with the community as soon - as possible. - Chances are if you have discovered the need for changes, someone - else in the community needs them also. - -
- -
- Summary - - - This section summarizes the key recommendations described in the - previous sections: - - Use Git - as the source control system. - Maintain your Metadata in layers that make sense - for your situation. - See the "Understanding - and Creating Layers" section for more information on - layers. - - Separate the project's Metadata and code by using - separate Git repositories. - See the - "Yocto Project Source Repositories" - section for information on these repositories. - See the - "Getting Set Up" - section for information on how to set up local Git - repositories for related upstream Yocto Project - Git repositories. - - Set up the directory for the shared state cache - (SSTATE_DIR) - where it makes sense. - For example, set up the sstate cache on a system used - by developers in the same organization and share the - same source directories on their machines. - - Set up an Autobuilder and have it populate the - sstate cache and source directories. - The Yocto Project community encourages you - to send patches to the project to fix bugs or add features. - If you do submit patches, follow the project commit - guidelines for writing good commit messages. - See the "How to Submit a Change" - section. - Send changes to the core sooner than later - as others are likely to run into the same issues. - For some guidance on mailing lists to use, see the list in the - "How to Submit a Change" - section. - For a description of the available mailing lists, see the - "Mailing Lists" - section in the Yocto Project Reference Manual. - - - -
+ + + Set up an Autobuilder: + Autobuilders are often the core of the development + environment. + It is here that changes from individual developers are brought + together and centrally tested and subsequent decisions about + releases can be made. + Autobuilders also allow for "continuous integration" style + testing of software components and regression identification + and tracking. + + See "Yocto Project Autobuilder" + for more information and links to buildbot. + The Yocto Project team has found this implementation + works well in this role. + A public example of this is the Yocto Project + Autobuilders, which we use to test the overall health of the + project. + + The features of this system are: + + + Highlights when commits break the build. + + + Populates an sstate cache from which + developers can pull rather than requiring local + builds. + + + Allows commit hook triggers, + which trigger builds when commits are made. + + + Allows triggering of automated image booting + and testing under the QuickEMUlator (QEMU). + + + Supports incremental build testing and + from-scratch builds. + + + Shares output that allows developer + testing and historical regression investigation. + + + Creates output that can be used for releases. + + + Allows scheduling of builds so that resources + can be used efficiently. + + + + + Set up Test Machines: + Use a small number of shared, high performance systems + for testing purposes. + Developers can use these systems for wider, more + extensive testing while they continue to develop + locally using their primary development system. + + + Document Policies and Change Flow: + The Yocto Project itself uses a hierarchical structure and a + pull model. + Scripts exist to create and send pull requests + (i.e. create-pull-request and + send-pull-request). + This model is in line with other open source projects where + maintainers are responsible for specific areas of the project + and a single maintainer handles the final "top-of-tree" merges. + + You can also use a more collective push model. + The gitolite software supports both the + push and pull models quite easily. + + + As with any development environment, it is important + to document the policy used as well as any main project + guidelines so they are understood by everyone. + It is also a good idea to have well structured + commit messages, which are usually a part of a project's + guidelines. + Good commit messages are essential when looking back in time and + trying to understand why changes were made. + + If you discover that changes are needed to the core + layer of the project, it is worth sharing those with the + community as soon as possible. + Chances are if you have discovered the need for changes, + someone else in the community needs them also. + + + Development Environment Summary: + Aside from the previous steps, some best practices exist + within the Yocto Project development environment. + Consider the following: + + + Use Git + as the source control system. + + + Maintain your Metadata in layers that make sense + for your situation. + See the "Understanding + and Creating Layers" section for more information on + layers. + + + Separate the project's Metadata and code by using + separate Git repositories. + See the + "Yocto Project Source Repositories" + section for information on these repositories. + See the + "Getting Set Up" + section for information on how to set up local Git + repositories for related upstream Yocto Project + Git repositories. + + + Set up the directory for the shared state cache + (SSTATE_DIR) + where it makes sense. + For example, set up the sstate cache on a system used + by developers in the same organization and share the + same source directories on their machines. + + + Set up an Autobuilder and have it populate the + sstate cache and source directories. + + + The Yocto Project community encourages you + to send patches to the project to fix bugs or add features. + If you do submit patches, follow the project commit + guidelines for writing good commit messages. + See the "How to Submit a Change" + section. + + + Send changes to the core sooner than later + as others are likely to run into the same issues. + For some guidance on mailing lists to use, see the list in the + "How to Submit a Change" + section. + For a description of the available mailing lists, see the + "Mailing Lists" + section in the Yocto Project Reference Manual. + + + + +
-- cgit v1.2.3-54-g00ecf